Austin Group Defect Tracker

Aardvark Mark III


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0000248 [1003.1(2008)/Issue 7] Shell and Utilities Objection Error 2010-04-30 20:20 2011-07-06 23:59
Reporter dwheeler View Status public  
Assigned To ajosey
Priority normal Resolution Open  
Status Under Review  
Name David A. Wheeler
Organization
User Reference
Section Shell & Utilities
Page Number 2789,3003,3260,3385,and more
Line Number 91004,99054-99063,108778,113329-113334,and more
Interp Status ---
Final Accepted Text
Summary 0000248: Fix the numerous filename/pathname processing errors in specification examples
Description Many examples in the specification fail, sometimes spectacularly, when given filenames or pathnames permitted by the specification. The specification currently permits filenames to include newline, return, tab, escape, leading "-", double-quotes, single-quotes, ampersand, asterisk, and other "interesting" characters. However, many of the specification's examples will fail to work correctly when such filenames are present.

In a few cases, there's a note that the example presumes that some of these will not occur, but there is no standard way to *ensure* that such filenames cannot occur. In addition, many attackers have learned to *intentionally* create such filenames, since there is no way to prevent them in general. Thus, multi-user systems, or systems that receive external data (e.g., via USB sticks and network drives) are still at risk if they used these examples. This incorrect filename handling can be the basis of security problems, as noted in:
 http://www.dwheeler.com/essays/fixing-unix-linux-filenames.html [^]
It's worth noting that this is not a new problem. Line 82265 noted that the specification has been changed before "to remove the unreliable use of find | xargs".


The examples will need to be examined for patterns such as these:

# WRONG. Fails if newlines/tabs/space/backslash/apostrophe/double-quote/
# ampersand in filenames:
find . -type f | xargs COMMAND

# WRONG. Fails if a file in the current directory begins with "-" or
# if there are no files in the current directory:
COMMAND *
for file in * ; do
 COMMAND "$file"
done



Please either:
1. Fix the various errors. Below are some of them, to get started. It's concerning that even the experts who developed this specification (and they're definitely experts!) made these mistakes. I believe this shows that it is too difficult to correctly use systems while conforming to this specification.
2. Change the rules on filenames to forbid certain filenames. In particular, forbidding control characters in filenames (at least bytes 1-31 and maybe 127) would completely eliminate a number of the problems. In addition, forbidding leading "-" would solve some more (though not as many as the control-characters one).

Desired Action Page 2789, Line 91004:
Change:
 find . −type f | xargs hash
To:
 find . -type f -exec hash {} \;
Rationale:
 This example tries to explain why hash won't work in this case,
 but the example should only have *that* error, not *other* errors as well.

Page 3003, line 99054-99063:
Remove this entire example.
I believe it is not possible to do what this example is attempting to do
while using standard POSIX utilities, at least by using pax.
This example attempts to determine if the filenames in a pax archive
are valid, but filenames can include newline, and pax does not have any
way to disambiguate these cases (e.g., by having a -0 option) in its output.
POSIX tools simply aren't capable of doing accurate filename checking
in this case, at least not without re-implementing (say, in C) a significant
portion of pax.

Page 3260, line 108778:
Replace:
  find . −type f | xargs type
with:
  find . −type f -exec type {} \;

Page 3385, lines 113329-113334:
Remove this example, or change it to work properly
(and that may not be easy).
Although it documents that it will not work if
filenames contain newline, there's no way to be guaranteed of that.
It also presumes that the filenames don't have {}, which isn't guaranteed either. If this was executed on a system where a filename DID contain a newline,
or where the source/target directories contained {},
*disaster* could ensue, and examples should not recommend
techniques that might be disastrous.
Most of the other examples on page 3385 should arguably be
removed as well, since there's no mechanism for guaranteeing that
filenames do not include newline (if xargs -0 is added, then they
can probably be easily repaired and kept).

Tags No tags attached.
Attached Files

- Relationships

-  Notes
(0000885)
Don Cragun (manager)
2011-07-06 23:59

We agree that there are problems in the examples listed.
The fixes for these issues will be delayed until 0000251 is resolved.
The resolution to 0000251 is expected to make many of these
examples much easier to implement reliably using standard features.

- Issue History
Date Modified Username Field Change
2010-04-30 20:20 dwheeler New Issue
2010-04-30 20:20 dwheeler Status New => Under Review
2010-04-30 20:20 dwheeler Assigned To => ajosey
2010-04-30 20:20 dwheeler Name => David A. Wheeler
2010-04-30 20:20 dwheeler Section => Shell & Utilities
2010-04-30 20:20 dwheeler Page Number => 2789,3003,3260,3385,and more
2010-04-30 20:20 dwheeler Line Number => 91004,99054-99063,108778,113329-113334,and more
2011-07-06 23:59 Don Cragun Note Added: 0000885


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker