Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0000903 [1003.1(2013)/Issue7+TC1] Shell and Utilities Editorial Enhancement Request 2014-12-05 14:42 2015-03-12 16:18
Reporter safinaskar View Status public  
Assigned To
Priority normal Resolution Duplicate  
Status Closed  
Name Askar Safin
Organization
User Reference
Section find, etc
Page Number 2013 edition, p. 2761, etc
Line Number 90443, etc
Interp Status ---
Final Accepted Text
Summary 0000903: Please, add find -print0, xargs -0, read -d and other such options
Description I want to add a proposal about http://austingroupbugs.net/view.php?id=251 [^] issue ("Forbid newline, or even bytes 1 through 31 (inclusive), in filenames")
Desired Action Please add options to standard utils which help to deal with "interesting" characters such as newline, etc.

Add option -print0 to find as present in GNU findutils.
Add option -0 to xargs as present in GNU findutils.
Add option -d to read as present in GNU bash.
Also, please explicitly standardize at least one way to specify null character as argument to -d option to "read".
"read -d '' A" and "read -d "$(printf '\0')" A" both work in bash 4.2, please standardize at least one of them.

This proposal will make the following working (you may add this to examples):

find ... -print0 | xargs -0 ...
find ... -print0 | while read -d "$(printf '\0')" A; do ... done

Add some similar options to "pax".

Also, please, standardize any other options you want, which may help to deal with "interesting" characters.
Tags No tags attached.
Attached Files

- Relationships
duplicate of 0000243Closedajosey 1003.1(2008)/Issue 7 Add -print0 to "find" 

-  Notes
(0002481)
dwheeler (reporter)
2014-12-05 15:13

 Yes, we need to deal with these problems. POSIX permits \n, \r, escape, and so on in filenames, but fails to provide standard mechanisms to DEAL with them, leading to well-known security vulnerabilities (see http://www.dwheeler.com/essays/fixing-unix-linux-filenames.html). [^]

I think you will find that these proposals are already recorded or overlap in #243, #244, and #245, which are long-standing open issues.
(0002482)
geoffclare (manager)
2014-12-05 15:25

 This bug should be closed as a duplicate of 243/244/245/251.

We made a decision long ago on the direction we would take on this issue. We are just waiting for a new edition with TC2 merged in so that we have a clean starting point to work out the detailed changes needed.
(0002483)
safinaskar (reporter)
2014-12-05 18:21

 ooops :)
I will use search
I will use search
I will use search
Also, dwheeler, your link is broken
(0002484)
dwheeler (reporter)
2014-12-06 01:48

 Technically the link isn't broken; the problem is that Mantis thought the closing paren was part of the link. Here's a working link:
http://www.dwheeler.com/essays/fixing-unix-linux-filenames.html [^]

I agree that 243/244/245 should be applied, *and* that certain bytes should be forbidden in filenames (in my opinion, all control chars should be forbidden). That would make it easy to write POSIX-compliant programs that work on current systems.

It's true that find "-print0" can be done with existing POSIX mechanisms, but POSIX should make it *easy* to write secure code, merely *possible*. Most developers do what is *easy*, even when it's insecure, so where possible let's make *secure* and *easy* the same thing.
(0002485)
joerg (reporter)
2014-12-06 12:30

 This proposal again unveils that introducing -print0 would require to add many changes to many other utilities as well in order to make it useful.

On the other side, SVr4 added "find . -exec cmd {} +" in 1988 (long before gfind added -print0) and I know of no recent find implementations that does not support EXECPLUS. Note that execplus is in POSIX since 2004.
(0002486)
dwheeler (reporter)
2014-12-06 18:46

 The updates to find, xargs, and read to support \0-separated data are *already* common and recommended practice, as they counter well-known vulnerability patterns.

The "find... {} +" is in POSIX, and has its use cases, but it is only useful in simple cases where there is a simple existing command that takes a file list. Many problems are difficult to express this way. In more complex cases it is important to be able to process information in multiple steps, and as-is find is ugly to use.

E.G., here's an example of an in-line shell command, where you'd have to use \' for single-quote - and let's face it, this is an ugly non-obvious hack:
 find . -exec sh -c '
  for file do
     ... # Beware of single-quotes in here!!!!
  done' sh {} +

The following loop has no head-busting quoting nonsense,
it's a simple and obvious loop,
and it doesn't require repeated forking (as find "+" will do in groups):
 find . -print0 | while IFS="" read -r -d "" file ; do ...
   COMMAND "$file"
 done

The latter could be even simpler if read supported a simple option to do this:
 find . -print0 | while read -0 file ; do ...
   COMMAND "$file"
 done

In the very long term forbidding control characters from filenames would help, but that is a long-term play. I believe we should support the long-term play, but *also* support mechanisms to handle the current situation; they will be around a long time.

- Issue History
Date Modified Username Field Change
2014-12-05 14:42 safinaskar New Issue
2014-12-05 14:42 safinaskar Name => Askar Safin
2014-12-05 14:42 safinaskar Section => find, etc
2014-12-05 14:42 safinaskar Page Number => 2013 edition, p. 2761, etc
2014-12-05 14:42 safinaskar Line Number => 90443, etc
2014-12-05 15:13 dwheeler Note Added: 0002481
2014-12-05 15:25 geoffclare Note Added: 0002482
2014-12-05 18:21 safinaskar Note Added: 0002483
2014-12-06 01:48 dwheeler Note Added: 0002484
2014-12-06 12:30 joerg Note Added: 0002485
2014-12-06 18:46 dwheeler Note Added: 0002486
2015-03-12 16:15 Don Cragun Relationship added duplicate of 0000243
2015-03-12 16:18 Don Cragun Interp Status => ---
2015-03-12 16:18 Don Cragun Status New => Closed
2015-03-12 16:18 Don Cragun Resolution Open => Duplicate
2016-04-03 02:28 dannyniu Issue Monitored: dannyniu

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker