Austin Group Defect Tracker

Aardvark Mark III


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001154 [1003.1(2016)/Issue7+TC2] Shell and Utilities Objection Enhancement Request 2017-06-25 15:58 2017-10-24 06:12
Reporter jobsnijders View Status public  
Assigned To
Priority normal Resolution Open  
Status New  
Name Job Snijders
Organization
User Reference
Section rm
Page Number 3198-3202
Line Number 107289
Interp Status ---
Final Accepted Text
Summary 0001154: Add 'rm -v'
Description Many rm implementations provide an extension option '-v' that makes rm print the files and directories which it successfully deleted. Currently the burden is on the user to determine which set of files and directories has been deleted, which leads to a time-of-check-to-time-of-use-to-time-to-observe race between obtaining a file listing and using rm on that set of files.

One may attempt to obtain similar functionality to "rm -rfv *" by issuing "find -d * -exec rm -rf {} \; -exec echo {} \;" but that is a lot of typing.
Desired Action In "Base Specifications Issue 7 with TC2 Merged",

at page 3198 line 107245 [rm SYNOPSIS], change:

    rm [−iRr] file...
    rm -f [-iRr] [file...]

to:

    rm [−iRrv] file...
    rm -f [-iRrv] [file...]

At line 107284 [DESCRIPTION], change:

    If this fails for any reason, rm shall write a diagnostic message to standard error, do nothing more with the current file, and go on to any remaining files.

to:

    If rm successfully performed an action on the current file, and the -v option is specified, rm shall write a message containing the name of the current file to the standard output. If the action fails for any reason, rm shall write a diagnostic message to standard error, do nothing more with the current file, and go on to any remaining files.

After line 107298 [OPTIONS], insert a new paragraph:

    -v Print names of files as they are processed

After line 107404 [RATIONALE], add a paragraph:

    The addition of the −v option allows the use of rm to report which files have been deleted.
Tags No tags attached.
Attached Files

- Relationships

-  Notes
(0003801)
jobsnijders (reporter)
2017-06-25 16:05

Example output from macosx, netbsd and freebsd:

    $ mkdir dir && touch file dir/file
    $ rm -rfv dir file
    dir/file
    dir
    file
(0003802)
kre (reporter)
2017-06-25 17:37

I think if this were to be added (and no objection to that) it will
need better wording, just "Print names of files as they are processed"
says almost nothing (before the attempt is made to remove, them, after...)

And the relationship between -f (which suppresses most (or all) error
messages, and -v which deliberately requests them need to be explained.
And perhaps any relationship with -i (does -v really tell you, again,
the name of a file you just answered 'y' to removing and got no error
message about?)
(0003803)
jobsnijders (reporter)
2017-06-25 18:52

Thank you for your feedback.

I'm not sure there is an immediate relation with "-f". "-f" indeed relates to _errors_, where as "-v" prints what succeeded. On most platforms running "rm -iv" is indeed a bit verbose, since no relation exists between the two flags.

You are right that "Print names of files as they are processed" is a bit ambiguous. All "-v" implementations I am aware of specifically print _after_ a successful deletion (through a system call to unlink() or rmdir()). Perhaps "Print names of files as they are successfully deleted"?

I've collected the sentences used in "rm" man pages under [OPTIONS] from various platforms:

FreeBSD 11: "-v Be verbose when deleting files, showing them as they are removed."

DragonFly 4.8.0: "-v Be verbose when deleting files, showing them as they are removed."

NetBSD 7.1: "-v Cause rm to be verbose, showing files as they are processed."

GNU coreutils rm: "-v Print the name of each file before removing it." [red. reading the source code it appears the man page is wrong and the name is printed after successful removal]

OSX/OpenDarwin: "-v Be verbose when deleting files, showing them as they are removed."

Debian/Redhat/Suse: "-v explain what is being done"

Minix: "-v Tell what rm is removing"
(0003812)
wahern (reporter)
2017-08-02 06:13
edited on: 2017-08-02 06:13

Processing the output of -v doesn't handle the case of file names with embedded whitespace like \n. Is it worthwhile to define a new extension that doesn't address this issue? Wouldn't a simple loop like the following suffice, which is both complete in itself (no additional code necessary for post-processing of output) and which correctly handles embedded whitespace in file names?
 
  while [ $? -eq 0 ]; do
    set -- *
    test $# -gt 0 || break
    rm -fr -- "$@"
  done

More importantly, unless the application ensures mutual exclusion so that files aren't created by another process after a file name has been printed, it's not clear to me that -v can even solve TOCTTOU issues. If the application does ensure mutual exclusion, there's no TOCTTOU situation to worry about.

(0003866)
jobsnijders (reporter)
2017-10-21 11:19
edited on: 2017-10-21 11:20

In "Base Specifications Issue 7 with TC2 Merged", at page 3198 line 107245 [rm SYNOPSIS], change:

    rm [−iRr] file...
    rm -f [-iRr] [file...]

to:

    rm [−iRrv] file...
    rm -f [-iRrv] [file...]

At line 107284 [DESCRIPTION], change:

    If this fails for any reason, rm shall write a diagnostic message to standard error, do nothing more with the current file, and go on to any remaining files.

to:

    If rm successfully performed an action on the current file, and the -v option is specified, rm shall write a message containing the name of the current file to the standard output. If the action fails for any reason, rm shall write a diagnostic message to standard error, do nothing more with the current file, and go on to any remaining files.

After line 107298 [OPTIONS], insert a new paragraph:

    -v Print file after it has been unlinked.

After line 107404 [RATIONALE], add a paragraph:

    The addition of the −v option allows the use of rm to report which files have been deleted.

(0003867)
jobsnijders (reporter)
2017-10-21 11:22

The wording has been discussed on the mailing list, all of part of the Desired Action section of the bug report should be replaced with the new proposed wording. I'd like to thank Don Cragun, Robert Elz, Scott Lurndal and Stephane Chazelas for their feedback.
(0003868)
nick (manager)
2017-10-23 21:23

The STDOUT section (page 3200 line 107331) also needs to be updated.

At line 107331 replace
    None.

with
    If the -v option is specified, the name of any file successfully unlinked shall be printed, followed by a <newline>.
(0003869)
stephane (reporter)
2017-10-24 06:12
edited on: 2017-10-24 06:14

Re: Note: 0001154

It should rather be something like:

} If the -v option is specified, for each file (including
} directories if -r is specified) that is successfully removed, a
} message to that effect should be printed on stdout including the
} path of the file in an unspecified format. When stdout is a tty
} device (at least), implementations are encouraged to make the
} message as unambiguous as possible.

In particular:
- it should be "path" or "path name", not "name"
- I used "removed" instead of "unlink" as it could be ambiguous
  for files of type "directory" where the unlink() system call
  is not involved (or where removing a directory unlinks ".",
  ".." and the directory itself).
- different implementations have different formats. It's
  pointless to try and print a terse newline delimited path name
  only as anyway the output is not post-processable as file
  paths are sequences of arbitrary bytes and may contain newline
  characters and PATH_MAX is generally bigger than LINE_MAX.
- disambiguate messages for instance by outputting a
  "foo<newline>bar " file as $'foo\nbar ' like GNU rm does is
  useful even if it's not always possible with Unicode (with
  many characters with the same visual representation, combining
  characters, invisible characters...) or even for ASCII when
  fonts use the same or similar glyph for l and 1 or I (which a
  rm implementations can't know about).

Reposting what I said on the ML:

Date: Wed, 28 Jun 2017 23:14:07 +0100
From: Stephane Chazelas <stephane.chazelas@gmail.com>
To: Scott Lurndal <scott@lurndal.org>
Cc: Job Snijders <job@ntt.net>, Austin Group Bug Tracker
 <noreply@msnkbrown.net>, austin-group-l@opengroup.org
Subject: Re: [1003.1(2016)/Issue7+TC2 0001154]: Add 'rm -v'

2017-06-27 16:15:50 -0700, Scott Lurndal:
> On Tue, Jun 27, 2017 at 11:57:14PM +0200, Job Snijders wrote:
> > On Mon, Jun 26, 2017 at 11:06:14AM -0700, Scott Lurndal wrote:
> > > On Sun, Jun 25, 2017 at 06:52:16PM +0000, Austin Group Bug Tracker wrote:
> > > > You are right that "Print names of files as they are processed" is a bit
> > > > ambiguous. All "-v" implementations I am aware of specifically print
> > > > _after_ a successful deletion (through a system call to unlink() or
> > > > rmdir()). Perhaps "Print names of files as they are successfully deleted"?
> > >
> > > Although deleted is imprecise here - the directory entry is removed
> > > but the file may still exist via a different pathname on the same
> > > device (i.e. a hard link).
> >
> > Do you have a suggestion how to improve the phrasing?
>
> As the rest of the description of 'rm' refers to 'directory entries',
> that seems the appropriate term in this instance instead of 'files'.
>
> E.g. "Print directory entries as they are successfully removed".
[...]

In the case of "rm -r", that's potentially ambiguous. When a
directory is removed, that's possibly 3 entries being removed:
"." and ".." in the directory itself and the entry for the
directory in its parent, but rm -v should only display the
latter (and ignore the case of ".", ".." being successfully
unlinked but not the directory which should not happen on modern
systems).

Also, implementations don't just display the name, they display
the full path constructed by appending components to the base
file passed as argument.

The STDOUT section would need to be modified as well (as all
implementations seem to be using stdout and not stderr).

The format should probably be left unspecifid, even in the POSIX
locale, also allowing applications to do transformations on the
file names (like substiting non-printable characters) since it
varies between implementations.

In any case, even with the simple format of FreeBSD, that output
is not post-processable in scripts since it's newline delimited.
That's only intended for user interaction.

FreeBSD:

$ mkdir -p 1/2/3
$ touch 1/2/file
$ rm -rv 1
1/2/3
1/2/file
1/2
1


GNU:

$ LC_ALL=fr_FR.UTF-8 rm -rv 1
removed directory '1/2/3'
'1/2/file' supprimé
removed directory '1/2'
removed directory '1'

$ touch "$(printf '\b')"
$ LC_ALL=fr_FR.UTF-8 /tmp/cu/bin/rm -rv "$(printf '\b')"
''$'\b' supprimé


- Issue History
Date Modified Username Field Change
2017-06-25 15:58 jobsnijders New Issue
2017-06-25 15:58 jobsnijders Name => Job Snijders
2017-06-25 15:58 jobsnijders Section => rm
2017-06-25 15:58 jobsnijders Page Number => 3198-3202
2017-06-25 15:58 jobsnijders Line Number => 107289
2017-06-25 16:00 jobsnijders Issue Monitored: jobsnijders
2017-06-25 16:05 jobsnijders Note Added: 0003801
2017-06-25 17:37 kre Note Added: 0003802
2017-06-25 18:52 jobsnijders Note Added: 0003803
2017-08-02 06:13 wahern Note Added: 0003812
2017-08-02 06:13 wahern Note Edited: 0003812
2017-08-02 06:13 wahern Note Edited: 0003812
2017-10-21 11:19 jobsnijders Note Added: 0003866
2017-10-21 11:20 jobsnijders Note Edited: 0003866
2017-10-21 11:22 jobsnijders Note Added: 0003867
2017-10-23 21:23 nick Note Added: 0003868
2017-10-24 06:12 stephane Note Added: 0003869
2017-10-24 06:14 stephane Note Edited: 0003869


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker