Austin Group Defect Tracker

Aardvark Mark III


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001133 [1003.1(2016)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2017-03-27 10:27 2018-09-27 15:15
Reporter Rocco83 View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Resolved  
Name Daniele Palumbo
Organization
User Reference
Section find
Page Number -
Line Number -
Interp Status ---
Final Accepted Text Note: 0004127
Summary 0001133: find clarification on -xdev behavior for mounted filesystem within primary
Description Current find description for -xdev:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html [^]
---
-xdev
    The primary shall always evaluate as true; it shall cause find not to continue descending past directories that have a different device ID (st_dev, see the stat() function defined in the System Interfaces volume of POSIX.1-2008). If any -xdev primary is specified, it shall apply to the entire expression even if the -xdev primary would not normally be evaluated.
---

-xdev definition do not have a unique understanding from OS implementation on how to deal with filesystem mounted within primary, Solaris find and GNU find are currently behaving in different way.
Both should be POSIX standard compliant, therefore the same behavior is expected.
https://en.wikipedia.org/wiki/POSIX#POSIX-oriented_operating_systems [^]

It has to be clarified if that mount point itself has to be listed or skipped, by POSIX definition.
It is important to recall that the mount point itself have a different st_dev then primary (see evidence below).

Consider the following example, with mountpoint on /mnt/testmount

Solaris 11.3:
root@testserver:~# pkg info entire
Name: entire
Summary: entire incorporation including Support Repository Update (Oracle Solaris 11.3.9.4.0).
[...]
Version: 0.5.11 (Oracle Solaris 11.3.9.4.0)
Build Release: 5.11
Branch: 0.175.3.9.0.4.0
Packaging Date: June 10, 2016 12:51:48 AM
[...]
root@testserver:~# zfs list /mnt/testmount
NAME USED AVAIL REFER MOUNTPOINT
rpool/mounttest 31K 46.2G 31K /mnt/testmount
root@testserver:~# find /mnt -xdev -type d
/mnt
root@testserver:~#

RHEL 7:
testserver ~ # uname -a
Linux testserver 3.10.0-229.el7.x86_64 #1 SMP Thu Jan 29 18:37:38 EST 2015 x86_64 x86_64 x86_64 GNU/Linux
testserver ~ # rpm -qf /usr/bin/find
findutils-4.5.11-3.el7.x86_64
testserver ~ # df|grep test
/dev/mapper/myvg-mounttest 999320 2572 927936 1% /mnt/testmount
testserver ~ # find /mnt/ -xdev
/mnt/
/mnt/testmount
testserver ~ #

Again RHEL7:
testserver ~ # stat /mnt/|grep Device
Device: fd01h/64769d Inode: 8196 Links: 3
testserver ~ # stat /mnt/testmount/|grep Device
Device: fd07h/64775d Inode: 2 Links: 2
testserver ~ #

So the Device ID is different (fd01h/... vs fd07h/...), as expected.
Desired Action The core is the following sentence:
"it shall cause find not to continue descending past directories that have a different device ID (see st_dev"
sentence.

1) clarify which of the two behavior is technically correct, to list or skip the mount point itself
2) Have a more clear statement for -xdev find XCU definition

Given all of the above skip of the mount point seems the correct behavior, as different ID is listed.
If confirmed, to fix #2, the following rewording is suggested:
---
-xdev
    The primary shall always evaluate as true; restricts the search to the directory with the same device ID of primary (st_dev, see the stat() function defined in the System Interfaces volume of POSIX.1-2008). If any -xdev primary is specified, it shall apply to the entire expression even if the -xdev primary would not normally be evaluated.
---
Tags issue8
Attached Files

- Relationships
related to 0001210Interpretation Required FTW_MOUNT and symbolic links to directories on other file systems 

-  Notes
(0003654)
shware_systems (reporter)
2017-03-27 19:11

As a logical entry in the directory of the device where the find traversal begins, I'd expect when -xdev is set a mount point name to be listed similar to symbolic links when neither -H or -L are specified, as the intent of the wording. That the st_dev of the mount point target differs does not affect the mount name entry in this regard. When -H or -L are specified with -xdev, I'd expect a symbolic link target with a mount point reference to a different st_dev in any part of the substitute path to keep that symbolic link from being part of the output, but not a direct mount name.

However, a mount need not have an actual physical file entry stored on the media, of any standard or implementation-defined file type as its inode target. If the implementation maintains a separate table of mount points, and checks each open() or creat() attempt, and symbolic link resolutions, against that table and the stored dirents for name collisions, the find implementation may simply be limiting itself to the entries on the media when -xdev specified. A name in the table would not be part of the output then.

Either behavior I could see being construed as conforming. The standard does not require st_dev values be stored in FILE records on media; they're usually synthesized in some manner for stat() calls as something configuration dependent, not static values that could be considered portable. In terms of resolving the ambiguity, I can see making it a requirement mount names show when
neither -H or -L specified, whether -xdev set or not, and hidden when either option and -xdev is specified, treating the direct name as a symbolic link body, essentially. This may be a breaking change, but covers both observed behaviors.
(0003656)
joerg (reporter)
2017-03-29 13:48
edited on: 2017-03-29 13:49

SVSvr4 and descendent find implementations

.... and

Schily find (sfind) which is based on libfind

do not list mount points that return a different st_dev with stat(2).

This does not include possible mount points that are currently not mounted
but it applies to all mount points that are currently mounted.

(0004125)
Don Cragun (manager)
2018-09-13 16:02

We discussed this during the Austin Group conference call on September 13, 2018. We believe that the current wording in the standard requires mount points to be included in the output from <tt>find some_directory -xdev</tt> but to not list files found by traversing those mount points. This allows one to make a backup of a directory containing mount points and have those mount points included in the backup without including any files found under that mount point.

Unfortunately, this is not the way System V find behaved and there is not discussion in the rationale explaining why this change to historic behavior was made. We would guess that the documented behavior of the System V find matches what is in the standard even though that was not the way System V find behaved. We believe that this was a bug in the System V find and not an incorrect requirement in the standard.
(0004127)
geoffclare (manager)
2018-09-21 14:44
edited on: 2018-09-21 14:47

Proposed changes:

On page 255 line 8562 section <ftw.h>, add after FTW_MOUNT:
FTW_XDEV
The walk does not descend below directories that have a different device ID than the starting directory.

On page 1397 line 46455 section nftw(), change:
If clear, nftw() shall report all files encountered during the walk.
to:
If clear, nftw() shall report all files encountered during the walk, unless FTW_XDEV is set.

(Note that bug 0001210 has changes to the first part of the FTW_MOUNT
description to use wording similar to the FTW_XDEV wording below.)

On page 1397 line 46457 section nftw(), add after FTW_PHYS:
FTW_XDEV
If set, nftw() shall not descend below directories that have a different device ID (st_dev) than path; that is, when a directory with a different device ID is encountered, nftw() shall report the directory itself (unless FTW_MOUNT is set) but shall not report any files below the directory. If clear, nftw() shall report all files encountered during the walk, unless FTW_MOUNT is set.

Note: If both FTW_MOUNT and FTW_XDEV are set, nftw() obeys both flags but the end result is the same as if FTW_XDEV were clear.

On page 1400 line 46568 section nftw(), add a new paragraph to APPLICATION USAGE:
When restricting the walk to files on one file system, it can sometimes be desirable for the crossing points themselves to be reported and sometimes for them not to be reported. (Crossing points are mount points and, if FTW_PHYS is clear, symbolic links to directories on other file systems.) With FTW_XDEV nftw() reports them and with FTW_MOUNT it does not. However, with FTW_MOUNT it also does not report symbolic links to non-directory files on other file systems (if FTW_PHYS is clear). If there is a need for an application to exclude crossing points but include symbolic links to non-directory files on other file systems, this can be achieved by using FTW_XDEV and performing a check such as the following in fn():
    if (tflag == FTW_D && sb->st_dev != saved_dev)
        return 0;
(where <tt>saved_dev</tt> is the st_dev value for path).

On page 2797 line 91941 section find, add before -xdev:
-mount
The primary shall always evaluate as true; it shall cause find to act only on files that have the same device ID (st_dev, see [xref to stat()]) as the path operand below which they are encountered and cause find not to descend below directories that have a different device ID than that path operand. If any -mount primary is specified, it shall apply to the entire expression even if the -mount primary would not normally be evaluated.

On page 2797 line 91942 section find, change:
The primary shall always evaluate as true; it shall cause find not to continue descending past directories that have a different device ID (st_dev, see [xref to stat()]).
to:
The primary shall always evaluate as true; it shall cause find not to descend below directories that have a different device ID (st_dev, see [xref to stat()]) than the path operand below which they are encountered; that is, when a directory with a different device ID is encountered, find shall act on the directory itself (unless -mount is specified) but shall not act on any files below the directory.

On page 2801 line 92114 section find, add a new paragraph to APPLICATION USAGE:
When restricting the search to files on one file system, it can sometimes be desirable for the crossing points themselves to be acted on and sometimes for them not to be acted on. (Crossing points are mount points and, if the -L option is specified, symbolic links to directories on other file systems.) The -xdev primary acts on them and the -mount primary does not. However, -mount also does not act on symbolic links to non-directory files on other file systems (if -L is specified). If there is a need for an application to exclude crossing points but include symbolic links to non-directory files on other file systems, this can be achieved by using two find commands as follows:
    find -L dir -mount -type d -print
    find -L dir -xdev ! -type d -print
(in a subshell whose output is piped to sort, if the order matters).

If both -mount and -xdev are specified, find obeys both primaries but the end result is the same as if -xdev were not specified.

On page 2804 line 92246 section find, add a new paragraph to RATIONALE:
Historically, many find implementations supported -mount and -xdev as synonymous primaries and earlier versions of this standard only required support for -xdev. However, the behavior of find with -xdev differed from that of the nftw() function with FTW_MOUNT as regards whether the mount point itself was included or excluded. Therefore the standard now requires support for both primaries with slightly differing behaviors: -mount behaves in the manner of nftw() with the traditional FTW_MOUNT flag, and -xdev in the manner of nftw() with a new FTW_XDEV flag.

On page 3076 line 102580 section pax, change:
When traversing the file hierarchy specified by a pathname, pax shall not descend into directories that have a different device ID (st_dev; see [xref to stat()]).
to:
When traversing the file hierarchy specified by a pathname, pax shall not descend below directories that have a different device ID (st_dev, see [xref to stat()]) than the specified pathname; that is, when a directory with a different device ID is encountered, pax shall process (archive or copy) the directory itself but shall not process any files below the directory.

On page 3094 line 103290 section pax, add a new paragraph to APPLICATION USAGE:
When restricting file hierarchy traversal to one file system, it can sometimes be desirable for the crossing points themselves to be processed (archived or copied) and sometimes for them not to be processed. (Crossing points are mount points and, if the -L option is specified, symbolic links to directories on other file systems.) With the -X option pax processes them, but there is no standard way to have pax not process them. However, this can be achieved by using find to do the hierarchy traversal and piping the output of find to pax (with the -d option); see the APPLICATION USAGE for [xref to find].



- Issue History
Date Modified Username Field Change
2017-03-27 10:27 Rocco83 New Issue
2017-03-27 10:27 Rocco83 Name => Daniele Palumbo
2017-03-27 10:27 Rocco83 Section => find
2017-03-27 10:27 Rocco83 Page Number => -
2017-03-27 10:27 Rocco83 Line Number => -
2017-03-27 19:11 shware_systems Note Added: 0003654
2017-03-29 13:48 joerg Note Added: 0003656
2017-03-29 13:49 joerg Note Edited: 0003656
2018-09-13 16:02 Don Cragun Note Added: 0004125
2018-09-21 14:44 geoffclare Note Added: 0004127
2018-09-21 14:47 geoffclare Note Edited: 0004127
2018-09-21 14:52 geoffclare Relationship added related to 0001210
2018-09-27 15:15 geoffclare Interp Status => ---
2018-09-27 15:15 geoffclare Final Accepted Text => Note: 0004127
2018-09-27 15:15 geoffclare Status New => Resolved
2018-09-27 15:15 geoffclare Resolution Open => Accepted As Marked
2018-09-27 15:16 geoffclare Tag Attached: issue8


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker