Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001457 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2021-03-09 21:21 2023-03-22 14:31
Reporter steffen View Status public  
Assigned To ajosey
Priority normal Resolution Accepted As Marked  
Status Applied  
Name Steffen Nurpmeso
Organization
User Reference
Section Vol. 3: Shell and Utilities.
Page Number (page or range of pages)
Line Number (Line or range of lines)
Interp Status ---
Final Accepted Text Note: 0006079
Summary 0001457: Add readlink(1), realpath(1) utility
Description Some POSIX utilities (like find) offer operators to work with symbolic links.
It is not possible to read content of symbolic links by itself as far as i know, as is offered by readlink(1) which seems to be widely available, for a long time.
The current standard does not mention this utility as having been obsoleted in the past either, so i am asking for inclusion.
Desired Action Add the readlink utility.
The most minimal form seems to be from OpenBSD, it states:


NAME

readlink — display target of symbolic link on standard output
SYNOPSIS
readlink [-fn] file
DESCRIPTION

The readlink utility when invoked with the pathname of a symbolic link as its argument dereferences the symbolic link and prints the name of target on standard output. If the -f option is not specified and readlink is invoked with an argument other than the pathname of a symbolic link, it exits with a nonzero exit code without printing anything.

The options are as follows:

-f
    Canonicalize by following every symlink in every component of the given path recursively. readlink will resolve both absolute and relative paths and return the absolute pathname corresponding to file. The argument does not need to be a symbolic link.
-n
    Do not print a trailing newline character.

EXIT STATUS

The readlink utility exits 0 on success, and >0 if an error occurs.
Tags issue8
Attached Files

- Relationships

-  Notes
(0005700)
geoffclare (manager)
2022-02-22 15:58
edited on: 2022-02-24 09:34

Suggested changes...

On page 3194 insert a new page for readlink:

NAME
readlink - display target of symbolic link on standard output

SYNOPSIS
readlink [-fn] file

DESCRIPTION
If the -f option is not specified and the file operand names a symbolic link, the readlink utility shall not follow the symbolic link when resolving file and shall write the contents of the symbolic link to standard output. If the -f option is not specified and file does not refer to a symbolic link, readlink shall write a diagnostic message to standard error and exit with non-zero status.

If the -f option is specified:
  • If the file operand names a symbolic link, readlink shall write to standard output the absolute pathname that would be returned by a call to the realpath() function with the pathname as its first argument.

  • If the file operand names a file that is not a symbolic link, or if the path prefix of file resolves, with symbolic links followed, to an existing directory and the final component of file does not exist as a directory entry in that directory, readlink shall write to standard output the absolute pathname that would be returned by a call to the realpath() function with the path prefix of file as its first argument, followed by a <slash> character and the final component of file. If file is non-empty and only has one pathname component, it shall be treated as if it had a path prefix of "./".

  • If file has more than one pathname component and the path prefix of file does not resolve, with symbolic links followed, to an existing directory, readlink shall write a diagnostic message to standard error and exit with non-zero status.

If the realpath() function would return a null pointer, readlink shall write a diagnostic message to standard error and exit with non-zero status.

If the -n option is not specified, the output to standard output in all cases described above shall be followed by a <newline> character.

OPTIONS
The readlink utility shall conform to [xref to XBD 12.2].

The following options shall be supported:

-f
Canonicalize the pathname specified by file, as specified in the DESCRIPTION section.

-n
Do not output a trailing <newline> character.

OPERANDS
file
A pathname of a symbolic link to be read or, if the -f option is specified, a pathname to be canonicalized.

STDIN
Not used.

INPUT FILES
None.

ENVIRONMENT VARIABLES
The following environment variables shall affect the execution of readlink:

LANG
Provide a default value for the internationalization variables that are unset or null. (See [xref to XBD 8.2] for the precedence of internationalization variables used to determine the values of locale categories.)

LC_ALL
If set to a non-empty string value, override the values of all the other internationalization variables.

LC_CTYPE
Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments).

LC_MESSAGES
Determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error.

[XSI]NLSPATH
Determine the location of message catalogs for the processing of LC_MESSAGES.[/XSI]

ASYNCHRONOUS EVENTS
Default.

STDOUT
See DESCRIPTION.

STDERR
The standard error shall be used only for diagnostic messages.

OUTPUT FILES
None.

EXTENDED DESCRIPTION
None.

EXIT STATUS
The following exit values shall be returned:

 0 Successful completion.
>0 An error occurred.

CONSEQUENCES OF ERRORS
Default.

APPLICATION USAGE
As the final component of file does not need to exist when the -f option is used, a separate existence check has to be performed in applications that need to canonicalize a pathname and also check that the pathname names an existing file.

EXAMPLES
None.

RATIONALE
The readlink utility was added because using <tt>ls -l</tt> to obtain the contents of a symbolic link is difficult if the output includes more than one occurrence of the string " -> ".

Although the behavior of readlink with -f is specified by reference to the realpath() function, which is part of the XSI option, non-XSI implementations that do not support realpath() are nevertheless required to implement readlink in accordance with the requirements described in this standard for realpath().

FUTURE DIRECTIONS
None.

SEE ALSO
ln, ls, pwd

XBD Chapter 8, Section 12.2

XSH readlink(), realpath()

CHANGE HISTORY
First released in Issue 8.


On page 2901 line 95844 section ln, add readlink to SEE ALSO.

(0005702)
steffen (reporter)
2022-02-22 22:19

Re #5701

That is a pity. Then despites Geoff's awesome work on providing a fully fledged standard entry including -f based on refined realpath() usage, readlink should stay and -f alone be dropped.
And i should apologize for not looking everywhere.
(That -f of Net and FreeBSD is really strange.)
(0005703)
geoffclare (manager)
2022-02-23 08:32
edited on: 2022-02-24 09:36

I noticed that the GNU coreutils man page for readlink says "Note realpath(1) is the preferred command to use for canonicalization functionality." So perhaps we should add a realpath utility in place of readlink -f.

(0005704)
emaste (reporter)
2022-02-23 13:31

I added a comment about the -f option on FreeBSD, but deleted it moments later after realizing I was mistaken. FreeBSD documents stat and readlink in the same man page and the option summary is unclear about which options apply to which program.

Here is an excerpt of the stat/readlink man page that describes only readlink:

     readlink [-fn] [file ...]

     When invoked as readlink, only the target of the symbolic link is
     printed. If the given argument is not a symbolic link and the -f option
     is not specified, readlink will print nothing and exit with an error. If
     the -f option is specified, the output is canonicalized by following
     every symlink in every component of the given path recursively. readlink
     will resolve both absolute and relative paths, and return the absolute
     pathname corresponding to file. In this case, the argument does not need
     to be a symbolic link.

     -n Do not force a newline to appear at the end of each piece of
             output.

From NetBSD's man page:

     readlink [-fnqsv] [file ...]

     When invoked as readlink, only the target of the symbolic link is
     printed. If the given argument is not a symbolic link and the -f option
     is not specified, readlink will print nothing and exit with an error. If
     the -f option is specified, the output is canonicalized by following
     every symlink in every component of the given path recursively. readlink
     will resolve both absolute and relative paths, and return the absolute
     pathname corresponding to file. In this case, the argument does not need
     to be a symbolic link.

     -n Do not force a newline to appear at the end of each piece
                   of output.

     -q Suppress failure messages if calls to stat(2) or lstat(2)
                   fail. When run as readlink, error messages are
                   automatically suppressed.

     -s ... When run as readlink, suppress
                   error messages.

     -v Turn off quiet mode.

And OpenBSD:

     readlink [-fn] file

     -f Canonicalize by following every symlink in every component of the
             given path recursively. readlink will resolve both absolute and
             relative paths and return the absolute pathname corresponding to
             file. The argument does not need to be a symbolic link.

     -n Do not print a trailing newline character.
(0005705)
calestyo (reporter)
2022-02-23 13:38

Re: #5703

"Note realpath(1) is the preferred command to use for canonicalization functionality."

=> but *just* for the canonicalization, AFAIU, realpath cannot print the resolved link (i.e. it's destination)
(0005706)
geoffclare (manager)
2022-02-24 09:28

Looking at the GNU coreutils man page for realpath(1), it seems that its default behaviour is the same as readlink -f in that all components except the last need to exist, but it has -e and -m options which alter the existence requirements. It also has a -L option which eliminates ".." components before following symlinks. I assume these features are the reason the readlink man page says realpath is preferred over readlink -f.

Even though there is no longer any obstacle to standardising readlink -f, I think we should still consider adding realpath (alongside readlink without -f).

Solaris 11.4 has the GNU coreutils realpath. Do BSD systems have a realpath utility?
(0005721)
mirabilos (reporter)
2022-02-25 21:24

mksh has a realpath builtin, please don’t add an incompatible requirement if you do ;-)

It takes an optional first argument "--" (skipped) and a mandatory second argument (file); no options, and any other arguments cause an error message as well.

I’ve recently learnt about some differences between BSD and POSIX realpath on the C level (from reading the OpenSSH mailing list). The realpath in mksh is specified to allow referring to nōnexistant files in existing directories, which I understand is very much not a POSIX thing. Without that, though, I’d consider it good as useless (one can always add a test -e afterwards).

https://github.com/MirBSD/mksh/blob/5d135a8ee38d84c5a4f6c175b9a831d5acbc641b/misc.c#L1886 [^] in case of interest.

I understand other OSes have diverging realpath utilities. mksh has a mode where a utility can call the builtin or an external one depending on the existence of the latter and, optionally, presence of flags. The latter is implemented for realpath, so “realpath -h” calls coreutils’ realpath on my GNU system, but not on the BSD system without an external realpath utility. This is a compromise.

“readlink” is important. I’d argue for standardised readlink without and with -f but not adding realpath (unless the latter can be done in a way that doesn’t blow up (remember the story of cat(1) that went out and came back waving flags) mksh or, worse, make its implementation incompatible).
(0005898)
geoffclare (manager)
2022-07-21 10:41
edited on: 2022-07-21 10:57

New suggested changes...

On page 3194 insert new pages for readlink and realpath:

NAME
readlink - display the contents of a symbolic link

SYNOPSIS
readlink [-n] file

DESCRIPTION
If the file operand names a symbolic link, the readlink utility shall not follow the symbolic link when resolving file and shall write the contents of the symbolic link to standard output. If the -n option is not specified, the output to standard output shall be followed by a <newline> character.

If file does not name a symbolic link, readlink shall write a diagnostic message to standard error and exit with non-zero status.

OPTIONS
The readlink utility shall conform to [xref to XBD 12.2].

The following option shall be supported:

-n
Do not output a trailing <newline> character.

OPERANDS
file
A pathname of a symbolic link to be read.

STDIN
Not used.

INPUT FILES
None.

ENVIRONMENT VARIABLES
The following environment variables shall affect the execution of readlink:

LANG
Provide a default value for the internationalization variables that are unset or null. (See [xref to XBD 8.2] for the precedence of internationalization variables used to determine the values of locale categories.)
LC_ALL
If set to a non-empty string value, override the values of all the other internationalization variables.
LC_CTYPE
Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments).
LC_MESSAGES
Determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error.
[XSI]NLSPATH
Determine the location of messages objects and message catalogs.[/XSI]

ASYNCHRONOUS EVENTS
Default.

STDOUT
See DESCRIPTION.

STDERR
The standard error shall be used only for diagnostic messages.

OUTPUT FILES
None.

EXTENDED DESCRIPTION
None.

EXIT STATUS
The following exit values shall be returned:

 0 Successful completion.

>0 An error occurred.

CONSEQUENCES OF ERRORS
Default.

APPLICATION USAGE
None.

EXAMPLES
None.

RATIONALE
The readlink utility was added because using <tt>ls -l</tt> to obtain the contents of a symbolic link is difficult if the output includes more than one occurrence of the string " -> ".

The -f option found in many implementations was not included, as the realpath utility provides equivalent functionality with a choice of behaviors.

FUTURE DIRECTIONS
None.

SEE ALSO
ln, ls, pwd, realpath

XBD Chapter 8, Section 12.2

XSH readlink()

CHANGE HISTORY
First released in Issue 8.


NAME
realpath - resolve a pathname

SYNOPSIS
realpath [-E|-e] file

DESCRIPTION
The realpath utility shall canonicalize the pathname specified by the file operand as follows:

If a call to the realpath() function with the specified pathname as its first argument would succeed, the canonicalized pathname shall be the pathname that would be returned by that realpath() call. Otherwise:

  • If the -e option is specified, the canonicalization shall fail.

  • If the -E option is specified, then if a call to the realpath() function with the specified pathname as its first argument would encounter an error condition other than [ENOENT], the canonicalization shall fail; if the call would encounter an [ENOENT] error, realpath shall expand all symbolic links that would be encountered in an attempt to resolve the specified pathname using the algorithm specified in [xref to XBD 4.13 Pathname Resolution], except that any trailing <slash> characters that are not also leading <slash> characters shall be ignored. If this expansion succeeds and the path prefix of the expanded pathname resolves to an existing directory, the canonicalized pathname shall be the expanded pathname. In all other cases, the canonicalization shall fail. If the expanded pathname is not empty, does not begin with a <slash>, and has exactly one pathname component, it shall be treated as if it had a path prefix of "./".

  • If no options are specified, realpath shall canonicalize the specified pathname in an unspecified manner such that the resulting absolute pathname does not contain any components that refer to files of type symbolic link and does not contain any components that are dot or dot-dot.


Upon successful canonicalization, realpath shall write the canonicalized pathname, followed by a <newline> character, to standard output.

If canonicalization fails, or the canonicalized pathname is empty, nothing shall be written to standard output, a diagnostic message shall be written to standard error, and realpath shall exit with non-zero status.

OPTIONS
The realpath utility shall conform to [xref to XBD 12.2].

The following options shall be supported:

-E
Do not treat it as an error if attempting to resolve the last component of the canonicalized form of the file operand results in an [ENOENT] error condition.
-e
Treat it as an error if attempting to resolve the last component of the canonicalized form of the file operand results in an [ENOENT] error condition.
Specifying more than one of the mutually-exclusive options -E and -e shall not be considered an error. The last option specified shall determine the behavior of the utility.

OPERANDS
file
A pathname to be canonicalized.

STDIN
Not used.

INPUT FILES
None.

ENVIRONMENT VARIABLES
The following environment variables shall affect the execution of realpath:

LANG
Provide a default value for the internationalization variables that are unset or null. (See [xref to XBD 8.2] for the precedence of internationalization variables used to determine the values of locale categories.)
LC_ALL
If set to a non-empty string value, override the values of all the other internationalization variables.
LC_CTYPE
Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments).
LC_MESSAGES
Determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error.
[XSI]NLSPATH
Determine the location of messages objects and message catalogs.[/XSI]

ASYNCHRONOUS EVENTS
Default.

STDOUT
See DESCRIPTION.

STDERR
The standard error shall be used only for diagnostic messages.

OUTPUT FILES
None.

EXTENDED DESCRIPTION
None.

EXIT STATUS
The following exit values shall be returned:

 0 Successful completion.

>0 An error occurred.

CONSEQUENCES OF ERRORS
Default.

APPLICATION USAGE
If neither the -e nor the -E option is specified, some implementations behave as if -e had been specified and others as if -E had been specified, but there are also implementations where the behavior differs from both of these. For example, the mksh shell has an internal implementation of realpath that canonicalizes /dir/regular_file/.. to /dir, whereas the realpath() function would return an [ENOTDIR] error in this case. Portable applications should always specify either -e or -E.

EXAMPLES
None.

RATIONALE
The realpath utility was added in preference to a -f option found in some implementations of the readlink utility because it allows the application to specify whether or not a missing final component is to be treated as an error.

The behavior with the -E option when file does not resolve (with symbolic links followed) to an existing file is not the same as simply calling realpath() with the path prefix of the file operand and writing the resulting pathname, a <slash>, and the last component of file to standard output. For example, if /tmp/nofile does not exist, and file is A/B where A is an existing directory and B is a symbolic link to /tmp/nofile, realpath with -E will output /tmp/nofile, but if B is a symbolic link to /tmp/nofile/foo, realpath with -E will treat this as an error. In both cases <tt>realpath("A/B")</tt> would fail with errno set to [ENOENT]. Even though <tt>realpath("A")</tt> would succeed, in neither case is anything ending /B the result.

Trailing <slash> characters (that follow a non-<slash>) are handled differently with -E than with -e. With -e they are handled as for the realpath() function. With -E they are sometimes effectively ignored, and they are never included in the output. For example, if /tmp/nofile does not exist and /tmp/regfile is an existing regular file:
$ realpath -E /tmp/nofile/
/tmp/nofile
$ realpath -E /tmp/regfile/
realpath: /tmp/regfile/: Not a directory

Although the behavior of the realpath utility is specified by reference to the realpath() function, which is part of the XSI option, non-XSI implementations that do not support realpath() are nevertheless required to implement realpath in accordance with the requirements described in this standard for realpath().

FUTURE DIRECTIONS
None.

SEE ALSO
ln, ls, pwd, readlink

XBD Chapter 8, Section 12.2

XSH Section 2.3, realpath()

CHANGE HISTORY
First released in Issue 8.

On page 2901 line 95844 section ln, add readlink and realpath to SEE ALSO.

On page 2932 line 96992 section ls, add readlink to SEE ALSO.

(0005899)
geoffclare (manager)
2022-07-21 10:45
edited on: 2022-07-21 10:47

My thanks to kre for helping to ensure that my proposed realpath page correctly describes the behaviour both of his new implementation for NetBSD and of the GNU coreutils version (once they add a no-op -E option).

Update: actually, -E won't quite be a no-op, as it will need to turn off -e.

(0005900)
kre (reporter)
2022-07-21 11:15

The NetBSD code (standard BSD licence, so available to everyone for
anything, close enough) which implements Note: 0005898 is available at

   http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/realpath/?only_with_tag=MAIN [^]

just follow the links for the source file, the man page, or the Makefile
and then click the download link on the most recent revision of what that
gives you (depending on your browser, some kind of modified click might be
needed to save the file, otherwise it will just display in the browser).

(https:// should work as well, if you prefer).

This should be fairly easy to build on any BSD derived platform (may need
some minor updates to deal with <sys/cdefs.h> and the __RCSID() macro).

This version originated at FreeBSD, but there's not a lot of that code
remaining.
(0005901)
kre (reporter)
2022-07-21 11:45

Also, wrt the now long deleted note#5701, I totally understand how
that could have happened - the combined stat(1)+readlink(1) man page
was a total mess, and almost impossible to decipher.

At NetBSD we have now split it into two entirely separate pages,
stat(1) and readlink(1) - the two (conceptually) are totally
unrelated utilities.

It just happens that they are the same binary, which looks at argv[0].
readlink (as it exists in BSD) is just stat with a particular set of
options (which depends upon the options to readlink) - readlink could
as easily have been a script that calls stat - except that for some
uses it is used a lot, and adding a sh invocation to every call would
be costly.

But that note 5701 interpreted that messy man page incorrectly doesn't
affect the conclusion to omit readlink -f from the standard. The coreutils
readlink -f is (effectively) realpath -E (as specified here) and the BSD
one is realpath -e - they are not compatible, and adding more options to
readlink to "fix" that would be a sub-optimal solution (what's more, the
coreutils version could not be implemented as a call to stat, no matter
what options are given stat(1) never returns data about files that don't
exist (it would have to invent that, and that's not going to happen.)
(0005921)
geoffclare (manager)
2022-07-29 14:16

The readlink and realpath additions have been made in the Issue8NewAPIs branch in gitlab, based on Note: 0005898.
(0006079)
geoffclare (manager)
2022-11-21 16:12

Make the changes from "Additional APIs for Issue 8, Part 2" (Austin/1273).
(0006219)
eblake (manager)
2023-03-22 12:55

Updating title to reflect agreed-on action
(0006221)
eblake (manager)
2023-03-22 14:31

GNU coreutils readlink has -q (--quiet) and -v (--verbose) options, and currently defaults to -q, which gives a non-zero status for a non-symlink but with no diagnostic message. This is in conflict with the proposed wording that mandates that readlink output a diagnostic. Should we amend this proposal to either standardize -v/-q, or to permit coreutils' current behavior? I've also opened a bug with the coreutils folks to see if they are willing to switch their default (perhaps only when POSIXLY_CORRECT is set?):
https://lists.gnu.org/archive/html/bug-coreutils/2023-03/msg00035.html [^]

- Issue History
Date Modified Username Field Change
2021-03-09 21:21 steffen New Issue
2021-03-09 21:21 steffen Status New => Under Review
2021-03-09 21:21 steffen Assigned To => ajosey
2021-03-09 21:21 steffen Name => Steffen Nurpmeso
2021-03-09 21:21 steffen Section => Vol. 3: Shell and Utilities.
2021-03-09 21:21 steffen Page Number => (page or range of pages)
2021-03-09 21:21 steffen Line Number => (Line or range of lines)
2021-03-10 04:11 emaste Issue Monitored: emaste
2022-02-22 15:58 geoffclare Note Added: 0005700
2022-02-22 16:01 geoffclare Note Edited: 0005700
2022-02-22 16:09 geoffclare Note Edited: 0005700
2022-02-22 16:11 geoffclare Project 1003.1(2008)/Issue 7 => 1003.1(2016/18)/Issue7+TC2
2022-02-22 16:13 geoffclare Note Edited: 0005700
2022-02-22 21:39 emaste Note Added: 0005701
2022-02-22 21:41 emaste Note Deleted: 0005701
2022-02-22 22:19 steffen Note Added: 0005702
2022-02-22 22:34 calestyo Issue Monitored: calestyo
2022-02-23 08:32 geoffclare Note Added: 0005703
2022-02-23 13:31 emaste Note Added: 0005704
2022-02-23 13:38 calestyo Note Added: 0005705
2022-02-24 09:28 geoffclare Note Added: 0005706
2022-02-24 09:32 geoffclare Note Edited: 0005700
2022-02-24 09:34 geoffclare Note Edited: 0005700
2022-02-24 09:36 geoffclare Note Edited: 0005703
2022-02-25 21:24 mirabilos Note Added: 0005721
2022-07-21 10:41 geoffclare Note Added: 0005898
2022-07-21 10:42 geoffclare Note Edited: 0005898
2022-07-21 10:45 geoffclare Note Added: 0005899
2022-07-21 10:47 geoffclare Note Edited: 0005899
2022-07-21 10:57 geoffclare Note Edited: 0005898
2022-07-21 11:15 kre Note Added: 0005900
2022-07-21 11:45 kre Note Added: 0005901
2022-07-29 14:16 geoffclare Note Added: 0005921
2022-11-21 16:12 geoffclare Note Added: 0006079
2022-11-21 16:13 geoffclare Interp Status => ---
2022-11-21 16:13 geoffclare Final Accepted Text => Note: 0006079
2022-11-21 16:13 geoffclare Status Under Review => Resolved
2022-11-21 16:13 geoffclare Resolution Open => Accepted As Marked
2022-11-21 16:14 geoffclare Tag Attached: issue8
2022-11-24 10:09 geoffclare Status Resolved => Applied
2023-03-22 12:55 eblake Note Added: 0006219
2023-03-22 12:55 eblake Summary Add readlink(1) utility => Add readlink(1), realpath(1) utility
2023-03-22 14:31 eblake Note Added: 0006221


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker