Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001616 [Issue 8 drafts] Shell and Utilities Editorial Enhancement Request 2022-11-08 23:03 2023-02-22 13:30
Reporter illiliti View Status public  
Assigned To
Priority normal Resolution Open  
Status New   Product Version Draft 2.1
Name Mark Lundblad
Organization
User Reference
Section Shell and Utilities
Page Number -
Line Number -
Final Accepted Text
Summary 0001616: Standardize mktemp utility
Description The current standard has no reliable and convenient utility to create temporary file or directory, despite that low-level interfaces, mkstemp/mkdtemp, are available for this use case. The mktemp(1) utility is a perfect candidate to fill this gap. Its interface is fairly simple and mostly consistent across various implementations, however not without some differences. I evaluated GNU, LSB, NetBSD, FreeBSD, OpenBSD, Illumos, Solaris, sbase, toybox, busybox implementations and here is what I found:

- FreeBSD, NetBSD accept multiple templates. Others do not.
- FreeBSD does not specify -p option. Others do.
- LSB does not specify -d and -p options. Others do.
- FreeBSD, NetBSD accept parameter for -t option. Others do not.
- FreeBSD, NetBSD use -t option to specify prefix. Others use it to explicitly tell mktemp to create temporary file in temporary directory rather than in current directory.
- All have different default value for template.
- All have different requirements regarding how many Xs template should contain, but it seems they all accept six Xs.
- sbase, Illumos, Solaris override -p parameter with TMPDIR value if it is set. Others do not.
- GNU overrides -p parameter with TMPDIR value or /tmp if parameter is empty string. Others do not.
- OpenBSD, Illumos, Solaris(not sure), toybox do not allow '/' in template if -p option is passed. Others do.

Given that, I think mktemp interface is not completely lost and below is my attempt to make it portable.
Desired Action Standardize mktemp utility. Here is the most minimal and portable example:

NAME
     mktemp - create temporary unique file or directory

SYNOPSIS
     mktemp [-dqu] [-p directory] [template]

DESCRIPTION
     mktemp creates a temporary file based on a template as if by calling
     mkstemp(3). If no template is specified, an implementation-defined value
     shall be used instead. If template does not contain exactly six Xs at the
     end, the behavior is unspecified. If template does not contain '/', the file
     shall be created in the current directory. Otherwise, the file shall be
     created by its path. The filename of created temporary file shall be written
     to the standard output.

OPTIONS
     -d Create directory instead of file as if by calling mkdtemp(3).

     -p directory
             Change directory where temporary file will be created. With this
             option, it is unspecified what happens if template contains '/'. If
             TMPDIR is set, it is unspecified whether its value take precedence
             over passed parameter or vice versa. If passed parameter is empty
             string, the behavior is unspecified.

     -q Fail silently if an error occurs.

     -u Operate in unsafe mode. A unique name is generated, but the
             temporary file shall be unlinked before mktemp exits. Use of this
             option is not encouraged.

SEE ALSO
     mkdtemp(3), mkstemp(3)
Tags No tags attached.
Attached Files

- Relationships

-  Notes
(0006038)
steffen (reporter)
2022-11-08 23:30

While i accept it as natural as it is so widely used, please note that POSIX says for pathchk(1):

  The pathchk utility was new for the ISO POSIX-2: 1993 standard. It, along with the set −C(noclobber) option added to the shell, replaces the mktemp, validfnam, and create utilities that appeared in early proposals.

So what you do is

if ( set -C ) >/dev/null 2>&1; then
  set +C
else
  # For heaven's sake auto-redirect on SunOS/Solaris
  if [ -f /usr/xpg4/bin/sh ] && [ -x /usr/xpg4/bin/sh ]; then
    exec /usr/xpg4/bin/sh "${0}" "${@}"
  else
    synopsis 1 'sh(1)ell without "set -C" (for safe temporary file creation)'
  fi
fi

and then

old_umask=`umask`
umask 077
i=1
while :; do
  tmpfile="${tmpdir}/MYNAME-${i}.tmp"
  (
    set -C
    : > "${tmpfile}"
  ) >/dev/null 2>&1 && break
  i=`expr ${i} + 1`
  if [ ${i} -gt ${max} ]; then
    echo >&2 'Cannot create a temporary file within '"${tmpdir}"
    exit ${EX_TEMPFAIL}
  fi
done
trap "exit ${EX_TEMPFAIL}" HUP INT QUIT PIPE TERM
trap "trap \"\" HUP INT QUIT PIPE TERM EXIT; rm -f ${tmpfile}" EXIT
umask ${old_umask}

..after finding a tmpdir and doing some other setup.
(0006041)
illiliti (reporter)
2022-11-09 13:18

Yeah, your example should work, but it is not simple. Look at how simple mktemp is:

tmpfile=$(mktemp)

or directory:

tmpdir=$(mktemp -d)

mktemp is impossible to misuse unlike shell. Plus it is already more-or-less portable and widespread on all open-source systems.

Also unrelated, but it seems that your example does not work with yash when tmpfile is a dangling symlink. Looks like a yash bug. Reported.
(0006166)
ormaaj (reporter)
2023-02-22 13:30
edited on: 2023-02-22 19:10

Here's a few more data points.

The uutils mktemp is basically same as coreutils, as expected:

(ins)root 5 (7104) 0 ~ # uu-mktemp --help
create a temporary file or directory.

Usage: uu-mktemp [OPTION]... [TEMPLATE]

Arguments:
  [template]

Options:
  -d, --directory Make a directory instead of a file
  -u, --dry-run do not create anything; merely print a name (unsafe)
  -q, --quiet Fail silently if an error occurs.
      --suffix <SUFFIX> append SUFFIX to TEMPLATE; SUFFIX must not contain a path separator. This option is implied if TEMPLATE does not end with X.
  -p, --tmpdir [<DIR>] interpret TEMPLATE relative to DIR; if DIR is not specified, use $TMPDIR ($TMP on windows) if set, else /tmp. With this option, TEMPLATE must not be an absolute name; unlike with -t, TEMPLATE may contain slashes, but mktemp creates only the final component
  -t Generate a template (using the supplied prefix and TMPDIR (TMP on windows) if set) to create a filename template [deprecated]
  -h, --help Print help information
  -V, --version Print version information

Bash's "example" loadable mktemp:

(ins)root 5 (7104) 0 ~ # bash -c 'enable -f mktemp{,}; type mktemp; help mktemp'
mktemp is a shell builtin
mktemp: mktemp [-d] [-q] [-t prefix] [-u] [-v varname] [template] ...
    Make unique temporary file name

    Take each supplied filename template and overwrite a portion of it
    to create a filename, which is unique and may be used by the calling
    script. TEMPLATE is a string ending in some number of 'X's. If
    TEMPLATE is not supplied, shtmp.XXXXXX is used and $TMPDIR is used as
    the name of the containing directory. Files are created u+rw; directories
    are created u+rwx.

    Options, if supplied, have the following meanings:

        -d Create a directory instead of a file
        -q Do not print error messages about file creation failure
        -t PREFIX Use PREFIX as the directory in which to create files
        -u Do not create anything; simply print a name
        -v VAR Store the generated name into shell variable VAR

    Any PREFIX supplied with -t is ignored if TEMPLATE is supplied.

    The return status is true if the file or directory was created successfully;
    false if an error occurs or VAR is invalid or readonly.

And the AST libcmd builtin under ksh93 (edit: u+m also has this one):

(ins)root 5 (7104) 0 ~ # ksh93v -c 'builtin mktemp; type mktemp; mktemp --man'
mktemp is a shell builtin
NAME
  mktemp - make temporary file or directory

SYNOPSIS
  mktemp [ options ] [ prefix [ directory ] ]

DESCRIPTION
  mktemp creates a temporary file with optional base name prefix prefix. If prefix is omitted then tmp is used and --tmp is implied. A consecutive string of trailing X's in prefix is replaced by a pseudorandom combination of [0-9a-zA-Z]characters, otherwise the first 5 characters of prefix is catenated with a
  pseudorandom string to construct a file name component of 14 characters. If directory is specified or if prefix contains a directory prefix then that directory overrides any of the directories described below. A temporary file will have mode rw------- and a temporary directory will have mode rwx------, subject to
  umask(1). Generated paths have these attributes:
    * Lower case to avoid clashes on case ignorant filesystems.
    * Pseudo-random part to deter denial of service attacks.
    * Default pseudo-random part (no specific X... template) formatted to accomodate 8.3 filesystems.

  A consecutive trailing sequence of X's in prefix is replaced by the pseudo-random part. If there are no X's then the pseudo-random part is appended to the prefix.

OPTIONS
  -d, --directory Create a directory instead of a regular file.
  -m, --mode=mode Set the mode of the created temporary to mode. mode is symbolic or octal mode as in chmod(1). Relative modes assume an initial mode of u=rwx.
  -p, --default=directory
                  Use directory if the TMPDIR environment variable is not defined. Implies --tmp.
  -q, --quiet Suppress file and directory error diagnostics.
  -R, --regress=seed
                  The pseudo random generator is seeded with seed instead of process/system specific transient data. Use for testing only. A seed of 0 is silently changed to 1.
  -t, --tmp|temporary-directory
                  Create a path rooted in a temporary directory.
  -u, --unsafe|dry-run
                  Check for file/directory existence but do not create. Use this for testing only.

SEE ALSO
  mkdir(1), pathtemp(3), mktemp(3)

IMPLEMENTATION
  version mktemp (AT&T Research) 2012-12-12
  author Glenn Fowler <glenn.s.fowler@gmail.com>
  author David Korn <dgkorn@gmail.com>
  copyright Copyright (c) 1992-2015 AT&T Intellectual Property
  license http://www.eclipse.org/org/documents/epl-v10.html [^]

In addition ksh93 has the `>;word` redirect which opens a temporary file and renames it to word if the command returned success. It can serve a similar purpose.

The canonical method I thought everyone knew is m4's mkstemp ( https://mywiki.wooledge.org/BashFAQ/062#Using_m4 [^] )

Another reasonably widely-available tool - xfs_io from xfsprogs - can be used to open a file with O_TMPFILE. It works on several platforms including with some non-xfs filesystems.


- Issue History
Date Modified Username Field Change
2022-11-08 23:03 illiliti New Issue
2022-11-08 23:03 illiliti Name => Mark Lundblad
2022-11-08 23:03 illiliti Section => Shell and Utilities
2022-11-08 23:03 illiliti Page Number => -
2022-11-08 23:03 illiliti Line Number => -
2022-11-08 23:30 steffen Note Added: 0006038
2022-11-09 13:18 illiliti Note Added: 0006041
2023-02-22 13:30 ormaaj Note Added: 0006166
2023-02-22 19:10 ormaaj Note Edited: 0006166


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker