Austin Group Defect Tracker

Aardvark Mark III


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001294 [1003.1(2016)/Issue7+TC2] Shell and Utilities Comment Omission 2019-10-01 08:03 2019-12-13 10:06
Reporter Konrad_Schwarz View Status public  
Assigned To
Priority normal Resolution Open  
Status New  
Name Konrad Schwarz
Organization Siemens AG
User Reference
Section c99
Page Number https://pubs.opengroup.org/onlinepubs/9699919799/utilities/c99.html [^]
Line Number https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/utilities/c99.html#tag_20_11_13_03 [^]
Interp Status ---
Final Accepted Text
Summary 0001294: POSIX recognizes the existence of dynamically loadable, executable object files, but provides no way of producing them.
Description The interface specified in https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/basedefs/dlfcn.h.html [^] allows run-time access to "executable object files", but I could not find
a conforming way to produce them.
Desired Action Add _POSIX_SHARED_OBJECT and _POSIX_V8_SHARED_OBJECT_{CFLAGS,LDFLAGS} options to getconf analogously to the "Threaded Programming Environment" table
in c99.

For the GCC toolchain, I would expect _POSIX_V8_SHARED_OBJECT_CFLAGS to be
-fpic and ..._LDFLAGS to be -shared.
Tags No tags attached.
Attached Files

- Relationships

-  Notes
(0004586)
GarrettWollman (reporter)
2019-10-01 17:14

There is no portable way to generate dynamically loadable object files because some dynamic linker implementations require that an exhaustive list of exported symbols be provided by the programmer (distinct from the symbol table in the object files), or require other similar interface information that cannot be deduced from the object files alone.
(0004587)
shware_systems (reporter)
2019-10-01 20:05

From c99 Description:
"If there are no options that prevent link editing (such as -c or -E), and all input files compile and link without error, the resulting executable file shall be written according to the -o outfile option (if present) or to the file a.out."

When this is true, what makes such an executable file a program is the presence of main(), either from a provided object or static library, or the one in standard libraries libl or liby. If main() is not found the file is a library that is expected to be useable by dlopen() the same as dlopen(NULL, flags) references the current program file.

This is portable without needing any extra command line flag. The wording of dlfcn.h and interfaces could emphasize better that what is allowed there as implementation-defined is in addition to the above expectation, not a substitute.

Also, if an external linker is used, it is the responsibility of the compilation phase to synthesize from the input .o and .c sources the information that linker requires. If the compiler cannot do this, necessitating the maintenance of files like an exports list, they are disqualified from being considered conforming, and the object format is suspect for being insufficently sophisticated to begin with. The caveat in the Input Files section for c99 that additional formats may be defined I see as for optional data, not for any additional requirements for producing a valid executable.
(0004589)
alanc (reporter)
2019-10-01 22:22

For platforms which support multiple compilers, how would you specify to getconf which compiler you want the flags for? For instance, on Solaris, _POSIX_V8_SHARED_OBJECT_LDFLAGS would need to be -G for Studio compilers but -shared for GNU & LLVM compilers.
(0004591)
Konrad_Schwarz (reporter)
2019-10-02 07:38
edited on: 2019-10-17 16:13

Re Note: 0004586: are these dynamic linker implementations relevant to POSIX? (Can POSIX assume an "ELF-only" world at this point?).

But this is a fair point -- I recognize that any serious engineering
organization will invest in far more in-depth handling of toolchain
issues that the generic getconf interface offers. OTOH, this is
true for c99 in general, so conversely, should c99 (or any hypothetical
successors like c11) be retired?

I would argue no; I think it is beneficial to have a standardized
(fairly simple) interface to the compilation tool chain, which,
at the least, gives you a starting point for platform (or toolchain)
dependent optimizations.

Re Note: 0004589: You would use whatever non-standard mechanism
that platform offers -- if any -- to map the standard c99 utility
to one of the compilers the platform supposedly supports.

(0004625)
eblake (manager)
2019-10-17 16:07

Re: Note: 0004591 asking about ELF-only. The Cygwin platform tries to comply with POSIX (where possible) but uses PE-COFF not ELF (thanks to the underlying Windows OS). And yes, Cygwin has support for loadable libraries. Thus, POSIX cannot assume ELF-only.
(0004634)
Konrad_Schwarz (reporter)
2019-11-05 11:20
edited on: 2019-11-05 11:34

Re: Note: 0004625: on purely formal grounds, I think Cygwin doesn't count,
as it is not a POSIX system.

However, the GNU ld --export-all-symbols option would seem to do the right thing, even on Cygwin.

So Cygwin could probably support this proposal.

(Also, it is quite likely that any system supporting libtool could support
this extension).

(0004635)
shware_systems (reporter)
2019-11-05 17:05

Cygwin does count, as do the apps created for Windows POSIX add-on, as the internals of .o files and application or library files linked from these, as a.out modules, are left unspecified. As such, COFF and OMF are as valid as ELF in that these all maintain associations of symbolic names with object or function addresses that a dlsym() implementation can reference.

This said, the consensus in the phone call discussions is an option like --export-all-symbols is missing from the c99 utility that ensures all names from source .o files with extern scope are represented in an a.out module, whether a main() declaration is processed or not. Addition of this switch is seen as more forward compatible than the Desired Action, and is using the Solaris cc compiler's -G switch as example/model of existing practice. See etherpad for details.
(0004636)
joerg (reporter)
2019-11-05 17:10

From my understanding, the option --export-all-symbols is not relevant for us as long as we do not standardize linker map files.

My impression is that --export-all-symbols is a workaround, since on UNIX, all global symbols remain global unless there is a linker map file.
(0004637)
shware_systems (reporter)
2019-11-05 17:41

yes, map and listing files of any sort are unspecified, as are linker utilities to begin with as unnecessary. That many toolchains include a separate linker is their election, but only needed if a compiler other than c99 is provided that requires it's use. As an option name it is descriptive of functionality c99 is seen as lacking, that's all, not that it is the model for removing that lack.
(0004638)
Konrad_Schwarz (reporter)
2019-11-06 08:27
edited on: 2019-11-06 10:09

Re: Note: 0004636 and Note: 0004637

I think you are missing the point here: the key difference
in Windows PE-COFF DLLs vs. ELF shared objects is that
DLLs have an explicit symbol export list, whereas shared objects
export everything by default. Obviously, there are various ways
of changing this, but I am talking about the default case here.

Now with DLLs, you either use a separate file
(a module definition file---not an linker map file)
to specify which symbols
are exported, or you mark up your code with declspec(dllexport), etc.
POSIX does not want you to do that, because ELF shared objects don't
require it: by default they export all non-static symbols, just as
in the static linking case.

However, the GCC toolchain for PE-COFF targets provides the
--export-all-symbols flag
which basically replicates the shared object behavior for PE-COFF DLLs.
It also supports "direct linking to a DLL" (see the manual),
so the .lib import file used by Microsoft toolchains is not required.
So the toolchain flow for GCC for a PE-COFF-based target turns out
to be pretty much identical to the ELF case.

Hence, an implementation of getconf on Cygwin can return
-Xl,--export-all-symbols or similar in the proposed
_POSIX_V8_SHARED_OBJECT_LDFLAGS tag.

Just to make this clear:
-Xl,--export-all-symbols is a compiler flag that gcc
and its POSIX-specified alter ego c99 understand.

Finally, note that GCC ld will automatically assume
--export-all-symbols
if no symbols would otherwise be exported. This means that
the "best" solution would probably be for
_POSIX_V8_SHARED_OBJECT_LDFLAGS to not include
-Xl,--export-all-symbols at all.

(0004686)
geoffclare (manager)
2019-12-13 10:06
edited on: 2019-12-13 10:18

The following changes were agreed in the 2019-12-12 teleconference.

When the c99 page is converted to a c17 or c2x page for Issue 8, make additional changes as if the following had been applied to the c99 page before the conversion.

On page 2540 line 82224 section c99, change:
<tt>[-L directory] [-l library]</tt>
to:
<tt>[-L directory] [-l library] [-R directory]</tt>

On page 2542 line 82227 section c99, change:
The system conceptually consists of a compiler and link editor.
to:
The system conceptually consists of a compilation phase, encompassing Translation Phases 1 through 7 of the ISO C standard, and a linkage phase, for handling Phase 8 of the ISO C standard and extensions described here. In addition, the compilation phase can be split into a separate preprocessing operation, handling Translation Phases 1 through 4, and a processing operation, handling Phases 5 though 7. Whether a single utility or multiple utilities for handling phases separately is provided by an implementation is left unspecified.

On page 2542 line 82228 section c99, change:
The input files referenced by pathname operands and -l option-arguments shall be compiled and linked to produce an executable file. (It is unspecified whether the linking occurs entirely within the operation of c99; some implementations may produce objects that are not fully resolved until the file is executed.)
to:
The input files referenced by pathname operands and -l option-arguments shall be compiled and linked to produce an executable file or, if the -G option is specified, a shared library file. It is unspecified whether the linking of an executable file occurs entirely within the operation of c99; when a pathname operand or -l option-argument names a shared library, an executable object may be produced that is not fully resolved until the file is executed.

On page 2542 line 82236 section c99, change:
If there are no options that prevent link editing (such as -c or -E), and all input files compile and link without error, the resulting executable file shall be written according to the -o outfile option (if present) or to the file a.out.

The executable file shall be created ...
to:
If there are no options that prevent link editing (such as -c or -E), and all input files compile and link without error, the resulting executable file or shared library file shall be written according to the -o outfile option, if present. If -o outfile is not specified, a resulting executable file shall be written to the file a.out; if the file to be written is a shared library file, the behavior is unspecified.

Executable files shall be created ...

On page 2542 line 82246 section c99, change:
The order of specifying the -L and -l options, ...
to:
The order of specifying the -L, -l and -R options, ...

Add three new options inserted in alphabetic order:
-B mode
If mode is "dynamic", produce a dynamically linked executable file. If the -B option is present with -c, -E or -G, the result is unspecified.
-G
Create a shared library or create object files suitable for inclusion in such a shared library. Compilations shall be performed in a manner suitable for the creation of shared libraries (for example, by producing position-independent code).

If -c is also specified, create object files suitable for inclusion in a shared library.

If -c is not specified, create a shared library. In this case the application shall ensure that the file named by the -o outfile option-argument includes an element named "so" or an implementation-defined element denoting a shared library, where elements in the last component of outfile are separated by <period> characters, for example libx.so.1; if no -o option is included in the options or the file named by the -o outfile option does not contain an element named "so" or an implementation-defined element denoting a shared library, the result is unspecified. If a pathname operand or -l option-argument names a shared library and that shared library defines an object used by the library being created, it shall become a dependency of the created shared library.

If the -G option is present with -B or -E, the result is unspecified.
-R directory
If the object file format supports it, specify a directory to be searched for shared libraries when an executable file or shared library being created by c99 is subsequently executed, or loaded using dlopen(). If directory contains any <colon> or <dollar-sign> characters, the behavior is unspecified. If an implementation provides a means for setting a default load time search location or locations, the -R option shall take precedence.

The directory named by directory shall not be searched by a process performing dynamic loading if either of the following are true:
  • the real and effective user IDs of that process are different and the directory has write permission for a user ID outside the set of the effective user ID of that process and any implementation-specific user IDs used for directories containing system libraries

  • the real and effective group IDs of that process are different and the directory has write permission for group IDs other than the effective group ID of that process.

Directories named in -R options shall be searched in the order specified, before the default system library locations are searched.

If a directory specified by a -R option contains files with names starting with any of the strings "libc.", "libl.", "libpthread.", "libm.", "librt.", [OB]"libtrace.",[/OB] "libxnet.", or "liby.", the result is unspecified.

If the -R option is present with -c or -E, the result is unspecified.

Change the description of the -l library option on P2543, L82286-82290 from:
Search the library named liblibrary.a. A library shall be searched when its name is encountered, so the placement of a -l option is significant. Several standard libraries can be specified in this manner, as described in the EXTENDED DESCRIPTION section. Implementations may recognize implementation-defined suffixes other than .a as denoting libraries.
to:
Search for the library named liblibrary.a or liblibrary.so. When searching for a library, the linker shall look at each directory specified by -L options that appear on the command line before this -l option, in the order given, and then the system default libraries. If liblibrary.a and liblibrary.so both exist in a directory, c99 shall use liblibrary.so if either -B dynamic or -G is specified. Once a library has been found (shared or static) in a directory, later directories in the list shall not be considered. A library shall be searched when its name is encountered, so the placement of a -l option is significant. Several standard libraries can be specified in this manner, as described in the EXTENDED DESCRIPTION section. Implementations may recognize implementation-defined suffixes other than .a and .so as denoting libraries.
(Note to the Editor: The liblibrary.a on P2543, L82286 seems to be a typo. The "library" in that string is the option-argument to the -l option. Therefore, it needs to be in italics as shown in the replacement text above.)

Change the description of the -o outfile option on P2543, L82297-82298 from:
Use the pathname outfile, instead of the default a.out, for the executable file produced. If the -o option is present with -c or -E, the result is unspecified.
to:
Name the output file to be produced. If the -o option is present with -c or -E, the result is unspecified.

When creating an executable file, use the pathname outfile, instead of the default a.out, for the executable file produced.

When creating a shared library, use the pathname outfile as the name of the shared library. If no -o outfile option is specified when creating a shared library, the result is unspecified.

On page 2543 line 82304 section c99, change:
Multiple instances of the -D, -I, -L, -l, and -U options can be specified.
to:
Multiple instances of the -D, -I, -L, -l, -R, and -U options can be specified.

On page 2544 line 82320 section c99, after applying bug 667 change INPUT FILES from:
Each input file shall be one of the following: a text file containing a C-language source program, a text file containing the output of c99 -E, an object file in the format produced by c99 -c, or a library of object files, in the format produced by archiving zero or more object files, using ar. Implementations may supply additional utilities that produce files in these formats. Additional input file formats are implementation-defined.
to:
Each input file shall be one of the following:
  • A text file containing a C-language source program or the output from c99 -E

  • An object file in the format produced by c99 -c

  • A library of object files in the format produced by archiving zero or more object files using ar

  • A shared library in the format produced by c99 -G

Implementations may supply additional utilities that produce files in these formats. Additional input file formats are implementation-defined.

Change the description of file.a on P2544, L82310-82312 from:
A library of object files typically produced by the ar utility, and passed directly to the link editor. Implementations may recognize implementation-defined suffixes other than .a as denoting object file libraries.
to:
A library of static object files typically produced by the ar utility, and referenced during the link-edit phase. Implementations may recognize implementation-defined suffixes other than .a as denoting static object file libraries.

and add a new entry for file.so after P2544, L82315 with the description:
A library of shared object files typically produced by the c99 utility with the -G option, and referenced during the link-edit phase. Implementations may recognize implementation-defined suffixes other than .so as denoting shared object file libraries.

On page 2546 line 82403 section c99, change:
It is unspecified whether the libraries libc.a, libl.a, libm.a, libpthread.a, librt.a, [OB]libtrace.a,[/OB] libxnet.a, or liby.a exist as regular files. The implementation may accept as -l option-arguments names of objects that do not exist as regular files.
to:
The libraries c, l, m, pthread, rt, [OB]trace,[/OB] xnet, and y shall be found as shared libraries when specified as the option-argument to the -l option and may also be found as static libraries but, except for the shared library version of the c library, need not exist as regular files. The implementation may accept as -l option-arguments names of additional implementation-defined libraries that do not exist as regular files.

On page 2550 line 82588 section c99, add new examples:
5. The following example shows how to create a shared library that does not depend on any other shared library:
c99 -G -c foo.c bar.c
c99 -G -o foobar.so foo.o bar.o

6. The following example shows how to create a dynamic executable that loads application specific shared libraries by searching a specified list of directories when it is executed:
c99 -G -c foo.c
c99 -G -o /path/to/dir1/foo.so foo.o
c99 -G -c bar.c
c99 -G -o /path/to/dir2/bar.so bar.o 
c99 -B dynamic -L /path/to/dir1 -L /path/to/dir2 -R /path/to/dir1 \
    -R /path/to/dir2 -o foobar foobar.c -l foo -l bar

On page 2551 line 82609 section c99, add a new paragraph to RATIONALE:
The shared library version of the c library is required to exist as a regular file because the dynamic linker needs to be able to load at least one library at execution time. Other standard shared libraries need not exist in their own right if the interfaces the standard requires them to provide exist in the c library; all that is required is that they are "found" when specified as -l option-arguments. Static versions of the standard libraries need not exist as regular files, even if they are found as static libraries when specified as -l option-arguments.

Add the following new paragraphs to the C99 RATIONALE section after P2551, L82629:
When the -R option is not included when an executable file or shared library is being created, some implementations use the environment variables LD_RUN_PATH and LD_LIBRARY_PATH to determine the directories to be searched for shared libraries.

Some implementations permit placeholders preceded by a <dollar-sign> character ('$'), such as $ORIGIN, in the -R directory option-argument to be evaluated at load time. Some implementations accept a colon separated list of directories for the path to search for shared libraries, with the same effect as specifying the -R option multiple times. However, these features are not universal.

The name of a shared library usually contains an element named "so". Other implementation-defined elements are allowed for backwards compatibility with historical systems, and so that tools can be developed on conforming systems to create libraries for multiple environments. For example, Microsoft systems use the filename extension ".dll" (and do not allow following text) to denote a shared library. The standard allows additional characters to be used in the name of a library following an "so" element to permit shared library versioning information to be at the end of the library filename rather than requiring that any such strings appear before the final element of the library name.

The decision to standardize on "so" as a required element in a shared library name was intentional, as the alternative would have been standardizing things such as a new make macro $(SHLIB_EXT) that would otherwise be needed to write a portable makefile that can compile shared libraries despite not having a standardized element name.

If a combination of direct and indirect dependencies of a shared library would require different versions of another shared library, options that are not specified by the standard (such as -B direct) will probably need to be used when linking that shared library, so that at runtime the intended versions are found.



- Issue History
Date Modified Username Field Change
2019-10-01 08:03 Konrad_Schwarz New Issue
2019-10-01 08:03 Konrad_Schwarz Name => Konrad Schwarz
2019-10-01 08:03 Konrad_Schwarz Organization => Siemens AG
2019-10-01 08:03 Konrad_Schwarz Section => c99
2019-10-01 08:03 Konrad_Schwarz Page Number => https://pubs.opengroup.org/onlinepubs/9699919799/utilities/c99.html [^]
2019-10-01 08:03 Konrad_Schwarz Line Number => https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/utilities/c99.html#tag_20_11_13_03 [^]
2019-10-01 17:14 GarrettWollman Note Added: 0004586
2019-10-01 20:05 shware_systems Note Added: 0004587
2019-10-01 22:22 alanc Note Added: 0004589
2019-10-02 07:38 Konrad_Schwarz Note Added: 0004591
2019-10-17 16:07 eblake Note Added: 0004625
2019-10-17 16:13 Don Cragun Note Edited: 0004591
2019-11-05 11:20 Konrad_Schwarz Note Added: 0004634
2019-11-05 11:34 Konrad_Schwarz Note Edited: 0004634
2019-11-05 17:05 shware_systems Note Added: 0004635
2019-11-05 17:10 joerg Note Added: 0004636
2019-11-05 17:41 shware_systems Note Added: 0004637
2019-11-06 08:27 Konrad_Schwarz Note Added: 0004638
2019-11-06 08:28 Konrad_Schwarz Note Edited: 0004638
2019-11-06 10:09 Konrad_Schwarz Note Edited: 0004638
2019-12-13 10:06 geoffclare Note Added: 0004686
2019-12-13 10:18 geoffclare Note Edited: 0004686


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker