Austin Group Defect Tracker

Aardvark Mark III


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001134 [1003.1(2016)/Issue7+TC2] System Interfaces Editorial Enhancement Request 2017-03-31 07:06 2019-01-16 13:33
Reporter nmav View Status public  
Assigned To
Priority normal Resolution Open  
Status New  
Name Nikos Mavrogiannopoulos
Organization Red Hat
User Reference
Section getentropy
Page Number 0
Line Number 0
Interp Status ---
Final Accepted Text
Summary 0001134: Add getentropy interface
Description Software often requires a source of pseudo random data, that provides high quality unpredictable random data, suitable for cryptographic use.
Various operating systems provide interfaces via device files, such as
/dev/urandom, and /dev/random, as well as system calls like getrandom(),
getentropy() and others. However, there is no common convention in
systems resulting to much of compatibility code.

The proposed interface below is based on the getentropy description as entered
the GLIBC in 2016, however, it existed in OpenBSD and other systems
since long time.


SYNOPSIS
       #include <unistd.h>

       int getentropy(void *buffer, size_t length);

DESCRIPTION
The getentropy() function writes length bytes of data starting at the location pointed to by buffer. The output is unpredictable high quality random data, generated by a cryptographically secure pseudo-random number generator.
The maximum permitted value for the length argument is 256.

A successful call to getentropy() always provides the requested number of bytes of entropy.

RETURN VALUE
       On success, this function returns zero, On error, -1 is
returned, and errno is set appropriately.

ERRORS
       EFAULT Part or all of the buffer specified by buffer and length
is not in valid addressable memory.

       EIO Length is greater than 256.

       ENOSYS The system does not provide the necessary system calls for this operation.

NOTES

The getentropy () call is not a cancellation point.
Desired Action Add the getentropy interface(). This interface is related to the withdrawn #859.
Tags No tags attached.
Attached Files

- Relationships
related to 0000859Closed 1003.1(2013)/Issue7+TC1 Add posix_random family of interfaces 

-  Notes
(0003657)
EdSchouten (reporter)
2017-04-01 06:25

Though I'm still a fan of having an interface adopted that is guaranteed to work (like arc4random_buf()), any progress in this area is good.

Instead of hardcoding the 256-byte upper limit, would it make sense to add a definition 'GETENTROPY_MAX', for which we state that it has to be at least 256 bytes in size? Leaving it undefined either means there is no limit or it's compile-time undefined. We should probably also add a sysconf() option to query the run-time limit.

The description of EFAULT should probably be removed, as we don't document it for other calls specifically.

EIO is inconsistent with the rest of the functions part of the specification. EINVAL is typically used to denote that an argument passed to the function is outside of bounds.
(0003660)
nmav (reporter)
2017-04-06 09:46

> Though I'm still a fan of having an interface adopted that is guaranteed to work (like arc4random_buf()), any progress in this area is good.

Certainly, though this interface is quite orthogonal to that. It will allow crypto libraries to seed their own PRNG using the system's PRNG. A posix_random() function with the properties of arc4random_buf() would be certainly a nice complement to this one, but not necessary.

> Instead of hardcoding the 256-byte upper limit, would it make sense to add a definition 'GETENTROPY_MAX', for which we state that it has to be at least 256 bytes in size? Leaving it undefined either means there is no limit or it's compile-time undefined. We should probably also add a sysconf() option to query the run-time limit.

I like that proposal and the EIO comments, especially the part where it requires it to be more than 256 bytes (to prevent systems which set values like 1). Feel free to post a note with an updated interface incorporating your comments.
(0003662)
shware_systems (reporter)
2017-04-06 20:00

IIRC, Bug 859 was tentatively approved partly because PRNGs can be implemented entirely in software. Perceived randomness was therefore a function of algorithm quality and usage. An interface like this more assumes motherboards or CPUs all have peripherals that are designed to generate these uniformly random sequences independently, to guarantee they are 'cryptographically secure' in the black box sense, and few commercial systems, if any, that I'm aware of actually provide this type of hardware. Many have hardware that can provide values random enough looking, possibly, to be a PRNG seed value, but not securely so or dedicated to this purpose. This lack of hardware is why, I believe, many PKCS key generators rely on the effective randomness of an operator moving a mouse around as their seed source.

That said, I agree adding something along these lines is desirable for uniformity of usage, but I don't see it as a candidate for the Base. It should be marked part of the CRYPT Option Group, and the ENOSYS error should use wording consistent with the other members of that group. No POSIX system is required to use a system call paradigm, after all; hardware accesses can be coded directly into the C library.
(0003663)
nmav (reporter)
2017-04-07 08:47

I do not believe that the comment above is accurate. There is no hardware requirement such as "random generator" peripherals, or any such assumption whatsoever, to provide a cryptographic random generator. In fact, in almost none of the systems that today provide that functionality, such hardware is present. Implementing a CSPRNG requires the OS being able to gather several bits of information unpredictable to an adversary to seed a software PRNG. That is possible today even with the most constrained systems.
(0003664)
schwarze (reporter)
2017-04-07 10:35

Since nobody else from OpenBSD commented so far: Is it really wise to standardize getentropy() before standardizing arc4random()? We have seen application software abuse getentropy() as a random number generator before, and this order of standardization might possibly precipitate that kind of abuse. Wouldn't it be a better idea to standardize arc4random[_buf]() first, which is *much* more relevant for application software?

I'm not at all saying that i oppose including this interface, in particular since the use case given by nmav, "allow crypto libraries to seed their own PRNG", is probably relevant. Would it make sense to include a statement to the effect, "This function is not intended to be used by application programs as a random number generator. Its use should be restricted to seeding pseudo random number generator libraries" in the DESCRIPTION, or at least in the APPLICATION USAGE?

Besides, as far as i know, the restriction to 256 bytes was imposed deliberately and not as a matter of laziness, because that amount of data is sufficient to seed PRNGs, and allowing more would merely complicate the implementation, increase the risk of depleting the actual entropy available, and encourage abuse as a random-number generator. For that reason, GETENTROPY_MAX seems like a bad idea, not only for reasons of needless complication and incompatibility with existing implementations, but also because it encourages abuse of the function. I strongly agree with nmav's argument that getentropy() and arc4random_buf() are complimentary, and that argument implies that GETENTROPY_MAX blurs the picture rather than helping anything.

Please take all this with a grain of salt; i'm not a specialist in cryptography. I'm merely asking questions based on what i heard in discussions with the original developers of getentropy().
(0003667)
EdSchouten (reporter)
2017-04-07 17:32

I agree with Ingo. POSIX should aim at standardising functional APIs that people can use freely in their applications. It currently feels as if we're attempting to standardise the low-level intrinsic, while we should aim to add the high-level API instead.
(0003668)
steffen (reporter)
2017-04-07 18:54

I totally and completely disagree with #3667. Anyone can place mediocre high-level pseudo random code on top of a good low-level interface. I think it gets increasingly impossible to create something good on a mediocre or non-existing low-level entropy source interface.

I for one adore POSIX because of the low-level interface that (would) allow(ed) me to use -nostdinc (and not to forget -fno-exceptions -fno-rtti). Except for getdirentries() almost anything one needs is available to create truly portable applications like so, which i think cannot be rated high enough!

This is true even if vendors seem to favor more and more to put money into the Linux kernel than in their own complete distributions, in order to effectively safe money on the research side -- this is nothing but a degenerative motion towards a monoculture, and in real-life monocultures cannot survive. (In fact "safe even more money", if one looks at the manuals of some commercial vendors, which never got enough time nor love.)

I regret that we don't get a high-level interface in addition, i.e., that the corresponding issue has been closed: because like this vendors could exchange algorithms as the necessities arise, and could provide optimised variants. And because errors would be encapsulated in the good maintained C library, rather than shipped in uncountable packages which implement their own / have a copy of some pseudo-random generator because they need the functionality.

But it should be in addition, because in practice many tasks require good random entropy and systems do provide it since decades: /dev/u?random on Linux and FreeBSD, /dev/arandom on OpenBSD, increasingly often sysctl(2)s or direct system calls which provide the same data but without requiring a file-descriptor.
It would be logical to lift this to a standardized level, in my opinion.
(0003669)
shware_systems (reporter)
2017-04-07 19:01

Re: 3663 & 3667:
It's described as referencing device files, implying to me actual devices (such as fingerprint or retinal scanners as biometric "entropy" just starting to be in common use), not bits it is hoped is unpredictable masquerading as one. Many of the bits current systems use have modulo or similar problems that allow them to be brute force attacked, or can be compromised in other ways because they're not black boxes.

The RC4 algorithm I believe arc4random() is based on (I haven't looked at the code) has been shown to be susceptible to brute force attacks too, so is no longer recommended by NIST for any new code, as are the current interfaces in the CRYPT group. Moore's law has caught up with algorithms using less than 64 bits of seed, pretty much. The movement is heading towards governments requiring platforms support algorithms that use over 256 bits as seed sizes, in the public documents I've come across.
See http://www.pdflib.com/en/knowledge-base/pdf-20/deprecated-features/ [^] for the plans for PDFs, being voted on now or soon at ISO, as an example of this movement.

This is one of the faster evolving technologies, actually, and eventually I expect the experts on cryptography employed by various national bodies, and who have access to their classified documents that might pertain also, to weigh in directly on what should or shouldn't be part of the standard for Issue 8. It may even be a formal ISO-TR work activity, similar to how the regex group was a separate effort for Issue 6, but this is far from being decided yet.

An interface like this or a work-alike I don't see as being so controversial someone might want to exclude it, but who knows what some politicians might decide to enact as laws whatever our aims, given how much cyber-security issues are being bandied about various media nowadays. By the time Issue 8 is that advanced in addressing the non-security related bug reports, there may well be new hardware requirements and algorithms the industry, and then the standard or this interface, will have to accommodate.
(0003670)
EdSchouten (reporter)
2017-04-07 19:04

The security of the RC4 algorithm is not up for discussion. The API is called arc4random*() for historical reasons. There is no requirement that it's actually implemented using RC4. OpenBSD uses Salsa/ChaCha20, for example.
(0003671)
shware_systems (reporter)
2017-04-08 04:45

*shrug* FreeBSD uses both an Arc4 variant, in its libc, and ChaCha, in an OpenBSD compatibility module, so it is part of the history of arc4random(). The point is many interfaces that have seen common use in this area are not as secure nor as random as advertised, and are subject to being superseded by factors outside the control of POSIX or implementors.
(0003734)
nmav (reporter)
2017-05-24 07:39

Please avoid unrelated discussion for APIs which are not part of the proposal. The proposal is about adding getentropy().
(0004137)
ajosey (manager)
2018-09-30 18:18

See the New Work item section of http://www.opengroup.org/austin/docs/austin_sd6.txt [^]

The recommended criteria for development of new interfaces to enable
them to be considered for inclusion in a future revision are as follows:

1.There must be a written specification that has undergone a formal
consensus based approval process and is suitable for inclusion.

Parties interested in submitting new work items through one of the
three organizations within the Austin Group (The Open Group, IEEE, ISO/IEC)
should contact the appropriate Organizational Representative for further
information and advice on how each organization handles new work items.
Submissions from other organizations will also be considered.
Items 2 through 4 below apply to all submissions regardless of
origin.

2.There must be an implementation, preferably a reference implementation.

3.The specification must be "sponsored" by one of three organizations
(The Open Group, IEEE, ISO/IEC) within the Austin Group, i.e. they would
support and champion its inclusion.

4.Submitters must provide an outline plan of the editing instructions to
merge the document with the Austin Group specifications, and assistance
to the Austin Group editors as required to complete the merger.
For an example, see

https://collaboration.opengroup.org/platform/single_unix_specification/protected/mailarch.php?soph=N&action=show&archive=austin-group-l&num=434 [^] [^]

or
https://collaboration.opengroup.org/sophocles/mailarch.php?soph=Y&action=show&archive=austin-group-l&num=434 [^] [^]

Andrew J will take the action to forward to The Open Group Organizational Rep.
(0004211)
ajosey (manager)
2019-01-16 13:33
edited on: 2019-01-16 13:33

The Open Group Base Working group has agreed to sponsor this as a new work item. The next stage will be to develop a manual page entry for review and eventual approval at The Open Group.

Is there further input needed beyond that in the Description of this bug report?


- Issue History
Date Modified Username Field Change
2017-03-31 07:06 nmav New Issue
2017-03-31 07:06 nmav Name => Nikos Mavrogiannopoulos
2017-03-31 07:06 nmav Organization => Red Hat
2017-03-31 07:06 nmav Section => getentropy
2017-03-31 07:06 nmav Page Number => 0
2017-03-31 07:06 nmav Line Number => 0
2017-03-31 07:17 Don Cragun Relationship added related to 0000859
2017-04-01 06:25 EdSchouten Note Added: 0003657
2017-04-06 09:46 nmav Note Added: 0003660
2017-04-06 20:00 shware_systems Note Added: 0003662
2017-04-07 08:47 nmav Note Added: 0003663
2017-04-07 10:35 schwarze Note Added: 0003664
2017-04-07 17:32 EdSchouten Note Added: 0003667
2017-04-07 18:54 steffen Note Added: 0003668
2017-04-07 19:01 shware_systems Note Added: 0003669
2017-04-07 19:04 EdSchouten Note Added: 0003670
2017-04-08 04:45 shware_systems Note Added: 0003671
2017-05-24 07:39 nmav Note Added: 0003734
2018-09-30 18:18 ajosey Note Added: 0004137
2019-01-16 13:33 ajosey Interp Status => ---
2019-01-16 13:33 ajosey Note Added: 0004211
2019-01-16 13:33 ajosey Note Edited: 0004211


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker