Austin Group Defect Tracker

Aardvark Mark III


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001134 [1003.1(2016)/Issue7+TC2] System Interfaces Editorial Enhancement Request 2017-03-31 07:06 2017-05-24 07:39
Reporter nmav View Status public  
Assigned To
Priority normal Resolution Open  
Status New  
Name Nikos Mavrogiannopoulos
Organization Red Hat
User Reference
Section getentropy
Page Number 0
Line Number 0
Interp Status ---
Final Accepted Text
Summary 0001134: Add getentropy interface
Description Software often requires a source of pseudo random data, that provides high quality unpredictable random data, suitable for cryptographic use.
Various operating systems provide interfaces via device files, such as
/dev/urandom, and /dev/random, as well as system calls like getrandom(),
getentropy() and others. However, there is no common convention in
systems resulting to much of compatibility code.

The proposed interface below is based on the getentropy description as entered
the GLIBC in 2016, however, it existed in OpenBSD and other systems
since long time.


SYNOPSIS
       #include <unistd.h>

       int getentropy(void *buffer, size_t length);

DESCRIPTION
The getentropy() function writes length bytes of data starting at the location pointed to by buffer. The output is unpredictable high quality random data, generated by a cryptographically secure pseudo-random number generator.
The maximum permitted value for the length argument is 256.

A successful call to getentropy() always provides the requested number of bytes of entropy.

RETURN VALUE
       On success, this function returns zero, On error, -1 is
returned, and errno is set appropriately.

ERRORS
       EFAULT Part or all of the buffer specified by buffer and length
is not in valid addressable memory.

       EIO Length is greater than 256.

       ENOSYS The system does not provide the necessary system calls for this operation.

NOTES

The getentropy () call is not a cancellation point.
Desired Action Add the getentropy interface(). This interface is related to the withdrawn #859.
Tags No tags attached.
Attached Files

- Relationships
related to 0000859Closed 1003.1(2013)/Issue7+TC1 Add posix_random family of interfaces 

-  Notes
(0003657)
EdSchouten (reporter)
2017-04-01 06:25

Though I'm still a fan of having an interface adopted that is guaranteed to work (like arc4random_buf()), any progress in this area is good.

Instead of hardcoding the 256-byte upper limit, would it make sense to add a definition 'GETENTROPY_MAX', for which we state that it has to be at least 256 bytes in size? Leaving it undefined either means there is no limit or it's compile-time undefined. We should probably also add a sysconf() option to query the run-time limit.

The description of EFAULT should probably be removed, as we don't document it for other calls specifically.

EIO is inconsistent with the rest of the functions part of the specification. EINVAL is typically used to denote that an argument passed to the function is outside of bounds.
(0003660)
nmav (reporter)
2017-04-06 09:46

> Though I'm still a fan of having an interface adopted that is guaranteed to work (like arc4random_buf()), any progress in this area is good.

Certainly, though this interface is quite orthogonal to that. It will allow crypto libraries to seed their own PRNG using the system's PRNG. A posix_random() function with the properties of arc4random_buf() would be certainly a nice complement to this one, but not necessary.

> Instead of hardcoding the 256-byte upper limit, would it make sense to add a definition 'GETENTROPY_MAX', for which we state that it has to be at least 256 bytes in size? Leaving it undefined either means there is no limit or it's compile-time undefined. We should probably also add a sysconf() option to query the run-time limit.

I like that proposal and the EIO comments, especially the part where it requires it to be more than 256 bytes (to prevent systems which set values like 1). Feel free to post a note with an updated interface incorporating your comments.
(0003662)
shware_systems (reporter)
2017-04-06 20:00

IIRC, Bug 859 was tentatively approved partly because PRNGs can be implemented entirely in software. Perceived randomness was therefore a function of algorithm quality and usage. An interface like this more assumes motherboards or CPUs all have peripherals that are designed to generate these uniformly random sequences independently, to guarantee they are 'cryptographically secure' in the black box sense, and few commercial systems, if any, that I'm aware of actually provide this type of hardware. Many have hardware that can provide values random enough looking, possibly, to be a PRNG seed value, but not securely so or dedicated to this purpose. This lack of hardware is why, I believe, many PKCS key generators rely on the effective randomness of an operator moving a mouse around as their seed source.

That said, I agree adding something along these lines is desirable for uniformity of usage, but I don't see it as a candidate for the Base. It should be marked part of the CRYPT Option Group, and the ENOSYS error should use wording consistent with the other members of that group. No POSIX system is required to use a system call paradigm, after all; hardware accesses can be coded directly into the C library.
(0003663)
nmav (reporter)
2017-04-07 08:47

I do not believe that the comment above is accurate. There is no hardware requirement such as "random generator" peripherals, or any such assumption whatsoever, to provide a cryptographic random generator. In fact, in almost none of the systems that today provide that functionality, such hardware is present. Implementing a CSPRNG requires the OS being able to gather several bits of information unpredictable to an adversary to seed a software PRNG. That is possible today even with the most constrained systems.
(0003664)
schwarze (reporter)
2017-04-07 10:35

Since nobody else from OpenBSD commented so far: Is it really wise to standardize getentropy() before standardizing arc4random()? We have seen application software abuse getentropy() as a random number generator before, and this order of standardization might possibly precipitate that kind of abuse. Wouldn't it be a better idea to standardize arc4random[_buf]() first, which is *much* more relevant for application software?

I'm not at all saying that i oppose including this interface, in particular since the use case given by nmav, "allow crypto libraries to seed their own PRNG", is probably relevant. Would it make sense to include a statement to the effect, "This function is not intended to be used by application programs as a random number generator. Its use should be restricted to seeding pseudo random number generator libraries" in the DESCRIPTION, or at least in the APPLICATION USAGE?

Besides, as far as i know, the restriction to 256 bytes was imposed deliberately and not as a matter of laziness, because that amount of data is sufficient to seed PRNGs, and allowing more would merely complicate the implementation, increase the risk of depleting the actual entropy available, and encourage abuse as a random-number generator. For that reason, GETENTROPY_MAX seems like a bad idea, not only for reasons of needless complication and incompatibility with existing implementations, but also because it encourages abuse of the function. I strongly agree with nmav's argument that getentropy() and arc4random_buf() are complimentary, and that argument implies that GETENTROPY_MAX blurs the picture rather than helping anything.

Please take all this with a grain of salt; i'm not a specialist in cryptography. I'm merely asking questions based on what i heard in discussions with the original developers of getentropy().
(0003667)
EdSchouten (reporter)
2017-04-07 17:32

I agree with Ingo. POSIX should aim at standardising functional APIs that people can use freely in their applications. It currently feels as if we're attempting to standardise the low-level intrinsic, while we should aim to add the high-level API instead.
(0003668)
steffen (reporter)
2017-04-07 18:54

I totally and completely disagree with #3667. Anyone can place mediocre high-level pseudo random code on top of a good low-level interface. I think it gets increasingly impossible to create something good on a mediocre or non-existing low-level entropy source interface.

I for one adore POSIX because of the low-level interface that (would) allow(ed) me to use -nostdinc (and not to forget -fno-exceptions -fno-rtti). Except for getdirentries() almost anything one needs is available to create truly portable applications like so, which i think cannot be rated high enough!

This is true even if vendors seem to favor more and more to put money into the Linux kernel than in their own complete distributions, in order to effectively safe money on the research side -- this is nothing but a degenerative motion towards a monoculture, and in real-life monocultures cannot survive. (In fact "safe even more money", if one looks at the manuals of some commercial vendors, which never got enough time nor love.)

I regret that we don't get a high-level interface in addition, i.e., that the corresponding issue has been closed: because like this vendors could exchange algorithms as the necessities arise, and could provide optimised variants. And because errors would be encapsulated in the good maintained C library, rather than shipped in uncountable packages which implement their own / have a copy of some pseudo-random generator because they need the functionality.

But it should be in addition, because in practice many tasks require good random entropy and systems do provide it since decades: /dev/u?random on Linux and FreeBSD, /dev/arandom on OpenBSD, increasingly often sysctl(2)s or direct system calls which provide the same data but without requiring a file-descriptor.
It would be logical to lift this to a standardized level, in my opinion.
(0003669)
shware_systems (reporter)
2017-04-07 19:01

Re: 3663 & 3667:
It's described as referencing device files, implying to me actual devices (such as fingerprint or retinal scanners as biometric "entropy" just starting to be in common use), not bits it is hoped is unpredictable masquerading as one. Many of the bits current systems use have modulo or similar problems that allow them to be brute force attacked, or can be compromised in other ways because they're not black boxes.

The RC4 algorithm I believe arc4random() is based on (I haven't looked at the code) has been shown to be susceptible to brute force attacks too, so is no longer recommended by NIST for any new code, as are the current interfaces in the CRYPT group. Moore's law has caught up with algorithms using less than 64 bits of seed, pretty much. The movement is heading towards governments requiring platforms support algorithms that use over 256 bits as seed sizes, in the public documents I've come across.
See http://www.pdflib.com/en/knowledge-base/pdf-20/deprecated-features/ [^] for the plans for PDFs, being voted on now or soon at ISO, as an example of this movement.

This is one of the faster evolving technologies, actually, and eventually I expect the experts on cryptography employed by various national bodies, and who have access to their classified documents that might pertain also, to weigh in directly on what should or shouldn't be part of the standard for Issue 8. It may even be a formal ISO-TR work activity, similar to how the regex group was a separate effort for Issue 6, but this is far from being decided yet.

An interface like this or a work-alike I don't see as being so controversial someone might want to exclude it, but who knows what some politicians might decide to enact as laws whatever our aims, given how much cyber-security issues are being bandied about various media nowadays. By the time Issue 8 is that advanced in addressing the non-security related bug reports, there may well be new hardware requirements and algorithms the industry, and then the standard or this interface, will have to accommodate.
(0003670)
EdSchouten (reporter)
2017-04-07 19:04

The security of the RC4 algorithm is not up for discussion. The API is called arc4random*() for historical reasons. There is no requirement that it's actually implemented using RC4. OpenBSD uses Salsa/ChaCha20, for example.
(0003671)
shware_systems (reporter)
2017-04-08 04:45

*shrug* FreeBSD uses both an Arc4 variant, in its libc, and ChaCha, in an OpenBSD compatibility module, so it is part of the history of arc4random(). The point is many interfaces that have seen common use in this area are not as secure nor as random as advertised, and are subject to being superseded by factors outside the control of POSIX or implementors.
(0003734)
nmav (reporter)
2017-05-24 07:39

Please avoid unrelated discussion for APIs which are not part of the proposal. The proposal is about adding getentropy().

- Issue History
Date Modified Username Field Change
2017-03-31 07:06 nmav New Issue
2017-03-31 07:06 nmav Name => Nikos Mavrogiannopoulos
2017-03-31 07:06 nmav Organization => Red Hat
2017-03-31 07:06 nmav Section => getentropy
2017-03-31 07:06 nmav Page Number => 0
2017-03-31 07:06 nmav Line Number => 0
2017-03-31 07:17 Don Cragun Relationship added related to 0000859
2017-04-01 06:25 EdSchouten Note Added: 0003657
2017-04-06 09:46 nmav Note Added: 0003660
2017-04-06 20:00 shware_systems Note Added: 0003662
2017-04-07 08:47 nmav Note Added: 0003663
2017-04-07 10:35 schwarze Note Added: 0003664
2017-04-07 17:32 EdSchouten Note Added: 0003667
2017-04-07 18:54 steffen Note Added: 0003668
2017-04-07 19:01 shware_systems Note Added: 0003669
2017-04-07 19:04 EdSchouten Note Added: 0003670
2017-04-08 04:45 shware_systems Note Added: 0003671
2017-05-24 07:39 nmav Note Added: 0003734


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker