Every time I call RAND_bytes and RAND_pseudo_bytes, with the same seed, it returns different random numbers and I don't understand why. It said that the PRNG automatically seeds itself from /dev/urandom in Linux, but how does it work in Windows?
Why does the same seed lead to different random numbers?
Why does the same seed lead to different random numbers?
You can read about the general design of the rand subsystem at Random Numbers on the OpenSSL wiki. The reason the same seed produces different random numbers is...
It depends on the generator. If you are using the default generator, then you are using md_rand. If you look at the source code for md_rand.c, then you will see rand_bytes adds entropy at each invocation with system calls to functions like time.
On Linux rand_bytes also adds the result of getpid; and on Windows it adds the result of GetSystemTime and SystemTimeToFileTime.
Adding entropy at each invocation is a good design practice for RNGs. Also see When Good Randomness Goes Bad: Virtual Machine Reset Vulnerabilities
and Hedging Deployed Cryptography and When Virtual is Harder than Real: Security Challenges in Virtual Machine Based Computing Environments.
Related
I am trying to use fortran intrinsic PRNG in a MPI code.
I understand from this link that GFortran implement the PRNG using xorshift1024* which has a period of 2^1024 - 1. It also says:
Note that in a multi-threaded program (e.g. using OpenMP directives),
each thread will have its own random number state.
Then reading this I found:
When a new thread uses
RANDOM_NUMBER for the first time, the seed is copied from the master
seed, and forwarded N * 2^512 steps to guarantee that the random
stream does not alias any other stream in the system, where N is the
number of threads that have used RANDOM_NUMBER so far during the
program execution
If this is an automatic feature of GFortran, it works only in OpenMP? What if I want to have parallel PRNG using MPI? How can I ensure the portability of the code to other compilers?
In other words: Is there any way to do what GFortran says it does (i.e. guarantee real parallel PRNG) in a portable way using the fortran intrinsic instructions?
NOTE: I was using the PRNG of Numerical Recipes in MPI. That worked well for some years, but now I am getting some errors in some assumptions on the integer model that Numerical Recipes says goes beyond fortran... so I don't see how to solve that and that is way I want to use the intrinsic PRNG if is possible.
Note that the use of xorshoft1024* is a very new feature in GFortran, it's only available in the development trunk version, no released version has it yet at the time of writing this. It will be released as part of GCC 7, probably in spring 2017.
So when you're using MPI, each MPI rank is a separate process and the random number generators in each process is completely separate with no communication between the PRNG's in different processes (unless you handle it yourself with MPI, of course). The thing with forwarding the PRNG stream 2^512 steps happens only when using the PRNG from multiple threads within the same process.
That being said, xorshift1024* has a fairly long period (2^1024-1), and the first time the PRNG is used in a process (again, think MPI rank) it is initialized with random data from the OS (/dev/urandom on POSIX systems), unless it has already been explicitly initialized with RANDOM_SEED. So in practice I think you'll be fine, it's exceedingly unlikely that the PRNG streams for different MPI ranks will alias.
And no, the above describes the PRNG in GFortran version 7. If you want something portable you cannot rely on anything beyond what the standard guarantees. Beyond the parallel aspects, for portable high quality random numbers you're probably better of with using a known good PRNG rather than relying on the one provided by the compiler (I have personal experience of at least one compiler producing poor quality random numbers with the RANDOM_NUMBER intrinsic, but I'll refrain from naming the vendor since it was many years ago and they might have fixed it since, if they are even in business anymore, I don't know).
(If you find the semantics of the new xorshift1024* implementation difficult, blame a) me, since I devised it and implemented it b) the Fortran standard which makes it impossible to have a parallel PRNG with simple semantics)
If you want to have a portable version of a multi-stream random number generator for a Fortran program, there is a multi-stream Fortran version of the Mersenne Twister. See http://theo.phys.sci.hiroshima-u.ac.jp/~ishikawa/PRNG/mt_stream_en.html . It uses the concept of advancing the PRNG by a very large number of steps for the different threads. It's setup and configured by subroutine calls so you should be able to use it from various multi-threading environments.
I have an embedded device using the standard glibc srand()/rand() functions to generate pseudo-random values. For the same seed x I need to get the same values for rand() on a Windows machine. To my surpise however, this isn't as simple as it seems.
Glibc apparently used two algorithms for it's random number generator. An older one, described here, and a newer one, which my embedded device uses. How can I implement the latter on another platform? My knowlegde of C isn't thorough enough to understand what exactly is happening there.
We need to migrate to a better RNG or RBG for some key value generation which will be further used for encryption of the data.
Which will be the most suitable algorithm? Shall I consider NIST doc for this?
Any pseudo random number generator that produces a Gaussian distribution and that has a wide output (say at least 32 bits) should be enough for creating keys. It's up to you to determine your needs and then find a matching RNG.
For more info, see http://www.random.org/randomness.
Depending on the language you choose to implement this, I'm sure you can find source code for pseudo-RNG on the Web, if the one built-in into your system isn't good enough.
As we are a programming site, I would seriously look at the secure random number generators at your disposal in your particular runtime environment. In general you will have to rely on system resources to generate randoms, at least to seed the pseudo random number generator. The only possible exception are CPU specific random instructions, such as the ones used on the latest Intel CPU's (hopefully well-tested secure RNGs will become a main feature of CPU's).
Within many programming environments there is very little choice but to use OpenSSL or /dev/random for seeding. In general it is hard to find useful information about the random number generator. Sometimes the RNG is really not suitable at all (e.g. the native PHP version).
If possible, try to find something that conforms to NIST requirements.
I am looking for an equivalent to drand48 on Windows. To all who do not know, the following is not equivalent:
(double)rand()/RAND_MAX;
Firstly, rand returns values including RAND_MAX
Secondly, on Windows RAND_MAX=32767 which is a too short period for my application.
My purpose is to generate noise for a simulation. It is desirable to use a pseudo-random generator with the same period as drand48.
Firstly, note that you appear to be confusing the resolution with the period. On Windows, rand will return values from 0 to 32767, but this does not mean that the same values will repeat every 32768 calls. So rand should be perfectly adequate unless you need more than 16 bits of resolution. (The resolution and the period are the same in drand48, but not in all pseudorandom number generators.)
If you do not need the exact behaviour of drand48, rand_s would be the simplest option. It has a 32-bit resolution, less than drand48 but enough for most purposes. It generates a cryptographically secure random number, so there is no fixed period. One possible issue is that it will be much slower than drand48.
If you want the same behaviour of drand48, the algorithm is documented and should be easy to re-implement, or you could use the source code from FreeBSD (link to source code browser on http://fxr.watson.org/).
7 years late to the party. Sorry.
Gnu Scientific Library is a good solution to this problem. The library employs several high-quality generating algorithms.
https://www.gnu.org/software/gsl/doc/html/rng.html
It might not be an exact answer to your question, but still it is a solution. Use the CryptGenRandom(it's form the WinAPI).
I want to produce a random cryptographic key on Windows. Where can I obtain entropy?
I would like my entropy function to work without a network connection and to be reliable on Windows 2000 and upwards. Even sources which may or may not provide a small amount of entropy could be useful as all the sources will be pooled.
This is my initial list of functions:
GetCurrentProcessID,
GetCurrentThreadID,
GetTickCount,
GetLocalTime,
QueryPerformanceCounter,
GlobalMemoryStatus,
GetDiskFreeSpace,
GetComputerName,
GetUserName,
GetCursorPos,
GetMessageTime,
GetSystemInfo,
CryptGenRandom,
GetProcessHandleCount,
GetProcessMemoryInfo.
Although early versions of the CryptGenRandom function may contain weaknesses later versions follow secure standards (see remarks on the CrypGenRandom page.)
It is weak to just use time as your seed. There is an answer under What is the most secure seed for random number generation? which explains that the unpredictable random seed may only need 128 bits to produce a secure PRNG. It is therefore probably unnecessary to find more sources than those listed in the question, and normally the CryptGenRandom function will already contain and generate enough entropy for itself that the caller does not need to do any of this.
CryptGenRandom and the function CryptAcquireContext which must preceed it can be called from Delphi like this.
If its an option you can ask user to move mouse pointer for a while.
The only external source that most machines have is Mic In/Line In, call waveInOpen+waveInPrepareHeader+waveInAddBuffer+waveInStart. How random that is probably depends on the hardware...