We need to migrate to a better RNG or RBG for some key value generation which will be further used for encryption of the data.
Which will be the most suitable algorithm? Shall I consider NIST doc for this?
Any pseudo random number generator that produces a Gaussian distribution and that has a wide output (say at least 32 bits) should be enough for creating keys. It's up to you to determine your needs and then find a matching RNG.
For more info, see http://www.random.org/randomness.
Depending on the language you choose to implement this, I'm sure you can find source code for pseudo-RNG on the Web, if the one built-in into your system isn't good enough.
As we are a programming site, I would seriously look at the secure random number generators at your disposal in your particular runtime environment. In general you will have to rely on system resources to generate randoms, at least to seed the pseudo random number generator. The only possible exception are CPU specific random instructions, such as the ones used on the latest Intel CPU's (hopefully well-tested secure RNGs will become a main feature of CPU's).
Within many programming environments there is very little choice but to use OpenSSL or /dev/random for seeding. In general it is hard to find useful information about the random number generator. Sometimes the RNG is really not suitable at all (e.g. the native PHP version).
If possible, try to find something that conforms to NIST requirements.
Related
I am doing a lot of Metropolis-Hastings Markov chain Monte Carlo (MCMC).
Most codes I have in use, use Mersenne Twister (MT) as pseudo random number generator (PRNG).
However, I recently read, that MT is outdated and probably shouldn't be used anymore as it fails some tests and is relatively slow. So I am willing to switch.
Numpy now defaults to PCG (https://www.pcg-random.org/), which claims to be good. Other sites are rather critical. E.g. http://pcg.di.unimi.it/pcg.php.
It seems everyone praises its own work.
There is some good information already here: Pseudo-random number generator
But many answers are already a bit dated and I want to formulate my question a bit more specific.
As I said: the main use case is Metropolis-Hastings MCMC.
Therefore, I need:
uniformly distributed numbers in half-open and open intervals
around 2^50 samples, apparently per rule of thumb the PRNG should have a period of at least 2^128
sufficient quality of random numbers (whatever this might mean)
a reasonable fast PRNG (for a fixed runtime faster code means more accuracy for MCMC)
I do not need
cryptographically security
As I am by no means an expert, of course usability counts also. So I would welcome an available C++ implementation (this seems to be standard), which is sufficiently easy to use for the novice.
Every time I call RAND_bytes and RAND_pseudo_bytes, with the same seed, it returns different random numbers and I don't understand why. It said that the PRNG automatically seeds itself from /dev/urandom in Linux, but how does it work in Windows?
Why does the same seed lead to different random numbers?
Why does the same seed lead to different random numbers?
You can read about the general design of the rand subsystem at Random Numbers on the OpenSSL wiki. The reason the same seed produces different random numbers is...
It depends on the generator. If you are using the default generator, then you are using md_rand. If you look at the source code for md_rand.c, then you will see rand_bytes adds entropy at each invocation with system calls to functions like time.
On Linux rand_bytes also adds the result of getpid; and on Windows it adds the result of GetSystemTime and SystemTimeToFileTime.
Adding entropy at each invocation is a good design practice for RNGs. Also see When Good Randomness Goes Bad: Virtual Machine Reset Vulnerabilities
and Hedging Deployed Cryptography and When Virtual is Harder than Real: Security Challenges in Virtual Machine Based Computing Environments.
I'm asked to design a true random generator using VHDL.With lot of struggle I could only design a PRNGs not TRNG. Is it possible to generate number perfectly random??? Please suggest me in this. I'm really clueless!
There is NO such thing as a "true" random number generator. This is one of my favorite pseudo-random generators however, and would be fun to implement in VHDL.
http://en.wikipedia.org/wiki/Xorshift
Also, see this: http://en.wikipedia.org/wiki/Random_number_generation#.22True.22_random_numbers_vs._pseudorandom_numbers
The only thing that I can think of to get you "better" randomness would be to do something like write a file and then read a file. The scheduler on the host PC might have enough entropy associated with it to cause some variance in the time it takes for these operations and you could use that time as a key to seed your algorithm.
Since you are asking about VHDL, you want to design special-purpose hardware. Now if you operate hardware in a way which should never be done for digital logic, you might get some kind of "true" random behavior.
If, e.g., you design a circuit with a D-type flip-flop that is clocked when its data input changes its level, the output becomes metastable, i.e. is some time undefined (between 0 and 1), before it becomes stable as 0 or 1 again. How long this takes, depends among others on the electric noise, e.g. is random. I could imagine that you can use such effects to make a random generator.
Contrary to the claims of most of the other answers, there are several TRNG designs for FPGAs mostly based on ring oscillators or self-timed rings, see e.g.
B. Yang, "True Random Number Generators for FPGAs," PhD thesis, KU Leuven, N. Mentens, and I. Verbauwhede (promotors), 2018.
and
VHDL TRNG designs
thank u all for ur replies.I'm thinking to use a register holding different values and take it each time the repetition starts. the idea is to provide different seed values so I can get random values. Since I'm new to VHDL coding, Im not sure if this works but just a try from my side if I can do like this. Any suggestions are welcomed on this.
You're not going to get a true random number generator out of an FPGA / VHDL. The best you can hope for is a h/w PRNG that's readable from some register somewhere.
You might choose to implement one of the PRNG algorithms out there. You're then going to have to trust the algorithm designer and then trust which ever VHDL implementation you go with (your own or one you acquire off someone else). You might start by looking at:
http://en.wikipedia.org/wiki/List_of_pseudorandom_number_generators#Cryptographic_algorithms
I want to produce a random cryptographic key on Windows. Where can I obtain entropy?
I would like my entropy function to work without a network connection and to be reliable on Windows 2000 and upwards. Even sources which may or may not provide a small amount of entropy could be useful as all the sources will be pooled.
This is my initial list of functions:
GetCurrentProcessID,
GetCurrentThreadID,
GetTickCount,
GetLocalTime,
QueryPerformanceCounter,
GlobalMemoryStatus,
GetDiskFreeSpace,
GetComputerName,
GetUserName,
GetCursorPos,
GetMessageTime,
GetSystemInfo,
CryptGenRandom,
GetProcessHandleCount,
GetProcessMemoryInfo.
Although early versions of the CryptGenRandom function may contain weaknesses later versions follow secure standards (see remarks on the CrypGenRandom page.)
It is weak to just use time as your seed. There is an answer under What is the most secure seed for random number generation? which explains that the unpredictable random seed may only need 128 bits to produce a secure PRNG. It is therefore probably unnecessary to find more sources than those listed in the question, and normally the CryptGenRandom function will already contain and generate enough entropy for itself that the caller does not need to do any of this.
CryptGenRandom and the function CryptAcquireContext which must preceed it can be called from Delphi like this.
If its an option you can ask user to move mouse pointer for a while.
The only external source that most machines have is Mic In/Line In, call waveInOpen+waveInPrepareHeader+waveInAddBuffer+waveInStart. How random that is probably depends on the hardware...
Let us assume we're generating very large (e.g. 128 or 256bit) numbers to serve as keys for a block cipher.
Let us further assume that we wear tinfoil hats (at least when outside).
Being so paranoid, we want to be sure of our available entropy, but we don't entirely trust any particular source. Maybe the government is rigging our coins. Maybe these dice are ever so subtly weighted. What if the hardware interrupts feeding into /dev/random are just a little too consistent? (Besides being paranoid, we're lazy enough that we don't want to generate it all by hand...)
So, let's mix them all up.
What are the secure method(s) for doing this? Presumably just concatenating a few bytes from each source isn't entirely secure -- if one of the sources is biased, it might, in theory, lend itself to such things as a related-key attack, for example.
Is running SHA-256 over the concatenated bytes sufficient?
(And yes, at some point soon I am going to pick up a copy of Cryptography Engineering. :))
Since you mention /dev/random -- on Linux at least, /dev/random is fed by an algorithm that does very much what you're describing. It takes several variously-trusted entropy sources and mixes them into an "entropy pool" using a polynomial function -- for each new byte of entropy that comes in, it's xor'd into the pool, and then the entire pool is stirred with the mixing function. When it's desired to get some randomness out of the pool, the entire pool is hashed with SHA-1 to get the output, then the pool is mixed again (and actually there's some more hashing, folding, and mutilating going on to make sure that reversing the process is about as hard as reversing SHA-1). At the same time, there's a bunch of accounting going on -- each time some entropy is added to the pool, an estimate of the number of bits of entropy it's worth is added to the account, and each time some bytes are extracted from the pool, that number is subtracted, and the random device will block (waiting on more external entropy) if the account would go below zero. Of course, if you use the "urandom" device, the blocking doesn't happen and the pool simply keeps getting hashed and mixed to produce more bytes, which turns it into a PRNG instead of an RNG.
Anyway... it's actually pretty interesting and pretty well commented -- you might want to study it. drivers/char/random.c in the linux-2.6 tree.
Using a hash function is a good approach - just make sure you underestimate the amount of entropy each source contributes, so that if you are right about one or more of them being less than totally random, you haven't weakened your key unduly.
This isn't dissimilar to the approach used in key stretching (though you have no need for multiple iterations here).
I've done this before, and my approach was just to XOR them, byte-by-byte, against each other.
Running them through some other algorithm, like SHA-256, is terribly inefficient, so it's not practical, and I think it would be not really useful and possibly harmful.
If you do happen to be incredibly paranoid, and have a tiny bit of money, it might be fun to buy a "true" (depending on how convinced you are by Quantum Mechanics) a Quantum Random Number Generator.
-- Edit:
FWIW, I think the method I describe above (or something similar) is effectively a One-Time Pad from the point of view of either sources, assuming one of them is random, and therefore unattackable assuming they are independant and out to get you. I'm happy to be corrected on this if someone takes issue with it, and I encourage anyone not taking issue with it to question it anyway, and find out for yourself.
If you have a source of randomness but you're not sure whether it is biased or not, then there are a lot of different algorithms. Depending on how much work you want to do, the entropy you waste from the original source differes.
The easiest algorithm is the (improved) van Neumann algorithm. You can find the details in this pdf:
http://security1.win.tue.nl/~bskoric/physsec/files/PhysSec_LectureNotes.pdf
at page 27.
I also recommend you to read this document if you're interested in how to produce uniformly randomness from a given souce, how true random number generators work, etc!