For security, I need to use 256 bits input as the rand seed, but it seems there is no satisfied API or functions, for example, in Golang std library, the seed should be int64, or 64 bits integer.
// Seed uses the provided seed value to initialize the default Source to a
// deterministic state. If Seed is not called, the generator behaves as
// if seeded by Seed(1). Seed values that have the same remainder when
// divided by 2³¹-1 generate the same pseudo-random sequence.
// Seed, unlike the Rand.Seed method, is safe for concurrent use.
func Seed(seed int64) { globalRand.Seed(seed) }
above code is just an inappropriate example, you can ignore it,
crypto/rand is much safer, as we can see in the std library doc.
Is there a way to use 256 bits rand seeds in Golang?
THANKS A LOT !!!
Is there a way to use 256 bits rand seeds in [math/rand] Go[]?
No, for obvious reasons.
Related
Is there a way to generate random bytes for crypto in Go deterministically, from a high-entropy seed?
I found crypto/rand, which is safe for crypto but not deterministic.
I found math/rand, which can be initialized with a seed, but is not safe for crypto.
I found x/crypto/chacha20, and I was wondering if it would be safe to use XORKeyStream with a src value of 1s. The seed would be the key and nonce, which could be generated with crypto/rand.
Edit
As an example of what I'm after, cryptonite, which is the main Haskell crypto library, has a function drgNewSeed that you can use to make a random generator from a seed.
Yes, XORKeyStream would be fine for this, and is a good design for a CSPRNG. The entire point of a stream cipher is that it generates a stream of "effectively random" values given a seed (the key and the IV). Those stream values are then XORed with the plaintext. "Effectively random" in this context means that there is no "efficient algorithm" (one that runs in polynomial time) that can distinguish this sequence from a "truly random" sequence. And that's what you want.
There's no need to pull in ChaCha20, though. You can use a built-in cipher like AES. Any block cipher can be converted into a stream cipher using one of several modes, such as CTR, OFB, or CFB. The differences between these modes don't really matter for this problem.
// Defining some seed, split across a "key" and an "iv"
key, _ := hex.DecodeString("6368616e676520746869732070617373")
iv, _ := hex.DecodeString("0123456789abcdef0123456789abcdef")
// We can turn a block cipher into a stream cipher, and AES is handy
block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}
// Convert block cipher into a stream cipher using a streaming mode like CTR
// OFB or CFB would work, too
stream := cipher.NewCTR(block, iv)
for x := 0; x < 10; x++ {
// Create a fixed value of the size you want
value := []byte{0}
// Transform it to a random value
stream.XORKeyStream(value, value)
fmt.Printf("%d\n", value)
}
Playground
There are several other approaches you can use here. You can use a secure hash like SHA-256 to hash a counter (pick a random 128-bit number and keep incrementing it, hashing each value). Or you could hash the previous result (I have heard a little bit of controversy over whether it's possible for repeated hashes to impact the security of the hash. See https://crypto.stackexchange.com/questions/19392/any-weakness-when-performing-sha-256-repeatedly and https://crypto.stackexchange.com/questions/15481/will-repeated-rounds-of-sha-512-provide-random-numbers/15488 for more.)
You can also use a block cipher to do the same thing, by encrypting a counter or the previous output. That's pretty close to what the stream cipher modes are doing. You can just do it by hand, too.
If you want to dig into this, you might search for "csprng stream cipher" at crypto.stackexchange.com. That's a better place to ask for crypto advice, but IMO this is a programming-specific question, so did belong here.
In random number generation, first, limited randomness harvested from the computer's unpredictable elements. This physical randomness then cleaned from possible biases, like hashing, then the resulting smaller true randomness is stretched into many by using a pseudorandom number generator (PRNG).
PRNG's are deterministic which means that if the initial value (the initial true randomness - the seed) is known then the rest will be known. Keep this always secret!
We are not lost, the important design goal of the PRNGs is that the outputs should not be predictable from any other output. This is a strong requirement indicating that it should be impossible to learn the internal states only by looking at the outputs.
Go's crypto/rand uses the underlying system functionalities to get the physical randomness.
On Linux and FreeBSD, Reader uses getrandom(2) if available, /dev/urandom otherwise. On OpenBSD, Reader uses getentropy(2). On other Unix-like systems, Reader reads from /dev/urandom. On Windows systems, Reader uses the CryptGenRandom API. On Wasm, Reader uses the Web Crypto API.
Then one can use possible good Deterministic RBG like the Hash-DRGB, HMAC-DRGB, and CTR-DRGB as defined in NIST 800-90
You can use the x/crypto/chacha20 to generate the long deterministic random sequence. Keep the key and nonce fixed and secret then you will have a deterministic DRGB. It is very fast, and seekable, too.
Some low-level languages like C require the programmer to set seed (usually srand(time(0)) if the user wants a different sequence of random numbers whenever the program runs. If it is not set, the program generates the same sequence of random numbers for each run.
Some high-level languages automatically set the seed if it is not set at first.
In Julia, if I want to generate a new sequence of random numbers each time, should I call srand()?
If you call Julia's srand() without providing a seed, Julia will use system entropy for seeding (essentially using a random seed).
On startup (specifically during initialisation of the Random module), Julia calls srand() without arguments. This means the global RNG is initialised randomly.
That means there's usually no need to call srand() in your own code unless you want to make the point that your random results are not meant to be reproducible.
Julia seeds the random number generator automatically, you use srand with a known seed, in order to recreate the same pseudo random sequence deterministically (useful for testing for example), but if you want to generate a different random sequence each time, all you need is to call rand.
help?> srand
search: srand sprand sprandn isreadonly StepRange StepRangeLen ClusterManager AbstractRNG AbstractUnitRange CartesianRange
srand([rng=GLOBAL_RNG], seed) -> rng
srand([rng=GLOBAL_RNG]) -> rng
Reseed the random number generator: rng will give a reproducible sequence
of numbers if and only if a seed is provided. Some RNGs
don't accept a seed, like RandomDevice. After the call to srand, rng is
equivalent to a newly created object initialized with the
same seed.
I am using the rgsl library in Rust that wraps functions from the C GSL math libraries. I was using a random number generator function, but I am always getting the same exact value whenever I generate a new random number. I imagine that the number should vary upon each run of the function. Is there something that I am missing? Do I need to set a new random seed each time or such?
extern crate rgsl;
use rgsl::Rng;
fn main() {
rgsl::RngType::env_setup();
let t = rgsl::rng::default();
let r = Rng::new(&t).unwrap()
let val = rgsl::randist::binomial::binomial(&r, 0.01f64, 1u32);
print!("{}",val);
}
The value I keep getting is 1, which seems really high considering the probability of obtaining a 1 is 0.01.
The documentation for env_setup explains everything you need to know:
This function reads the environment variables GSL_RNG_TYPE and GSL_RNG_SEED and uses their values to set the corresponding library variables gsl_rng_default and gsl_rng_default_seed
If you don’t specify a generator for GSL_RNG_TYPE then gsl_rng_mt19937 is used as the default. The initial value of gsl_rng_default_seed is zero.
(Emphasis mine)
Like all software random number generators, this is really an algorithm that produces pseudo random numbers. The algorithm and the initial seed uniquely identify a sequence of these numbers. Since the seed is always the same, the first (and second, third, ...) number in the sequence will always be the same.
So if I want to generate a new series of random numbers, then I need to change the seed each time. However, if I use the rng to generate a set of random seeds, then I will get the same seeds each time.
That's correct.
Other languages don't seem to have this constraint, meaning that the seed can be manually set if desired, but is otherwise is random.
A classical way to do this is to seed your RNG with the current time. This produces an "acceptable" seed for many cases. You can also get access to true random data from the operating system and use that as a seed or mix it in to produce more random data.
Is there no way to do this in Rust?
This is a very different question. If you just want a random number generator in Rust, use the rand crate. This uses techniques like I described above.
You could even do something crazy like using random values from the rand crate to seed your other random number generator. I just assumed that there is some important reason you are using that crate instead of rand.
Is there a way to save the random state in the golang math/rand package?
I would like to serialize it and store it for later use, but the random state is an interface and the concrete struct underneath the interface is unexported (so json.Marshall apparently cannot be used).
As an alternative to saving the rand.Source object, I thought about just saving the underlying int64 seed value. You can set it with rand.Seed, but I don't see a way to obtain the seed value so that it can be retained for later use.
You can make your own random number source which can be marshalled.
One way would be to simply copy the code from http://golang.org/src/math/rand/rng.go and adapt it by adding code that can marshal the state in whatever way you want. That'll give you your own rand.Source that you can use in rand.New(myRandSource) to generate random numbers.
As you already noted, the math/rand package does not give you insight into the current state (seed) of the (pseudo-)random number generator. If you would need this, you would have to implement it on your own (as mentioned by Anonymous).
If you would: Ok, let's say you know that the internal state of the random number generator is equivalent to where a seed value of 1234 would be set. How is this better than if the seed is any other concrete or "random" number?
Here's a tip how to "simulate" access to the seed value of the generator:
Let's say you've already created your Rand object, you've set it up and used it (already generated some random numbers). This Rand object may as well be the default/global of the math/rand package, it doesn't have to be a distinct Rand.
You reach a point where you would like to save the "sate" of the random number generator in order so that later you can repeat the exact pseudo-random sequence from this point. This would need to get the current seed, and later when you want to repeat the sequence from this point, you would just set the stored seed to the Rand object. Problem: the seed you can't access.
But what you can do is set a new seed and then you know that the internal seed is the one you just set! So to simulate a GetSeed() (on the default Rand of the math/rand package):
func GetSeed() int64 {
seed := time.Now().UnixNano() // A new random seed (independent from state)
rand.Seed(seed)
return seed
}
To simulate a GetSeed() or any Rand (not the default one):
func GetSeed2(r rand.Rand) int64 {
seed := time.Now().UnixNano() // A new random seed (independent from state)
r.Seed(seed)
return seed
}
Preserving the "pseudo" part of the randomness
The above proposed GetSeed() uses the current time to "re-seed" the Rand object. This will alter the pseudo-random sequence based on the time it was called at. This may or may not be ok (in most of the times this is not a problem).
But if it is, we can avoid this if we use the Rand object itself to designate the new seed like this (by doing this the new seed will only depend on the current state - the current seed):
func GetSeed() int64 {
seed := rand.Int63() // A new pseudo-random seed: determined by current state/seed
rand.Seed(seed)
return seed
}
func GetSeed2(r rand.Rand) int64 {
seed := r.Int63() // A new pseudo-random seed: determined by current state/seed
r.Seed(seed)
return seed
}
Notes:
The GetSeed() functionality is designed to used occasionally, e.g. when you want to save a game. Normal usage would be to generate thousands of random numbers and call GetSeed() once only.
As Anonymous pointed out, with the current algorithm of the pseudo-random number generator, in some extreme situations (for example you call GetSeed() after each generated random number) this may result in a cycle in the generated random numbers and would return the previously generated sequence of random numbers. The length of the sequence may be around a couple of thousands. Here is an example of this where the length of the sequence is 8034: Repetition Go Playground Example.
But again: this is not a normal usage, GetSeed() is called intentionally after each random number generation; and this only applies if you use Rand itself to generate the new seed. If you re-seed your Rand object with the current time for example, repetition will not occur.
I wonder is there any cheap and effective function to generate pseudo-random numbers by their indices? With something like that implementation:
var rand = new PseudoRandom(seed); // all sequences for same seeds are equal
trace(rand.get(index1)); // get int number by index1, for example =0x12345678
trace(rand.get(index2));
...
trace(rand.get(index1)); // must return the SAME number, =0x12345678
Probably it isn't about randomness but about good (fast and close as much as possible to uniform distribution) hashing where initial seed used as salt.
You could build such random number generator out of the stream cipher Salsa20. One of the nice features of Salsa20 is that you can jump ahead to any offset very cheaply. And Salsa20 is fast, typically less than 20 cycles per byte. Since the cipher is indistinguishable from a truly random stream, uniformity should be excellent.
Since you probably don't need cryptographically secure random numbers, you could even reduce the number of rounds to something like 8 instead of the usual 20 rounds.
Another option is to just use the ideas behind Salsa20, how to mix up a state array (Bernstein calls that a hashing function), to build your own random number generator.