How to generate a fixed length random number in Go? - random

What is the fastest and simplest way to generate fixed length random numbers in Go?
Say to generate 8-digits long numbers, the problem with rand.Intn(100000000) is that the result might be far less than 8-digits, and padding it with leading zeros doesn't look like a good answer to me.
I.e., I care about the the quality of the randomness more in the sense of its length. So I'm thinking, for this specific problem, would the following be the fastest and simplest way to do it?
99999999 - rand.Int63n(90000000)
I.e., I guess Int63n might be better for my case than Intn. Is it ture, or it is only a wishful thinking? Regarding randomness of the full 8-digits, would the two be the same, or there is really one better than the other?
Finally, any better way than above?
UPDATE:
Please do not provide low + rand(hi-low) as the answer, as everyone knows that. It is equivalent of what I'm doing now, and it doesn't answer my real question, "Regarding randomness of the full 8-digits, would the two be the same, or there is really one better than the other? "
If nobody can answer that, I'll plot a 2-D scatter plot between the two and find out myself...
Thanks

This is a general purpose function for generating numbers within a range
func rangeIn(low, hi int) int {
return low + rand.Intn(hi-low)
}
See it on the Playground
In your specific case, trying to generate 8 digit numbers, the range would be (10000000, 99999999)

It depend on value range you want to use.
If you allow value range [0-99999999] and padding zero ip number of char < 8, then use fmt like fmt.Sprintf("%08d",rand.Intn(100000000)).
If you dont want padding, which value in range [10000000, 99999999], then give it a base like ranNumber := 10000000 + rand.Intn(90000000)`

See it on Playground
crypto/rand package is used to generate number.
func generateRandomNumber(numberOfDigits int) (int, error) {
maxLimit := int64(int(math.Pow10(numberOfDigits)) - 1)
lowLimit := int(math.Pow10(numberOfDigits - 1))
randomNumber, err := rand.Int(rand.Reader, big.NewInt(maxLimit))
if err != nil {
return 0, err
}
randomNumberInt := int(randomNumber.Int64())
// Handling integers between 0, 10^(n-1) .. for n=4, handling cases between (0, 999)
if randomNumberInt <= lowLimit {
randomNumberInt += lowLimit
}
// Never likely to occur, kust for safe side.
if randomNumberInt > int(maxLimit) {
randomNumberInt = int(maxLimit)
}
return randomNumberInt, nil
}

I recently needed to do something like this, but with a certain byte length (rather than number of digits) and with numbers larger than max int64 (so using math/big.Int). Here was my general solution:
See on the Playground (with added code comments)
func generateRandomBigInt(numBytes int) (*big.Int, error) {
value := make([]byte, numBytes)
_, err := rand.Reader.Read(value)
if err != nil {
return nil, err
}
for true {
if value[0] != 0 {
break
}
firstByte := value[:1]
_, err := rand.Reader.Read(firstByte)
if err != nil {
return nil, err
}
}
return (&big.Int{}).SetBytes(value), nil
}

Related

Using Go deterministicly generate RSA Private Key with custom io.Reader

For reasons probably best left unanswered I need to generate infinite RSA public/private keys. Note this is not being used for anything highly secure, so please don't tell me not to do this, yes I know its not ideal. By "infinite" I mean I need a unknown number of them think billions to trillions, and creating them before being used is not possible.
Since this would consume infinite space and take infinite time to generate I need to do it at runtime.
However I also need to have for a given input the same key pair. This means I need to deterministically recreate the RSA key given the input.
I am using Go and normally you create keys using the following,
k, err := rsa.GenerateKey(rand.Reader, 2048)
Of course the catch is that rand.Reader is supplied by crypto/rand and as such there is no way to seed this.
I thought that it would be possible to provide my own reader implementation to achieve my goal. I looked through the source of GenerateKey and noted that it is looking for prime numbers, so I implemented my own reader, such that I could control the "random" primes returned, allowing me to generate the same key when required,
type Reader struct {
data []byte
sum int
primes []int
}
func NewReader(toRead string) *Reader {
primes := sieveOfEratosthenes(10_000_000)
return &Reader{[]byte(toRead), 0, primes}
}
func (r *Reader) Read(p []byte) (n int, err error) {
r.sum = r.sum + 1
if r.sum >= 100_000 {
return r.primes[rand.Intn(len(r.primes))], io.EOF
}
return r.primes[rand.Intn(len(r.primes))], nil
}
func sieveOfEratosthenes(N int) (primes []int) {
b := make([]bool, N)
for i := 2; i < N; i++ {
if b[i] == true {
continue
}
primes = append(primes, i)
for k := i * i; k < N; k += i {
b[k] = true
}
}
return
}
I can then call into generate key like so
k, err := rsa.GenerateKey(NewReader(""), 2048)
Which compiles, but crashes at runtime due to nil pointers. I am fairly comfortable with Go, but the implementation of RSA for this is beyond my understanding. Looking for either a better way to achieve this, or perhaps what I need to do to get my reader working.
Note, the only hard requirement's I have here are being able to generate the same key for a given input, and use rsa.GenerateKey or a drop in compatible replacement. The input can be anything really, so long as I get the same key as the output.
Here is a Go playground link demonstrating where I am currently https://go.dev/play/p/jd1nAoPR5aD
The Read method is not doing what is expected. It does not fill the input p byte slice with random bytes. If you look at the implementation for Unix of crypto/rand.Read method, it passes the input byte slice into another reader. So basically you what you need to fill the byte slice with random numbers. For example:
func (r *Reader) Read(p []byte) (n int, err error) {
i := 0
b := p
for i < len(b) {
if len(b) < 4 {
b[0] = 7
b = b[1:]
} else {
binary.LittleEndian.PutUint32(b, uint32(rand.Intn(len(r.primes))))
b = b[4:]
}
}
return len(p), nil
}
Here is the link to playground.
UPD
As mentioned in the answer by Erwin, there is a function called MaybeReadRand that with 50% chance read 1 byte from rand reader to make this function nondeterministic. But you can get around by adding if statement in Read method: if length of the input slice is 1, then ignore everything and return. Otherwise, feed the input slice with prime numbers:
func (r *Reader) Read(p []byte) (n int, err error) {
i := 0
b := p
if len(p) == 1 {
println("maybeReadRand")
return 1, nil
}
for i < len(b) {
if len(b) < 4 {
b[0] = 7
b = b[1:]
} else {
binary.LittleEndian.PutUint32(b, uint32(r.primes[r.i]))
r.i++
b = b[4:]
}
}
return len(p), nil
}
In this snippet I am creating 2 keys, and both of them are equal.
Someone posted an answer here, and I was about to award it to them as it got me the final piece I needed to resolve the question.
In short, I didn't read how the read was being called properly and I needed to populate the input slice. Changing the read to the below,
func (r *Reader) Read(p []byte) (n int, err error) {
i := 0
b := p
for i < len(b) {
if len(b) < 4 {
b[0] = 7
b = b[1:]
} else {
binary.LittleEndian.PutUint32(b, uint32(rand.Intn(len(r.primes))))
b = b[4:]
}
}
return len(p), nil
}
resolves the crashes, and since it is now using math/rand I can set the seed for the list of primes allowing me to ensure it is deterministic. Everything solved without needing anything from 3rd parties. Excellent.
To whoever posted that, if you post again I will award you the accepted answer.
You have made a strange assumption. rand.Reader returns completely random data. It doesn't return prime numbers. rsa.GenerateKey uses the data internally to come up with -very probable- prime numbers. But it doesn't depend on any particular pattern in the reader. In fact, it prefers there to be no pattern (and "being a prime number" is definitely a pattern)
Sure, you're free to feed rsa.GenerateKeym, or excerpts from Shakespeare's works, but rsa.GenerateKey is not going to make use of the fact that it is.
So running a sieve of Eratosthenes is a waste of time (literally).
If rsa.GenerateKey allowed you to deterministically generate keys, then you could directly generate bytes using a pseudo-random number generator that uses a seed, directly.
But, that's not the case. Look at the source code, line 287 of rsa.go:
func GenerateMultiPrimeKey(random io.Reader, nprimes int, bits int) (*PrivateKey, error) {
randutil.MaybeReadByte(random) // <------- this line here is the important one
The function GenerateMultiPrimeKey is directly invoked by GenerateKey.
Now what does randutil.MaybeReadByte do? The documentation is clear:
MaybeReadByte reads a single byte from r with ~50% probability. This
is used to ensure that callers do not depend on non-guaranteed
behaviour, e.g. assuming that rsa.GenerateKey is deterministic w.r.t.
a given random stream.
In short, it ensures that it is not possible to deterministically generate a particular RSA key.
If you still needed to do that, you would need to copy the source code of GenerateMultiPrimeKey to your own library, and remove the line randutil.MaybeReadByte. Or you'd need to rethink what your requirement really is, a level higher than you're currently thinking, and see if there is a different way to achieve your requirement. (Have a look at the "XY problem")
If you need a deterministic-random io.Reader, here's how to do it with the standard library
import (
"io"
"math/rand"
)
func main() {
const seed = 1
var myRand = rand.New(rand.NewSource(seed))
}
rand.NewSource creates a deterministic seeded random source, and rand.New wraps it to create a more featureful *Rand object, which includes an implementation of io.Reader.
You can also reset the *Rand by calling myRand.Seed(seed).

How can I identify a matching pattern for a given number using go?

I'm trying to identify pattern matches for a given telephone number range to use in a Cisco Communications Manager platform.
Essentially, an 'X' matches the numbers 0-9 in a telephone number, and you can specify a range of digits using the [x-y] notation.
Given a telephone number range of 02072221000-02072221149 consisting of 150 numbers, this would create and should output two patterns: 020722210XX and 020722211[0-4]X
Obviously I'd like it to work on any range provided. I just can't seem to get my head around how to generate those patterns given the number ranges.
Any thoughts would be greatly appreciated.
Many thanks.
I believe I found a decent algorithm which should handle this for you. I apologize ahead of time if any of the explanation isn't detailed enough, but a lot of this came to intuition which can be hard to explain.
I started with more simplified cases, figuring out a method for how to get the fewest number of patterns from a comparison. For my examples I'll be comparing 211234 to 245245.
After a bit of thinking I worked out that you need to take the range of numbers from the smaller number up to 9 and handle the special case for the lowest digit in the smaller number. To explain in a bit more detail, in the number 211234 the ideal is to represent the last digit as an X but we can only do that for cases where the digit may be [0-9] the only case in this example where we can't use [0-9] is when our tens digit is 3 because we have a lower limit of 4. This logic then propagates up the rest of the number as we head toward the most significant digit. So for the tens digit in the next case we have a lower bound based on the previous example of 4 because we're handling the case when we allow a 3 specially. So for our tens range we end up with a 4-9 because the next digit over does not restrict our range.
In fact we won't be restricted until the most significant digit which is bounded by the numbers in the range between the numbers we're comparing. After working a few problems out by hand I noticed a bit of a pattern of the pyramid of Xs in the cases where the numbers digits were significantly apart:
compare: 211234
to: 245245
21123[4-9]
2112[4-9]X
211[3-9]XX
21[2-9]XXX
2[2-3]XXXX
24[0-4]XXX
245[0-1]XX
2452[0-3]X
24514[0-5]
This was my first hint as to how to handle it. Starting from the least significant moving up, taking advantage of the symmetry, but handling the case where we hit the "top of the pyramid". This example is easy though, there are many corner cases which will cause issues. For the sake of brevity I'm not going to go into detail for each but I'll give a short explanation for each:
What do you do when the 2 compared digits has one number between them, such as between 4 and 6?
In this case simply use the single digit in place of a range.
What do you do when the 2 compared digits have no number between them, such as between 4 and 5?
In this case throw away the row in which you'd handle the numbers between the digits as all cases will be handled explicitly.
What do you do when the minimum number in the range is 8?
In this case when we add 1 to the number to get a lower bound for the range we get a 9, which means we can simply substitute in a 9 rather than a range of [9-9]
What do you do when the minimum number in the range is 9?
In this case we simply don't bother handling that number as when handling the next digit up it should be covered by its use of X
I'm sure I'm missing some corner cases which I handle in the code which I simply didn't think to put in this list. I'm willing to clarify any part of the code if you just leave a comment asking.
Below is my stab at it in Go. It could probably be a bit more DRY but this is what I came up with after fiddling for a bit. I'm also pretty new to Go so please notify me of any spirit fouls in the comments and I'll correct them.
I don't guarantee this will handle every case, but it handled every case I threw at it. It's up to you to turn it into a script which takes in 2 strings ;)
Edit: I just realized via the example in the question (which for some reason I never ran) that this doesn't always condense the provided range in to the smallest number of outputs, but it should always give patterns which cover every case. Despite this drawback I think it's a good step in the right direction for you to work on top of. I'll update the answer if I find the time to get it to condense cases where the previous range is 1-9 and the special case is 0. The best means for which might end up being after the initial generation condensing these cases "manually".
package main
import (
"strconv"
"fmt"
)
func getStringFromMinAndMax(min int, max int) (string, bool){
minstr := strconv.Itoa(min)
maxstr := strconv.Itoa(max)
if max == min {
return minstr, false
}
if max < min{
return minstr, false
}
return "["+minstr+"-"+maxstr+"]", true
}
func main(){
str1 := "211234"
str2 := "245245"
diffLength := 0
for i := 0; i < len(str1); i++{
diffLength = i+1
number1, _ := strconv.Atoi(str1[:len(str1)-i-1])
number2, _ := strconv.Atoi(str2[:len(str2)-i-1])
if number1 == number2 {
break
}
}
elems := (diffLength * 2)-1
output := make([]*[]string, elems+1)
for i := 0; i < elems; i++ {
newSlice := make([]string, diffLength)
output[i] = &newSlice
}
for digit := 0; digit < diffLength; digit++ {
for j := 0; j < diffLength; j++ {
if j == digit {
if output[j] != nil {
min, _ := strconv.Atoi(string(str1[len(str1)-(digit+1)]))
max := 9
if digit == diffLength-1 {
max, _ = strconv.Atoi(string(str2[len(str1)-(digit+1)]))
max = max - 1
}
if digit != 0{
min = min+1
}
if min < 10 {
maxchar := strconv.Itoa(max)[0]
minchar := strconv.Itoa(min)[0]
newVal, safe := getStringFromMinAndMax(min, max)
if digit == diffLength-1 && !safe && (str1[len(str1)-(digit+1)] == maxchar || str2[len(str2)-(digit+1)] == minchar) {
output[j] = nil
} else {
(*output[j])[diffLength-digit-1] = newVal
}
} else {
output[j] = nil
}
}
if j != diffLength-1 && output[elems-1-j] != nil {
min := 0
max, _ := strconv.Atoi(string(str2[len(str1)-(digit+1)]))
if digit != 0{
max = max-1
}
if max >= 0{
newVal, _ := getStringFromMinAndMax(min, max)
(*output[elems-1-j])[diffLength-digit-1] = newVal
} else {
output[elems-1-j] = nil
}
}
} else {
if j > digit {
if output[j] != nil {
(*output[j])[diffLength-digit-1] = "X"
}
if j != diffLength-1 && output[elems-1-j] != nil {
(*output[elems-1-j])[diffLength-digit-1] = "X"
}
} else {
if output[j] != nil {
(*output[j])[diffLength-digit-1] = string(str1[len(str1)-digit-1])
}
if j != diffLength-1 && output[elems-1-j] != nil {
(*output[elems-1-j])[diffLength-digit-1] = string(str2[len(str2)-digit-1])
}
}
}
}
}
for _, list := range output {
if list != nil{
if len(str1) != diffLength{
fmt.Printf(str1[:len(str1)-diffLength])
}
for _, element := range *list {
fmt.Printf(element)
}
fmt.Printf("\n")
}
}
}
Footnotes:
diffLength is the number of characters on the end of the strings which differ, I couldn't think of a better way to get this number than what's in the script...
Me setting an output to nil is me saying, "This one will be handled explicitly, so throw it away"
j is a variable for which output I'm setting... But this also gets mirrored to the bottom, so I couldn't think of a concise name to give it thus I left it j.
digit is tracking which digit from the right we are modifying

How to create a big int with a secure random

I have this code in java and I need to reproduce in Go.
String nonce = new BigInteger(130, new SecureRandom()).toString(32);
Is the only way to generate a nonce for GDS amadeus soap header 4.
Thanks
Use package math/big and crypto/rand. The snippet looks like:
//Max random value, a 130-bits integer, i.e 2^130 - 1
max := new(big.Int)
max.Exp(big.NewInt(2), big.NewInt(130), nil).Sub(max, big.NewInt(1))
//Generate cryptographically strong pseudo-random between 0 - max
n, err := rand.Int(rand.Reader, max)
if err != nil {
//error handling
}
//String representation of n in base 32
nonce := n.Text(32)
A working example can be found at The Go Playground.
The accepted answer is wrong because the crypto/rand rand.Int function
returns a uniform random value in [0, max). It panics if max <= 0.
Here is an answer that doesn't skip the 2^130 - 1 value.
// Max value, a 130-bits integer, i.e 2^130 - 1
var max *big.Int = big.NewInt(0).Exp(big.NewInt(2), big.NewInt(130), nil)
// Generate cryptographically strong pseudo-random between [0, max)
n, err := rand.Int(rand.Reader, max)
if err != nil {
// error handling
}
fmt.Println(n)

Working with bitstrings and big.Int in Go

I'm new to Go and I'm working on a few exercises to get up to speed. How can I convert a string representing a sequence of bits to the appropriate datatype in Go?
For eg, I see that if its a bitstring representing a 64-bit number, I can do :-
val, err := strconv.ParseInt(bitstring, 2, 64)
However, if the bitstring represents a larger number(say 1024 or 2048 bits), how can I go about converting that number to the appropriate type in Go? I believe the type for managing big integers in Go is big.Int.
Yes, you may use the big.Int type, and its Int.SetString() method, passing 2 as the base.
Example:
i := big.NewInt(0)
if _, ok := i.SetString("10101010101010101010101010101010101010101010101010101010101010101010101010", 2); !ok {
fmt.Println("Invalid number!")
} else {
fmt.Println(i)
}
Output (try it on the Go playground):
12592977287652387236522

How to compare images with go?

In go image package, i don't see any methods which can be used to compare two images? Is it possible to do image comparison in go similar to ImageMagick?
If you are trying to compare two images and just need to boil it down to a single number, the following will work. This is useful in (for example) genetic algorithms, where you want to compare a set of candidates and choose the one that differs the least from a reference image:
Visit every pixel, break it down into its parts: R, G, B, & A (in go: image.At(x,y).RGBA())
Subtract the RGBA vals from their corresponding pixel vals in the reference image.
Square the differences, add them up.
Take the square root of the total sum.
This number will give you a rough idea of how much the images differ.
If you know that the two images are both instances of image.RGBA (or you can convert them), then you can do something even faster: just grab the bytes directly from RGBA.Pix. That's what I do here, and it's roughly 10x faster than doing img.At(x,y).RGBA() for every pixel pair:
func FastCompare(img1, img2 *image.RGBA) (int64, error) {
if img1.Bounds() != img2.Bounds() {
return 0, fmt.Errorf("image bounds not equal: %+v, %+v", img1.Bounds(), img2.Bounds())
}
accumError := int64(0)
for i := 0; i < len(img1.Pix); i++ {
accumError += int64(sqDiffUInt8(img1.Pix[i], img2.Pix[i]))
}
return int64(math.Sqrt(float64(accumError))), nil
}
func sqDiffUInt8(x, y uint8) uint64 {
d := uint64(x) - uint64(y)
return d * d
}
Try https://github.com/vitali-fedulov/images3. I wrote
this package to be able to find near duplicates. There is a live web-demo with the same algorithm, so you can get an idea how well the package suites your needs.
Inspired by George's answer.
The function below is not so fast, but it allows you to visually assess the difference in images.
func ImgCompare(img1, img2 image.Image) (int64, image.Image, error) {
bounds1 := img1.Bounds()
bounds2 := img2.Bounds()
if bounds1 != bounds2 {
return math.MaxInt64, nil, fmt.Errorf("image bounds not equal: %+v, %+v", img1.Bounds(), img2.Bounds())
}
accumError := int64(0)
resultImg := image.NewRGBA(image.Rect(
bounds1.Min.X,
bounds1.Min.Y,
bounds1.Max.X,
bounds1.Max.Y,
))
draw.Draw(resultImg, resultImg.Bounds(), img1, image.Point{0, 0}, draw.Src)
for x := bounds1.Min.X; x < bounds1.Max.X; x++ {
for y := bounds1.Min.Y; y < bounds1.Max.Y; y++ {
r1, g1, b1, a1 := img1.At(x, y).RGBA()
r2, g2, b2, a2 := img2.At(x, y).RGBA()
diff := int64(sqDiffUInt32(r1, r2))
diff += int64(sqDiffUInt32(g1, g2))
diff += int64(sqDiffUInt32(b1, b2))
diff += int64(sqDiffUInt32(a1, a2))
if diff > 0 {
accumError += diff
resultImg.Set(
bounds1.Min.X+x,
bounds1.Min.Y+y,
color.RGBA{R: 255, A: 255})
}
}
}
return int64(math.Sqrt(float64(accumError))), resultImg, nil
}
func sqDiffUInt32(x, y uint32) uint64 {
d := uint64(x) - uint64(y)
return d * d
}
With two of the current answers here, the images need to be the same size, or the comparison fails. A third answer here uses vitali-fedulov/images, which doesn't have any method to get the
difference between two images, only a Similar function that returns a bool determining if two images are similar. Further, the answer at Rosetta Code also fails if the images are different sizes.
So if I was to implement my own solution, first I would need to scale down the larger image. I found x/image/draw and nfnt/resize for that purpose, but I thought maybe I could find something, to kill two birds with one stone. To that end, I did find some packages that scale the images as needed, take a hash of each, and get the difference of the hashes. Here is corona10/goimagehash:
package main
import (
"github.com/corona10/goimagehash"
"image/jpeg"
"os"
)
func hash(name string) (*goimagehash.ImageHash, error) {
f, err := os.Open(name)
if err != nil {
return nil, err
}
defer f.Close()
i, err := jpeg.Decode(f)
if err != nil {
return nil, err
}
return goimagehash.AverageHash(i)
}
Example:
package main
func main() {
a, err := hash("mb.jpg")
if err != nil {
panic(err)
}
b, err := hash("hqdefault.jpg")
if err != nil {
panic(err)
}
d, err := a.Distance(b)
if err != nil {
panic(err)
}
println(d)
}

Resources