When writing a Go library that needs to use random numbers, what is the best way to initialize and consume random numbers?
I know that the std way to do this in an application is:
import (
"math/rand"
"time"
)
// do the initial seeding in an init fn
func init() {
// set the global seed and use the global fns
rand.Seed(time.Now().UTC().UnixNano())
}
func main() {
fmt.Println(rand.Int())
fmt.Println(rand.Intn(200))
}
So when I'm writing library code (not in the main package), should I just do the same:
package libfoo
func init() {
rand.Seed(time.Now().UTC().UnixNano())
}
func AwesomeFoo() {
r := rand.Intn(1000)
// ...
}
The application using my library might also do its own random number seeding and use rand.Intn, so my question really is - is there any downside to having a library seed the random number generator and some app code (or another library) do so as well?
Also is there any issue with the library using the "global" rand.Intn or rand.Int or should a library create it's own private Rand object via rand.New(src) and use that instead?
I don't have any particular reason for thinking this is unsafe, but I know enough about crypto and PRNGs to know that it is easy to get something wrong if you don't know what you're doing.
For example, here's a simple library for the Knuth (Fisher-Yates) shuffle that needs randomness: https://gist.github.com/quux00/8258425
What's best really just depends on the type of application you're writing and the type of library you want to create. If we're not sure, we can get the most flexibility by using a form of dependency injection through Go interfaces.
Consider the following naive Monte Carlo integrator that takes advantage of the rand.Source interface:
package monte
import (
"math/rand"
)
const (
DEFAULT_STEPS = 100000
)
type Naive struct {
rand *rand.Rand
steps int
}
func NewNaive(source rand.Source) *Naive {
return &Naive{rand.New(source), DEFAULT_STEPS}
}
func (m *Naive) SetSteps(steps int) {
m.steps = steps
}
func (m *Naive) Integrate1D(fn func(float64) float64, a, b float64) float64 {
var sum float64
for i := 0; i < m.steps; i++ {
x := (b-a) * m.rand.Float64()
sum += fn(x)
}
return (b-a)*sum/float64(m.steps)
}
We can then use this package to calculate the value of pi:
func main() {
m := monte.NewNaive(rand.NewSource(200))
pi := 4*m.Integrate1D(func (t float64) float64 {
return math.Sqrt(1-t*t)
}, 0, 1)
fmt.Println(pi)
}
In this case, the quality of our algorithm's results depend on the type of pseudorandom number generator used, so we need to provide a way for users to swap out one generator for another. Here we've defined an opaque type that takes a random number source in its constructor. By having their random number generator satisfy the rand.Source interface, our application writer can then swap out random number generators as needed.
However, there are many cases where this is exactly what we don't want to do. Consider a random password or key generator. In that case, what we really want is a high entropy source of truly random data, so we should just use the crypto/rand package internally and hide the details from our application writers:
package keygen
import (
"crypto/rand"
"encoding/base32"
)
func GenKey() (string, error) {
b := make([]byte, 20)
if _, err := rand.Read(b); err != nil {
return "", err
}
enc := base32.NewEncoding("ABCDEFGHIJKLMNOPQRSTUVWXYZ346789")
return enc.EncodeToString(b), nil
}
Hopefully that helps you make a decision. If the code is for your own applications or applications within a specific company rather than industry wide or public use, lean towards the library design that exposes the fewest internals and creates the fewest dependencies rather than the most general design, since that will ease maintenance and shorten implementation time.
Basically, if it feels like overkill, it probably is.
In the case of the Knuth Shuffle, the requirements are simply a decent psuedo-random number generator, so you could simply use an internally seeded rand.Rand object that's private to your package like so:
package shuffle
import (
"math/rand"
"time"
)
var r *rand.Rand
func init() {
r = rand.New(rand.NewSource(time.Now().UTC().UnixNano()))
}
func ShuffleStrings(arr []string) {
last := len(arr)-1
for i := range arr {
j := r.Intn(last)
arr[i], arr[j] = arr[j], arr[i]
}
}
Then the application doesn't have to worry about how it works:
package main
import (
"shuffle"
"fmt"
)
func main() {
arr := []string{"a","set","of","words"}
fmt.Printf("Shuffling words: %v\n", arr)
for i := 0; i<10; i++ {
shuffle.ShuffleStrings(arr)
fmt.Printf("Shuffled words: %v\n", arr)
}
}
This prevents the application from accidentally reseeding the random number generator used by your package by calling rand.Seed.
Don't seed the global random number generator. That should be left to package main.
If you care what your seed is, you should create your own private Rand object. If you don't care, you can just use the global source.
If you care about your numbers actually being random, you should use crypto/rand instead of math/rand.
Related
This question already has answers here:
Go rand.Intn same number/value
(3 answers)
Closed 2 years ago.
I am trying to write a function that generates a random sequence with an alphanumeric character, Unfortunately, the function returns the same random sequence when calling multiple times.
I even tried by seeding the rand with time.Now().UTC().UnixNano(), even though getting the same values again and again
Main Package:
package main
import (
"fmt"
"time"
"userpkg/random"
)
func main() {
fmt.Println(random.RandomHash(32))
fmt.Println(random.RandomHash(32))
fmt.Println(random.RandomHash(32))
fmt.Println(random.RandomHash(32))
}
Random Package
package random
func RandomHash(length int8) string {
rand.Seed(time.Now().UTC().UnixNano())
pool := []byte(`0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ`)
/* allocate a new slice array to store the hash */
buf := make([]byte, length)
for i := int8(0); i < length; i++ {
buf[i] = pool[rand.Intn(len(pool))]
}
rand.Shuffle(len(buf), func(i, j int) {
buf[i], buf[j] = buf[j], buf[i]
})
str := string(buf)
return str
}
Output :
Aau9hmA3YpDezPMIFUtgSUoQfwi7KuWK
Aau9hmA3YpDezPMIFUtgSUoQfwi7KuWK
Aau9hmA3YpDezPMIFUtgSUoQfwi7KuWK
Aau9hmA3YpDezPMIFUtgSUoQfwi7KuWK
Please guide me on how to solve this issue, Thanks
You need to seed the math/rand package once only. If you call the RandomHash() function "very fast", you will seed it to the same value, so it will use the same random values, resulting in the same result! On top of this, on the Go Playground the time is deterministic (it doesn't elapse unless e.g. time.Sleep() is called!).
Move the seeding outside of RandomHash(), e.g. to a package init() function:
func init() {
rand.Seed(time.Now().UnixNano())
}
func RandomHash(length int8) string {
// ...
}
Then each return value of RandomHash() will (likely) be different, e.g. (try it on the Go Playground):
Aau9hmA3YpDezPMIFUtgSUoQfwi7KuWK
8XhJlp6EAXqqbEcPLQL83pw8wUiJRl7D
HGWpHldhGWpzl2KY10ua15T04N1eoPp7
huRNzf4eD7IIuqYNjoMZB5z6r0RFRB64
Also see related question:
How to generate a random string of a fixed length in Go?
I am a beginner in Golang.
I have a problem with variable type assigning from user input.
When the user enters data like "2012BV352" I need to be able to ignore the BV and pass 2012352 to my next function.
There has a package name gopkg.in/validator.v2 in doc
But what it returns is whether or not the variable is safe or not.
I need to cut off the unusual things.
Any idea on how to achieve this?
You could write your own sanitizing methods and if it becomes something you'll be using more often, I'd package it out and add other methods to cover more use cases.
I provide two different ways to achieve the same result. One is commented out.
I haven't run any benchmarks so i couldn't tell you for certain which is more performant, but you could write your own tests if you wanted to figure it out. It would also expose another important aspect of Go and in my opinion one of it's more powerful tools... testing.
package main
import (
"fmt"
"log"
"regexp"
"strconv"
"strings"
)
// using a regex here which simply targets all digits and ignores everything else. I make it a global var and use MustCompile because the
// regex doesn't need to be created every time.
var extractInts = regexp.MustCompile(`\d+`)
func SanitizeStringToInt(input string) (int, error) {
m := extractInts.FindAllString(input, -1)
s := strings.Join(m, "")
return strconv.Atoi(s)
}
/*
// if you didn't want to use regex you could use a for loop
func SanitizeStringToInt(input string) (int, error) {
var s string
for _, r := range input {
if !unicode.IsLetter(r) {
s += string(r)
}
}
return strconv.Atoi(s)
}
*/
func main() {
a := "2012BV352"
n, err := SanitizeStringToInt(a)
if err != nil {
log.Fatal(err)
}
fmt.Println(n)
}
Is there a way to obtain the total number of finalizers registered using runtime.SetFinalizer and which have not yet run?
We are considering adding a struct with a registered finalizer to some of our products to release memory allocated using malloc, and the object could potentially have a relatively high allocation rate. It would be nice if we could monitor the number of finalizers, to make sure that they do not pile up and trigger out-of-memory errors (like they tend to with other garbage collectors).
(I'm aware that explicit deallocation would avoid this problem, but we cannot change the existing code, which does not call a Close function or something like that.)
You can keep keep a count of these objects by incrementing and decrementing a unexported package variable when a new object is created and finalized, respectively.
For example:
package main
import (
"fmt"
"runtime"
"sync/atomic"
)
var totalObjects int32
func TotalObjects() int32 {
return atomic.LoadInt32(&totalObjects)
}
type Object struct {
p uintptr // C allocated pointer
}
func NewObject() *Object {
o := &Object{
}
// TODO: perform other initializations
atomic.AddInt32(&totalObjects, 1)
runtime.SetFinalizer(o, (*Object).finalizer)
return o
}
func (o *Object) finalizer() {
atomic.AddInt32(&totalObjects, -1)
// TODO: perform finalizations
}
func main() {
fmt.Println("Total objects:", TotalObjects())
for i := 0; i < 100; i++ {
_ = NewObject()
runtime.GC()
}
fmt.Println("Total objects:", TotalObjects())
}
https://play.golang.org/p/n35QABBIcj
It's possible to make a wrapper on runtime.SetFinalizer which does the counting for you. Of course, it's a question of using it everywhere where you use SetFinalizer.
In case this is problematic, you can also modify SetFinalizer source code directly, but that requires a modified Go compiler.
Atomic integers are used as SetFinalizer may be called on different threads, and otherwise a counter may not be accurate as without those a race condition could possibly occur. Golang guarantees that finalizers are called from a single goroutine, so it's not needed for inner function.
https://play.golang.org/p/KKCH2UwTFYw
package main
import (
"fmt"
"reflect"
"runtime"
"sync/atomic"
)
var finalizersCreated int64
var finalizersRan int64
func SetFinalizer(obj interface{}, finalizer interface{}) {
finType := reflect.TypeOf(finalizer)
funcType := reflect.FuncOf([]reflect.Type{finType.In(0)}, nil, false)
f := reflect.MakeFunc(funcType, func(args []reflect.Value) []reflect.Value {
finalizersRan++
return reflect.ValueOf(finalizer).Call([]reflect.Value{args[0]})
})
runtime.SetFinalizer(obj, f.Interface())
atomic.AddInt64(&finalizersCreated, 1)
}
func main() {
v := "a"
SetFinalizer(&v, func(a *string) {
fmt.Println("Finalizer ran")
})
fmt.Println(finalizersRan, finalizersCreated)
runtime.GC()
fmt.Println(finalizersRan, finalizersCreated)
}
I'm new to Go and not sure why it prints the same number for rand.Intn(n int) int for every run:
package main
import (
"fmt"
"math/rand"
)
func main() {
fmt.Println(rand.Intn(10))
}
The docs says :
Intn returns, as an int, a non-negative pseudo-random number in [0,n) from the default Source. It panics if n <= 0.
And how do I properly seed the random number generation?
By calling the rand.Seed() function, passing it a (random) seed (typically the current unix timestamp). Quoting from math/rand package doc:
Top-level functions, such as Float64 and Int, use a default shared Source that produces a deterministic sequence of values each time a program is run. Use the Seed function to initialize the default Source if different behavior is required for each run.
Example:
rand.Seed(time.Now().UnixNano())
If rand.Seed() is not called, the generator behaves as if seeded by 1:
Seed uses the provided seed value to initialize the default Source to a deterministic state. If Seed is not called, the generator behaves as if seeded by Seed(1).
package main
import
(
"fmt"
"math/rand"
"time"
)
func randomGen(min, max int) int {
rand.Seed(time.Now().Unix())
return rand.Intn(max - min) + min
}
func main() {
randNum := randomGen(1, 10)
fmt.Println(randNum)
}
Under the package, math/rand, you can find a type Rand.
func New(src Source) *Rand - New returns a new Rand that uses random
values from src to generate other random values.
It actually needs a seed to generate it.
step 1: create a seed as a source using new Source.
time.Now().UnixNano() is used for the accuracy.
step 2: create a
type Rand from the seed
step 3: generate a random number.
Example:
package main
import (
"fmt"
"math/rand"
"time"
)
func main() {
source := rand.NewSource(time.Now().UnixNano())
r := rand.New(source)
fmt.Println(r.Intn(100))
}
Go has two packages for random numbers:
crypto/rand, which provides a way to get random bytes
math/rand, which has a nice algorithm for shuffling ints
I want to use the Perm algorithm from math/rand, but provide it with high-quality random numbers.
Since the two rand packages are part of the same standard library there should be a way to combine them in a way so that crypto/rand provides a good source of random numbers that is used by math/rand.Perm to generate a permutation.
Here (and on the Playground) is the code I wrote to connect these two packages:
package main
import (
cryptoRand "crypto/rand"
"encoding/binary"
"fmt"
mathRand "math/rand"
)
type cryptoSource struct{}
func (s cryptoSource) Int63() int64 {
bytes := make([]byte, 8, 8)
cryptoRand.Read(bytes)
return int64(binary.BigEndian.Uint64(bytes) >> 1)
}
func (s cryptoSource) Seed(seed int64) {
panic("seed")
}
func main() {
rnd := mathRand.New(&cryptoSource{})
perm := rnd.Perm(52)
fmt.Println(perm)
}
This code works. Ideally I don't want to define the cryptoSource type myself but just stick together the two rand packages so that they work together. So is there a predefined version of this cryptoSource type somewhere?
That's basically what you need to do. It's not often that you need a cryptographically secure source of randomness for the common usage of math/rand, so there's no adaptor provided. You can make the implementation slightly more efficient by allocating the buffer space directly in the value, rather than allocating a new slice on every call. However in the unlikely event that reading the OS random source fails, this will need to panic to prevent returning invalid results.
type cryptoSource [8]byte
func (s *cryptoSource) Int63() int64 {
_, err := cryptoRand.Read(s[:])
if err != nil {
panic(err)
}
return int64(binary.BigEndian.Uint64(s[:]) & (1<<63 - 1))
}