Go has two packages for random numbers:
crypto/rand, which provides a way to get random bytes
math/rand, which has a nice algorithm for shuffling ints
I want to use the Perm algorithm from math/rand, but provide it with high-quality random numbers.
Since the two rand packages are part of the same standard library there should be a way to combine them in a way so that crypto/rand provides a good source of random numbers that is used by math/rand.Perm to generate a permutation.
Here (and on the Playground) is the code I wrote to connect these two packages:
package main
import (
cryptoRand "crypto/rand"
"encoding/binary"
"fmt"
mathRand "math/rand"
)
type cryptoSource struct{}
func (s cryptoSource) Int63() int64 {
bytes := make([]byte, 8, 8)
cryptoRand.Read(bytes)
return int64(binary.BigEndian.Uint64(bytes) >> 1)
}
func (s cryptoSource) Seed(seed int64) {
panic("seed")
}
func main() {
rnd := mathRand.New(&cryptoSource{})
perm := rnd.Perm(52)
fmt.Println(perm)
}
This code works. Ideally I don't want to define the cryptoSource type myself but just stick together the two rand packages so that they work together. So is there a predefined version of this cryptoSource type somewhere?
That's basically what you need to do. It's not often that you need a cryptographically secure source of randomness for the common usage of math/rand, so there's no adaptor provided. You can make the implementation slightly more efficient by allocating the buffer space directly in the value, rather than allocating a new slice on every call. However in the unlikely event that reading the OS random source fails, this will need to panic to prevent returning invalid results.
type cryptoSource [8]byte
func (s *cryptoSource) Int63() int64 {
_, err := cryptoRand.Read(s[:])
if err != nil {
panic(err)
}
return int64(binary.BigEndian.Uint64(s[:]) & (1<<63 - 1))
}
Related
I am trying to convert a string to integer and then to calculate its log.
My first approach was to convert the string using strconv library, but I got an error about the length of the string to be converted.
After that, I used math/big library which worked fine. Now I am not able to apply math.Log()on the resulted big integer.
Code:
package main
import (
"fmt"
"math"
"math/big"
)
func main() {
bb := "11948904162160164791281681976941230184120142151411311314211115130161285142991119211447"
bi := big.NewInt(0)
if _, ok := bi.SetString(bb, 10); ok {
fmt.Println(math.Log(bi))
} else {
fmt.Printf("error parsing line %#v\n", bb)
}
}
Error:
cannot use bi (type *big.Int) as type float64 in argument to math.Log
There are very few situations in which you'd need a precision greater than the one provided by the standard float64 type.
But just to satisfy any "midnight crazy ideas" (or even some very in-depth scientific research!) anyone might run into, Rob Pike's implementations of some operations with big floats are probably the best you can get right now with Go. The log function can be found here.
So I'm super new to Golang and seeing as the big buzz around the language seems to be concurrency I figured a good way to get my toes wet would be to write a generalized map function. In psudo code:
map(F funtion,A array){
return([F(k) for k in A])
}
And obviously I want the calculations for each F(k) occur concurrently. For organization I have a main file with my implementation and a supporting file Mr with my required definitions.
.
├── main.go
└── Mr
└── Mr.go
Main is a simple test implementation which should convert an array of strings to an array of ints where each member of the result is the length of the corresponding string in the input array.
package main
import(
"fmt"
"./Mr"
)
func exfunc(i int, c chan int){
c<-i
}
func main(){
data := make(map[int]string)
data[1]="these"
data[2]="are"
data[3]="some"
data[4]="words"
data[5]="and a few more..."
f := func(w string)int{
return(len(w))
}
testMr := Mr.Map(f,data) // this is line 22 (matters later)
fmt.Println(testMr)
}
Which works great with my Mr.Map function given that I specify all the types explicitly.
package Mr
type result struct{
key,value int
}
func wrapper(f func(string) int,k int,v string, c chan result){
c <- result{k,f(v)}
}
func Map(f func(string) int,m map[int]string) map[int]int{
c := make(chan result)
ret := make(map[int]int)
n := 0
for k := range m{
go wrapper(f,k,m[k],c)
n++
}
for ;n>0; {
r := <-c
ret[r.key]=r.value
n--
}
return(ret)
}
However I was hoping that I would be able to generalize this mapping with the empty interface.
package Mr
type T interface{}
type result struct{
key,value T
}
func wrapper(f func(T) T,k T,v T, c chan result){
c <- result{k,f(v)}
}
func Map(f func(T) T,m map[T]T) map[T]T{
c := make(chan result)
ret := make(map[T]T)
n := 0
for k := range m{
go wrapper(f,k,m[k],c)
n++
}
for ;n>0; {
r := <-c
ret[r.key]=r.value
n--
}
return(ret)
}
Unfortunately when I run main with this generalize Mr.Map I get the following error.
# command-line-arguments
./main.go:22: cannot use f (type func(string) int) as type func(Mr.T) Mr.T in argument to Mr.Map
./main.go:22: cannot use data (type map[int]string) as type map[Mr.T]Mr.T in argument to Mr.Map
So yeah, obviously I understand what the errors are telling me but it seems wild that I would have to re-write my Map function for each possible combination of key and value types.
Is there a work around here, or is this just the nature of the beast?
No real workaround there, that's the nature of the beast.
The language was designed after struggling with C++ for some time, and the idea of the creators was simplifying all non-vital things but at the same time make key additions to make the language more expressive.
You can read a bit about their reasoning here, which I believe is quite interesting even if you don't agree with all the decisions they made:
https://commandcenter.blogspot.com.ar/2012/06/less-is-exponentially-more.html
In your example, if you wanted to, you could make your maps and functions use interface{} (which by the way is called the empty interface and not "nil" interface).
But of course you would lose compile-time type checking and would have to add casts all around.
You can also try to find an interface to express the commonalities of the types you want to use (which might not be so easy or even possible at all), and then build your mapping API around that interface.
The philosophy of Go is not compatible with generalized functions such as is the style with popular dynamic languages. To properly inform the compiler of what you are trying to do, you should express your needed map through an interface or simply by writing it for each type you are using it with.
Mapping requires allocating an array, iterating through a collection, and adding to the array some data element for each element in the collection. If you need a map for a slice of structs, as is common in the application layer, you can express this tersely in Go:
https://play.golang.org/p/pk3Tl_BdlD
Dynamic languages build a "type tree" of "generic" types that allow for terse programming, such as functions like map being called by one symbol over any possible type. This provides a ton of developer productivity because code can be written loosely to allow easy experimentation.
Go is designed for writing semi-permanent software. It performs well because it requires more information to be supplied to the compiler. Map is only about three lines of code, so the cost/benefit of developer productivity v. efficiency lands on the performance side for Go. Functions like map, reduce and filter should be written explicitly as needed.
To evaluate the language, I would encourage you to try to solve a problem with a Go program and see where that takes you.
I am looking for an elegant way to unzip a list of arguments in Go. I do not want to use a variadic function for that purpose because in my usecase when writing a function I already know the number of arguments and I want to keep that part simple. However in my usecase the parameters arrive as []interface{}.
I could not find a solution but hey maybe someone out there already knows how to do that?
package main
import (
"fmt"
)
// NON-VARIADIC greater
func greet(n1, n2 string) {
fmt.Printf("%s %s\n", n1, n2)
}
func main() {
l := []interface{}{"hello", "world"}
// works
greet(l[0].(string), l[1].(string))
// does not work: "./args.go:20: not enough arguments in call to greet"
//greet(l...)
// is there something more elegant to unzip the list?
}
You could create a "generic" caller using reflect package, although this comes with overhead and lacks type safety. Unless you have some special case situation and don't know what you want to call in the code, it would be wiser to use the snippet from your question which works, but you consider not elegant.
Example usage of reflect which could be your starting point:
package main
import (
"fmt"
"reflect"
)
func call(f interface{}, args []interface{}) {
// Convert arguments to reflect.Value
vs := make([]reflect.Value, len(args))
for n := range args {
vs[n] = reflect.ValueOf(args[n])
}
// Call it. Note it panics if f is not callable or arguments don't match
reflect.ValueOf(f).Call(vs)
}
func greet(n1, n2 string) {
fmt.Printf("%s %s\n", n1, n2)
}
func main() {
l := []interface{}{"hello", "world"}
call(greet, l)
}
// Output: hello world
https://play.golang.org/p/vbi3CChCdV
I'm not quite sure what you're trying to do. If you want a way to easily pass a slice of two empty interfaces to a function that accepts two strings, you can create a little helper:
func twoStrings(vs []interface{}) (string, string) {
return vs[0].(string), vs[1].(string)
}
Use it as
greet(twoStrings(l))
Playground: http://play.golang.org/p/R8KFwMUT_V.
But honestly, it seems like you're doing something wrong, trying to make the Go type system to do something it cannot do.
I'm trying to assign a value to a field, but my program panics with runtime error: invalid memory address or nil pointer dereference.
package main
type Node struct {
Value int
}
func (n *Node) SetValue(value int) {
n.Value = value
}
func main() {
var n *Node
n.SetValue(1)
}
This is reasonable since variable is nil.
But I've fount some Go internal structs are allowed to do this, e.g. bytes.Buffer
package main
import "bytes"
import "io"
import "os"
func main() {
var b bytes.Buffer
b.Write([]byte("Hello world"))
io.Copy(os.Stdout, &b)
}
Here is the `bytes.Buffer source code
func (b *Buffer) Write(p []byte) (n int, err error) {
b.lastRead = opInvalid
m := b.grow(len(p))
return copy(b.buf[m:], p), nil
}
Is it the thing only builtin structs can do or it's possible to accomplish this in my code?
EDIT
Here is the working example. Thanks #twotwotwo for suggestion.
package main
import "fmt"
type Node struct {
Value int
}
func (n *Node) SetValue(value int) {
n.Value = value
}
func main() {
var n Node
n.SetValue(1)
fmt.Println(n.Value)
}
The crucial thing is var b bytes.Buffer doesn't get you a nil pointer, it gets you a bytes.Buffer object with all its fields initialized with their zero values (in machine terms, with zero bytes). The spec says the zero value is "false for booleans, 0 for integers, 0.0 for floats, "" for strings, and nil for pointers, functions, interfaces, slices, channels, and maps"; follow that link for more detail.
It is possible to make your own structs whose zero values work and the Go team encourages it. struct Position { x, y int } is an easy example and Effective Go gives a more realistic one. But note that that doesn't make the nil pointer work; you would still need new(Node) or var n Node to allocate the zero Node. Same for bytes.Buffer.
Another common use of zero values: wherever your users create structs of your type directly (as folks do with, say, http.Server), the zero value is the default for any fields they don't specify. It's the default in a lot of other places: what you get for a not-found map key, if you receive from a closed channel, and probably others.
When writing a Go library that needs to use random numbers, what is the best way to initialize and consume random numbers?
I know that the std way to do this in an application is:
import (
"math/rand"
"time"
)
// do the initial seeding in an init fn
func init() {
// set the global seed and use the global fns
rand.Seed(time.Now().UTC().UnixNano())
}
func main() {
fmt.Println(rand.Int())
fmt.Println(rand.Intn(200))
}
So when I'm writing library code (not in the main package), should I just do the same:
package libfoo
func init() {
rand.Seed(time.Now().UTC().UnixNano())
}
func AwesomeFoo() {
r := rand.Intn(1000)
// ...
}
The application using my library might also do its own random number seeding and use rand.Intn, so my question really is - is there any downside to having a library seed the random number generator and some app code (or another library) do so as well?
Also is there any issue with the library using the "global" rand.Intn or rand.Int or should a library create it's own private Rand object via rand.New(src) and use that instead?
I don't have any particular reason for thinking this is unsafe, but I know enough about crypto and PRNGs to know that it is easy to get something wrong if you don't know what you're doing.
For example, here's a simple library for the Knuth (Fisher-Yates) shuffle that needs randomness: https://gist.github.com/quux00/8258425
What's best really just depends on the type of application you're writing and the type of library you want to create. If we're not sure, we can get the most flexibility by using a form of dependency injection through Go interfaces.
Consider the following naive Monte Carlo integrator that takes advantage of the rand.Source interface:
package monte
import (
"math/rand"
)
const (
DEFAULT_STEPS = 100000
)
type Naive struct {
rand *rand.Rand
steps int
}
func NewNaive(source rand.Source) *Naive {
return &Naive{rand.New(source), DEFAULT_STEPS}
}
func (m *Naive) SetSteps(steps int) {
m.steps = steps
}
func (m *Naive) Integrate1D(fn func(float64) float64, a, b float64) float64 {
var sum float64
for i := 0; i < m.steps; i++ {
x := (b-a) * m.rand.Float64()
sum += fn(x)
}
return (b-a)*sum/float64(m.steps)
}
We can then use this package to calculate the value of pi:
func main() {
m := monte.NewNaive(rand.NewSource(200))
pi := 4*m.Integrate1D(func (t float64) float64 {
return math.Sqrt(1-t*t)
}, 0, 1)
fmt.Println(pi)
}
In this case, the quality of our algorithm's results depend on the type of pseudorandom number generator used, so we need to provide a way for users to swap out one generator for another. Here we've defined an opaque type that takes a random number source in its constructor. By having their random number generator satisfy the rand.Source interface, our application writer can then swap out random number generators as needed.
However, there are many cases where this is exactly what we don't want to do. Consider a random password or key generator. In that case, what we really want is a high entropy source of truly random data, so we should just use the crypto/rand package internally and hide the details from our application writers:
package keygen
import (
"crypto/rand"
"encoding/base32"
)
func GenKey() (string, error) {
b := make([]byte, 20)
if _, err := rand.Read(b); err != nil {
return "", err
}
enc := base32.NewEncoding("ABCDEFGHIJKLMNOPQRSTUVWXYZ346789")
return enc.EncodeToString(b), nil
}
Hopefully that helps you make a decision. If the code is for your own applications or applications within a specific company rather than industry wide or public use, lean towards the library design that exposes the fewest internals and creates the fewest dependencies rather than the most general design, since that will ease maintenance and shorten implementation time.
Basically, if it feels like overkill, it probably is.
In the case of the Knuth Shuffle, the requirements are simply a decent psuedo-random number generator, so you could simply use an internally seeded rand.Rand object that's private to your package like so:
package shuffle
import (
"math/rand"
"time"
)
var r *rand.Rand
func init() {
r = rand.New(rand.NewSource(time.Now().UTC().UnixNano()))
}
func ShuffleStrings(arr []string) {
last := len(arr)-1
for i := range arr {
j := r.Intn(last)
arr[i], arr[j] = arr[j], arr[i]
}
}
Then the application doesn't have to worry about how it works:
package main
import (
"shuffle"
"fmt"
)
func main() {
arr := []string{"a","set","of","words"}
fmt.Printf("Shuffling words: %v\n", arr)
for i := 0; i<10; i++ {
shuffle.ShuffleStrings(arr)
fmt.Printf("Shuffled words: %v\n", arr)
}
}
This prevents the application from accidentally reseeding the random number generator used by your package by calling rand.Seed.
Don't seed the global random number generator. That should be left to package main.
If you care what your seed is, you should create your own private Rand object. If you don't care, you can just use the global source.
If you care about your numbers actually being random, you should use crypto/rand instead of math/rand.