I have a small golang program that calculates the ith fibonnaci number, however it appears to overflow for some numbers large numbers, even when the array is changed to a type of int64. Why is this happening?
package main
import "fmt"
func main() {
fib(555) //prints a negative number
}
func fib(num int) {
queue := []int{0, 1}
for i := 0; i < num; i++ {
next := queue[0] + queue[1]
queue[0] = queue[1]
queue[1] = next
}
fmt.Println(queue[len(queue)-1])
}
The Fibonacci sequence gets very large, very fast. You need to use the math/big package in order to calculate integers this large. Translating your algorithm gives us:
queue := []*big.Int{big.NewInt(0), big.NewInt(1)}
for i := 0; i < num; i++ {
next := new(big.Int).Add(queue[0], queue[1])
queue[0] = queue[1]
queue[1] = next
}
or more concisely:
for i := 0; i < num; i++ {
queue[0].Add(queue[0], queue[1])
queue[0], queue[1] = queue[1], queue[0]
}
https://play.golang.org/p/udIITdDPfrY
Which will output the following number with 555 as the input:
70411399558423479787498867358975911087747238266614004739546108921832817803452035228895708644544982403856194431208467
(this is off by 1 from the expected 555th Fibonacci number, since it's 0 indexed)
Because the 555th fibonacci number is
43516638122555047989641805373140394725407202037260729735885664398655775748034950972577909265605502785297675867877570
which is much too large even for an int64.
Related
I am new to go and now evaluate a demo function about slice with Fibonacci sequence
package main
import "fmt"
func fbn(n int) []uint64 {
fbnSlice := make([]uint64, n)
fbnSlice[0] = 1
fbnSlice[1] = 1
for i := 2; i < n; i++ {
fbnSlice[i] = fbnSlice[i-1] + fbnSlice[i-2]
}
return fbnSlice
}
func main() {
fnbSlice := fbn(5)
fmt.Println(fnbSlice)
}
It will print "[1 1 2 3 5]"
My doubt is how the slice add it's len to 5，not the 5th num, thanks!
make([]uint64, n)
Will make a slice of length n, filled with zeros. Hence, fbn(5) will produce a slice of length 5.
I have a slice of float64 containing some values and a float value epsilon, what I would like to do is:
assuming that the slice got already sorted I want to go through the slice of float64 and check that every value of the sequence is bigger than the next one of at least value epsilon.
If it’s not bigger than the value epsilon than we will append on a slice of slices a new slice containing all the numbers read and the next numbers will be put in a new slice until the same condition happens or we finish going through the slice.
INPUT:
Epsilon : 0,001
Slice of floats64: [0,4351 0,455 0,4356 0,4359 0,4362]
DESIRED OUTPUT:
Returned slices: [ 0,4351 0,4355 ] [ 0,4356 0,4359 0,4362 ]
This is how I've tried to implement this:
for i := 0; i < len(sliceFloat); i++ {
for j := i + 1; j < len(sliceFloat); j++ {
if sliceFloat[i] - sliceFloat[j] <= epsilon {
sliceOfSlices = append(sliceOfSlices, sliceFloat[i:j])
} else {
continue
}
}
}
return sliceOfSlices
This is the output that I get:
[[0.4351] [0.4351 0.4355] [0.4351 0.4355 0.4356] [0.4351 0.4355 0.4356 0.4359] [0.4355] [0.4355 0.4356] [0.4355 0.4356 0.4359] [0.4356] [0.4356 0.4359] [0.4359]]
What am I doing wrong and how can I fix this?
The test input you posted is clearly wrong: GIGO: Garbage in, garbage out.
Epsilon : 0,001
Slice of floats64: [0,4351 0,455 0,4356 0,4359 0,4362]
Your code does not attempt to fully implement the specification.
else {
continue
}
After fixing all the bugs:
package main
import "fmt"
func crescents(s []float64, epsilon float64) [][]float64 {
var ss [][]float64
for i, f := range s {
if i == 0 || f <= s[i-1]+epsilon {
ss = append(ss, []float64(nil))
}
ss[len(ss)-1] = append(ss[len(ss)-1], f)
}
return ss
}
func main() {
s := []float64{0.4351, 0.4355, 0.4356, 0.4359, 0.4362}
epsilon := 0.0001
ss := crescents(s, epsilon)
fmt.Println(s, epsilon)
fmt.Println(ss)
}
https://go.dev/play/p/h-SxeIWPuu-
[0.4351 0.4355 0.4356 0.4359 0.4362] 0.0001
[[0.4351 0.4355] [0.4356 0.4359 0.4362]]
I just read some short tutorials of Go and wrote a simple program sieve. Sieve uses sieve algorithm to print all the prime number that is smaller than 10000, which create a lot of go routines. I got the correct results but the program is very slow (5 seconds on my machine). I also wrote lua script and python script which implemented the same algorithm, and runs a lot faster (both are about 1 second on my machine).
Note that the purpose is to have idea of go routine's performance compared with coroutine in other languages, for example lua. The implementation is very inefficient, some comments pointed out that it's not correct way to implement Sieve of Eratosthenes. Yes, that's intentional. Some other replies pointed out that slowness is caused by print I/O. So I commented out print lines.
My question is why my sieve program implemented in Go is so slow?
Here is the code:
package main
import (
"fmt"
"sync"
)
type Sieve struct {
id int;
msg_queue chan int;
wg *sync.WaitGroup;
}
func NewSieve(id int) *Sieve {
sieve := new(Sieve)
sieve.id = id
sieve.msg_queue = make(chan int)
sieve.wg = new(sync.WaitGroup)
sieve.wg.Add(1)
return sieve
}
func (sieve *Sieve) run() {
defer sieve.wg.Done()
myprime := <-sieve.msg_queue
if myprime == 0 {
return
}
// fmt.Printf("Sieve (%d) is for prime number %d.\n", sieve.id, myprime)
next_sieve := NewSieve(sieve.id + 1)
go next_sieve.run()
for {
number := <-sieve.msg_queue
if number == 0 {
next_sieve.msg_queue <- number;
next_sieve.wg.Wait()
return
} else if number % myprime != 0 {
// fmt.Printf("id: %d, number: %d, myprime: %d, number mod myprime: %d\n", sieve.id, number, myprime, number % myprime)
next_sieve.msg_queue <- number
}
}
}
func driver() {
first := NewSieve(2)
go first.run()
for n := 2; n <= 10000; n++ {
first.msg_queue <- n
}
first.msg_queue <- 0
first.wg.Wait()
}
func main() {
driver()
}
As a comparison, here is the code of sieve.lua
function sieve(id)
local myprime = coroutine.yield()
// print(string.format("Sieve (%d) is for prime number %d", id, myprime))
local next_sieve = coroutine.create(sieve)
coroutine.resume(next_sieve, id + 1)
while true do
local number = coroutine.yield()
if number % myprime ~= 0 then
// print(string.format("id: %d, number: %d, myprime: %d, number mod myprime: %d", id, number, myprime, number % myprime))
coroutine.resume(next_sieve, number)
end
end
end
function driver()
local first = coroutine.create(sieve)
coroutine.resume(first, 2)
local n
for n = 2, 10000 do
coroutine.resume(first, n)
end
end
driver()
Meaningless microbenchmarks produce meaningless results.
You are timing print I/O.
You are incurring go routine and channel overhead for a small amount of work.
Here is a prime number sieve program in Go.
Output:
$ go version
go version devel +46be01f4e0 Sun Oct 13 01:48:30 2019 +0000 linux/amd64
$ go build sumprimes.go && time ./sumprimes
5736396
29.96µs
real 0m0.001s
user 0m0.001s
sys 0m0.000s
sumprimes.go:
package main
import (
"fmt"
"time"
)
const (
prime = 0x00
notprime = 0xFF
)
func oddPrimes(n uint64) (sieve []uint8) {
sieve = make([]uint8, (n+1)/2)
sieve[0] = notprime
p := uint64(3)
for i := p * p; i <= n; i = p * p {
for j := i; j <= n; j += 2 * p {
sieve[j/2] = notprime
}
for p += 2; sieve[p/2] == notprime; p += 2 {
}
}
return sieve
}
func sumPrimes(n uint64) uint64 {
sum := uint64(0)
if n >= 2 {
sum += 2
}
for i, p := range oddPrimes(n) {
if p == prime {
sum += 2*uint64(i) + 1
}
}
return sum
}
func main() {
start := time.Now()
var n uint64 = 10000
sum := sumPrimes(n)
fmt.Println(sum)
fmt.Println(time.Since(start))
}
Most of the time is spent in fmt.Printf.
Taking out the line:
fmt.Printf("id: %d, number: %d, myprime: %d, number mod myprime: %d\n", sieve.id, number, myprime, number%myprime)
reduces runtime from ~5.4 seconds to ~0.64 seconds on one test I ran.
Taking out the unnecessary sync.WaitGroups reduces the time a bit further, to ~0.48 seconds. See the version without sync.WaitGroup here. You're still doing a lot of channel operations, which languages with yield-value-from-coroutine operators do not need (though they have their own issues instead). This is not a good way to implement primality testing.
How can I generate a stream of unique random number in Go?
I want to guarantee there are no duplicate values in array a using math/rand and/or standard Go library utilities.
func RandomNumberGenerator() *rand.Rand {
s1 := rand.NewSource(time.Now().UnixNano())
r1 := rand.New(s1)
return r1
}
rng := RandomNumberGenerator()
N := 10000
for i := 0; i < N; i++ {
a[i] = rng.Int()
}
There are questions and solutions on how to generate a series of random number in Go, for example, here.
But I would like to generate a series of random numbers that does not duplicate previous values. Is there a standard/recommended way to achieve this in Go?
My guess is to (1) use permutation or to (2) keep track of previously generated numbers and regenerate a value if it's been generated before.
But solution (1) sounds like overkill if I only want a few number and (2) sounds very time consuming if I end up generating a long series of random numbers due to collision, and I guess it's also very memory-consuming.
Use Case: To benchmark a Go program with 10K, 100K, 1M pseudo-random number that has no duplicates.
You should absolutely go with approach 2. Let's assume you're running on a 64-bit machine, and thus generating 63-bit integers (64 bits, but rand.Int never returns negative numbers). Even if you generate 4 billion numbers, there's still only a 1 in 4 billion chance that any given number will be a duplicate. Thus, you'll almost never have to regenerate, and almost never never have to regenerate twice.
Try, for example:
type UniqueRand struct {
generated map[int]bool
}
func (u *UniqueRand) Int() int {
for {
i := rand.Int()
if !u.generated[i] {
u.generated[i] = true
return i
}
}
}
I had similar task to pick elements from initial slice by random uniq index. So from slice with 10k elements get 1k random uniq elements.
Here is simple head on solution:
import (
"time"
"math/rand"
)
func getRandomElements(array []string) []string {
result := make([]string, 0)
existingIndexes := make(map[int]struct{}, 0)
randomElementsCount := 1000
for i := 0; i < randomElementsCount; i++ {
randomIndex := randomIndex(len(array), existingIndexes)
result = append(result, array[randomIndex])
}
return result
}
func randomIndex(size int, existingIndexes map[int]struct{}) int {
rand.Seed(time.Now().UnixNano())
for {
randomIndex := rand.Intn(size)
_, exists := existingIndexes[randomIndex]
if !exists {
existingIndexes[randomIndex] = struct{}{}
return randomIndex
}
}
}
I see two reasons for wanting this. You want to test a random number generator, or you want unique random numbers.
You're Testing A Random Number Generator
My first question is why? There's plenty of solid random number generators available. Don't write your own, it's basically dabbling in cryptography and that's never a good idea. Maybe you're testing a system that uses a random number generator to generate random output?
There's a problem: there's no guarantee random numbers are unique. They're random. There's always a possibility of collision. Testing that random output is unique is incorrect.
Instead, you want to test the results are distributed evenly. To do this I'll reference another answer about how to test a random number generator.
You Want Unique Random Numbers
From a practical perspective you don't need guaranteed uniqueness, but to make collisions so unlikely that it's not a concern. This is what UUIDs are for. They're 128 bit Universally Unique IDentifiers. There's a number of ways to generate them for particular scenarios.
UUIDv4 is basically just a 122 bit random number which has some ungodly small chance of a collision. Let's approximate it.
n = how many random numbers you'll generate
M = size of the keyspace (2^122 for a 122 bit random number)
P = probability of collision
P = n^2/2M
Solving for n...
n = sqrt(2MP)
Setting P to something absurd like 1e-12 (one in a trillion), we find you can generate about 3.2 trillion UUIDv4s with a 1 in a trillion chance of collision. You're 1000 times more likely to win the lottery than have a collision in 3.2 trillion UUIDv4s. I think that's acceptable.
Here's a UUIDv4 library in Go to use and a demonstration of generating 1 million unique random 128 bit values.
package main
import (
"fmt"
"github.com/frankenbeanies/uuid4"
)
func main() {
for i := 0; i <= 1000000; i++ {
uuid := uuid4.New().Bytes()
// use the uuid
}
}
you can generate a unique random number with len(12) using UnixNano in golang time package :
uniqueNumber:=time.Now().UnixNano()/(1<<22)
println(uniqueNumber)
it's always random :D
1- Fast positive and negative int32 unique pseudo random numbers in 296ms using std lib:
package main
import (
"fmt"
"math/rand"
"time"
)
func main() {
const n = 1000000
rand.Seed(time.Now().UTC().UnixNano())
duplicate := 0
mp := make(map[int32]struct{}, n)
var r int32
t := time.Now()
for i := 0; i < n; {
r = rand.Int31()
if i&1 == 0 {
r = -r
}
if _, ok := mp[r]; ok {
duplicate++
} else {
mp[r] = zero
i++
}
}
fmt.Println(time.Since(t))
fmt.Println("len: ", len(mp))
fmt.Println("duplicate: ", duplicate)
positive := 0
for k := range mp {
if k > 0 {
positive++
}
}
fmt.Println(`n=`, n, `positive=`, positive)
}
var zero = struct{}{}
output:
296.0169ms
len: 1000000
duplicate: 118
n= 1000000 positive= 500000
2- Just fill the map[int32]struct{}:
for i := int32(0); i < n; i++ {
m[i] = zero
}
When reading it is not in order in Go:
for k := range m {
fmt.Print(k, " ")
}
And this just takes 183ms for 1000000 unique numbers, no duplicate (The Go Playground):
package main
import (
"fmt"
"time"
)
func main() {
const n = 1000000
m := make(map[int32]struct{}, n)
t := time.Now()
for i := int32(0); i < n; i++ {
m[i] = zero
}
fmt.Println(time.Since(t))
fmt.Println("len: ", len(m))
// for k := range m {
// fmt.Print(k, " ")
// }
}
var zero = struct{}{}
3- Here is the simple but slow (this takes 22s for 200000 unique numbers), so you may generate and save it to a file once:
package main
import "time"
import "fmt"
import "math/rand"
func main() {
dup := 0
t := time.Now()
const n = 200000
rand.Seed(time.Now().UTC().UnixNano())
var a [n]int32
var exist bool
for i := 0; i < n; {
r := rand.Int31()
exist = false
for j := 0; j < i; j++ {
if a[j] == r {
dup++
fmt.Println(dup)
exist = true
break
}
}
if !exist {
a[i] = r
i++
}
}
fmt.Println(time.Since(t))
}
Temporary workaround based on #joshlf's answer
type UniqueRand struct {
generated map[int]bool //keeps track of
rng *rand.Rand //underlying random number generator
scope int //scope of number to be generated
}
//Generating unique rand less than N
//If N is less or equal to 0, the scope will be unlimited
//If N is greater than 0, it will generate (-scope, +scope)
//If no more unique number can be generated, it will return -1 forwards
func NewUniqueRand(N int) *UniqueRand{
s1 := rand.NewSource(time.Now().UnixNano())
r1 := rand.New(s1)
return &UniqueRand{
generated: map[int]bool{},
rng: r1,
scope: N,
}
}
func (u *UniqueRand) Int() int {
if u.scope > 0 && len(u.generated) >= u.scope {
return -1
}
for {
var i int
if u.scope > 0 {
i = u.rng.Int() % u.scope
}else{
i = u.rng.Int()
}
if !u.generated[i] {
u.generated[i] = true
return i
}
}
}
Client side code
func TestSetGet2(t *testing.T) {
const N = 10000
for _, mask := range []int{0, -1, 0x555555, 0xaaaaaa, 0x333333, 0xcccccc, 0x314159} {
rng := NewUniqueRand(2*N)
a := make([]int, N)
for i := 0; i < N; i++ {
a[i] = (rng.Int() ^ mask) << 1
}
//Benchmark Code
}
}
The problem is: find the index of two numbers that nums[index1] + nums[index2] == target. Here is my attempt in golang (index starts from 1):
package main
import (
"fmt"
)
var nums = []int{0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 25182, 25184, 25186, 25188, 25190, 25192, 25194, 25196} // The number list is too long, I put the whole numbers in a gist: https://gist.github.com/nickleeh/8eedb39e008da8b47864
var target int = 16021
func twoSum(nums []int, target int) (int, int) {
if len(nums) <= 1 {
return 0, 0
}
hdict := make(map[int]int)
for i := 1; i < len(nums); i++ {
if val, ok := hdict[nums[i+1]]; ok {
return val, i + 1
} else {
hdict[target-nums[i+1]] = i + 1
}
}
return 0, 0
}
func main() {
fmt.Println(twoSum(nums, target))
}
The nums list is too long, I put it into a gist:
https://gist.github.com/nickleeh/8eedb39e008da8b47864
This code works fine, but I find the return 0,0 part is ugly, and it runs ten times slower than the Julia translation. I would like to know is there any part that is written terrible and affect the performance?
Edit:
Julia's translation:
function two_sum(nums, target)
if length(nums) <= 1
return false
end
hdict = Dict()
for i in 1:length(nums)
if haskey(hdict, nums[i])
return [hdict[nums[i]], i]
else
hdict[target - nums[i]] = i
end
end
end
In my opinion if no elements found adding up to target, best would be to return values which are invalid indices, e.g. -1. Although returning 0, 0 would be enough as a valid index pair can't be 2 equal indices, this is more convenient (because if you forget to check the return values and you attempt to use the invalid indices, you will immediately get a run-time panic, alerting you not to forget checking the validity of the return values). As so, in my solutions I will get rid of that i + 1 shifts as it makes no sense.
Benchmarking of different solutions can be found at the end of the answer.
If sorting allowed:
If the slice is big and not changing, and you have to call this twoSum() function many times, the most efficient solution would be to sort the numbers simply using sort.Ints() in advance:
sort.Ints(nums)
And then you don't have to build a map, you can use binary search implemented in sort.SearchInts():
func twoSumSorted(nums []int, target int) (int, int) {
for i, v := range nums {
v2 := target - v
if j := sort.SearchInts(nums, v2); v2 == nums[j] {
return i, j
}
}
return -1, -1
}
Note: Note that after sorting, the indices returned will be indices of values in the sorted slice. This may differ from indices in the original (unsorted) slice (which may or may not be a problem). If you do need indices from the original order (original, unsorted slice), you may store sorted and unsorted index mapping so you can get what the original index is. For details see this question:
Get the indices of the array after sorting in golang
If sorting is not allowed:
Here is your solution getting rid of that i + 1 shifts as it makes no sense. Slice and array indices are zero based in all languages. Also utilizing for ... range:
func twoSum(nums []int, target int) (int, int) {
if len(nums) <= 1 {
return -1, -1
}
m := make(map[int]int)
for i, v := range nums {
if j, ok := m[v]; ok {
return j, i
}
m[target-v] = i
}
return -1, -1
}
If the nums slice is big and the solution is not found fast (meaning the i index grows big) that means a lot of elements will be added to the map. Maps start with small capacity, and they are internally grown if additional space is required to host many elements (key-value pairs). An internal growing requires rehashing and rebuilding with the already added elements. This is "very" expensive.
It does not seem significant but it really is. Since you know the max elements that will end up in the map (worst case is len(nums)), you can create a map with a big-enough capacity to hold all elements for the worst case. The gain will be that no internal growing and rehashing will be required. You can provide the initial capacity as the second argument to make() when creating the map. This speeds up twoSum2() big time if nums is big:
func twoSum2(nums []int, target int) (int, int) {
if len(nums) <= 1 {
return -1, -1
}
m := make(map[int]int, len(nums))
for i, v := range nums {
if j, ok := m[v]; ok {
return j, i
}
m[target-v] = i
}
return -1, -1
}
Benchmarking
Here's a little benchmarking code to test execution speed of the 3 solutions with the input nums and target you provided. Note that in order to test twoSumSorted(), you first have to sort the nums slice.
Save this into a file named xx_test.go and run it with go test -bench .:
package main
import (
"sort"
"testing"
)
func BenchmarkTwoSum(b *testing.B) {
for i := 0; i < b.N; i++ {
twoSum(nums, target)
}
}
func BenchmarkTwoSum2(b *testing.B) {
for i := 0; i < b.N; i++ {
twoSum2(nums, target)
}
}
func BenchmarkTwoSumSorted(b *testing.B) {
sort.Ints(nums)
b.ResetTimer()
for i := 0; i < b.N; i++ {
twoSumSorted(nums, target)
}
}
Output:
BenchmarkTwoSum-4 1000 1405542 ns/op
BenchmarkTwoSum2-4 2000 722661 ns/op
BenchmarkTwoSumSorted-4 10000000 133 ns/op
As you can see, making a map with big enough capacity speeds up: it runs twice as fast.
And as mentioned, if nums can be sorted in advance, that is ~10,000 times faster!
If nums is always sorted, you can do a binary search to see if the complement to whichever number you're on is also in the slice.
func binary(haystack []int, needle, startsAt int) int {
pivot := len(haystack) / 2
switch {
case haystack[pivot] == needle:
return pivot + startsAt
case len(haystack) <= 1:
return -1
case needle > haystack[pivot]:
return binary(haystack[pivot+1:], needle, startsAt+pivot+1)
case needle < haystack[pivot]:
return binary(haystack[:pivot], needle, startsAt)
}
return -1 // code can never fall off here, but the compiler complains
// if you don't have any returns out of conditionals.
}
func twoSum(nums []int, target int) (int, int) {
for i, num := range nums {
adjusted := target - num
if j := binary(nums, adjusted, 0); j != -1 {
return i, j
}
}
return 0, 0
}
playground example
Or you can use sort.SearchInts which implements binary searching.
func twoSum(nums []int, target int) (int, int) {
for i, num := range nums {
adjusted := target - num
if j := sort.SearchInts(nums, adjusted); nums[j] == adjusted {
// sort.SearchInts returns the index where the searched number
// would be if it was there. If it's not, then nums[j] != adjusted.
return i, j
}
}
return 0, 0
}