Is there another way of testing if a big.Int is 0? - performance

I'm working with big.Ints and need to test for 0. Right now, I'm using zero = big.NewInt(0)and Cmp(zero)==0 which works fine, but I was wondering if there's a quicker way specifically for 0 (I need this program to be very fast)?

big.Int exposes Int.Bits() to access the raw bytes of the representation, which is a slice and it shares the same underlying array: the returned slice is not copied. So it's fast. It's exposed to "support implementation of missing low-level Int functionality".
Perfect, exactly what we want.
0. Testing for 0
Documentation of big.Int also states that "the zero value for an Int represents the value 0". So in the zero value (which represents 0) the slice will be empty (zero value for slices is nil and the length of a nil slice is 0). We can simply check that:
if len(i1.Bits()) == 0 {
}
Also note that there is an Int.BitLen() function returning this, which also states that "the bit length of 0 is 0". So we may also use this:
if i1.BitLen() == 0 {
}
Let's benchmark these solutions:
func BenchmarkCompare(b *testing.B) {
zero := big.NewInt(0)
i1 := big.NewInt(1)
i2 := big.NewInt(0)
for i := 0; i < b.N; i++ {
if i1.Cmp(zero) == 0 {
}
if i2.Cmp(zero) == 0 {
}
}
}
func BenchmarkBits(b *testing.B) {
i1 := big.NewInt(1)
i2 := big.NewInt(0)
for i := 0; i < b.N; i++ {
if len(i1.Bits()) == 0 {
}
if len(i2.Bits()) == 0 {
}
}
}
func BenchmarkBitLen(b *testing.B) {
i1 := big.NewInt(1)
i2 := big.NewInt(0)
for i := 0; i < b.N; i++ {
if i1.BitLen() == 0 {
}
if i2.BitLen() == 0 {
}
}
}
Benchmark results:
BenchmarkCompare-8 76975251 13.3 ns/op
BenchmarkBits-8 1000000000 0.656 ns/op
BenchmarkBitLen-8 1000000000 1.11 ns/op
Getting the bits and comparing the slice length to 0 is 20 times faster than comparing it to another big.Int representing 0, using Int.BitLen() is also 10 times faster.
1. Testing for 1
Something similar could be made to test if a big.Int value equals to 1, but probably not as fast as testing for zero: 0 is the most special value. Its internal representation is a nil slice, any other value requires a non-nil slice. Also: 0 has another special property: its absolute value is itself.
This absolute value property is important, because Int.Bits() returns the absolute value. So in case of a non-zero value checking just the bits slice is insufficient, as it carries no sign information.
So testing for 1 can be implemented by checking if the bits content represents 1, and the sign is positive:
func isOne(i *big.Int) bool {
bits := i.Bits()
return len(bits) == 1 && bits[0] == 1 && i.Sign() > 0
}
Let's benchmark this along with comparing the number to one := big.NewInt(1):
func BenchmarkCompareOne(b *testing.B) {
one := big.NewInt(1)
i1 := big.NewInt(0)
i2 := big.NewInt(1)
i3 := big.NewInt(2)
for i := 0; i < b.N; i++ {
if i1.Cmp(one) == 0 {
}
if i2.Cmp(one) == 0 {
}
if i3.Cmp(one) == 0 {
}
}
}
func BenchmarkBitsOne(b *testing.B) {
i1 := big.NewInt(0)
i2 := big.NewInt(1)
i3 := big.NewInt(2)
for i := 0; i < b.N; i++ {
if isOne(i1) {
}
if isOne(i2) {
}
if isOne(i3) {
}
}
}
And the benchmark results:
BenchmarkCompareOne-8 58631458 18.9 ns/op
BenchmarkBitsOne-8 715606629 1.76 ns/op
Not bad! Our way of testing for 1 is again 10 times faster.

Related

If else function with identical symbols

I have this:
func main() {
n1 := new(big.Int)
n1.SetString("1000000", 10)
n2 := new(big.Int)
n2.SetString("1000009", 10)
one := big.NewInt(1)
for i := n1; i.Cmp(n2) < 0; i.Add(i, one) {
//fmt.Println(i)
}
}
How do I make when n1 := 100000 <-- got 5 identical symbols or 122222 and etc just to keep count till there is only 4 identical symbols (example 100001) ... then go to next line.
I know I need to use ... if ... else ... but I have no idea how to check if there is only 4 identical symbols in result of n1
Any idea?
I don't know if this is exactly what you mean or not, but I can suggest you create a function which accepts a *big.Int and a specific number of identical symbols, and returns weather or not there's a character that is repeated n times:
func containsNIdenticalSymbols(n int, i *big.Int) bool {
counter := make(map[rune]int)
for _, char := range i.String() {
counter[char] ++
}
hasNumIdenticalSymbols := false
for _, count := range counter {
if count == n { hasNumIdenticalSymbols = true; break }
}
return hasNumIdenticalSymbols
}
And just && it with the i.Cmp(n2) < 0 condition:
numIdenticalSymbols := 4
// as you said "keep count till there is only 4 identical symbols"
for i := n1; i.Cmp(n2) < 0 && !containsNIdenticalSymbols(numIdenticalSymbols, i) ; i.Add(i, one) { ... }

how to simplimize my go script because always get time out in hackerrank

I have a test interview as a Go Developer and have to do some of the tasks on hackerrank.
I've done the task, but when I submit my script it always "times out".. maybe because there are a lot of loops that I use to do this function, and the task is :
So, my solution are :
Loop from a to b with a increment.
Define the digit sum with modulus by 10, sum the result with the leftover.
Define the square sum with converting int(a) to string then use for-range to sum the values.
checking if digit sum and square sum is a prime number, if so then count++
My script is :
func main() {
fmt.Printf("Jadi ada %d bilangan prima \n", luckyNumbers(1, 20))
}
func luckyNumbers(a int64, b int64) int64 {
count := 0
for min, max := a, b; min <= max; min++ {
squareSum := digitSquare(min)
digitSum := digitSum(min)
if isPrime(digitSum) && isPrime(squareSum) {
count++
}
}
return int64(count)
}
func digitSquare(number int64) int64 {
numStr := strconv.Itoa(int(number))
var firstDigit, secondDigit int
for _, digit := range numStr {
numInt, _ := strconv.Atoi(string(digit))
pow := int(math.Pow(float64(numInt), 2))
if firstDigit == 0 {
firstDigit += pow
} else {
secondDigit += pow
}
}
squareSum := int64(firstDigit + secondDigit)
return squareSum
}
func digitSum(number int64) int64 {
var remainder, sumResult int64 = 0, 0
for number != 0 {
remainder = number % 10
sumResult += remainder
number /= 10
}
return sumResult
}
func isPrime(num int64) bool {
if num < 2 {
return false
}
for i := int64(2); i <= int64(math.Sqrt(float64(num))); i++ {
if num%i == 0 {
return false
}
}
return true
}
The script above is the best script that I can make right now, I understand that I do a lot of iterations, so when I try to submit it will always show "time out". So I want to learn from you and want to see if there is a simpler script so that it can be submitted.
Thank you,
Regards

Why is strconv.ParseUint so slow compared to strconv.Atoi?

I'm benchmarking unmarshaling from string to int and uint with this code:
package main
import (
"strconv"
"testing"
)
func BenchmarkUnmarshalInt(b *testing.B) {
for i := 0; i < b.N; i++ {
UnmarshalInt("123456")
}
}
func BenchmarkUnmarshalUint(b *testing.B) {
for i := 0; i < b.N; i++ {
UnmarshalUint("123456")
}
}
func UnmarshalInt(v string) int {
i, _ := strconv.Atoi(v)
return i
}
func UnmarshalUint(v string) uint {
i, _ := strconv.ParseUint(v, 10, 64)
return uint(i)
}
Result:
Running tool: C:\Go\bin\go.exe test -benchmem -run=^$ myBench/main -bench .
goos: windows
goarch: amd64
pkg: myBench/main
BenchmarkUnmarshalInt-8 99994166 11.7 ns/op 0 B/op 0 allocs/op
BenchmarkUnmarshalUint-8 54550413 21.0 ns/op 0 B/op 0 allocs/op
Is it possible that the second (uint) is almost twice as slow as the first (int)?
Yes, it's possible. strconv.Atoi has a fast path when the input string length is less than 19 (or 10 if int is 32 bit). This allows it to be a lot faster because it doesn't need to check for overflow.
If you change your test number to "1234567890123456789" (assuming 64 bit int), then your int benchmark is slightly slower than the uint benchmark because the fast path can't be used. On my machine, it takes 37.6 ns/op for the signed version vs 31.5 ns/op for the unsigned version.
Here's the modified benchmark code (note I added a variable that sums up the parsed results, just in case the compiler got clever and optimized it away).
package main
import (
"fmt"
"strconv"
"testing"
)
const X = "1234567890123456789"
func BenchmarkUnmarshalInt(b *testing.B) {
var T int
for i := 0; i < b.N; i++ {
T += UnmarshalInt(X)
}
fmt.Println(T)
}
func BenchmarkUnmarshalUint(b *testing.B) {
var T uint
for i := 0; i < b.N; i++ {
T += UnmarshalUint(X)
}
fmt.Println(T)
}
func UnmarshalInt(v string) int {
i, _ := strconv.Atoi(v)
return i
}
func UnmarshalUint(v string) uint {
i, _ := strconv.ParseUint(v, 10, 64)
return uint(i)
}
For reference, the code for strconv.Atoi in the standard library is currently as follows:
func Atoi(s string) (int, error) {
const fnAtoi = "Atoi"
sLen := len(s)
if intSize == 32 && (0 < sLen && sLen < 10) ||
intSize == 64 && (0 < sLen && sLen < 19) {
// Fast path for small integers that fit int type.
s0 := s
if s[0] == '-' || s[0] == '+' {
s = s[1:]
if len(s) < 1 {
return 0, &NumError{fnAtoi, s0, ErrSyntax}
}
}
n := 0
for _, ch := range []byte(s) {
ch -= '0'
if ch > 9 {
return 0, &NumError{fnAtoi, s0, ErrSyntax}
}
n = n*10 + int(ch)
}
if s0[0] == '-' {
n = -n
}
return n, nil
}
// Slow path for invalid, big, or underscored integers.
i64, err := ParseInt(s, 10, 0)
if nerr, ok := err.(*NumError); ok {
nerr.Func = fnAtoi
}
return int(i64), err
}

What is the correct way of using bytes.Buffer to implement Fibonacci-like string concatenation in Go?

I tested simple string concatenation in Go with “+” and bytes.Buffer (both “WriteString” and “Write(bytes)”. The result shows that “+” is much slower than the other two, which makes sense.
However, when I use the three ways to implement Fibonacci-like string concatenation (i.e. a, b, ab, bab, abbab, bababbab, abbabbababbab), “+” performs the best. The sample codes and the benchmarking results are shown as follows.
String “+”
func Fibonacci(n int) string {
FiboResult := ""
prev_result := "a"
next_result := "b"
if n == 1{
FiboResult = "a"
}else if n == 2 {
FiboResult = "b"
}else{
for i := 3; i <= n; i++ {
FiboResult = prev_result + next_result
prev_result = next_result
next_result = FiboResult
}
}
return FiboResult
}
bytes.Buffer (WriteString)
func Fibonacci(n int) bytes.Buffer {
var FiboResult bytes.Buffer
var prev_result bytes.Buffer
prev_result.WriteString("a")
var next_result bytes.Buffer
next_result.WriteString("b")
if n == 1{
FiboResult.WriteString("a")
}else if n == 2 {
FiboResult.WriteString("b")
}else{
for i := 3; i <= n; i++ {
FiboResult.Reset()
FiboResult.WriteString(prev_result.String())
FiboResult.WriteString(next_result.String())
prev_result.Reset()
prev_result.WriteString(next_result.String())
next_result.Reset()
next_result.WriteString(FiboResult.String())
}
}
return FiboResult
}
the benchmarking results
I believe it is the overhead of bytes.Buffer.String() that make this happen. But I could not figure out how to use bytes.Buffer correctly in this case. Or how could I modify my code to avoid the problem? Hints, sample codes, or explanations are all appreciated. Many thanks in advance!
In Go, use the testing package for benchmarks.
Write reasonably efficient Go functions. Don't perform unnecessary conversions. Minimize allocations and copies. And so on. Allow for non-ASCII characters, for example Chinese characters. Allow for strings of more than one character. Consider using a byte slice. For example,
func fibonacciN(n int) uint64 {
f := uint64(0)
a, b := uint64(0), uint64(1)
for i := 0; i < n; i++ {
f, a, b = a, b, a+b
if a > b {
break
}
}
return f
}
func Fibonacci(a, b string, n int) string {
if n < 0 {
n = 0
}
switch n {
case 0:
return ""
case 1:
return a
case 2:
return b
}
f := make([]byte, len(a)*int(fibonacciN(n-1))+len(b)*int(fibonacciN(n)))
ab := a + b
copy(f[len(f)-len(ab):], ab)
for i := 4; i <= n; i++ {
end := len(f) - (len(a)*int(fibonacciN(i-3)) + len(b)*int(fibonacciN(i-2)))
start := len(f) - (len(a)*int(fibonacciN(i-1)) + len(b)*int(fibonacciN(i)))
copy(f[start:end], f[end:])
}
return string(f)
}
Benchmark functions. For example, with n = 20,
$ go test fib_test.go -bench=. -benchmem
goos: linux
goarch: amd64
BenchmarkPeterSO-8 1000000 1851 ns/op 13568 B/op 2 allocs/op
BenchmarkPlus-8 500000 2493 ns/op 18832 B/op 18 allocs/op
BenchmarkBuffer-8 100000 12773 ns/op 90256 B/op 60 allocs/op
PASS
$
fib_test.go:
package main
import (
"bytes"
"testing"
)
var benchN = 20
func fibonacciN(n int) uint64 {
f := uint64(0)
a, b := uint64(0), uint64(1)
for i := 0; i < n; i++ {
f, a, b = a, b, a+b
if a > b {
break
}
}
return f
}
func FibonacciPeterSO(a, b string, n int) string {
if n < 0 {
n = 0
}
switch n {
case 0:
return ""
case 1:
return a
case 2:
return b
}
f := make([]byte, len(a)*int(fibonacciN(n-1))+len(b)*int(fibonacciN(n)))
ab := a + b
copy(f[len(f)-len(ab):], ab)
for i := 4; i <= n; i++ {
end := len(f) - (len(a)*int(fibonacciN(i-3)) + len(b)*int(fibonacciN(i-2)))
start := len(f) - (len(a)*int(fibonacciN(i-1)) + len(b)*int(fibonacciN(i)))
copy(f[start:end], f[end:])
}
return string(f)
}
func BenchmarkPeterSO(b *testing.B) {
for i := 0; i < b.N; i++ {
FibonacciPeterSO("a", "b", benchN)
}
}
func FibonacciPlus(n int) string {
FiboResult := ""
prev_result := "a"
next_result := "b"
if n == 1 {
FiboResult = "a"
} else if n == 2 {
FiboResult = "b"
} else {
for i := 3; i <= n; i++ {
FiboResult = prev_result + next_result
prev_result = next_result
next_result = FiboResult
}
}
return FiboResult
}
func BenchmarkPlus(b *testing.B) {
for i := 0; i < b.N; i++ {
FibonacciPlus(benchN)
}
}
func FibonacciBuffer(n int) bytes.Buffer {
var FiboResult bytes.Buffer
var prev_result bytes.Buffer
prev_result.WriteString("a")
var next_result bytes.Buffer
next_result.WriteString("b")
if n == 1 {
FiboResult.WriteString("a")
} else if n == 2 {
FiboResult.WriteString("b")
} else {
for i := 3; i <= n; i++ {
FiboResult.Reset()
FiboResult.WriteString(prev_result.String())
FiboResult.WriteString(next_result.String())
prev_result.Reset()
prev_result.WriteString(next_result.String())
next_result.Reset()
next_result.WriteString(FiboResult.String())
}
}
return FiboResult
}
func BenchmarkBuffer(b *testing.B) {
for i := 0; i < b.N; i++ {
FibonacciBuffer(benchN)
}
}
var testN = benchN
func TestPeterSO(t *testing.T) {
for n := 0; n <= testN; n++ {
got := FibonacciPeterSO("a", "b", n)
want := FibonacciPlus(n)
if want != got {
t.Errorf("want: %s got: %s", want, got)
}
}
}
bytes.Buffer (or the newer and faster strings.Builder) wins over simple + string concatenation if you want to append "many" values, and obtain the result once in the end, because intermediate allocations are not needed compared to using + multiple times.
And you are not using bytes.Buffer this way: you just write one string into it, and you obtain its content and you reset it. That's just a roundtrip which turns out to be an overhead.
The problem here is that generating the Fibonacci string you are seeking, that requires prepending text to the buffer, not appending to it. And bytes.Buffer only supports appending to it, so using it like this is not a good fit at all.
Generating reverse with bytes.Buffer
Note that a prepend operation is basically an append operation if you generate the reverse of a string. Which means if we first would generate the reverse of the result, we could use bytes.Buffer to perform an append when otherwise a prepend would be needed. Of course the appended string would have to also be the reverse of what otherwise would be prepended.
And of course when we're done, we have to reverse the result to get what we originally wanted.
Also note that when building result in an iterative way, the successive intermediate result is the concatenation of the previous and the one before that. So to obtain the nth result, we can simply append the substring of what we already have! This is a nice optimization.
Here's how it would look like:
func FibonacciReverseBuf(n int) string {
switch n {
case 0:
return ""
case 1:
return "a"
case 2:
return "b"
}
prev, prev2 := 1, 1
buf := bytes.NewBufferString("ba")
for i := 3; i < n; i++ {
buf.Write(buf.Bytes()[:buf.Len()-prev2])
prev2, prev = prev, prev+prev2
}
// Reverse
b := buf.Bytes()
for i, j := 0, len(b)-1; i < j; i, j = i+1, j-1 {
b[i], b[j] = b[j], b[i]
}
return string(b)
}
Generating reverse with []byte and append()
Also note that since we're only appending, we can just as easily use a []byte and use the builtin append() function:
func FibonacciReverse(n int) string {
switch n {
case 0:
return ""
case 1:
return "a"
case 2:
return "b"
}
prev, prev2 := 1, 1
b := []byte("ba")
for i := 3; i < n; i++ {
b = append(b, b[:len(b)-prev2]...)
prev2, prev = prev, prev+prev2
}
// Reverse
for i, j := 0, len(b)-1; i < j; i, j = i+1, j-1 {
b[i], b[j] = b[j], b[i]
}
return string(b)
}
Preallocating and using copy() in a single []byte
Still, using append() may cause reallocations, because we don't know how big the buffer (the result) will be. So we start with a small buffer, and append() will increase it as needed. Also append() requires slice value (slice header) assignments. And we also have to reverse the result.
A much faster solution would be to get rid of those cons.
First let's calculate how big the result will be (this is essentially calculating the Fibonacci numbers), and allocate the required byte slice in one step.
If we do so, we can do the "prepend" operations by copying parts of our buffer (which is a []byte) to specific positions. So no append(), no reallocations, no reversing.
This is how it will look like:
func Fibonacci(n int) string {
switch n {
case 0:
return ""
case 1:
return "a"
case 2:
return "b"
}
fibs := make([]int, n)
fibs[0], fibs[1] = 1, 1
for i := 2; i < n; i++ {
fibs[i] = fibs[i-1] + fibs[i-2]
}
l := fibs[n-1]
b := make([]byte, l)
b[l-2], b[l-1] = 'a', 'b'
for i := 3; i < n; i++ {
copy(b[l-fibs[i]:], b[l-fibs[i-2]:])
}
return string(b)
}
Testing the output
To test if the above functions give the result we expect them to give, we may use the following testing function:
func TestFibonacci(t *testing.T) {
cases := []struct {
n int
exp string
}{
{0, ""},
{1, "a"},
{2, "b"},
{3, "ab"},
{4, "bab"},
{5, "abbab"},
{6, "bababbab"},
{7, "abbabbababbab"},
}
funcs := []struct {
name string
f func(int) string
}{
{"FibonacciReverseBuf", FibonacciReverseBuf},
{"FibonacciReverse", FibonacciReverse},
{"Fibonacci", Fibonacci},
}
for _, c := range cases {
for _, f := range funcs {
if got := f.f(c.n); got != c.exp {
t.Errorf("%s: Expected: %s, got: %s, n: %d",
f.name, c.exp, got, c.n)
}
}
}
}
Benchmark results
Benchmarking with n = 20:
BenchmarkFibonacciReverseBuf-4 200000 10739 ns/op 18024 B/op 10 allocs/op
BenchmarkFibonacciReverse-4 100000 13208 ns/op 28864 B/op 10 allocs/op
BenchmarkFibonacci-4 500000 3383 ns/op 13728 B/op 3 allocs/op
BenchmarkPeterSO-4 300000 4417 ns/op 13568 B/op 2 allocs/op
BenchmarkPlus-4 200000 6072 ns/op 18832 B/op 18 allocs/op
BenchmarkBuffer-4 50000 29608 ns/op 90256 B/op 60 allocs/op
We can see that this use of bytes.Buffer was much better than yours. Still, using concatenation was faster, because there aren't many concatenations here, they are small ones, and that doesn't require reversing in the end.
On the other hand my Fibonacci() solution outperformed all other presented solutions.

Why does Go map vs slice performance have 10x speed difference here

I just solved problem 23 on Project Euler, but I noticed a big difference between map[int]bool, and []bool in terms of performance.
I have a function that sums up the proper divisors of a number:
func divisorsSum(n int) int {
sum := 1
for i := 2; i*i <= n; i++ {
if n%i == 0 {
sum += i
if n/i != i {
sum += n / i
}
}
}
return sum
}
And then in main I do like this:
func main() {
start := time.Now()
defer func() {
elapsed := time.Since(start)
fmt.Printf("%s\n", elapsed)
}()
n := 28123
abundant := []int{}
for i := 12; i <= n; i++ {
if divisorsSum(i) > i {
abundant = append(abundant, i)
}
}
sums := map[int]bool{}
for i := 0; i < len(abundant); i++ {
for j := i; j < len(abundant); j++ {
if abundant[i]+abundant[j] > n {
break
}
sums[abundant[i]+abundant[j]] = true
}
}
sum := 0
for i := 1; i <= 28123; i++ {
if _, ok := sums[i]; !ok {
sum += i
}
}
fmt.Println(sum)
}
This code takes 450ms on my computer. But if I change the main code to below with slice of bool instead of map like this:
func main() {
start := time.Now()
defer func() {
elapsed := time.Since(start)
fmt.Printf("%s\n", elapsed)
}()
n := 28123
abundant := []int{}
for i := 12; i <= n; i++ {
if divisorsSum(i) > i {
abundant = append(abundant, i)
}
}
sums := make([]bool, n)
for i := 0; i < len(abundant); i++ {
for j := i; j < len(abundant); j++ {
if abundant[i]+abundant[j] < n {
sums[abundant[i]+abundant[j]] = true
} else {
break
}
}
}
sum := 0
for i := 0; i < len(sums); i++ {
if !sums[i] {
sum += i
}
}
fmt.Println(sum)
}
Now it takes only 40ms, below 1/10 of the speed from previous. I thought maps were supposed to have faster look ups. What is up with the performance difference here?
You can profile your code and see, but in general, there are two main reasons:
You pre-allocate sums in the second example to its desired size. This means it never has to grow, and all this is very efficient, there's no GC pressure, no reallocs, etc. Try creating the map with the desired size in advance and see how much it improves things.
I don't know the internal implementation of Go's hash map, but in general, random access of an array/slice by integer index is super efficient, and a hash table adds overhead on top of it, especially if it hashes the integers (it might do so to create better distribution).

Resources