Go variables being overwritten (bug?) - go

So bit of a weird one here. My question is, do people get the same results from running my code as I do? And if you do, is it a fault of my code (I'm a python programmer usually), or a bug in golang?
System info: Go version (1.1.2) linux x64 (fedora 19)
Background info on the code: What I'm doing is finding the highest cost route from the top of a triangle to the bottom, this is from project_euler 18 and 67
The bug: I set a variable called pathA, this is an integer list, plus a new int for the new value found from the triangle
e.g. 3, 7, 2 append 8 should equal 3, 2, 7, 8
and, it does! ... until I set pathB. pathB gets set correctly however suddenly pathA is the same value as pathB.
tl;dr one variable is being overwritten when I set another
My code is as follows:
package main
import (
"fmt"
)
func extendPaths(triangle, prePaths [][]int) [][]int {
nextLine := triangle[len(prePaths)]
fmt.Println("#####PrePaths: ", prePaths)
fmt.Println("#####nextLine: ", nextLine)
postPaths := [][]int{{}}
for i := 0; i < len(prePaths); i++ {
route := prePaths[i]
nextA := nextLine[i]
nextB := nextLine[i+1]
fmt.Println("Next A:", nextA, "Next B:", nextB, "\n")
pathA := append(route, nextA)
fmt.Println("pathA check#1:", pathA)
pathB := append(route, nextB)
fmt.Println("pathA check#2:", pathA, "\n")
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
postPaths = postPaths[1:]
prePaths = [][]int{postPaths[0]}
for i := 1; i < len(postPaths)-1; i += 2 {
if getSum(postPaths[i]) > getSum(postPaths[i+1]) {
prePaths = append(prePaths, postPaths[i])
} else {
prePaths = append(prePaths, postPaths[i+1])
}
}
prePaths = append(prePaths, postPaths[len(postPaths)-1])
return prePaths
}
func getSum(sumList []int) int {
total := 0
for i := 0; i < len(sumList); i++ {
total += sumList[i]
}
return total
}
func getPaths(triangle [][]int) {
prePaths := [][]int{{triangle[0][0]}}
for i := 0; i < len(triangle)-1; i++ {
prePaths = extendPaths(triangle, prePaths)
}
}
func main() {
triangle := [][]int{{3}, {7, 4}, {2, 4, 6}, {8, 5, 9, 3}}
getPaths(triangle)
}
This gives the output in my terminal shown below:
#####PrePaths: [[3]]
#####nextLine: [7 4]
Next A: 7 Next B: 4
pathA check#1: [3 7]
pathA check#2: [3 7]
#####PrePaths: [[3 7] [3 4]]
#####nextLine: [2 4 6]
Next A: 2 Next B: 4
pathA check#1: [3 7 2]
pathA check#2: [3 7 2]
Next A: 4 Next B: 6
pathA check#1: [3 4 4]
pathA check#2: [3 4 4]
#####PrePaths: [[3 7 2] [3 7 4] [3 4 6]]
#####nextLine: [8 5 9 3]
Next A: 8 Next B: 5
pathA check#1: [3 7 2 8]
pathA check#2: [3 7 2 5]
Next A: 5 Next B: 9
pathA check#1: [3 7 4 5]
pathA check#2: [3 7 4 9]
Next A: 9 Next B: 3
pathA check#1: [3 4 6 9]
pathA check#2: [3 4 6 3]
Here you can see that for the last 4 times that I set pathA, it is initially set correctly, but then gets overwritten by pathB.
Does anyone have any thoughts on this?
EDIT:
As pointed out by the comments below, what was needed was to make new slices and copy data from the originals. This was done using code from http://blog.golang.org/go-slices-usage-and-internals modified slightly:
func AppendInt(slice []int, data ...int) []int {
m := len(slice)
n := m + len(data)
if n > cap(slice) {
newSlice := make([]int, (n+1)*2)
copy(newSlice, slice)
slice = newSlice
}
slice = slice[0:n]
copy(slice[m:n], data)
return slice
}
I also changed the code on the other side, where I created the slices pathA and pathB. This changed to:
for i := 0; i < len(prePaths); i++ {
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := AppendInt(prePaths[i], nextA)
pathB := AppendInt(prePaths[i], nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
EDIT2:
It's quite early in the morning here, and I flat out made a mistake in my first edit, I did not fully understand your solution, after a bit of hacking I got there in the end:
This code does not work (pathA gets overwritten):
for i := 0; i < len(prePaths); i++ {
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := append(prePaths[i], nextA)
pathB := append(prePaths[i], nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
This code also does not work (pathA gets overwritten):
for i := 0; i < len(prePaths); i++ {
newRoute := make([]int, len(prePaths[i]), (cap(prePaths[i])+1)*2)
copy(newRoute, prePaths[i])
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := append(newRoute, nextA)
pathB := append(newRoute, nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
However, if I mix the 2 scenarios above into the code below, it works fine (pathA does not get overwritten):
for i := 0; i < len(prePaths); i++ {
newRoute := make([]int, len(prePaths[i]), (cap(prePaths[i])+1)*2)
copy(newRoute, prePaths[i])
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := append(newRoute, nextA)
pathB := append(prePaths[i], nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
So, my solution was to make a copy of the array, and have them both use different ones.

A slice is basically a structure consisting of 3 things:
A pointer to an array of the elements in the slice
The length of that array (the "capacity")
The number of elements actually stored in the array (the "length")
When you run the following code:
append(x, element)
It does the following:
Check if extending the slice will exceed the capacity of the underlying array. If so, allocate a larger one and copy the existing elements to the new array, and update the capacity.
Write the new element (or elements) to the end of the array and update the length.
Return the new slice.
In your code, you have the following:
pathA := append(route, nextA)
pathB := append(route, nextB)
Now there are two possibilities here:
len(route) == cap(route), and a new backing array will be allocated, with pathA and pathB having independent values.
len(route) < cap(route), so pathA and pathB end up sharing the same backing array. The last element in the array will be nextB, since that operation was run second.
It seems that the first case is true for the first few iterations of your loop, after which you hit the second case. You could avoid this by manually making a copy for one of your paths (allocate a slice with make(), and then use copy() to copy the old data).

Related

Error when using pointers to append into slice [][]int

While I was trying to solve a problem "Subset II" from LC, I came across a strange problem. The code generates a power set from a given set.
However, when I run the code it failed because one of the set wasn't correct.
The set [0,3,5,7] replaced by [0,3,5,9] (hence gets appended twice).
I have a print statement (highlighted in code) right before a set gets appended to res, and it prints the correct power set.
The only issue I could think is the use of pointers to append values into a slice, however since it's does not run concurrently I don't see why there would be a race condition.
Appreciate if someone can point out my mistake.
package main
import (
"fmt"
"sort"
)
func ValueCount( nums []int) map[int]int{
hm := make(map[int]int)
for _,v := range(nums){
if c, ok := hm[v]; ok {
hm[v] = c + 1
}else{
hm[v] = 1
}
}
return hm
}
func subsetsWithDup(nums []int) [][]int {
var res [][]int
res = append(res,[]int{})
sort.Ints(nums)
hashMap := ValueCount(nums)
var t []int
printTest(nums, t, &res, hashMap)
return res
}
func printTest(nums []int, t []int, res *[][]int, hm map[int]int) {
if len(nums) == 0 {
return
}
for i:= 0; i < len(nums); {
v := nums[i]
x := nums[i:]
for k:= 0; k< hm[v]; k++ {
var a,b []int
for z:= 0; z<k+1; z++ {
a = append(t,x[z])
}
fmt.Println(a) // <--------- Prints the values that gets appended to res
*res = append(*res, a)
b = a
printTest(nums[i+hm[v]:], b, res, hm)
}
i += hm[v]
}
}
func main(){
n := []int{9,0,3,5,7}
fmt.Println("Find the power set of:", n)
fmt.Println(subsetsWithDup(n))
}
// [0,3,5,7] changes to
// [0,3,5,9] in the output
The bug occurs on line 40:
a = append(t, x[z])
A quick fix would be to change this for loop:
for k := 0; k < hm[v]; k++ {
var a, b []int
for z := 0; z < k+1; z++ {
a = append(t, x[z])
}
fmt.Println(a) // <--------- Prints the values that gets appended to res
*res = append(*res, a)
b = a
printTest(nums[i+hm[v]:], b, res, hm)
}
To this:
for k := 0; k < hm[v]; k++ {
var a, b []int
a = make([]int, len(t))
copy(a, t)
for z := 0; z < k+1; z++ {
a = append(a, x[z])
}
fmt.Println(a) // <--------- Prints the values that gets appended to res
*res = append(*res, a)
b = a
printTest(nums[i+hm[v]:], b, res, hm)
}
It has to do with how Go uses slices as a data structure. When the first argument to the built-in append function was a slice argument, it copied some of the slice's internal data that wasn't intuitive to the programmer. It then modified the argument slice, t, and the newly created slice, a.
I'd recommend reading up on slice internals if you're interested in learning more.
Full program edited:
package main
import (
"fmt"
"sort"
)
func ValueCount(nums []int) map[int]int {
hm := make(map[int]int)
for _, v := range nums {
if c, ok := hm[v]; ok {
hm[v] = c + 1
} else {
hm[v] = 1
}
}
return hm
}
func subsetsWithDup(nums []int) [][]int {
var res [][]int
res = append(res, []int{})
sort.Ints(nums)
hashMap := ValueCount(nums)
var t []int
printTest(nums, t, &res, hashMap)
return res
}
func printTest(nums []int, t []int, res *[][]int, hm map[int]int) {
if len(nums) == 0 {
return
}
for i := 0; i < len(nums); {
v := nums[i]
x := nums[i:]
for k := 0; k < hm[v]; k++ {
var a, b []int
a = make([]int, len(t))
copy(a, t)
for z := 0; z < k+1; z++ {
a = append(a, x[z])
}
fmt.Println(a) // <--------- Prints the values that gets appended to res
*res = append(*res, a)
b = a
printTest(nums[i+hm[v]:], b, res, hm)
}
i += hm[v]
}
}
func main() {
n := []int{9, 0, 3, 5, 7}
fmt.Println("Find the power set of:", n)
fmt.Println(subsetsWithDup(n))
}
New output:
Find the power set of: [9 0 3 5 7]
[0]
[0 3]
[0 3 5]
[0 3 5 7]
[0 3 5 7 9]
[0 3 5 9]
[0 3 7]
[0 3 7 9]
[0 3 9]
[0 5]
[0 5 7]
[0 5 7 9]
[0 5 9]
[0 7]
[0 7 9]
[0 9]
[3]
[3 5]
[3 5 7]
[3 5 7 9]
[3 5 9]
[3 7]
[3 7 9]
[3 9]
[5]
[5 7]
[5 7 9]
[5 9]
[7]
[7 9]
[9]
[[] [0] [0 3] [0 3 5] [0 3 5 7] [0 3 5 7 9] [0 3 5 9] [0 3 7] [0 3 7 9] [0 3 9] [0 5] [0 5 7] [0 5 7 9] [0 5 9] [0 7] [0 7 9] [0 9] [3] [3 5] [3 5 7] [3 5 7 9] [3 5 9] [3 7] [3 7 9] [3 9] [5] [5 7] [5 7 9] [5 9] [7] [7 9] [9]]
Be very careful using (and reusing) slice results - especially when altering those slice values later. Since slices have backing arrays, the referenced data can change in very unexpected ways!
A quick fix to your problem is to copy slice results to a new slice. This ensures changes to the original slice do not introduce bugs (especially in a recursive algorithm).
To copy a slice:
func copyIntSlice(a []int) []int {
c := make([]int, len(a))
copy(c, a) // `a` can now grow/shrink/change without affecting `c`
return c
}
and just call this from your main code:
aCopy := copyIntSlice(a)
*res = append(*res, aCopy)
printTest(nums[i+hm[v]:], aCopy, res, hm)
https://play.golang.org/p/1p8Z4sV9foQ

Strange Behavior of append in two dimension slice

I am a bit new to Go Language, and trying to build a function which will subdivide a slice into a number of slices with almost equal sizes. In case the size of the main slice does not fit into the number of sub-slices, I plan to redistribute the remaining elements to the sub-slices in order.
I have built the following code:
package main
import (
"fmt"
stc "strconv"
"strings"
)
func main() {
myslice := make([]int, 12)
myslice = []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
fmt.Println("Original Slice = ", myslice)
newdiv := subslices(myslice, 4)
fmt.Println(newdiv)
}
func subslices(sl []int, dividnet int) [][]int {
var res [][]int
minsize := len(sl) / dividnet
for i := 0; i < dividnet; i++ {
res = append(res, sl[i*minsize:i*minsize+minsize])
}
for i := 0; i < dividnet; i++ {
fmt.Printf("res[%d] = %v\n", i, res[i])
}
fmt.Println(res)
if rem := len(sl) % dividnet; rem != 0 {
fmt.Println("remaining elements = ", rem)
for j := 0; j < rem; j++ {
tobeadd := sl[minsize*dividnet+j]
fmt.Println("element to be added = ", tobeadd)
fmt.Printf("res[%d] before append = %v\n", j, res[j])
res[j] = append(res[j], tobeadd)
fmt.Printf("res[%d] after append = %v\n", j, res[j])
}
}
return res
}
func gentwodim(x, y int) [][]int {
res := make([][]int, x)
for z := range res {
res[z] = make([]int, y)
}
for i := 0; i < x; i++ {
for j := 0; j < y; j++ {
res[i][j] = i + j
}
}
return res
}
A Sample in Go Play
sample of the above code
The output of the code is as follows:
res[0] = [1 2]
res[1] = [3 4]
res[2] = [5 6]
res[3] = [7 8]
[[1 2] [3 4] [5 6] [7 8]]
remaining elements = 2
element to be added = 9
res[0] before append = [1 2]
res[0] after append = [1 2 9] // up to this step the code works fine
element to be added = 10
res[1] before append = [9 4] // I did not get why res[1] is changed by replacing 3 with 9
res[1] after append = [9 4 10]
[[1 2 9] [9 4 10] [10 6] [7 8]]
However, after appending the first remaining element which shown in res[0] after append = [1 2 9], the 2nd sub-slice is changed as shown from res[1] = [3 4] to res[1] before append = [9 4]
I have tried to debug and understand what I have missed or coded wrong here, but could not.
I would appreciate your support.
Avoid the problem by distributing the "extra" values as you slice things up:
func subslices(sl []int, dividnet int) [][]int {
var res [][]int
var i int
for len(sl) > 0 {
i = len(sl) / (dividnet - len(res))
res = append(res, sl[:i])
sl = sl[i:]
}
return res
}

Does slice assignment in Go copy memory

Purpose: I have a big buffer, and I would like to have an array/slice of pointer pointing to different loc in the buffer.
What I am doing:
datPtrs := make([][]byte, n)
for i:=0; i<n; i++{
datPtrs[i] = bigBuf[i*m:(i+1)*m]
}
My Question:
Will this copy memory? My guess is not, but I cannot find anywhere to confirm this.
What is the best way/tool to find out whether there is memory copy or not?
Go slices are implemented as a struct:
src/runtime/slice.go:
type slice struct {
array unsafe.Pointer
len int
cap int
}
You are assigning/copying the slice struct, which does not copy the underlying array, only its pointer.
A simple illustration:
package main
import (
"fmt"
)
func main() {
buf := make([]byte, 8)
for i := range buf {
buf[i] = byte(i)
}
sub := buf[1:3]
fmt.Println(buf)
fmt.Println(sub)
for i := range sub {
sub[i] += 43
}
fmt.Println(buf)
fmt.Println(sub)
}
Playground: https://play.golang.org/p/4OzPwuNmUlY
Output:
[0 1 2 3 4 5 6 7]
[1 2]
[0 44 45 3 4 5 6 7]
[44 45]
See The Go Blog: Go Slices: usage and internals,
No
Slice is just a pointer to memory + len and cap
see: Why can not I duplicate a slice with `copy()` in Golang?
Like so:
package main
import (
"fmt"
)
func main() {
bigBuf := []byte{1, 2, 3, 4, 5}
datPtrs := make([][]byte, 2)
for i := 0; i < 2; i++ {
datPtrs[i] = bigBuf[i : i+1]
}
fmt.Println(bigBuf) // [1 2 3 4 5]
datPtrs[0][0] = 10
fmt.Println(bigBuf) // [10 2 3 4 5]
datPtrs[1][0] = 20
fmt.Println(bigBuf) // [10 20 3 4 5]
}

Unexpected behavior when passing a pointer to a slice in go

The following go program is supposed to generate all permutations of a slice of integers:
package main
import "fmt"
func permute(nums []int) [][]int {
var res [][]int
var s []int
permuteHlp(&res, nums, 0, s)
return res
}
func permuteHlp(res *[][]int, nums []int, i int, s []int) {
if i == len(nums) {
*res = append(*res, s)
return
}
for j := i; j < len(nums); j++ {
s = append(s, nums[j])
nums[i], nums[j] = nums[j], nums[i]
permuteHlp(res, nums, i+1, s)
s = s[:len(s)-1]
nums[i], nums[j] = nums[j], nums[i]
}
}
func main() {
x := []int{1,2,3,4}
y := permute(x)
fmt.Println(y)
}
The output is unexpected
[[1 2 4 3] [1 2 4 3] [1 3 4 2] [1 3 4 2] [1 4 2 3] [1 4 2 3] [2 1 4 3] [2 1 4 3] [2 3 4 1] [2 3 4 1] [2 4 1 3] [2 4 1 3] [3 2 4 1] [3 2 4 1] [3 1 4 2] [3 1 4 2] [3 4 2 1] [3 4 2 1] [4 2 1 3] [4 2 1 3] [4 3 1 2] [4 3 1 2] [4 1 2 3] [4 1 2 3]]
I don't understand what is wrong here. I would appreciate any help.
Thank you!
You're passing around a pointer to the the same slice. In the end you wind up with a bunch of pointers to the same slice in your results, so of course all the values will be identical - it's the same slice printed over and over.
It's also worth noting that a pointer to a slice is rarely what you want, as slices already contain a pointer to the underlying array.
There's no need for a pointer to the slice since slices are pointers themselves. "a slice is a reference to a contiguous segment of an array.", reference.
The strange behavior you're seeing is because you're using append, when a slice grows beyond its capacity it's required to create a new slice with increased capacity and copy all the contents of the original one (this is what append does behind the scenes), hence new slice is no longer pointing to the original underlying array.
Instead of modifying the incoming parameter, I suggest returning the slice as a return value for the function.
func permute(nums []int) [][]int {
res := permuteHlp(nums, 0, new([]int))
return res
}
I recommend you read the blog post in golang.org about slices internals, here
Edit:
I add a refactor, taking the algorithm from this answer.
package main
import (
"fmt"
)
func permutations(arr []int)[][]int{
var helper func([]int, int)
res := [][]int{}
helper = func(arr []int, n int){
if n == 1{
tmp := make([]int, len(arr))
copy(tmp, arr)
res = append(res, tmp)
} else {
for i := 0; i < n; i++{
helper(arr, n - 1)
if n % 2 == 1{
tmp := arr[i]
arr[i] = arr[n - 1]
arr[n - 1] = tmp
} else {
tmp := arr[0]
arr[0] = arr[n - 1]
arr[n - 1] = tmp
}
}
}
}
helper(arr, len(arr))
return res
}
func main() {
x := []int{1,2,3,4}
d := permutations(x)
fmt.Print(d)
}
Generally you won't want to have a pointer to a slice, instead, return a new one from the function, another thing to comment on, try not to use recursion if possible as golang doesn't have tail call optimization, and its loops perform amazingly. Hope it helps!

Working with maps in Golang

I am new to Go and doing a few exercises. One of them is to sort the numbers in an array by frequency, from most to least frequent.
Example:
Input: [2, 2, 5, 7, 4, 4, 4, 7, 2]
Output: [2, 4, 7, 5]
Note that [4, 2, 7, 5] would also be correct, since 4 and 2 have the same frequency.
For this purpose I am converting the array into a value value map, which here would look like this: [2:3][4:3][7:2][5:1] (2 and 3 have freq. of 3, 7 has the freq of 2,... )
Afterwards I would like to simply loop through the map and output the keys ordered by value. For that I use the following code, which apparently does not work. Why?
count := 0
max := -1
// break loop, if map is empty
for i := 0; i < 1; i-- {
if len(m) == 0 {
break
}
max = -1
// get key of biggest value
for k, v := range m {
if v > max {
max = k
}
}
// res (for result) is a slice of integers
res[count] = max
// remove key-value-pair from map
delete(m, max)
count++
}
return res
Please keep in mind that this is an exercise. I am very sure there are much better, build in ways to do this.
Your 'max' variable is meant to keep track of the maximum frequency seen so far. However when you do 'max = k' you're assigning a key.
You need to keep track of the maximum frequency and the key associated with that frequency in separate variables.
...
for k, v := range m {
if v > maxFreq {
maxFreq = v
mostFrequentKey = k
}
}
// res (for result) is a slice of integers
res[count] = mostFrequentKey
// remove key-value-pair from map
delete(m, mostFrequentKey)
count++
...
For sorted frequencies, use a map then a slice. For example,
package main
import (
"fmt"
"sort"
)
func main() {
Input := []int{2, 2, 5, 7, 4, 4, 4, 7, 2}
fmt.Println("Input: ", Input)
mFreq := make(map[int]int, len(Input))
for _, n := range Input {
mFreq[n]++
}
sFreq := make([][2]int, 0, len(mFreq))
for n, f := range mFreq {
sFreq = append(sFreq, [2]int{n, f})
}
sort.Slice(sFreq, func(i, j int) bool {
if sFreq[i][1] <= sFreq[j][1] {
if sFreq[i][1] < sFreq[j][1] {
return false
}
if sFreq[i][0] >= sFreq[j][0] {
return false
}
}
return true
},
)
Output := []int{2, 4, 7, 5}
fmt.Println("Output: ", Output)
fmt.Println("Frequencies:", sFreq)
}
Playground: https://play.golang.org/p/8tiSksz3S76
Output:
Input: [2 2 5 7 4 4 4 7 2]
Output: [2 4 7 5]
Frequencies: [[2 3] [4 3] [7 2] [5 1]]

Resources