Purpose: I have a big buffer, and I would like to have an array/slice of pointer pointing to different loc in the buffer.
What I am doing:
datPtrs := make([][]byte, n)
for i:=0; i<n; i++{
datPtrs[i] = bigBuf[i*m:(i+1)*m]
}
My Question:
Will this copy memory? My guess is not, but I cannot find anywhere to confirm this.
What is the best way/tool to find out whether there is memory copy or not?
Go slices are implemented as a struct:
src/runtime/slice.go:
type slice struct {
array unsafe.Pointer
len int
cap int
}
You are assigning/copying the slice struct, which does not copy the underlying array, only its pointer.
A simple illustration:
package main
import (
"fmt"
)
func main() {
buf := make([]byte, 8)
for i := range buf {
buf[i] = byte(i)
}
sub := buf[1:3]
fmt.Println(buf)
fmt.Println(sub)
for i := range sub {
sub[i] += 43
}
fmt.Println(buf)
fmt.Println(sub)
}
Playground: https://play.golang.org/p/4OzPwuNmUlY
Output:
[0 1 2 3 4 5 6 7]
[1 2]
[0 44 45 3 4 5 6 7]
[44 45]
See The Go Blog: Go Slices: usage and internals,
No
Slice is just a pointer to memory + len and cap
see: Why can not I duplicate a slice with `copy()` in Golang?
Like so:
package main
import (
"fmt"
)
func main() {
bigBuf := []byte{1, 2, 3, 4, 5}
datPtrs := make([][]byte, 2)
for i := 0; i < 2; i++ {
datPtrs[i] = bigBuf[i : i+1]
}
fmt.Println(bigBuf) // [1 2 3 4 5]
datPtrs[0][0] = 10
fmt.Println(bigBuf) // [10 2 3 4 5]
datPtrs[1][0] = 20
fmt.Println(bigBuf) // [10 20 3 4 5]
}
Related
package main
import (
"fmt"
)
func main() {
arr0 := []int{
1,2,3,4,5,
}
arr1 := []int{}
fmt.Println(arr0)
fmt.Println(arr1)
fmt.Println("transferring...")
transfer(&arr0, &arr1)
fmt.Println(arr0)
fmt.Println(arr1)
}
func transfer(arr0 *[]int, arr1 *[]int) {
tmp := make([]int, 0)
for i:=0;i<len(*arr0);i++ {
tmp = append(tmp, (*arr0)[i])
}
arr1 = &tmp
s := make([]int, 0)
arr0 = &s
}
For function of transfer, I intented to transfer elements of slice arr0 to slice arr1 and empty slice arr0
But it is not successful
Here is my output
[1 2 3 4 5]
[]
transferring...
[1 2 3 4 5]
[]
After transferring, I need the result below.
[]
[1 2 3 4 5]
But actually, arr0, and arr1 in the main function remain as it was!
can someone tell me why this is not ok?
I thought in the memory, it should be like this
after running transfer function
#jacobsa has given an excellent answer.
Just keep that in mind. You can achieve the same effect but with better performance. Golang offers an excellent opportunity for this.
package main
import (
"fmt"
)
func main() {
arr0 := []int{
1, 2, 3, 4, 5,
}
arr1 := []int{}
fmt.Println(arr0)
fmt.Println(arr1)
fmt.Println("transferring...")
transfer(&arr0, &arr1)
fmt.Println(arr0)
fmt.Println(arr1)
}
func transfer(arr0 *[]int, arr1 *[]int) {
*arr0, *arr1 = *arr1, *arr0
}
These two lines:
arr1 = &tmp
arr0 = &s
change the local variables arr1 and arr0 within the function. Those variables happen to be pointers, but they are just copies of the input pointers provided by main—they are not references to the input pointers.
If you changed the things the arr1 and arr0 pointers point to, rather than the pointers themselves, then you would see a change to the values provided by main:
*arr1 = tmp
*arr0 = s
The result of s is [1, 2, 3], I thought that slices hold a reference to the underlying array. Isn't that the case?
package main
import (
"fmt"
)
func main() {
s := []int{1, 2, 3}
ss := s[1:]
ss = append(ss, 4)
for _, v := range ss {
v += 10
}
for i := range ss {
ss[i] += 10
}
fmt.Println(s)
}
I thought that slices hold a reference to the underlying array. Isn't that the case?
Yes it is. But you created an array with length 3 with this statement:
s := []int{1, 2, 3}
When you appended an element to ss, that required an allocation of a new, longer array. So you lost the link between ss and s with this statement:
ss = append(ss, 4)
You can verify that by running this example:
package main
import (
"fmt"
)
func main() {
s := []int{1, 2, 3}
ss := s[1:]
ss[0] += 5
ss = append(ss, 4)
ss[0] += 100
fmt.Println(s)
}
Which prints [1 7 3].
If you change your initialization of s to have a length greater than three, then no new array allocation will be required, and the link between s and ss will be maintained:
package main
import (
"fmt"
)
func main() {
s := make([]int, 3, 4)
s[0], s[1], s[2] = 1, 2, 3
ss := s[1:]
ss[0] += 5
ss = append(ss, 4)
ss[0] += 100
fmt.Println(s)
}
Output: [1 107 3]
The answer that theorizes that the problem is a range copy of the slice is incorrect, which can be shown with this example:
package main
import (
"fmt"
)
func main() {
s := make([]int, 3, 4)
s[0], s[1], s[2] = 1, 2, 3
ss := s[1:]
ss = append(ss, 4)
for i := range ss {
ss[i] += 10
}
fmt.Println(s)
}
Output: [1 12 13]
It seems like you've created a copy of slice s, and you've made changes to slice copy ss thinking that the changes would also be passed to the slice that the copy was made from.
The first for loop is also walking through the elements of slice ss, but is not actually doing anything with them, because range also creates a copy when providing you element values, so it doesn't actually do anything!
It does seem like you're trying to do the following:
Append the value 4 to slice s
Take each value from index 1 of slice to the end of the slice, and add 10
If that's the case, this should help you accomplish that:
package main
import (
"fmt"
)
func main() {
s := []int{1, 2, 3}
s = append(s, 4)
for i := range s {
if i == 0 {
continue
}
s[i] += 10
}
fmt.Println(s)
}
You can see this on the Go playground: Link
I was having some issues with Golang slices.
I understand that a slice is a pointer to an underlying array, but some of the behaviour feels a little odd.
I was trying to remove an item from a slice I managed to do it by copying the slice is there a better way?
In the code below the original slice is changed.
package main
import (
"fmt"
)
func main() {
mySlice := []int{1,2,3,4,5,6}
pos := 3
slicePart1 := mySlice[:pos+1]
slicePart2 := mySlice[pos+2:]
fmt.Println(mySlice)
fmt.Println(slicePart1)
fmt.Println(slicePart2)
new := append(slicePart1,slicePart2...)
fmt.Println(new)
fmt.Println(mySlice)
}
For example,
package main
import "fmt"
func main() {
s := []int{1, 2, 3, 4, 5, 6}
fmt.Println(s)
i := 3
fmt.Println(i)
s = append(s[:i], s[i+1:]...)
fmt.Println(s)
}
Playground: https://play.golang.org/p/SVQEUE7Rrei
Output:
[1 2 3 4 5 6]
3
[1 2 3 5 6]
Or, if order is not important,
package main
import "fmt"
func main() {
s := []int{1, 2, 3, 4, 5, 6}
fmt.Println(s)
i := 3
fmt.Println(i)
s[i] = s[len(s)-1]
s = s[:len(s)-1]
fmt.Println(s)
}
Playground: https://play.golang.org/p/lVgKew3ZJNF
Output:
[1 2 3 4 5 6]
3
[1 2 3 6 5]
For several other ways, see SliceTricks.
how array shift function works with slices?
package main
import "fmt"
func main() {
s := []int{2, 3, 5, 7, 11, 13}
for k, v := range s {
x, a := s[0], s[1:] // get and remove the 0 index element from slice
fmt.Println(a) // print 0 index element
}
}
I found an example from slice tricks but can't get it right.
https://github.com/golang/go/wiki/SliceTricks
x, a := a[0], a[1:]
Edit can you please explain why x is undefined here?
Building upon the answer and merging with SliceTricks
import "fmt"
func main() {
s := []int{2, 3, 5, 7, 11, 13}
fmt.Println(len(s), s)
for len(s) > 0 {
x, s = s[0], s[1:] // undefined: x
fmt.Println(x) // undefined: x
}
fmt.Println(len(s), s)
}
For example,
package main
import "fmt"
func main() {
s := []int{2, 3, 5, 7, 11, 13}
fmt.Println(len(s), s)
for len(s) > 0 {
x := s[0] // get the 0 index element from slice
s = s[1:] // remove the 0 index element from slice
fmt.Println(x) // print 0 index element
}
fmt.Println(len(s), s)
}
Output:
6 [2 3 5 7 11 13]
2
3
5
7
11
13
0 []
References:
The Go Programming Language Specification: For statements
Addendum to answer edit to question:
Declare x,
package main
import "fmt"
func main() {
s := []int{2, 3, 5, 7, 11, 13}
fmt.Println(len(s), s)
for len(s) > 0 {
var x int
x, s = s[0], s[1:]
fmt.Println(x)
}
fmt.Println(len(s), s)
}
Output:
6 [2 3 5 7 11 13]
2
3
5
7
11
13
0 []
You can copy and paste my code for any slice type; it infers the type for x. It doesn't have to be changed if the type of s changes.
for len(s) > 0 {
x := s[0] // get the 0 index element from slice
s = s[1:] // remove the 0 index element from slice
fmt.Println(x) // print 0 index element
}
For your version, the type for x is explicit and must be changed if the type of s is changed.
for len(s) > 0 {
var x int
x, s = s[0], s[1:]
fmt.Println(x)
}
Just a quick explanation on how we implement shift-like functionality Go. It's actually a very manual process. Take this example:
catSounds := []string{"meow", "purr", "schnurr"}
firstValue := stuff[0] // meow
catSounds = catSounds[1:]
On the first line, we create our slice.
On the second line we get the first element of the slice.
On the third line, we re-assign the value of catSounds to everything currently in catSounds after the first element (catSounds[1:]).
So given all that, we can condense the second and third lines with a comma for brevity:
catSounds := []string{"meow", "purr", "schnurr"}
firstValue, catSounds := catSounds[0], catSounds[1:]
So bit of a weird one here. My question is, do people get the same results from running my code as I do? And if you do, is it a fault of my code (I'm a python programmer usually), or a bug in golang?
System info: Go version (1.1.2) linux x64 (fedora 19)
Background info on the code: What I'm doing is finding the highest cost route from the top of a triangle to the bottom, this is from project_euler 18 and 67
The bug: I set a variable called pathA, this is an integer list, plus a new int for the new value found from the triangle
e.g. 3, 7, 2 append 8 should equal 3, 2, 7, 8
and, it does! ... until I set pathB. pathB gets set correctly however suddenly pathA is the same value as pathB.
tl;dr one variable is being overwritten when I set another
My code is as follows:
package main
import (
"fmt"
)
func extendPaths(triangle, prePaths [][]int) [][]int {
nextLine := triangle[len(prePaths)]
fmt.Println("#####PrePaths: ", prePaths)
fmt.Println("#####nextLine: ", nextLine)
postPaths := [][]int{{}}
for i := 0; i < len(prePaths); i++ {
route := prePaths[i]
nextA := nextLine[i]
nextB := nextLine[i+1]
fmt.Println("Next A:", nextA, "Next B:", nextB, "\n")
pathA := append(route, nextA)
fmt.Println("pathA check#1:", pathA)
pathB := append(route, nextB)
fmt.Println("pathA check#2:", pathA, "\n")
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
postPaths = postPaths[1:]
prePaths = [][]int{postPaths[0]}
for i := 1; i < len(postPaths)-1; i += 2 {
if getSum(postPaths[i]) > getSum(postPaths[i+1]) {
prePaths = append(prePaths, postPaths[i])
} else {
prePaths = append(prePaths, postPaths[i+1])
}
}
prePaths = append(prePaths, postPaths[len(postPaths)-1])
return prePaths
}
func getSum(sumList []int) int {
total := 0
for i := 0; i < len(sumList); i++ {
total += sumList[i]
}
return total
}
func getPaths(triangle [][]int) {
prePaths := [][]int{{triangle[0][0]}}
for i := 0; i < len(triangle)-1; i++ {
prePaths = extendPaths(triangle, prePaths)
}
}
func main() {
triangle := [][]int{{3}, {7, 4}, {2, 4, 6}, {8, 5, 9, 3}}
getPaths(triangle)
}
This gives the output in my terminal shown below:
#####PrePaths: [[3]]
#####nextLine: [7 4]
Next A: 7 Next B: 4
pathA check#1: [3 7]
pathA check#2: [3 7]
#####PrePaths: [[3 7] [3 4]]
#####nextLine: [2 4 6]
Next A: 2 Next B: 4
pathA check#1: [3 7 2]
pathA check#2: [3 7 2]
Next A: 4 Next B: 6
pathA check#1: [3 4 4]
pathA check#2: [3 4 4]
#####PrePaths: [[3 7 2] [3 7 4] [3 4 6]]
#####nextLine: [8 5 9 3]
Next A: 8 Next B: 5
pathA check#1: [3 7 2 8]
pathA check#2: [3 7 2 5]
Next A: 5 Next B: 9
pathA check#1: [3 7 4 5]
pathA check#2: [3 7 4 9]
Next A: 9 Next B: 3
pathA check#1: [3 4 6 9]
pathA check#2: [3 4 6 3]
Here you can see that for the last 4 times that I set pathA, it is initially set correctly, but then gets overwritten by pathB.
Does anyone have any thoughts on this?
EDIT:
As pointed out by the comments below, what was needed was to make new slices and copy data from the originals. This was done using code from http://blog.golang.org/go-slices-usage-and-internals modified slightly:
func AppendInt(slice []int, data ...int) []int {
m := len(slice)
n := m + len(data)
if n > cap(slice) {
newSlice := make([]int, (n+1)*2)
copy(newSlice, slice)
slice = newSlice
}
slice = slice[0:n]
copy(slice[m:n], data)
return slice
}
I also changed the code on the other side, where I created the slices pathA and pathB. This changed to:
for i := 0; i < len(prePaths); i++ {
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := AppendInt(prePaths[i], nextA)
pathB := AppendInt(prePaths[i], nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
EDIT2:
It's quite early in the morning here, and I flat out made a mistake in my first edit, I did not fully understand your solution, after a bit of hacking I got there in the end:
This code does not work (pathA gets overwritten):
for i := 0; i < len(prePaths); i++ {
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := append(prePaths[i], nextA)
pathB := append(prePaths[i], nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
This code also does not work (pathA gets overwritten):
for i := 0; i < len(prePaths); i++ {
newRoute := make([]int, len(prePaths[i]), (cap(prePaths[i])+1)*2)
copy(newRoute, prePaths[i])
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := append(newRoute, nextA)
pathB := append(newRoute, nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
However, if I mix the 2 scenarios above into the code below, it works fine (pathA does not get overwritten):
for i := 0; i < len(prePaths); i++ {
newRoute := make([]int, len(prePaths[i]), (cap(prePaths[i])+1)*2)
copy(newRoute, prePaths[i])
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := append(newRoute, nextA)
pathB := append(prePaths[i], nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
So, my solution was to make a copy of the array, and have them both use different ones.
A slice is basically a structure consisting of 3 things:
A pointer to an array of the elements in the slice
The length of that array (the "capacity")
The number of elements actually stored in the array (the "length")
When you run the following code:
append(x, element)
It does the following:
Check if extending the slice will exceed the capacity of the underlying array. If so, allocate a larger one and copy the existing elements to the new array, and update the capacity.
Write the new element (or elements) to the end of the array and update the length.
Return the new slice.
In your code, you have the following:
pathA := append(route, nextA)
pathB := append(route, nextB)
Now there are two possibilities here:
len(route) == cap(route), and a new backing array will be allocated, with pathA and pathB having independent values.
len(route) < cap(route), so pathA and pathB end up sharing the same backing array. The last element in the array will be nextB, since that operation was run second.
It seems that the first case is true for the first few iterations of your loop, after which you hit the second case. You could avoid this by manually making a copy for one of your paths (allocate a slice with make(), and then use copy() to copy the old data).