I'm working my way through this "Tour of Go" and I'm doing this exercise here. I originally wrote the answer like this:
func Pic(dx, dy int) [][]uint8 {
outer := make([][]uint8, dy)
inner := make([]uint8, dx)
for y, _ := range outer {
for x, _ := range inner {
inner[x] = x^y // this is one of the image functions they give
}
outer[y] = inner
}
return outer
}
I thought this should work, but when I run Pic(5,5) I get:
[[0 4 8 12 16] [0 4 8 12 16] [0 4 8 12 16] [0 4 8 12 16] [0 4 8 12 16]]
Not right, obviously.
So, I finally figured out that I needed to move my inner := make([]uint8, dx) declaration inside of the first loop like so:
func Pic(dx, dy int) [][]uint8 {
outer := make([][]uint8, dy)
for y, _ := range outer {
inner := make([]uint8, dx)
for x, _ := range inner {
inner[x] = x^y // this is one of the image functions they give
}
outer[y] = inner
}
return outer
}
and now Pic(5,5) gives me the correct answer:
[[0 0 0 0 0] [0 1 2 3 4] [0 2 4 6 8] [0 3 6 9 12] [0 4 8 12 16]]
What's happening here? I think this has something to do with pointers or something, but I want to be sure I really understand what's going on. Obviously, in the first example, each time I reassigned something new to inner the values in outer that previously had inner assigned to them changed as well.
But what happens in the second example? Each time we go through the range outer loop the inner variable gets re-declared, but what happens to the values of outer that previously had inner assigned to them? Are they now somehow "cut off" and only exist as copies?
I'm mainly confused because the assignment line outer[y] = inner doesn't change in either example, so I don't understand the mechanics of how in one of them it assigns a copy of the value and in the other it assigns what I would think of as a pointer.
In the first case, you never "reassigned something new to inner". You created one slice of uint8, called inner, which you filled with values (overwriting those values several times), and you made every element of outer contain a copy of inner. But remember, a slice is only a view into an array that exists somewhere (in your case, the backing array is created anonymously by make). Copying a slice doesn't copy the values in the array, it just copies the slice header, meaning there are now multiple views onto the same storage (this is fine, and safe).
In the second case, you called make for every row of outer, so five different backing arrays were created, with inner pointing into a different one each time. This time, when you assigned inner to an element of outer, you got five genuinely different slice headers, instead of five copies of the same one.
In short: a slice is a value, but it's a value that contains a pointer. The pointer might point to an array that has a name, or it might point to an entirely anonymous array. There's nothing to worry about; Go will make sure that anything that is pointed-to stays alive.
First, consider this:
inner := make([]uint8, dx)
This statement allocates an array large enough to hold dx elements, creates a slice from it, and stores that slice in the variable inner. A slice contains three values: a pointer to the underlying array, capacity of that array, and the length of the slice. So inner contains a pointer to the actual data.
When you redeclare and assign inner in the for loop, a new array is created to store the data, inner becomes a slice pointing to that array, and then you fill out the individual elements of inner. Then you assign inner to an element of outer, and that element of outer now gets that slice, pointing to the initialized data. In the next iteration, a new array is created, inner is assigned to it, etc.
So it's got to do with pointers, in particular, the fact that a slice contains a pointer to the underlying array, and that when you assign a variable to a slice, that variable is assigned three values, the pointer to array, length, and capacity,
Related
I am learning Go by going through A Tour of Go. One of the exercises there asks me to create a 2D slice of dy rows and dx columns containing uint8. My current approach, which works, is this:
a:= make([][]uint8, dy) // initialize a slice of dy slices
for i:=0;i<dy;i++ {
a[i] = make([]uint8, dx) // initialize a slice of dx unit8 in each of dy slices
}
I think that iterating through each slice to initialize it is too verbose. And if the slice had more dimensions, the code would become unwieldy. Is there a concise way to initialize 2D (or n-dimensional) slices in Go?
There isn't a more concise way, what you did is the "right" way; because slices are always one-dimensional but may be composed to construct higher-dimensional objects. See this question for more details: Go: How is two dimensional array's memory representation.
One thing you can simplify on it is to use the for range construct:
a := make([][]uint8, dy)
for i := range a {
a[i] = make([]uint8, dx)
}
Also note that if you initialize your slice with a composite literal, you get this for "free", for example:
a := [][]uint8{
{0, 1, 2, 3},
{4, 5, 6, 7},
}
fmt.Println(a) // Output is [[0 1 2 3] [4 5 6 7]]
Yes, this has its limits as seemingly you have to enumerate all the elements; but there are some tricks, namely you don't have to enumerate all values, only the ones that are not the zero values of the element type of the slice. For more details about this, see Keyed items in golang array initialization.
For example if you want a slice where the first 10 elements are zeros, and then follows 1 and 2, it can be created like this:
b := []uint{10: 1, 2}
fmt.Println(b) // Prints [0 0 0 0 0 0 0 0 0 0 1 2]
Also note that if you'd use arrays instead of slices, it can be created very easily:
c := [5][5]uint8{}
fmt.Println(c)
Output is:
[[0 0 0 0 0] [0 0 0 0 0] [0 0 0 0 0] [0 0 0 0 0] [0 0 0 0 0]]
In case of arrays you don't have to iterate over the "outer" array and initialize "inner" arrays, as arrays are not descriptors but values. See blog post Arrays, slices (and strings): The mechanics of 'append' for more details.
Try the examples on the Go Playground.
There are two ways to use slices to create a matrix. Let's take a look at the differences between them.
First method:
matrix := make([][]int, n)
for i := 0; i < n; i++ {
matrix[i] = make([]int, m)
}
Second method:
matrix := make([][]int, n)
rows := make([]int, n*m)
for i := 0; i < n; i++ {
matrix[i] = rows[i*m : (i+1)*m]
}
In regards to the first method, making successive make calls doesn't ensure that you will end up with a contiguous matrix, so you may have the matrix divided in memory. Let's think of an example with two Go routines that could cause this:
The routine #0 runs make([][]int, n) to get allocated memory for matrix, getting a piece of memory from 0x000 to 0x07F.
Then, it starts the loop and does the first row make([]int, m), getting from 0x080 to 0x0FF.
In the second iteration it gets preempted by the scheduler.
The scheduler gives the processor to routine #1 and it starts running. This one also uses make (for its own purposes) and gets from 0x100 to 0x17F (right next to the first row of routine #0).
After a while, it gets preempted and routine #0 starts running again.
It does the make([]int, m) corresponding to the second loop iteration and gets from 0x180 to 0x1FF for the second row. At this point, we already got two divided rows.
With the second method, the routine does make([]int, n*m) to get all the matrix allocated in a single slice, ensuring contiguity. After that, a loop is needed to update the matrix pointers to the subslices corresponding to each row.
You can play with the code shown above in the Go Playground to see the difference in the memory assigned by using both methods. Note that I used runtime.Gosched() only with the purpose of yielding the processor and forcing the scheduler to switch to another routine.
Which one to use? Imagine the worst case with the first method, i.e. each row is not next in memory to another row. Then, if your program iterates through the matrix elements (to read or write them), there will probably be more cache misses (hence higher latency) compared to the second method because of worse data locality. On the other hand, with the second method it may not be possible to get a single piece of memory allocated for the matrix, because of memory fragmentation (chunks spread all over the memory), even though theoretically there may be enough free memory for it.
Therefore, unless there's a lot of memory fragmentation and the matrix to be allocated is huge enough, you would always want to use the second method to get advantage of data locality.
With Go 1.18 you get generics.
Here is a function that uses generics to allow to create a 2D slice for any cell type.
func Make2D[T any](n, m int) [][]T {
matrix := make([][]T, n)
rows := make([]T, n*m)
for i, startRow := 0, 0; i < n; i, startRow = i+1, startRow+m {
endRow := startRow + m
matrix[i] = rows[startRow:endRow:endRow]
}
return matrix
}
With that function in your toolbox, your code becomes:
a := Make2D[uint8](dy, dx)
You can play with the code on the Go Playground.
Here a consive way to do it:
value := [][]string{}{[]string{}{"A1","A2"}, []string{}{"B1", "B2"}}
PS.: you can change "string" to the type of element you're using in your slice.
Could someone help explain the Golang internals of why this code is mutating the original array a?
func main() {
a := []int{1,2,3,4}
b := a
b = append(b[0:1], b[2:]...)
fmt.Println(b)
fmt.Println(a)
}
Output:
[1 3 4]
[1 3 4 4]
I thought b := a would be passing by value. Thanks in advance.
That's how slices work. A slice is just a pointer(+size+capacity), the actual data is stored in the array.
When you copy a slice, the underlying array is not copied. Then you end up with two slices pointing to the same array. Mutating the values of one slice will become visible via the other slice.
See Go Slices: usage and internals for more details.
If you want to leave the original slice untouched, first make a deep copy. For example like this
b := append([]int{}, a...) // deep copy
(Live demo)
Slices are basically wrapper over arrays. Slices doesn't have their own data they just hold the reference to the arrays. In your given code you are assigning a to b now they both are indicating the same array. And so when you are mutating the slice b the slice a is also being mutated.
You can use copy method to copy elements from one array to another.
// copy returns the count of total copied elements
count := copy(b /*destination*/ , a /*source*/)
But make sure to allocate an array with the same length of source array.
Example is given below:
func main() {
a := []int{1,2,3,4}
b := make([]int, len(a))
_ = copy(b, a)
a[0] = 2
fmt.Println(b)
fmt.Println(a)
}
For example:
func test(requiredIp []int, i int) []int {
requiredIp = append(requiredIp[0:i], requiredIp[i+1:]...)
return requiredIp
}
func main(){
requiredIp := []int{1,2,4,5,6}
fmt.Println(test(requiredIp,0)) // output:[2 4 5 6]
fmt.Println("original", requiredIp) // output:[2 4 5 6,6]
}
Why original slice has two 6 ?
A slice is a struct type with three fields:
A pointer to (an address of) the underlying array holding the data.
The length — how many elements are there in the slice.
The capacity — how many elements it's possible to store into the underlying array without reallocating it.
As everything in Go, slices are passed by value. This happens when you assign a slice value to a variable or when you pass it as a parameter in a function/method call.
What is pased by value (that is, copied) is that structure with three fields.
The pointer which is copied, obviously points at the same data block as the one in the original slice.
What happens in your code is the following:
The original slice is []int{1,2,4,5,6}.
It has length and capacity equal to 5.
It is passed to a functon, test.
The slice available there via the function's parameter requiredIp is initially identical to the one which is passed in a call.
You reslice that slice by evaluating requiredIp[0:i], and since in your call i equals to 0, you evaluate requiredIp[0:0].
That expression creates a slice with the backing array and the capacity of the original slice and length 0.
You then reslice the original slice once again — with the expression requiredIp[i+1:], which, in your call is requiredIp[1:].
The result shares the backing array with the original, has the capacity 4 and the contents []int{2,4,5,6}.
You then append the slice obtained on the previous step to the one obtained in the step before the previous one. That's where it gets interesting.
Consider that the slice being appended to points at the 0th element of the original slice and has length 0 and capacity 5. It means it has room for 5 elements.
That is, the slice's backing array still holds [1, 2, 4, 5, 6].
The slice being appended shares the same backing array with the slice being appended to, just it points at the 1st element, not the 0th.
The code of append sees it's told to append 4 elements and checks to see whether the target slice has enough capacity to hold them, and it has.
So append merely copies [2, 4, 5, 6] from and to the same backing array, overwriting 4 elements starting from index 0 in it. Effectively, it's elements [2, 4, 5, 6] moved one element left.
The resulting backing array now contains [2, 4, 5, 6, 6]: the first 4 element are the 4 elements which were the last, moved by one element left, overwriting what there were at indices 0 through 3.
You now return the reslting slice to the caller. That new slice value shares the backing array with all the slice values involved in the example, but remember that it has its length set to 4 — because the append appended 4 elements to the slice of length 0.
You print the original slice and the one returned from your function. They differ only in their length: the original has it equals to 5, and the returned from the function — to 4. The rest of their fields are the same: they both have capacity 5 and share the same backing array by both pointing its 0th element.
The difference in the lengths explains why "the original" slice seemingly has "extra" 6 in it. In fact in has it exactly where it were left off before test was called.
If, for some reason you really wanted to "detach" the slice produced by test from the original one by forcing it to allocate the new backing array, there are several possibilities:
Merely append to an unallocated slice — for instance,
append([]int(nil), 1, 2, 3, 4) would allocate a fresh backing array.
In your particular case that would not really be a solution as with i > 0 the append would have to operate on a non-empty slice. This can be dealt, for exmaple, with two appends:
s := append([]int(nil), input[0:i]...)
s = append(s, input[i+1:]...)
…or with allocating a new slice and copying:
s := make([]int, len(input)-1)
copy(s, input[0:i])
copy(s[i:], input[i+1:])
When reslicing the original slice, be sure to also artifically reset its capacity:
return append(input[0:i:i], input[i+1:]...)
Here, the first slice would have the same capacity as its length, i, and appending even a single element to it would force the append to allocate a new backing array, copy over these i elements to it and then copy over what is being appended.
Further reading:
https://blog.golang.org/slices-intro
https://blog.golang.org/slices
https://blog.golang.org/strings
…and actually consider starting with Effective Go.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
According to the Tour of Go, in a Go slice s, the expression s[lo:hi] evaluates to a slice of the elements from lo through hi-1, inclusive:
package main
import "fmt"
func main() {
p := []int{0, // slice position 0
10, // slice position 1
20, // slice position 2
30, // slice position 3
40, // slice position 4
50} // slice position 5
fmt.Println(p[0:3]) // => [0 10 20]
}
In my code example above, "p[0:3]" would seem to intuitively "read" as: "the slice from position 0 to position 3", equating to [0, 10, 20, 30]. But of course, it actually equates to [0 10 20].
So my question is: what is the design rationale for the upper value evaluating to hi-1 rather than simply hi? It feels unintuitive, but there must be some reason for it that I'm missing, and I'm curious what that might be.
Thanks in advance.
This is completely a matter of convention, and there are certainly other ways to do it (for example, Matlab uses arrays whose first index is 1). The choice really comes down to what properties you want. As it turns out, using 0-indexed arrays where slicing is inclusive-exclusive (that is, a slice from a to b includes element a and excludes element b) has some really nice properties, and thus it's a very common choice. Here are a few advantages.
Advantages of 0-indexed arrays and inclusive-exclusive slicing
(note that I'm using non-Go terminology, so I'll talk about arrays in the way that C or Java would talk about them. Arrays are what Go calls slices, and slices are sub-arrays (ie, "the slice from index 1 to index 4"))
Pointer arithmetic works. If you're in a language like C, arrays are really just pointers to the first element in the array. Thus, if you use 0-indexed arrays, then you can say that the element at index i is just the element pointed at by the array pointer plus i. For example, if we have the array [3 2 1] with the address of the array being 10 (and assuming that each value takes up one byte of memory), then the address of the first element is 10 + 0 = 10, the address of the second is 10 + 1 = 11, and so on. In short, it makes the math simple.
The length of a slice is also the place to slice it. That is, for an array arr, arr[0:len(arr)] is just arr itself. This comes in handy a lot in practice. For example, if I call n, _ := r.Read(arr) (where n is the number of bytes read into arr), then I can just do arr[:n] to get the slice of arr corresponding to the data that was actually written into arr.
Indices don't overlap. This means that if I have arr[0:i], arr[i:j], arr[j:k], arr[k:len(arr)], these slices fully cover arr itself. You may not often find yourself partitioning an array into sub-slices like this, but it has a number of related advantages. For example, consider the following code to split an array based on non-consecutive integers:
func consecutiveSlices(ints []int) [][]int {
ret := make([][]int, 0)
i, j := 0, 1
for j < len(ints) {
if ints[j] != ints[j-1] + 1 {
ret = append(ret, ints[i:j])
i = j
}
}
ret = append(ret, ints[i:j])
}
(this code obviously doesn't handle some edge cases well, but you get the idea)
If we were to try to write the equivalent function using inclusive-inclusive slicing, it would be significantly more complicated.
If anyone can think of any more, please feel free to edit this answer and add them.
The Go Programming Language Specification
Slice types
Slice expressions
For a string, array, pointer to array, or slice a, the primary
expression
a[low : high]
constructs a substring or slice. The indices low and high select which
elements of operand a appear in the result. The result has indices
starting at 0 and length equal to high - low.
For convenience, any of the indices may be omitted. A missing low
index defaults to zero; a missing high index defaults to the length of
the sliced operand
For arrays or strings, the indices are in range if 0 <= low <= high <=
len(a), otherwise they are out of range. For slices, the upper index
bound is the slice capacity cap(a) rather than the length. A constant
index must be non-negative and representable by a value of type int;
for arrays or constant strings, constant indices must also be in
range. If both indices are constant, they must satisfy low <= high. If
the indices are out of range at run time, a run-time panic occurs.
For q := p[m:n], q is a slice of p starting at index m for a length of n-m elements.
I thought that in GO language, slices are passed by reference. But why the following code doesn't change the content of slice c? Am I missing something? Thank you.
package main
import (
"fmt"
)
func call(c []int) {
c = append(c, 1)
fmt.Println(c)
}
func main() {
c := make([]int, 1, 5)
fmt.Println(c)
call(c)
fmt.Println(c)
}
The result printed is:
[0]
[0 1]
[0]
while I was expecting
[0]
[0 1]
[0 1]
The length of the slice is kept in the slice header which is not passed by reference. You can think of a slice as a struct containing a pointer to the array, a length, and a capacity.
When you appended to the slice, you modified index 1 in the data array and then incremented the length in the slice header. When you returned, c in the main function had a length of 1 and so printed the same data.
The reason slices work this way is so you can have multiple slices pointing to the same data. For example:
x := []int{1,2,3}
y := x[:2] // [1 2]
z := x[1:] // [2 3]
All three of those slices point to overlapping data in the same underlying array.
Go is always pass by value. Certain types are reference types, like pointers, maps, channels; or partially reference types, like slices (which consists of a reference to the underlying array and also the values of the length and capacity). But regardless of type everything is passed by value. Thus assigning to a local variable never affects anything outside.