I'm taking the tour on Golang site, and I'm trying to digest one of the examples. It is unclear how it works:
package main
import "fmt"
func main() {
s := []int{2, 3, 5, 7, 11, 13}
printSlice(s)
// Slice the slice to give it zero length.
s = s[:0]
printSlice(s)
// Extend its length.
s = s[:4]
printSlice(s)
// Drop its first two values.
s = s[2:]
printSlice(s)
}
func printSlice(s []int) {
fmt.Printf("len=%d cap=%d %v\n", len(s), cap(s), s)
}
The output is:
len=6 cap=6 [2 3 5 7 11 13]
len=0 cap=6 []
len=4 cap=6 [2 3 5 7]
len=2 cap=4 [5 7]
After the first slice, s = s[:0] the slice length is 0. Then there is another slicing of s = s[:4]. Although the length is 0, this seems to work. But how this happens? Shouldn't the underlaying array be in accessible from s?
What confuses me more is, the next time we slice it, s = s[2:] we slice the old value of s (which is 4 elements) and not the original array.
Can someone shed some lights what is the difference between the two cases?
A slice is basically a pointer to memory with some additional information:
1) the number of elements currently used and
2) the capacity, i.e. the remaining length it can occupy.
At the start we create a slice with 6 integers, this makes go create the underlying int array with a total size of 6 as well.
here is your memory locations with addresses (content does not matter here)
* * * * * *
[0][1][2][3][4][5]
^
s points to the start of the memory
len(s) = 6
cap(s) = 6
Next we say: make this slice's len be 0, this is the s = s[:0] which takes a sub-slice of s at position 0 with length 0. Note that s[0:0] is the same, you can omit the first 0.
[0][1][2][3][4][5]
^
s still points to the start of the memory
len(s) = 0
cap(s) = 6
Since the capacity is still the same, we might as well make the length 4 by saying s = s[:4].
* * * *
[0][1][2][3][4][5]
^
s still points to the start of the memory
len(s) = 4
cap(s) = 6
Then we take a sub-slice that does not start at the beginning of the memory by doing s = s[2:].
* *
[0][1][2][3][4][5]
^
s now points to the original address plus two!
len(s) = 2
cap(s) = 4
Leon addressed me to the Go's blog post, where they address exactly my question.
This is the snippet which helped me better understanding this concept:
A slice is a descriptor of an array segment. It consists of a pointer to the array, the length of the segment, and its capacity (the maximum length of the segment).
A slice cannot be grown beyond its capacity. Attempting to do so will cause a runtime panic, just as when indexing outside the bounds of a slice or array. Similarly, slices cannot be re-sliced below zero to access earlier elements in the array.
Slices can be extended if the array has more elements in it, but it can not access elements below 0 of the slice. It's a window to the underlaying array. The blog post explains it in more depth.
Related
I am learning Go by going through A Tour of Go. One of the exercises there asks me to create a 2D slice of dy rows and dx columns containing uint8. My current approach, which works, is this:
a:= make([][]uint8, dy) // initialize a slice of dy slices
for i:=0;i<dy;i++ {
a[i] = make([]uint8, dx) // initialize a slice of dx unit8 in each of dy slices
}
I think that iterating through each slice to initialize it is too verbose. And if the slice had more dimensions, the code would become unwieldy. Is there a concise way to initialize 2D (or n-dimensional) slices in Go?
There isn't a more concise way, what you did is the "right" way; because slices are always one-dimensional but may be composed to construct higher-dimensional objects. See this question for more details: Go: How is two dimensional array's memory representation.
One thing you can simplify on it is to use the for range construct:
a := make([][]uint8, dy)
for i := range a {
a[i] = make([]uint8, dx)
}
Also note that if you initialize your slice with a composite literal, you get this for "free", for example:
a := [][]uint8{
{0, 1, 2, 3},
{4, 5, 6, 7},
}
fmt.Println(a) // Output is [[0 1 2 3] [4 5 6 7]]
Yes, this has its limits as seemingly you have to enumerate all the elements; but there are some tricks, namely you don't have to enumerate all values, only the ones that are not the zero values of the element type of the slice. For more details about this, see Keyed items in golang array initialization.
For example if you want a slice where the first 10 elements are zeros, and then follows 1 and 2, it can be created like this:
b := []uint{10: 1, 2}
fmt.Println(b) // Prints [0 0 0 0 0 0 0 0 0 0 1 2]
Also note that if you'd use arrays instead of slices, it can be created very easily:
c := [5][5]uint8{}
fmt.Println(c)
Output is:
[[0 0 0 0 0] [0 0 0 0 0] [0 0 0 0 0] [0 0 0 0 0] [0 0 0 0 0]]
In case of arrays you don't have to iterate over the "outer" array and initialize "inner" arrays, as arrays are not descriptors but values. See blog post Arrays, slices (and strings): The mechanics of 'append' for more details.
Try the examples on the Go Playground.
There are two ways to use slices to create a matrix. Let's take a look at the differences between them.
First method:
matrix := make([][]int, n)
for i := 0; i < n; i++ {
matrix[i] = make([]int, m)
}
Second method:
matrix := make([][]int, n)
rows := make([]int, n*m)
for i := 0; i < n; i++ {
matrix[i] = rows[i*m : (i+1)*m]
}
In regards to the first method, making successive make calls doesn't ensure that you will end up with a contiguous matrix, so you may have the matrix divided in memory. Let's think of an example with two Go routines that could cause this:
The routine #0 runs make([][]int, n) to get allocated memory for matrix, getting a piece of memory from 0x000 to 0x07F.
Then, it starts the loop and does the first row make([]int, m), getting from 0x080 to 0x0FF.
In the second iteration it gets preempted by the scheduler.
The scheduler gives the processor to routine #1 and it starts running. This one also uses make (for its own purposes) and gets from 0x100 to 0x17F (right next to the first row of routine #0).
After a while, it gets preempted and routine #0 starts running again.
It does the make([]int, m) corresponding to the second loop iteration and gets from 0x180 to 0x1FF for the second row. At this point, we already got two divided rows.
With the second method, the routine does make([]int, n*m) to get all the matrix allocated in a single slice, ensuring contiguity. After that, a loop is needed to update the matrix pointers to the subslices corresponding to each row.
You can play with the code shown above in the Go Playground to see the difference in the memory assigned by using both methods. Note that I used runtime.Gosched() only with the purpose of yielding the processor and forcing the scheduler to switch to another routine.
Which one to use? Imagine the worst case with the first method, i.e. each row is not next in memory to another row. Then, if your program iterates through the matrix elements (to read or write them), there will probably be more cache misses (hence higher latency) compared to the second method because of worse data locality. On the other hand, with the second method it may not be possible to get a single piece of memory allocated for the matrix, because of memory fragmentation (chunks spread all over the memory), even though theoretically there may be enough free memory for it.
Therefore, unless there's a lot of memory fragmentation and the matrix to be allocated is huge enough, you would always want to use the second method to get advantage of data locality.
With Go 1.18 you get generics.
Here is a function that uses generics to allow to create a 2D slice for any cell type.
func Make2D[T any](n, m int) [][]T {
matrix := make([][]T, n)
rows := make([]T, n*m)
for i, startRow := 0, 0; i < n; i, startRow = i+1, startRow+m {
endRow := startRow + m
matrix[i] = rows[startRow:endRow:endRow]
}
return matrix
}
With that function in your toolbox, your code becomes:
a := Make2D[uint8](dy, dx)
You can play with the code on the Go Playground.
Here a consive way to do it:
value := [][]string{}{[]string{}{"A1","A2"}, []string{}{"B1", "B2"}}
PS.: you can change "string" to the type of element you're using in your slice.
The capacity parameter in making a slice in Go does not make much sense to me. For example,
aSlice := make([]int, 2, 2) //a new slice with length and cap both set to 2
aSlice = append(aSlice, 1, 2, 3, 4, 5) //append integers 1 through 5
fmt.Println("aSlice is: ", aSlice) //output [0, 0, 1, 2, 3, 4, 5]
If the slice allows inserting more elements than the capacity allows, why do we need to set it in the make() function?
The builtin append() function uses the specified slice to append elements to if it has a big enough capacity to accomodate the specified elements.
But if the passed slice is not big enough, it allocates a new, big enough slice, copies the elements from the passed slice to the new slice and append the elements to that new slice. And returns this new slice. Quoting from the append() documentation:
The append built-in function appends elements to the end of a slice. If it has sufficient capacity, the destination is resliced to accommodate the new elements. If it does not, a new underlying array will be allocated. Append returns the updated slice. It is therefore necessary to store the result of append, often in the variable holding the slice itself:
When making a slice with make if the length and capacity are the same, the capacity can be omitted, in which case it is defaulted to the specified length:
// These 2 declarations are equivalent:
s := make([]int, 2, 2)
s := make([]int, 2)
Also note that append() appends elements after the last element of the slice. And the above slices already have len(s) == 2 right after declaration so if you append even just 1 element to it, it will cause a reallocation as seen in this example:
s := make([]int, 2, 2)
fmt.Println(s, len(s), cap(s))
s = append(s, 1)
fmt.Println(s, len(s), cap(s))
Output:
[0 0] 2 2
[0 0 1] 3 4
So in your example what you should do is something like this:
s := make([]int, 0, 10) // Create a slice with length=0 and capacity=10
fmt.Println(s, len(s), cap(s))
s = append(s, 1)
fmt.Println(s, len(s), cap(s))
Output:
[] 0 10
[1] 1 10
I recommend the following blog articles if you want to understand slices in more details:
Go Slices: usage and internals
Arrays, slices (and strings): The mechanics of 'append'
It is mainly an optimization, and it is not unique to go, similar structures in other languages have this as well.
When you append more than the capacity, the runtime needs to allocate more memory for the new elements. This is costly and can also cause memory fragmentation.
By specifying the capacity, the runtime allocates what is needed in advance, and avoids reallocations. However if you do not know the estimated capacity in advance or it changes, you do not have to set it, and the runtime reallocates what is needed and grows the capacity itself.
I read go slice usage and internals and Slice and Effective go#slice but there is nothing about slicing a slice with 3 number like this : slice[a:b:c]
For example this code :
package main
import "fmt"
func main() {
var s = []string{"a", "b", "c", "d", "e", "f", "g"}
fmt.Println(s[1:2:6], len(s[1:2:6]), cap(s[1:2:6]))
fmt.Println(s[1:2:5], len(s[1:2:5]), cap(s[1:2:5]))
fmt.Println(s[1:2], len(s[1:2]), cap(s[1:2]))
}
go playground result is this :
[b] 1 5
[b] 1 4
[b] 1 6
I can understand that the third one is something about capacity, but what is the exact meaning of this?
Do I miss something in documents?
The syntax has been introduced in Go 1.2, as I mentioned in "Re-slicing slices in Golang".
It is documented in Full slice expressions:
a[low : high : max]
constructs a slice of the same type, and with the same length and elements as the simple slice expression a[low : high].
Additionally, it controls the resulting slice's capacity by setting it to max - low.
Only the first index may be omitted; it defaults to 0.
After slicing the array a:
a := [5]int{1, 2, 3, 4, 5}
t := a[1:3:5]
the slice t has type []int, length 2, capacity 4, and elements
t[0] == 2
t[1] == 3
The design document for that feature had the following justification:
It would occasionally be useful, for example in custom []byte allocation managers, to be able to hand a slice to a caller and know that the caller cannot edit values beyond a given subrange of the true array.
The addition of append to the language made this somewhat more important, because append lets programmers overwrite entries between len and cap without realizing it or even mentioning cap.
2022: svitanok adds for Go 1.19+:
while the capacity of a "derivative" slice doesn't exceed the one specified by the third index during its creation the slice is still "from" the same spot in the memory as its original ("true") slice, so the changes applied to it will affect the original slice.
And if then, for example, you append to this derivative slice the amount of elements that would cause its capacity to be increased, this new slice will occupy a different place in the memory, and so the changes made to it will not affect the slice it originated from.
In a slice expression slice[a:b:c] or aSlice[1:3:5]
a:b or 1:3 -> gives length
a:c or 1:5 -> gives capacity
We can extract both length and capacity from a slice expression with 3 numbers/indices, without looking at the source slice/array.
expression| aSlice[low:high:max] or aSlice[a:b:c] or aSlice[1:3:7]
------------------------------------------------------------------------
Length | len(aSlice[low:high]) or len(aSlice[a:b]) or len(aSlice[1:3])
Capacity | len(aSlice[low:max]) or len(aSlice[a:c]) or len(aSlice[1:7])
------------------------------------------------------------------------
Read more here at Slice Expressions
Playground
Actually Go slice have a pointer and pointing to the array and it holds length and capacity of the array and we can show it like will be
pointer:length:capacity
and append is used for adding same new length.
sl1 := make([]int, 6)
fmt.Println(sl1)
sl2 := append(sl1, 1)
fmt.Println(sl2)
[0 0 0 0 0 0]
[0 0 0 0 0 0 1]
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
According to the Tour of Go, in a Go slice s, the expression s[lo:hi] evaluates to a slice of the elements from lo through hi-1, inclusive:
package main
import "fmt"
func main() {
p := []int{0, // slice position 0
10, // slice position 1
20, // slice position 2
30, // slice position 3
40, // slice position 4
50} // slice position 5
fmt.Println(p[0:3]) // => [0 10 20]
}
In my code example above, "p[0:3]" would seem to intuitively "read" as: "the slice from position 0 to position 3", equating to [0, 10, 20, 30]. But of course, it actually equates to [0 10 20].
So my question is: what is the design rationale for the upper value evaluating to hi-1 rather than simply hi? It feels unintuitive, but there must be some reason for it that I'm missing, and I'm curious what that might be.
Thanks in advance.
This is completely a matter of convention, and there are certainly other ways to do it (for example, Matlab uses arrays whose first index is 1). The choice really comes down to what properties you want. As it turns out, using 0-indexed arrays where slicing is inclusive-exclusive (that is, a slice from a to b includes element a and excludes element b) has some really nice properties, and thus it's a very common choice. Here are a few advantages.
Advantages of 0-indexed arrays and inclusive-exclusive slicing
(note that I'm using non-Go terminology, so I'll talk about arrays in the way that C or Java would talk about them. Arrays are what Go calls slices, and slices are sub-arrays (ie, "the slice from index 1 to index 4"))
Pointer arithmetic works. If you're in a language like C, arrays are really just pointers to the first element in the array. Thus, if you use 0-indexed arrays, then you can say that the element at index i is just the element pointed at by the array pointer plus i. For example, if we have the array [3 2 1] with the address of the array being 10 (and assuming that each value takes up one byte of memory), then the address of the first element is 10 + 0 = 10, the address of the second is 10 + 1 = 11, and so on. In short, it makes the math simple.
The length of a slice is also the place to slice it. That is, for an array arr, arr[0:len(arr)] is just arr itself. This comes in handy a lot in practice. For example, if I call n, _ := r.Read(arr) (where n is the number of bytes read into arr), then I can just do arr[:n] to get the slice of arr corresponding to the data that was actually written into arr.
Indices don't overlap. This means that if I have arr[0:i], arr[i:j], arr[j:k], arr[k:len(arr)], these slices fully cover arr itself. You may not often find yourself partitioning an array into sub-slices like this, but it has a number of related advantages. For example, consider the following code to split an array based on non-consecutive integers:
func consecutiveSlices(ints []int) [][]int {
ret := make([][]int, 0)
i, j := 0, 1
for j < len(ints) {
if ints[j] != ints[j-1] + 1 {
ret = append(ret, ints[i:j])
i = j
}
}
ret = append(ret, ints[i:j])
}
(this code obviously doesn't handle some edge cases well, but you get the idea)
If we were to try to write the equivalent function using inclusive-inclusive slicing, it would be significantly more complicated.
If anyone can think of any more, please feel free to edit this answer and add them.
The Go Programming Language Specification
Slice types
Slice expressions
For a string, array, pointer to array, or slice a, the primary
expression
a[low : high]
constructs a substring or slice. The indices low and high select which
elements of operand a appear in the result. The result has indices
starting at 0 and length equal to high - low.
For convenience, any of the indices may be omitted. A missing low
index defaults to zero; a missing high index defaults to the length of
the sliced operand
For arrays or strings, the indices are in range if 0 <= low <= high <=
len(a), otherwise they are out of range. For slices, the upper index
bound is the slice capacity cap(a) rather than the length. A constant
index must be non-negative and representable by a value of type int;
for arrays or constant strings, constant indices must also be in
range. If both indices are constant, they must satisfy low <= high. If
the indices are out of range at run time, a run-time panic occurs.
For q := p[m:n], q is a slice of p starting at index m for a length of n-m elements.
I have an array and a slice pointing to it, like shown as follows:
package main
import "fmt"
func main() {
array_str := []string{"0a","1b","2c","3d","4e"}
slice_str:=array_str[1:4]
fmt.Println("Initially :")
fmt.Println("Printing 1 :Array :",array_str)
fmt.Println("Printing 1 :Slice:",slice_str)
//Step 1.Changing Slice and it get reflected in array
fmt.Println("\nAfter Alteration:")
slice_str[0]="alterd_1b"
fmt.Println("Printing 2 :Array :",array_str)
fmt.Println("Printing 2 :Slice:",slice_str)
fmt.Println("len of slice_str:",len(slice_str)," cap of slice_str:",cap(slice_str),"len of array_str:",len(array_str))
//Step 2.appending to slice and it get reflected
slice_str = append(slice_str,"apnded_elemnt")
fmt.Println("\nAfter Apending:")
fmt.Println("Printing 3 :Array :",array_str)//"4e" is replaced with "apnded_elemnt" in array !!
fmt.Println("Printing 3 :Slice:",slice_str)
fmt.Println("len of slice_str:",len(slice_str)," cap of slice_str:",cap(slice_str),"len of array_str:",len(array_str))
//Step 3.Again appending to slice so that lentght of slice is growing further to underlaying array
slice_str = append(slice_str,"outgrown_elemnt")
fmt.Println("\nAfter OUT GROWING:")
fmt.Println("Printing 4 :Array :",array_str)//obeviously out grown elemnt not added to array that is fine
fmt.Println("Printing 4 :Slice:",slice_str)
fmt.Println("len of slice_str:",len(slice_str)," cap of slice_str:",cap(slice_str),"len of array_str:",len(array_str))
//How Capacity Become 8 here ???
//Step 4 .Now Changing Slice element which is in Range of array to verify it reflect on array:
fmt.Println("\nAfter Post out grown Alteration:")
slice_str[0]="again_alterd_1b"
fmt.Println("Printing 2 :Array :",array_str)//Change in slice is not reflectd in array .Why ?
fmt.Println("Printing 2 :Slice:",slice_str)
}
Playground: http://play.golang.org/p/3z52HXHQ7s
Questions:
In Step 3: why does cap of the slice jumped from 4 to 8?
In Step 4: after the slice is out grown, changes to the element of the slice, which is in the range of the array, is not reflected to the array and vice versa. Why is it not happening after it is grown out? What actually happens when the slice grows out?
See here: http://blog.golang.org/slices
Short answers: 1) it grows by doubling (while short). If you append once you might append a second time too and this avoids allocations. 2) That's how slice growing works. An array cannot grow, so a new, larger array is allocated, the old one copied and you are handed a slice pointing to the larger copy.
(The documentation on the golang.org website is really helpful, readable, short and precise. I'd like to recommend to look at golang.org first before asking here.)
The capacity is multiplied by 2 because it is less consuming. Indeed, memory allocation is very consuming, and it's better to allocate a lot of memory a single time than exactly what is needed every time.
Let's just compare, first with a simple example using the concatenation: every time, Go allocates just what is needed.
var n = time.Now()
var s = ""
for i := 0; i < 1000000; i++ {
s += "a"
}
fmt.Println(time.Now().Sub(n))
// 47.7s
Now, let's do the same but this time using the bytes.Buffer type:
var n = time.Now()
var b = bytes.NewBufferString("")
for i := 0; i < 1000000; i++ {
b.WriteString("a")
}
fmt.Println(time.Now().Sub(n))
// 18.5ms
The difference is the way Buffer allocates memory: when there is not enough capacity, it allocates twice the current capacity:
buf = makeSlice(2*cap(b.buf) + n)
Source
This works the same with slices (I was just not able to find the source code to prove it...). So yes, you may be losing some space, but this is for a much better efficiency!
You're second question is a bit more tricky for me, so I hope #Volker's answer will be clear enough for you !