I can't seem to wrap my head around the notation of indices when working with Go slices.
Given a slice s.
s := []int{1, 2, 3, 4, 5}
I now want to create a new slice s2 = [2 3].
s2 := s[1:3] // s2 = [2 3]
Now, what is the thought process that I should go through when accessing this value? Am I reading values starting from index 1 up to and including the third element of the slice? Or am I reading values from index 1 up to and excluding index 3?
I am not starting at index 1 and going up to index 3 and neither am I starting at position 1 and going up to position 3 as both of these would result in s2 having 3 elements.
What is the idea behind this notation?
Relevant section from the spec: Slice expressions.
For a string, array, pointer to array, or slice a, the primary expression
a[low : high]
constructs a substring or slice. The indices low and high select which elements of operand a appear in the result. The result has indices starting at 0 and length equal to high - low.
So s2 := s[1:3] creates a new slice with length 3 - 1 = 2, so it will contain 2 elements: s[1] and s[2].
When slicing a slice, low should be the index of the first element you want to include (inclusive), and high should be the index of the last element that will not be incuded (high is exclusive).
So if you want the result to include the elements [2, 3], you need to provide slicing indices 1 and 3:
s2 := s[1:3] // will be [2, 3]
What might be confusing is that the elements in your slice start with 1, but the index starts with 0.
For reasoning behind the inclusive-exclusive indices, see related question: In a Go slice, why does s[lo:hi] end at element hi-1?
Related
For example:
func test(requiredIp []int, i int) []int {
requiredIp = append(requiredIp[0:i], requiredIp[i+1:]...)
return requiredIp
}
func main(){
requiredIp := []int{1,2,4,5,6}
fmt.Println(test(requiredIp,0)) // output:[2 4 5 6]
fmt.Println("original", requiredIp) // output:[2 4 5 6,6]
}
Why original slice has two 6 ?
A slice is a struct type with three fields:
A pointer to (an address of) the underlying array holding the data.
The length — how many elements are there in the slice.
The capacity — how many elements it's possible to store into the underlying array without reallocating it.
As everything in Go, slices are passed by value. This happens when you assign a slice value to a variable or when you pass it as a parameter in a function/method call.
What is pased by value (that is, copied) is that structure with three fields.
The pointer which is copied, obviously points at the same data block as the one in the original slice.
What happens in your code is the following:
The original slice is []int{1,2,4,5,6}.
It has length and capacity equal to 5.
It is passed to a functon, test.
The slice available there via the function's parameter requiredIp is initially identical to the one which is passed in a call.
You reslice that slice by evaluating requiredIp[0:i], and since in your call i equals to 0, you evaluate requiredIp[0:0].
That expression creates a slice with the backing array and the capacity of the original slice and length 0.
You then reslice the original slice once again — with the expression requiredIp[i+1:], which, in your call is requiredIp[1:].
The result shares the backing array with the original, has the capacity 4 and the contents []int{2,4,5,6}.
You then append the slice obtained on the previous step to the one obtained in the step before the previous one. That's where it gets interesting.
Consider that the slice being appended to points at the 0th element of the original slice and has length 0 and capacity 5. It means it has room for 5 elements.
That is, the slice's backing array still holds [1, 2, 4, 5, 6].
The slice being appended shares the same backing array with the slice being appended to, just it points at the 1st element, not the 0th.
The code of append sees it's told to append 4 elements and checks to see whether the target slice has enough capacity to hold them, and it has.
So append merely copies [2, 4, 5, 6] from and to the same backing array, overwriting 4 elements starting from index 0 in it. Effectively, it's elements [2, 4, 5, 6] moved one element left.
The resulting backing array now contains [2, 4, 5, 6, 6]: the first 4 element are the 4 elements which were the last, moved by one element left, overwriting what there were at indices 0 through 3.
You now return the reslting slice to the caller. That new slice value shares the backing array with all the slice values involved in the example, but remember that it has its length set to 4 — because the append appended 4 elements to the slice of length 0.
You print the original slice and the one returned from your function. They differ only in their length: the original has it equals to 5, and the returned from the function — to 4. The rest of their fields are the same: they both have capacity 5 and share the same backing array by both pointing its 0th element.
The difference in the lengths explains why "the original" slice seemingly has "extra" 6 in it. In fact in has it exactly where it were left off before test was called.
If, for some reason you really wanted to "detach" the slice produced by test from the original one by forcing it to allocate the new backing array, there are several possibilities:
Merely append to an unallocated slice — for instance,
append([]int(nil), 1, 2, 3, 4) would allocate a fresh backing array.
In your particular case that would not really be a solution as with i > 0 the append would have to operate on a non-empty slice. This can be dealt, for exmaple, with two appends:
s := append([]int(nil), input[0:i]...)
s = append(s, input[i+1:]...)
…or with allocating a new slice and copying:
s := make([]int, len(input)-1)
copy(s, input[0:i])
copy(s[i:], input[i+1:])
When reslicing the original slice, be sure to also artifically reset its capacity:
return append(input[0:i:i], input[i+1:]...)
Here, the first slice would have the same capacity as its length, i, and appending even a single element to it would force the append to allocate a new backing array, copy over these i elements to it and then copy over what is being appended.
Further reading:
https://blog.golang.org/slices-intro
https://blog.golang.org/slices
https://blog.golang.org/strings
…and actually consider starting with Effective Go.
This question already has answers here:
Why does append modify passed slice
(5 answers)
Closed 10 months ago.
When running the following program:
package main
import "fmt"
func main() {
edges := [][]int{{1,2}, {2,3}, {3,4}, {1,4}, {1,5}}
printSlice2d(edges)
_ = append(edges[:0], edges[1:]...)
printSlice2d(edges)
}
func printSlice2d(s [][]int) {
fmt.Printf("len=%d cap=%d %v\n", len(s), cap(s), s)
}
I get the following output:
len=5 cap=5 [[1 2] [2 3] [3 4] [1 4] [1 5]]
len=5 cap=5 [[2 3] [3 4] [1 4] [1 5] [1 5]]
I don't understand why edges is being changed by calling append. I would expect the first and second lines to be the same. I've checked the specification, but I can't seem to find anything that would explain this behaviour this kind of input.
Slice uses array internally to store the data.
append function does not create a new array if it is not necessary to increase array size beyond its current capacity. Instead it copies new data elements into existing array. After that function returns reference to a new slice that internally uses the same array.
You can read more in this article - https://blog.golang.org/go-slices-usage-and-internals
As mentioned earlier, re-slicing a slice doesn't make a copy of the
underlying array. The full array will be kept in memory until it is no
longer referenced. Occasionally this can cause the program to hold all
the data in memory when only a small piece of it is needed.
This is what is going on in this line:
_ = append(edges[:0], edges[1:]...)
second append function argument (edges[1:]...) copies 4 items of edges into temp var. Its value - [{2,3}, {3,4}, {1,4}, {1,5}]
these values are copied into array that edges uses internally to store data. That overrides all items except of last one. This is where edges is mutated.
append returns reference to a new slice that internally uses the same array to store the data as edges
returned slice is ignored and will be garbage collected, but that does not matter for edges.
That is why you see changed values when you check edges after performing append on it.
edges[:0] is a slice of length 0 starting at index 0 of the underlying array of edges.
To this slice, you append another slice, the slice of length 4 starting at index of the underlying array ofedges`. That gives you the first 4 elements of the result you see in the second line.
Then you print edges, which is a slice with an underlying array of 5 elements, whose last 4 you just shifted one element lower. The last element is duplicated. in the edges array.
If you look at the result of the append, then you'd see a slice with length 4, cap 5, the first 4 elements of the underlying array of edges.
If you expected the two lines to be the same, maybe you tried to do:
append(edges[:1],edges[1:]...)
I'm taking the tour on Golang site, and I'm trying to digest one of the examples. It is unclear how it works:
package main
import "fmt"
func main() {
s := []int{2, 3, 5, 7, 11, 13}
printSlice(s)
// Slice the slice to give it zero length.
s = s[:0]
printSlice(s)
// Extend its length.
s = s[:4]
printSlice(s)
// Drop its first two values.
s = s[2:]
printSlice(s)
}
func printSlice(s []int) {
fmt.Printf("len=%d cap=%d %v\n", len(s), cap(s), s)
}
The output is:
len=6 cap=6 [2 3 5 7 11 13]
len=0 cap=6 []
len=4 cap=6 [2 3 5 7]
len=2 cap=4 [5 7]
After the first slice, s = s[:0] the slice length is 0. Then there is another slicing of s = s[:4]. Although the length is 0, this seems to work. But how this happens? Shouldn't the underlaying array be in accessible from s?
What confuses me more is, the next time we slice it, s = s[2:] we slice the old value of s (which is 4 elements) and not the original array.
Can someone shed some lights what is the difference between the two cases?
A slice is basically a pointer to memory with some additional information:
1) the number of elements currently used and
2) the capacity, i.e. the remaining length it can occupy.
At the start we create a slice with 6 integers, this makes go create the underlying int array with a total size of 6 as well.
here is your memory locations with addresses (content does not matter here)
* * * * * *
[0][1][2][3][4][5]
^
s points to the start of the memory
len(s) = 6
cap(s) = 6
Next we say: make this slice's len be 0, this is the s = s[:0] which takes a sub-slice of s at position 0 with length 0. Note that s[0:0] is the same, you can omit the first 0.
[0][1][2][3][4][5]
^
s still points to the start of the memory
len(s) = 0
cap(s) = 6
Since the capacity is still the same, we might as well make the length 4 by saying s = s[:4].
* * * *
[0][1][2][3][4][5]
^
s still points to the start of the memory
len(s) = 4
cap(s) = 6
Then we take a sub-slice that does not start at the beginning of the memory by doing s = s[2:].
* *
[0][1][2][3][4][5]
^
s now points to the original address plus two!
len(s) = 2
cap(s) = 4
Leon addressed me to the Go's blog post, where they address exactly my question.
This is the snippet which helped me better understanding this concept:
A slice is a descriptor of an array segment. It consists of a pointer to the array, the length of the segment, and its capacity (the maximum length of the segment).
A slice cannot be grown beyond its capacity. Attempting to do so will cause a runtime panic, just as when indexing outside the bounds of a slice or array. Similarly, slices cannot be re-sliced below zero to access earlier elements in the array.
Slices can be extended if the array has more elements in it, but it can not access elements below 0 of the slice. It's a window to the underlaying array. The blog post explains it in more depth.
I read go slice usage and internals and Slice and Effective go#slice but there is nothing about slicing a slice with 3 number like this : slice[a:b:c]
For example this code :
package main
import "fmt"
func main() {
var s = []string{"a", "b", "c", "d", "e", "f", "g"}
fmt.Println(s[1:2:6], len(s[1:2:6]), cap(s[1:2:6]))
fmt.Println(s[1:2:5], len(s[1:2:5]), cap(s[1:2:5]))
fmt.Println(s[1:2], len(s[1:2]), cap(s[1:2]))
}
go playground result is this :
[b] 1 5
[b] 1 4
[b] 1 6
I can understand that the third one is something about capacity, but what is the exact meaning of this?
Do I miss something in documents?
The syntax has been introduced in Go 1.2, as I mentioned in "Re-slicing slices in Golang".
It is documented in Full slice expressions:
a[low : high : max]
constructs a slice of the same type, and with the same length and elements as the simple slice expression a[low : high].
Additionally, it controls the resulting slice's capacity by setting it to max - low.
Only the first index may be omitted; it defaults to 0.
After slicing the array a:
a := [5]int{1, 2, 3, 4, 5}
t := a[1:3:5]
the slice t has type []int, length 2, capacity 4, and elements
t[0] == 2
t[1] == 3
The design document for that feature had the following justification:
It would occasionally be useful, for example in custom []byte allocation managers, to be able to hand a slice to a caller and know that the caller cannot edit values beyond a given subrange of the true array.
The addition of append to the language made this somewhat more important, because append lets programmers overwrite entries between len and cap without realizing it or even mentioning cap.
2022: svitanok adds for Go 1.19+:
while the capacity of a "derivative" slice doesn't exceed the one specified by the third index during its creation the slice is still "from" the same spot in the memory as its original ("true") slice, so the changes applied to it will affect the original slice.
And if then, for example, you append to this derivative slice the amount of elements that would cause its capacity to be increased, this new slice will occupy a different place in the memory, and so the changes made to it will not affect the slice it originated from.
In a slice expression slice[a:b:c] or aSlice[1:3:5]
a:b or 1:3 -> gives length
a:c or 1:5 -> gives capacity
We can extract both length and capacity from a slice expression with 3 numbers/indices, without looking at the source slice/array.
expression| aSlice[low:high:max] or aSlice[a:b:c] or aSlice[1:3:7]
------------------------------------------------------------------------
Length | len(aSlice[low:high]) or len(aSlice[a:b]) or len(aSlice[1:3])
Capacity | len(aSlice[low:max]) or len(aSlice[a:c]) or len(aSlice[1:7])
------------------------------------------------------------------------
Read more here at Slice Expressions
Playground
Actually Go slice have a pointer and pointing to the array and it holds length and capacity of the array and we can show it like will be
pointer:length:capacity
and append is used for adding same new length.
sl1 := make([]int, 6)
fmt.Println(sl1)
sl2 := append(sl1, 1)
fmt.Println(sl2)
[0 0 0 0 0 0]
[0 0 0 0 0 0 1]
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
According to the Tour of Go, in a Go slice s, the expression s[lo:hi] evaluates to a slice of the elements from lo through hi-1, inclusive:
package main
import "fmt"
func main() {
p := []int{0, // slice position 0
10, // slice position 1
20, // slice position 2
30, // slice position 3
40, // slice position 4
50} // slice position 5
fmt.Println(p[0:3]) // => [0 10 20]
}
In my code example above, "p[0:3]" would seem to intuitively "read" as: "the slice from position 0 to position 3", equating to [0, 10, 20, 30]. But of course, it actually equates to [0 10 20].
So my question is: what is the design rationale for the upper value evaluating to hi-1 rather than simply hi? It feels unintuitive, but there must be some reason for it that I'm missing, and I'm curious what that might be.
Thanks in advance.
This is completely a matter of convention, and there are certainly other ways to do it (for example, Matlab uses arrays whose first index is 1). The choice really comes down to what properties you want. As it turns out, using 0-indexed arrays where slicing is inclusive-exclusive (that is, a slice from a to b includes element a and excludes element b) has some really nice properties, and thus it's a very common choice. Here are a few advantages.
Advantages of 0-indexed arrays and inclusive-exclusive slicing
(note that I'm using non-Go terminology, so I'll talk about arrays in the way that C or Java would talk about them. Arrays are what Go calls slices, and slices are sub-arrays (ie, "the slice from index 1 to index 4"))
Pointer arithmetic works. If you're in a language like C, arrays are really just pointers to the first element in the array. Thus, if you use 0-indexed arrays, then you can say that the element at index i is just the element pointed at by the array pointer plus i. For example, if we have the array [3 2 1] with the address of the array being 10 (and assuming that each value takes up one byte of memory), then the address of the first element is 10 + 0 = 10, the address of the second is 10 + 1 = 11, and so on. In short, it makes the math simple.
The length of a slice is also the place to slice it. That is, for an array arr, arr[0:len(arr)] is just arr itself. This comes in handy a lot in practice. For example, if I call n, _ := r.Read(arr) (where n is the number of bytes read into arr), then I can just do arr[:n] to get the slice of arr corresponding to the data that was actually written into arr.
Indices don't overlap. This means that if I have arr[0:i], arr[i:j], arr[j:k], arr[k:len(arr)], these slices fully cover arr itself. You may not often find yourself partitioning an array into sub-slices like this, but it has a number of related advantages. For example, consider the following code to split an array based on non-consecutive integers:
func consecutiveSlices(ints []int) [][]int {
ret := make([][]int, 0)
i, j := 0, 1
for j < len(ints) {
if ints[j] != ints[j-1] + 1 {
ret = append(ret, ints[i:j])
i = j
}
}
ret = append(ret, ints[i:j])
}
(this code obviously doesn't handle some edge cases well, but you get the idea)
If we were to try to write the equivalent function using inclusive-inclusive slicing, it would be significantly more complicated.
If anyone can think of any more, please feel free to edit this answer and add them.
The Go Programming Language Specification
Slice types
Slice expressions
For a string, array, pointer to array, or slice a, the primary
expression
a[low : high]
constructs a substring or slice. The indices low and high select which
elements of operand a appear in the result. The result has indices
starting at 0 and length equal to high - low.
For convenience, any of the indices may be omitted. A missing low
index defaults to zero; a missing high index defaults to the length of
the sliced operand
For arrays or strings, the indices are in range if 0 <= low <= high <=
len(a), otherwise they are out of range. For slices, the upper index
bound is the slice capacity cap(a) rather than the length. A constant
index must be non-negative and representable by a value of type int;
for arrays or constant strings, constant indices must also be in
range. If both indices are constant, they must satisfy low <= high. If
the indices are out of range at run time, a run-time panic occurs.
For q := p[m:n], q is a slice of p starting at index m for a length of n-m elements.