Array Cap overwritten - go

GO MASTERs! could you advise?
with make([]int,len,cap), does GO allow array initialization at the same time to declaration?
myArray := make([]int, 3, 10){1,2,3}
I see cap(myArray) has changed as same to len(myArray) in the below case. What is fundamental behind?
<ex>
myArray := make([]int, 3, 10)
myArray = []int{1,2,3}
fmt.Println("len:"len(myArray),", cap:",cap(myArray))
len:3, cap:3
Why cap(myArray) is 3 instead of 10 as I declaration?
I initialized individual element directly. And, it works as I wanted
<ex>
myArray := make([]int, 3, 10)
myArray[0] = 1
myArray[1] = 2
myArray[2] = 3
fmt.Println("len:"len(myArray),", cap:",cap(myArray))
len:3, cap:10

I believe you're getting this result because in the first example you declare a slice with a length of 3 and a cap of 10. However, in the following statement you assign a completely different slice to that variable. Since the slice your assigning was declared/initialized using 'composite-literal' syntax and it has only 3 items, that is the capacity it is initialized with. In your second example, you assign values to each index of the slice you created with make, retaining the length and capacity properties.
To try and give more clarification... In C# or Java or C++ this would be like doing something like this;
List<string> myStrings = new List<string>();
myStrings.Add("A string");
myStrings = new List<string>();
As you might assume, after assigning a new list to the myStrings variable the old list is lost. So to give the simple answer, you're not just overwriting the cap, you're overwriting the whole slice. The value you have in your assignment that overwrites it, causes a slice to be created with a cap of 3 which is why you get that result.

So there's a few things at play here. I would strongly recommend the goblog's explanation of this: https://blog.golang.org/slices (go down to the section labeled Capacity). Also, just a (tiny) bit of semantics, but those are actually slices, not arrays :)
To get to the point, when you first declared your slice, you had a len of 3 and a capacity of 10. No problem with that. But then you redeclare myArray, and set it to be []int{1,2,3}. No problems there either, this is valid.
The problem comes when you expected your previous len/cap to remain the same; your slice is just a pointer to some space in memory. So you've now changed your underlying pointer (see the goblog). The capacity and length of the slice is now changed as well (to the default-- again, see the goblog). You can see the same changing of len/cap if you were to add more elements:
myArray := make([]int, 3, 10)
fmt.Println("len:"len(myArray),", cap:",cap(myArray)) // len: 3 cap: 10
myArray := []int{1, 2, 3}
fmt.Println("len:", len(myArray), "cap:", cap(myArray)) // len: 3 cap: 3
myArray = append(myArray, 4)
fmt.Println("len:", len(myArray), "cap:", cap(myArray)) // len: 4 cap: 8; automatically increases the cap

Related

Slice copy mutating original slice

Could someone help explain the Golang internals of why this code is mutating the original array a?
func main() {
a := []int{1,2,3,4}
b := a
b = append(b[0:1], b[2:]...)
fmt.Println(b)
fmt.Println(a)
}
Output:
[1 3 4]
[1 3 4 4]
I thought b := a would be passing by value. Thanks in advance.
That's how slices work. A slice is just a pointer(+size+capacity), the actual data is stored in the array.
When you copy a slice, the underlying array is not copied. Then you end up with two slices pointing to the same array. Mutating the values of one slice will become visible via the other slice.
See Go Slices: usage and internals for more details.
If you want to leave the original slice untouched, first make a deep copy. For example like this
b := append([]int{}, a...) // deep copy
(Live demo)
Slices are basically wrapper over arrays. Slices doesn't have their own data they just hold the reference to the arrays. In your given code you are assigning a to b now they both are indicating the same array. And so when you are mutating the slice b the slice a is also being mutated.
You can use copy method to copy elements from one array to another.
// copy returns the count of total copied elements
count := copy(b /*destination*/ , a /*source*/)
But make sure to allocate an array with the same length of source array.
Example is given below:
func main() {
a := []int{1,2,3,4}
b := make([]int, len(a))
_ = copy(b, a)
a[0] = 2
fmt.Println(b)
fmt.Println(a)
}

Golang: Make function and third param

What is the difference between:
x := make([]int, 5, 10)
x := make([]int, 5)
x := [5]int{}
I know that make allocates an array and returns a slice that refers to that array. I don't understand where it can be used?
I can't find a good example that will clarify the situation.
x := make([]int, 5) Makes slice of int with length 5 and capacity 5 (same as length).
x := make([]int, 5, 10) Makes slice of int with length 5 and capacity 10.
x := [5]int{} Makes array of int with length 5.
Slices
If you need to append more items than capacity of slice using append function, go runtime will allocate new underlying array and copy existing one to it. So if you know about estimated length of your slice, better to use explicit capacity declaration. It will consume more memory for underlying array at the beginning, but safe cpu time for many allocations and array copying.
You can explore how len and cap changes while append, using that simple test on Go playground
Every time when cap value changed, new array allocated
Arrays
Array size is fixed, so if you need to grow array you have to create new one with new length and copy your old array into it by your own.
There are some great articles about slices and arrays in go:
http://blog.golang.org/go-slices-usage-and-internals
http://blog.golang.org/slices
The second line will allocate 10 int's worth memory at the very beginning, but returning you a slice of 5 int's. The second line does not stand less memory, it saves you another memory allocation if you need to expand the slice to anything not more than 10 * load_factor.

Slices in Go: why does it allow appending more than the capacity allows?

The capacity parameter in making a slice in Go does not make much sense to me. For example,
aSlice := make([]int, 2, 2) //a new slice with length and cap both set to 2
aSlice = append(aSlice, 1, 2, 3, 4, 5) //append integers 1 through 5
fmt.Println("aSlice is: ", aSlice) //output [0, 0, 1, 2, 3, 4, 5]
If the slice allows inserting more elements than the capacity allows, why do we need to set it in the make() function?
The builtin append() function uses the specified slice to append elements to if it has a big enough capacity to accomodate the specified elements.
But if the passed slice is not big enough, it allocates a new, big enough slice, copies the elements from the passed slice to the new slice and append the elements to that new slice. And returns this new slice. Quoting from the append() documentation:
The append built-in function appends elements to the end of a slice. If it has sufficient capacity, the destination is resliced to accommodate the new elements. If it does not, a new underlying array will be allocated. Append returns the updated slice. It is therefore necessary to store the result of append, often in the variable holding the slice itself:
When making a slice with make if the length and capacity are the same, the capacity can be omitted, in which case it is defaulted to the specified length:
// These 2 declarations are equivalent:
s := make([]int, 2, 2)
s := make([]int, 2)
Also note that append() appends elements after the last element of the slice. And the above slices already have len(s) == 2 right after declaration so if you append even just 1 element to it, it will cause a reallocation as seen in this example:
s := make([]int, 2, 2)
fmt.Println(s, len(s), cap(s))
s = append(s, 1)
fmt.Println(s, len(s), cap(s))
Output:
[0 0] 2 2
[0 0 1] 3 4
So in your example what you should do is something like this:
s := make([]int, 0, 10) // Create a slice with length=0 and capacity=10
fmt.Println(s, len(s), cap(s))
s = append(s, 1)
fmt.Println(s, len(s), cap(s))
Output:
[] 0 10
[1] 1 10
I recommend the following blog articles if you want to understand slices in more details:
Go Slices: usage and internals
Arrays, slices (and strings): The mechanics of 'append'
It is mainly an optimization, and it is not unique to go, similar structures in other languages have this as well.
When you append more than the capacity, the runtime needs to allocate more memory for the new elements. This is costly and can also cause memory fragmentation.
By specifying the capacity, the runtime allocates what is needed in advance, and avoids reallocations. However if you do not know the estimated capacity in advance or it changes, you do not have to set it, and the runtime reallocates what is needed and grows the capacity itself.

golang slice, slicing a slice with slice[a:b:c]

I read go slice usage and internals and Slice and Effective go#slice but there is nothing about slicing a slice with 3 number like this : slice[a:b:c]
For example this code :
package main
import "fmt"
func main() {
var s = []string{"a", "b", "c", "d", "e", "f", "g"}
fmt.Println(s[1:2:6], len(s[1:2:6]), cap(s[1:2:6]))
fmt.Println(s[1:2:5], len(s[1:2:5]), cap(s[1:2:5]))
fmt.Println(s[1:2], len(s[1:2]), cap(s[1:2]))
}
go playground result is this :
[b] 1 5
[b] 1 4
[b] 1 6
I can understand that the third one is something about capacity, but what is the exact meaning of this?
Do I miss something in documents?
The syntax has been introduced in Go 1.2, as I mentioned in "Re-slicing slices in Golang".
It is documented in Full slice expressions:
a[low : high : max]
constructs a slice of the same type, and with the same length and elements as the simple slice expression a[low : high].
Additionally, it controls the resulting slice's capacity by setting it to max - low.
Only the first index may be omitted; it defaults to 0.
After slicing the array a:
a := [5]int{1, 2, 3, 4, 5}
t := a[1:3:5]
the slice t has type []int, length 2, capacity 4, and elements
t[0] == 2
t[1] == 3
The design document for that feature had the following justification:
It would occasionally be useful, for example in custom []byte allocation managers, to be able to hand a slice to a caller and know that the caller cannot edit values beyond a given subrange of the true array.
The addition of append to the language made this somewhat more important, because append lets programmers overwrite entries between len and cap without realizing it or even mentioning cap.
2022: svitanok adds for Go 1.19+:
while the capacity of a "derivative" slice doesn't exceed the one specified by the third index during its creation the slice is still "from" the same spot in the memory as its original ("true") slice, so the changes applied to it will affect the original slice.
And if then, for example, you append to this derivative slice the amount of elements that would cause its capacity to be increased, this new slice will occupy a different place in the memory, and so the changes made to it will not affect the slice it originated from.
In a slice expression slice[a:b:c] or aSlice[1:3:5]
a:b or 1:3 -> gives length
a:c or 1:5 -> gives capacity
We can extract both length and capacity from a slice expression with 3 numbers/indices, without looking at the source slice/array.
expression| aSlice[low:high:max] or aSlice[a:b:c] or aSlice[1:3:7]
------------------------------------------------------------------------
Length | len(aSlice[low:high]) or len(aSlice[a:b]) or len(aSlice[1:3])
Capacity | len(aSlice[low:max]) or len(aSlice[a:c]) or len(aSlice[1:7])
------------------------------------------------------------------------
Read more here at Slice Expressions
Playground
Actually Go slice have a pointer and pointing to the array and it holds length and capacity of the array and we can show it like will be
pointer:length:capacity
and append is used for adding same new length.
sl1 := make([]int, 6)
fmt.Println(sl1)
sl2 := append(sl1, 1)
fmt.Println(sl2)
[0 0 0 0 0 0]
[0 0 0 0 0 0 1]

Out growing Slice and Underlying array

I have an array and a slice pointing to it, like shown as follows:
package main
import "fmt"
func main() {
array_str := []string{"0a","1b","2c","3d","4e"}
slice_str:=array_str[1:4]
fmt.Println("Initially :")
fmt.Println("Printing 1 :Array :",array_str)
fmt.Println("Printing 1 :Slice:",slice_str)
//Step 1.Changing Slice and it get reflected in array
fmt.Println("\nAfter Alteration:")
slice_str[0]="alterd_1b"
fmt.Println("Printing 2 :Array :",array_str)
fmt.Println("Printing 2 :Slice:",slice_str)
fmt.Println("len of slice_str:",len(slice_str)," cap of slice_str:",cap(slice_str),"len of array_str:",len(array_str))
//Step 2.appending to slice and it get reflected
slice_str = append(slice_str,"apnded_elemnt")
fmt.Println("\nAfter Apending:")
fmt.Println("Printing 3 :Array :",array_str)//"4e" is replaced with "apnded_elemnt" in array !!
fmt.Println("Printing 3 :Slice:",slice_str)
fmt.Println("len of slice_str:",len(slice_str)," cap of slice_str:",cap(slice_str),"len of array_str:",len(array_str))
//Step 3.Again appending to slice so that lentght of slice is growing further to underlaying array
slice_str = append(slice_str,"outgrown_elemnt")
fmt.Println("\nAfter OUT GROWING:")
fmt.Println("Printing 4 :Array :",array_str)//obeviously out grown elemnt not added to array that is fine
fmt.Println("Printing 4 :Slice:",slice_str)
fmt.Println("len of slice_str:",len(slice_str)," cap of slice_str:",cap(slice_str),"len of array_str:",len(array_str))
//How Capacity Become 8 here ???
//Step 4 .Now Changing Slice element which is in Range of array to verify it reflect on array:
fmt.Println("\nAfter Post out grown Alteration:")
slice_str[0]="again_alterd_1b"
fmt.Println("Printing 2 :Array :",array_str)//Change in slice is not reflectd in array .Why ?
fmt.Println("Printing 2 :Slice:",slice_str)
}
Playground: http://play.golang.org/p/3z52HXHQ7s
Questions:
In Step 3: why does cap of the slice jumped from 4 to 8?
In Step 4: after the slice is out grown, changes to the element of the slice, which is in the range of the array, is not reflected to the array and vice versa. Why is it not happening after it is grown out? What actually happens when the slice grows out?
See here: http://blog.golang.org/slices
Short answers: 1) it grows by doubling (while short). If you append once you might append a second time too and this avoids allocations. 2) That's how slice growing works. An array cannot grow, so a new, larger array is allocated, the old one copied and you are handed a slice pointing to the larger copy.
(The documentation on the golang.org website is really helpful, readable, short and precise. I'd like to recommend to look at golang.org first before asking here.)
The capacity is multiplied by 2 because it is less consuming. Indeed, memory allocation is very consuming, and it's better to allocate a lot of memory a single time than exactly what is needed every time.
Let's just compare, first with a simple example using the concatenation: every time, Go allocates just what is needed.
var n = time.Now()
var s = ""
for i := 0; i < 1000000; i++ {
s += "a"
}
fmt.Println(time.Now().Sub(n))
// 47.7s
Now, let's do the same but this time using the bytes.Buffer type:
var n = time.Now()
var b = bytes.NewBufferString("")
for i := 0; i < 1000000; i++ {
b.WriteString("a")
}
fmt.Println(time.Now().Sub(n))
// 18.5ms
The difference is the way Buffer allocates memory: when there is not enough capacity, it allocates twice the current capacity:
buf = makeSlice(2*cap(b.buf) + n)
Source
This works the same with slices (I was just not able to find the source code to prove it...). So yes, you may be losing some space, but this is for a much better efficiency!
You're second question is a bit more tricky for me, so I hope #Volker's answer will be clear enough for you !

Resources