I thought that in GO language, slices are passed by reference. But why the following code doesn't change the content of slice c? Am I missing something? Thank you.
package main
import (
"fmt"
)
func call(c []int) {
c = append(c, 1)
fmt.Println(c)
}
func main() {
c := make([]int, 1, 5)
fmt.Println(c)
call(c)
fmt.Println(c)
}
The result printed is:
[0]
[0 1]
[0]
while I was expecting
[0]
[0 1]
[0 1]
The length of the slice is kept in the slice header which is not passed by reference. You can think of a slice as a struct containing a pointer to the array, a length, and a capacity.
When you appended to the slice, you modified index 1 in the data array and then incremented the length in the slice header. When you returned, c in the main function had a length of 1 and so printed the same data.
The reason slices work this way is so you can have multiple slices pointing to the same data. For example:
x := []int{1,2,3}
y := x[:2] // [1 2]
z := x[1:] // [2 3]
All three of those slices point to overlapping data in the same underlying array.
Go is always pass by value. Certain types are reference types, like pointers, maps, channels; or partially reference types, like slices (which consists of a reference to the underlying array and also the values of the length and capacity). But regardless of type everything is passed by value. Thus assigning to a local variable never affects anything outside.
Related
Could someone help explain the Golang internals of why this code is mutating the original array a?
func main() {
a := []int{1,2,3,4}
b := a
b = append(b[0:1], b[2:]...)
fmt.Println(b)
fmt.Println(a)
}
Output:
[1 3 4]
[1 3 4 4]
I thought b := a would be passing by value. Thanks in advance.
That's how slices work. A slice is just a pointer(+size+capacity), the actual data is stored in the array.
When you copy a slice, the underlying array is not copied. Then you end up with two slices pointing to the same array. Mutating the values of one slice will become visible via the other slice.
See Go Slices: usage and internals for more details.
If you want to leave the original slice untouched, first make a deep copy. For example like this
b := append([]int{}, a...) // deep copy
(Live demo)
Slices are basically wrapper over arrays. Slices doesn't have their own data they just hold the reference to the arrays. In your given code you are assigning a to b now they both are indicating the same array. And so when you are mutating the slice b the slice a is also being mutated.
You can use copy method to copy elements from one array to another.
// copy returns the count of total copied elements
count := copy(b /*destination*/ , a /*source*/)
But make sure to allocate an array with the same length of source array.
Example is given below:
func main() {
a := []int{1,2,3,4}
b := make([]int, len(a))
_ = copy(b, a)
a[0] = 2
fmt.Println(b)
fmt.Println(a)
}
I need to populate a struct that has a member of type [8]uint8. This needs be populated with a byte array of type []byte initialized to length 8. The simplistic approach does not work:
Data: [8]uint8(RequestFrame(0x180, r)),
gives
cannot convert .. (type []byte) to type [8]uint8
Since both arrays are structurally identical it would be nice if this could be done with casting/assignment rather than copying?
Background
The problem with your "simplistic approach" is that a slice
(of any type) is a struct-typed value consisting of a pointer
and two integers; the pointer contains the address of the
underlying (backing) data array, and the integers contain
what len() and cap() builtins return for that slice.
In other words, a slice is sort of a view into an array.
Then, in Go, there is no concept of a type cast; there are only
type conversions, and these conversions may only happen between
types with the same underlying representation¹.
Since a slice and an array may not have the same underlying
representation (array is literally a contiguous block of memory
of the size just enough to contain all the array's elements),
your alleged type conversion may not be legal.
Possible solutions
There are two possible solutions.
The simplest is to just copy the data from the slice's
backing array into a newly-allocated array:
var (
src = []byte{1, 2, 3, 4, 5, 6, 7, 8}
dst [8]uint8
)
copy(dst[:], src[:8])
Note that there exists an inherent disparity between slice an
array types: an array type encodes both the type of its elements
and its length (that is, the length is a part of the type),
while a slice type only encodes the type of its elements
(and may be of any length at runtime).
This means that you might need to have a check before such
copying that makes sure the source slice has exactly 8
elements, that is, len(src) == len(dst).
This invariant may be enforced by some other code, but I think
I'd warn you up front about this: if src has less than 8
elements, the src[:8] expression will panic at runtime,
and if it contains more, then there's the question of whether
copying just the first 8 of them is exactly what's needed.
The second approach (admittedly messier) is to just re-use
the underlying array of the slice:
import "unsafe"
var (
src = []byte{1, 2, 3, 4, 5, 6, 7, 8}
dstPtr *[8]uint8
)
if len(src) != len(*dstPtr) {
panic("boom")
}
dstPtr = (*[8]uint8)(unsafe.Pointer(&src[0]))
Here, we've just taken the address of the first element
contained in the slice's underlying array and peformed
a "dirty" two-phase type-conversion to make the obtained
pointer to be of type *[8]uint8—that is, "an address of
an array of 8 uint8s".
Note two caveats:
The resulting pointer now points to
the same memory block the original slice does.
It means it's now possible to mutate that memory both through the
slice and the pointer we obtained.
As soon as you'll decide to assign the array's data
to a variable of type [8]uint8 (and passing it as an argument
to a function's parameter of that type), you will dereference
that pointer (like with *dstPtr), and at that moment
the array's data will be copied.
I'm specifically mentioning this as often people resort
to hacks like this one to pull the backing array out of
a slice precisely in an attempt to not copy the memory.
TL;DR
Copy the data (after supposedly verifying the
len(src) == len(dst) invariant holds).
Copying 8 bytes is fast (on a typical 64-bit CPU this will be
a single MOV instruction, or two at most), and the code will
be straightforward.
Only resort to hacks from the second solution when you really
need to optimize on some critical hot path.
In that case, comment the solution extensively and watch for
not accidentally dereferencing your pointer.
¹ There are notable exceptions to this rule:
A []byte is type-convertible to string, and vice-versa.
A string is type-convertible to []rune, and vice-versa.
An int is type-convertible to string (but since Go 1.15 go vet gives a warning about it, and this feature may probably be prohibited in the future).
You can copy the contents of your byte slice into your uint8 array very simply by using copy, like this:
package main
import (
"fmt"
)
func main() {
slice := []byte{1, 2, 3, 4, 5, 6, 7, 8}
array := [8]uint8{}
copy(array[:], slice)
fmt.Println(array)
}
Outputs
[1 2 3 4 5 6 7 8]
Try it out on the playground.
But may I ask why you are using an array? It's usually better to just use slices, unless you have a really good reason.
Starting from Go 1.17 you are able to use type conversion directly, from a slice to an array pointer:
a := make([]byte, 8)
b := (*[8]uint8)(a) // b is pointer to [8]uint8
The you can just dereference to obtain a non-pointer [8]uint8 type.
a := make([]byte, 8)
b := *(*[8]uint8)(a) // b is [8]uint8
Notes:
unlike copy, the conversion approach does not incur in extra allocations (not yours, nor any possibly done by copy), because it simply yields a pointer to the existing backing array. Though dereferencing the array pointer will make a copy.
the conversion panics if the length of the array is greater than the slice's
a := make([]byte, 5)
b := (*[10]byte)(a) // panics
the pointer points to the slice's underlying array, therefore the same values will be visible by indexing either:
a := []byte{0xa1, 0xa2}
b := (*[2]uint8)(a)
fmt.Printf("%x\n", a[0]) // a1
b[0] = 0xff
fmt.Printf("%x\n", a[0]) // ff
you can convert from byte to uint8, including type literals derived from them, because byte is an alias (identical to) of uint8.
Related: How do you convert a slice into an array?
I have the following multivariate array:
x := [2][3]int{
{3, 2, 1},
{3, 2, 1},
}
Both rows and columns are fixed-size.
I'm trying to check that the rows are sorted, and I undertand the sort function requires arrays without a known size. How can I ask the go to treat the item with a fixed-, known size as if it had an unknown size?
var allTrue bool = true
for i := range x {
t := sort.Ints(x[i]) == []int{1, 2, 3}
allTrue = allTrue && t
}
I get:
./main.go:xx:yy: sort.Ints(x[i]) used as value
./main.go:xx:yy: cannot use x[i] (type [3]int) as type []int in argument to sort.Ints
Am I reading this error message correctly?
Notwithstanding this other answer, which provides guidance regarding appropriate slicing to use the sort package, I have added this answer to provide more description on some further issues with the code you have posted. I hope this aids your understanding of the Go language.
Introduction to Slices
I undertand the sort function requires arrays without a known size [sic]
As others have said, this is not the terminology to describe this concept in Go. All Go arrays are of fixed size, as defined by the language spec. As you know, arrays are of type [N]T for an array which contains some non-negative number N of elements of type T. This is fixed at compile time and never changes during runtime of your program.
"Arrays without a known size" most closely maps to slices. Slices are distinct types in Go which allow for representation of sequences of data of a particular type, where their length is managed dynamically by the Go runtime. They are of type []T for elements of type T. In particular, their size is not part of their type definition and can change at runtime. For some slice variable x []T, the implementation provides:
an internal backing array of similar elemental type, where the implementation manages the allocation of memory and expansion of the array as the slice length increases
its length len(x) – denoting the number of elements the slice currently contains
its capacity cap(x) – the total length of the slice plus the additional extent of the backing array, which may extend beyond the length due to slicing operations restricting the view on the array or a larger array being allocated by the runtime to allow for appending more items to the slice.
See the Tour of Go and the language spec on slices for more details.
Resolving the problem with the code
As noted above, slices are of distinct type to arrays, so you cannot use something of type [N]T for some N and T where something of type []T is required.
sort.Ints sorts a slice of integers in-place – it has type signature func Ints(a []int). Your call sort.Ints(x[i]) indexes the array x at index i, which will return an array of type [3]int. This is incompatible with the sort function and leads to the compile-time error you observe.
To obtain a slice from an array, you use a slice expression. Such expressions allow arrays, slices and some other types to be used to construct new slices.
Slice expressions are given in the form a[low : high] where low and high are optional integers providing indices into the backing array or slice which specify the range to return in the new slice. The language spec link above has more details which I recommend you read; suffice to say the simplest slice expression a[:] for some array or slice a is syntactic sugar to mean a[0:len(a)-1], i.e. transform the array/slice into a slice of the same length.
Using this trick, obtain a slice of type []int from your multi-dimensional array by slicing: x[i][:]:
x[i] returns an array of type [3]int, as before
slicing the returned array returns a slice of type []int, which is compatible with sort.Ints.
sort.Ints does not return a value, and slices are not comparable
Even if you fix these issues with your code, there remain two issues with the following line:
t := sort.Ints(x[i]) == []int{1, 2, 3}
sort.Ints sorts in-place; it does not return a value, so the equality test is meaningless.
sort.Ints operates on slices, which are not comparable. It is not possible to call A == B where either A or B is a slice, unless either A or B is the special identifier nil. This is a subtle point which is covered in the language spec. (Aside: read that page, as you will note arrays are comparable.)
As you cannot compare slices directly using the == equality operator, verifying element-wise equality of slices requires:
Slices to be of the same length (dissimilar lengths implies one slice has more elements than the other)
Elements at each index of one slice are identical to other slices.
(I am ignoring the fact that one slice may have a dissimilar capacity to another, as we only care about the element-wise equality.)
This can be verified by looping through one of the slices and verifying elements at each index correspond at the same index in the other slice. This example code provides an example of that (playground link):
package main
import (
"fmt"
)
func CheckEquality(a, b []int) bool {
// Slices of dissimilar length are not equal
if len(a) != len(b) {
return false
}
for i, el := range a {
if b[i] != el {
return false
}
}
return true
}
func main() {
var mySlice = []int{1, 2, 3, 4, 5}
var mySlice2 = []int{1, 2, 3, 4, 5} // same as mySlice
var otherSlice = []int{5, 6, 7, 8, 9} // dissimilar slice
var longSlice = []int{1, 2, 3, 4, 5, 6, 7, 8, 9}
fmt.Println(CheckEquality(mySlice, mySlice2)) // Expect true
fmt.Println(CheckEquality(mySlice, otherSlice)) // Expect false
fmt.Println(CheckEquality(mySlice, longSlice)) // Expect false
}
You have to slice the array with the [:] operator before you can use it with the sort package. See: A Tour of Go: Slices and the "sort" package godoc.
Also, you can check whether the slice is sorted more efficiently than sorting it, using sort.IntsAreSorted([]int), like this.
var allTrue bool = true
for i := range x {
if !sort.IntsAreSorted(x[i][:]) {
allTrue = false
break
}
}
You can get a slice from an array by using the [:] operator, e.g.:
arr := [3]int{1, 2, 3}
slc := arr[:] // []int{1, 2, 3}, backed by arr
As such, you could use the sort.Ints(...) function:
sort.Ints(x[0][:])
sort.Ints(x[1][:])
// Now both elements of "x" are sorted.
This code is in builti.go:
// The append built-in function appends elements to the end of a slice. If
// it has sufficient capacity, the destination is resliced to accommodate the
// new elements. If it does not, a new underlying array will be allocated.
// Append returns the updated slice. It is therefore necessary to store the
// result of append, often in the variable holding the slice itself:
// slice = append(slice, elem1, elem2)
// slice = append(slice, anotherSlice...)
// As a special case, it is legal to append a string to a byte slice, like this:
// slice = append([]byte("hello "), "world"...)
func append(slice []Type, elems ...Type) []Type
The last line made me feel very confused. I do not know the meaning of ...Type .
These are other codes:
package main
import "fmt"
func main() {
s := []int{1,2,3,4,5}
s1 := s[:2]
s2 := s[2:]
s3 := append(s1, s2...)
fmt.Println(s1, s2, s3)
}
The result is
[1 2] [3 4 5] [1 2 3 4 5]
I guess the function of ... is to pick all elements from elems, but I haven't found an official explanation. What is it?
The code in builtin.go serves as documentation. The code is not compiled.
The ... specifies that the final parameter of the function is variadic. Variadic functions are documented in the Go Language specification. In short, variadic functions can be called with any number of arguments for the final parameter.
The Type part is a stand-in for any Go type.
I read go slice usage and internals and Slice and Effective go#slice but there is nothing about slicing a slice with 3 number like this : slice[a:b:c]
For example this code :
package main
import "fmt"
func main() {
var s = []string{"a", "b", "c", "d", "e", "f", "g"}
fmt.Println(s[1:2:6], len(s[1:2:6]), cap(s[1:2:6]))
fmt.Println(s[1:2:5], len(s[1:2:5]), cap(s[1:2:5]))
fmt.Println(s[1:2], len(s[1:2]), cap(s[1:2]))
}
go playground result is this :
[b] 1 5
[b] 1 4
[b] 1 6
I can understand that the third one is something about capacity, but what is the exact meaning of this?
Do I miss something in documents?
The syntax has been introduced in Go 1.2, as I mentioned in "Re-slicing slices in Golang".
It is documented in Full slice expressions:
a[low : high : max]
constructs a slice of the same type, and with the same length and elements as the simple slice expression a[low : high].
Additionally, it controls the resulting slice's capacity by setting it to max - low.
Only the first index may be omitted; it defaults to 0.
After slicing the array a:
a := [5]int{1, 2, 3, 4, 5}
t := a[1:3:5]
the slice t has type []int, length 2, capacity 4, and elements
t[0] == 2
t[1] == 3
The design document for that feature had the following justification:
It would occasionally be useful, for example in custom []byte allocation managers, to be able to hand a slice to a caller and know that the caller cannot edit values beyond a given subrange of the true array.
The addition of append to the language made this somewhat more important, because append lets programmers overwrite entries between len and cap without realizing it or even mentioning cap.
2022: svitanok adds for Go 1.19+:
while the capacity of a "derivative" slice doesn't exceed the one specified by the third index during its creation the slice is still "from" the same spot in the memory as its original ("true") slice, so the changes applied to it will affect the original slice.
And if then, for example, you append to this derivative slice the amount of elements that would cause its capacity to be increased, this new slice will occupy a different place in the memory, and so the changes made to it will not affect the slice it originated from.
In a slice expression slice[a:b:c] or aSlice[1:3:5]
a:b or 1:3 -> gives length
a:c or 1:5 -> gives capacity
We can extract both length and capacity from a slice expression with 3 numbers/indices, without looking at the source slice/array.
expression| aSlice[low:high:max] or aSlice[a:b:c] or aSlice[1:3:7]
------------------------------------------------------------------------
Length | len(aSlice[low:high]) or len(aSlice[a:b]) or len(aSlice[1:3])
Capacity | len(aSlice[low:max]) or len(aSlice[a:c]) or len(aSlice[1:7])
------------------------------------------------------------------------
Read more here at Slice Expressions
Playground
Actually Go slice have a pointer and pointing to the array and it holds length and capacity of the array and we can show it like will be
pointer:length:capacity
and append is used for adding same new length.
sl1 := make([]int, 6)
fmt.Println(sl1)
sl2 := append(sl1, 1)
fmt.Println(sl2)
[0 0 0 0 0 0]
[0 0 0 0 0 0 1]