It's not very predictable to know whether the underlying original array is getting mutated or whether its the copy of the original array that is getting mutated when slices are passed around
a = [3]int {0, 1, 2}
s = a[:]
s[0] = 10
a[0] == s[0] // true
s = append(s, 3)
s[0] = 20
a[0] == s[0] // false
Let' say today I had a processing of this kind
a = [3]int {0, 1, 2}
s = some_func(a[:]) // returns slice
process(s) // a is getting mutated because so far some_func hasn't caused the underlying array to be copied
and now tomorrow
a = [3]int {0, 1, 2}
s = some_func(a[:]) // returns slice, does append operations
process(s) // a is not getting mutated because some_func caused the underlying array to be copied
What are the best practices for slices then?
If a function really does modify a slice's underlying array in place, and promises that it always modifies the underlying array in place, that function should in general take the slice argument by value and not return an updated slice:1
// Mutate() modifies (the backing array of) s in place to achieve $result.
// See below for why it returns an int.
func Mutate(s []T) int {
// code
}
If a function may modify the underlying array in place but may return a slice that uses a new array, the function should return a new slice value, or take a pointer to a slice:
// Replace() operates on a slice of T, but may return a totally new
// slice of T.
func Replace(s []T) []T {
// code
}
When this function returns, you should assume that the underlying array, if you have hold of it, may or may not be in use:
func callsReplace() {
var arr [10]T
s := Replace(arr[:])
// From here on, do not use variable arr directly as
// we don't know if it is s's backing array, or not.
// more code
}
But Mutate() promises to modify the array in place. Note that Mutate will often need to return the number of array elements actually updated:
func callsMutate() {
var arr [10]T
n := Mutate(arr[:])
// now work with arr[0] through arr[n]
// more code
}
1Of course, it could take a pointer to the array object, and modify the array in place, but that's less flexible since the array size is then baked in to the type.
Related
Could someone help explain the Golang internals of why this code is mutating the original array a?
func main() {
a := []int{1,2,3,4}
b := a
b = append(b[0:1], b[2:]...)
fmt.Println(b)
fmt.Println(a)
}
Output:
[1 3 4]
[1 3 4 4]
I thought b := a would be passing by value. Thanks in advance.
That's how slices work. A slice is just a pointer(+size+capacity), the actual data is stored in the array.
When you copy a slice, the underlying array is not copied. Then you end up with two slices pointing to the same array. Mutating the values of one slice will become visible via the other slice.
See Go Slices: usage and internals for more details.
If you want to leave the original slice untouched, first make a deep copy. For example like this
b := append([]int{}, a...) // deep copy
(Live demo)
Slices are basically wrapper over arrays. Slices doesn't have their own data they just hold the reference to the arrays. In your given code you are assigning a to b now they both are indicating the same array. And so when you are mutating the slice b the slice a is also being mutated.
You can use copy method to copy elements from one array to another.
// copy returns the count of total copied elements
count := copy(b /*destination*/ , a /*source*/)
But make sure to allocate an array with the same length of source array.
Example is given below:
func main() {
a := []int{1,2,3,4}
b := make([]int, len(a))
_ = copy(b, a)
a[0] = 2
fmt.Println(b)
fmt.Println(a)
}
I have the following function that generateс all subsets of a given array.
The idea is simple - I start with a results array that contains an empty set (slice) and for each element in the input array nums go over all previously generated sets, add the current element of nums to them and add the resulting new sets back to the results array. Nothing particularly interesting.
func subsets(nums []int) [][]int {
result := [][]int{{}}
for _, n := range nums {
newSets := [][]int{}
for _, set := range result {
newSets = append(newSets, append(set, n))
}
result = append(result, newSets...)
}
return result
}
The problem is that using append(newSets, append(set, n)) corrupts the result slice, of which set is a member. I modified the function a bit with some debug code (see below) and also found a workaround (the commented code) which doesn't cause the same behavior.
I very much suspect that this is caused by something that's passed by reference instead of being copied (I am appending the elements of newSets to result). The problem is that I can't find it. :( I never change the result within a loop that iterates over it. I also work with new instances of newSets for each loop. So I'm not sure what's causing it. Please advise. :)
func subsets(nums []int) [][]int {
result := [][]int{{}}
for _, n := range nums {
newSets := [][]int{}
var before, after []int
for _, set := range result {
lastResultIdx := len(result)-1
if lastResultIdx > 0 {
before = make([]int, len(result[lastResultIdx]))
copy(before, result[lastResultIdx])
}
//ns := []int{}
//for _,v := range set {
// ns = append(ns, v)
//}
//ns = append(ns, n)
//newSets = append(newSets, ns)
newSets = append(newSets, append(set, n))
if lastResultIdx > 0 {
after = result[lastResultIdx]
if before[len(before)-1]!=after[len(after)-1] {
fmt.Println(n, "before", before, "after", after)
}
}
}
result = append(result, newSets...)
}
return result
}
func main() {
subsets([]int{0, 1, 2, 3, 4})
}
The problem is here:
append(newSets, append(set, n))
The problem is not that it is a nested append. The problem is that you're assuming append(set,n) will return a new slice. That is not always the case. A slice is a view on an array, and when you add new elements to the slice, if the addition did not result in reallocation of the array, the returned slice is the same slice you passed in, with len field incremented. So when you're going through your results array, you're modifying the elements that are already there, and at the same time, adding them again as if they are different results.
To solve, when you get an element of the result, create a new slice, copy elements of the result to it, append the new element and then add the new slice to result.
The problem is simple enough: append takes a slice argument—[]T for some type T—plus of course the element(s) to append, and returns a []T result. But []T, if non-nil, consists of two parts: a slice header that points to some backing array and carries a current length and capacity, plus the backing array. When append does its job, it has a choice:
modify the backing array in place, and return a new slice header that re-uses the existing backing array, or
create a new backing array, copy the original values to the new backing array, and return a new slice header that uses the new backing array.
Whenever append copies the backing array, your code works. Whenever it re-uses the backing array, your code may or may not work, depending on whether some other slice header is using the same backing array.
Suppose your backing array has length 5 for instance, and one of the existing slice headers reads "length 1, capacity 5" with element 0 of the backing array holding zero. That is, the existing slice header h contains [0]. Now you call append(h, 1). The append operation re-uses the backing array and puts 1 in the second element and returns a new slice header h1 that contains [0, 1]. Now you take h again, append 2, and make a two-element slice h2 holding [0, 2]. But this re-uses the same backing array that h1 re-used so now h1 also holds [0, 2].
To solve the problem without modifying your algorithm much, you need either:
a variant of append that always copies, or
a variant of append one int to a slice of ints that always copies.
The latter is simpler:
func setPlusInt(set []int, n int) []int {
return append(append([]int(nil), set...), n)
}
which lets you replace one line of your existing code.
(I made one other trivial change here and added enough to provide a working example in the Go Playground.)
(An alternate solution is to set up each of your own slice headers to offer no extra capacity, so that append must always copy. I have not illustrated this method.)
I have the following multivariate array:
x := [2][3]int{
{3, 2, 1},
{3, 2, 1},
}
Both rows and columns are fixed-size.
I'm trying to check that the rows are sorted, and I undertand the sort function requires arrays without a known size. How can I ask the go to treat the item with a fixed-, known size as if it had an unknown size?
var allTrue bool = true
for i := range x {
t := sort.Ints(x[i]) == []int{1, 2, 3}
allTrue = allTrue && t
}
I get:
./main.go:xx:yy: sort.Ints(x[i]) used as value
./main.go:xx:yy: cannot use x[i] (type [3]int) as type []int in argument to sort.Ints
Am I reading this error message correctly?
Notwithstanding this other answer, which provides guidance regarding appropriate slicing to use the sort package, I have added this answer to provide more description on some further issues with the code you have posted. I hope this aids your understanding of the Go language.
Introduction to Slices
I undertand the sort function requires arrays without a known size [sic]
As others have said, this is not the terminology to describe this concept in Go. All Go arrays are of fixed size, as defined by the language spec. As you know, arrays are of type [N]T for an array which contains some non-negative number N of elements of type T. This is fixed at compile time and never changes during runtime of your program.
"Arrays without a known size" most closely maps to slices. Slices are distinct types in Go which allow for representation of sequences of data of a particular type, where their length is managed dynamically by the Go runtime. They are of type []T for elements of type T. In particular, their size is not part of their type definition and can change at runtime. For some slice variable x []T, the implementation provides:
an internal backing array of similar elemental type, where the implementation manages the allocation of memory and expansion of the array as the slice length increases
its length len(x) – denoting the number of elements the slice currently contains
its capacity cap(x) – the total length of the slice plus the additional extent of the backing array, which may extend beyond the length due to slicing operations restricting the view on the array or a larger array being allocated by the runtime to allow for appending more items to the slice.
See the Tour of Go and the language spec on slices for more details.
Resolving the problem with the code
As noted above, slices are of distinct type to arrays, so you cannot use something of type [N]T for some N and T where something of type []T is required.
sort.Ints sorts a slice of integers in-place – it has type signature func Ints(a []int). Your call sort.Ints(x[i]) indexes the array x at index i, which will return an array of type [3]int. This is incompatible with the sort function and leads to the compile-time error you observe.
To obtain a slice from an array, you use a slice expression. Such expressions allow arrays, slices and some other types to be used to construct new slices.
Slice expressions are given in the form a[low : high] where low and high are optional integers providing indices into the backing array or slice which specify the range to return in the new slice. The language spec link above has more details which I recommend you read; suffice to say the simplest slice expression a[:] for some array or slice a is syntactic sugar to mean a[0:len(a)-1], i.e. transform the array/slice into a slice of the same length.
Using this trick, obtain a slice of type []int from your multi-dimensional array by slicing: x[i][:]:
x[i] returns an array of type [3]int, as before
slicing the returned array returns a slice of type []int, which is compatible with sort.Ints.
sort.Ints does not return a value, and slices are not comparable
Even if you fix these issues with your code, there remain two issues with the following line:
t := sort.Ints(x[i]) == []int{1, 2, 3}
sort.Ints sorts in-place; it does not return a value, so the equality test is meaningless.
sort.Ints operates on slices, which are not comparable. It is not possible to call A == B where either A or B is a slice, unless either A or B is the special identifier nil. This is a subtle point which is covered in the language spec. (Aside: read that page, as you will note arrays are comparable.)
As you cannot compare slices directly using the == equality operator, verifying element-wise equality of slices requires:
Slices to be of the same length (dissimilar lengths implies one slice has more elements than the other)
Elements at each index of one slice are identical to other slices.
(I am ignoring the fact that one slice may have a dissimilar capacity to another, as we only care about the element-wise equality.)
This can be verified by looping through one of the slices and verifying elements at each index correspond at the same index in the other slice. This example code provides an example of that (playground link):
package main
import (
"fmt"
)
func CheckEquality(a, b []int) bool {
// Slices of dissimilar length are not equal
if len(a) != len(b) {
return false
}
for i, el := range a {
if b[i] != el {
return false
}
}
return true
}
func main() {
var mySlice = []int{1, 2, 3, 4, 5}
var mySlice2 = []int{1, 2, 3, 4, 5} // same as mySlice
var otherSlice = []int{5, 6, 7, 8, 9} // dissimilar slice
var longSlice = []int{1, 2, 3, 4, 5, 6, 7, 8, 9}
fmt.Println(CheckEquality(mySlice, mySlice2)) // Expect true
fmt.Println(CheckEquality(mySlice, otherSlice)) // Expect false
fmt.Println(CheckEquality(mySlice, longSlice)) // Expect false
}
You have to slice the array with the [:] operator before you can use it with the sort package. See: A Tour of Go: Slices and the "sort" package godoc.
Also, you can check whether the slice is sorted more efficiently than sorting it, using sort.IntsAreSorted([]int), like this.
var allTrue bool = true
for i := range x {
if !sort.IntsAreSorted(x[i][:]) {
allTrue = false
break
}
}
You can get a slice from an array by using the [:] operator, e.g.:
arr := [3]int{1, 2, 3}
slc := arr[:] // []int{1, 2, 3}, backed by arr
As such, you could use the sort.Ints(...) function:
sort.Ints(x[0][:])
sort.Ints(x[1][:])
// Now both elements of "x" are sorted.
This code is in builti.go:
// The append built-in function appends elements to the end of a slice. If
// it has sufficient capacity, the destination is resliced to accommodate the
// new elements. If it does not, a new underlying array will be allocated.
// Append returns the updated slice. It is therefore necessary to store the
// result of append, often in the variable holding the slice itself:
// slice = append(slice, elem1, elem2)
// slice = append(slice, anotherSlice...)
// As a special case, it is legal to append a string to a byte slice, like this:
// slice = append([]byte("hello "), "world"...)
func append(slice []Type, elems ...Type) []Type
The last line made me feel very confused. I do not know the meaning of ...Type .
These are other codes:
package main
import "fmt"
func main() {
s := []int{1,2,3,4,5}
s1 := s[:2]
s2 := s[2:]
s3 := append(s1, s2...)
fmt.Println(s1, s2, s3)
}
The result is
[1 2] [3 4 5] [1 2 3 4 5]
I guess the function of ... is to pick all elements from elems, but I haven't found an official explanation. What is it?
The code in builtin.go serves as documentation. The code is not compiled.
The ... specifies that the final parameter of the function is variadic. Variadic functions are documented in the Go Language specification. In short, variadic functions can be called with any number of arguments for the final parameter.
The Type part is a stand-in for any Go type.
I thought that in GO language, slices are passed by reference. But why the following code doesn't change the content of slice c? Am I missing something? Thank you.
package main
import (
"fmt"
)
func call(c []int) {
c = append(c, 1)
fmt.Println(c)
}
func main() {
c := make([]int, 1, 5)
fmt.Println(c)
call(c)
fmt.Println(c)
}
The result printed is:
[0]
[0 1]
[0]
while I was expecting
[0]
[0 1]
[0 1]
The length of the slice is kept in the slice header which is not passed by reference. You can think of a slice as a struct containing a pointer to the array, a length, and a capacity.
When you appended to the slice, you modified index 1 in the data array and then incremented the length in the slice header. When you returned, c in the main function had a length of 1 and so printed the same data.
The reason slices work this way is so you can have multiple slices pointing to the same data. For example:
x := []int{1,2,3}
y := x[:2] // [1 2]
z := x[1:] // [2 3]
All three of those slices point to overlapping data in the same underlying array.
Go is always pass by value. Certain types are reference types, like pointers, maps, channels; or partially reference types, like slices (which consists of a reference to the underlying array and also the values of the length and capacity). But regardless of type everything is passed by value. Thus assigning to a local variable never affects anything outside.