Concatenate 2 slices in golang - go

I have 2 slices,
s1 := []int{1, 2, 3, 4, 5}
s2 := []int{3, 4, 5, 6, 7}
I want the resultant
s3 = []int{1, 2, 3, 4, 5, 3, 4, 5, 6, 7}
I am doing something like:
for _, x := range s1 {
s2 = append(s2, x)
}
This looks very trivial question but trust me I don't find a one line solution to this.
How can we go about this?

This is what the builtin append() function is for: to append values (which may be values of a slice) to the end of another. And the result is the concatenation.
If you want s3 to be "independent" of s1 and s2, then append s1 to an empty or nil slice, then append s2 to the result:
s3 := append(append([]int{}, s1...), s2...)
fmt.Println(s3)
If s3 may use / overlap s1, you can simply append s2 to s1:
s4 := append(s1, s2...)
fmt.Println(s4)
Output in both cases (try it on the Go Playground):
[1 2 3 4 5 3 4 5 6 7]
Note: What this "overlapping" means is that if you append values to s1, if it has enough capacity, no new slice will be allocated, s1 will be resliced to have enough length to accommodate the elements you want to append to it. This may have surprising side-effects if used unwisely, like in this example:
arr := [...]int{1, 2, 3, 4, 5, 6, 7, 0, 0, 0}
s1 := arr[:5]
s2 := arr[2:7]
fmt.Println("s1:", s1, "cap:", cap(s1))
fmt.Println("s2:", s2)
s3 := append(s1, s2...)
fmt.Println("s3:", s3)
fmt.Println("s2:", s2)
Output is (try it on the Go Playground):
s1: [1 2 3 4 5] cap: 10
s2: [3 4 5 6 7]
s3: [1 2 3 4 5 3 4 5 6 7]
s2: [3 4 5 3 4]
What may be surprising here is that when we appended s2 to s1 and stored the result in s3 (which is what we expected), contents (elements) of s2 also changed. The reason for this is that append() saw that s1 has enough capacity to append s2 to it (elements of s2), so it did not create a new array, it just resliced s1 and added elements "in-place". But the area where the additional elements were written is the exact same memory where elements of s2 reside, so elements of s2 also got overwritten.

I would recommend you to give a look at this page: https://github.com/golang/go/wiki/SliceTricks
It's an official documentation by golang for slices. For your problem you just need to use append :)

Related

Why appending on slice modified another slice? [duplicate]

This question already has answers here:
Why does append() modify the provided slice? (See example)
(1 answer)
Golang append changing append parameter [duplicate]
(2 answers)
Why does append modify passed slice
(5 answers)
Deleting element of slice modifies original value [duplicate]
(1 answer)
Closed last year.
package main
import "fmt"
func main() {
src := []int{0, 1, 2, 3, 4, 5, 6}
a := src[:3]
b := src[3:]
a = append(a, 9)
fmt.Println(a, b)
}
output:
[0 1 2 9] [9 4 5 6]
Did append modified the underlay array as []int{0, 1, 2, 9, 4, 5, 6}?
Slice a was copied as a new slice with a new underlay array with value [0, 1, 2, 9] and slice b still pointing to the old array that was modified.
Thanks for any hints, much appreciated
Slice a was copied as a new slice with a new underlay array with value [0, 1, 2, 9] and slice b still pointing to the old array that was modified.
a := src[:3] created a slice (a pointer to the src head, length=3, capacity=7)
b := src[3:] created a slice(a pointer to the src[3],length=4, capacity=4)
a and b shares the same memory created by src
a = append(a, 9),when appending to the same slice, as long as it does not exceed cap, it is the same array that is modified
Did append modified the underlay array as []int{0, 1, 2, 9, 4, 5, 6}
YES
If the append exceed the cap of a, new array will be allocated and data will be copied the the new array
try this out:
package main
import "fmt"
func main() {
src := []int{0, 1, 2, 3, 4, 5, 6}
a := src[:3]
b := src[3:]
a = append(a, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9)
fmt.Println(a, b)
}
Output:
[0 1 2 9 9 9 9 9 9 9 9 9 9] [3 4 5 6]

Appending to two slice with same underlying array, why the result?

Here is a code snippet from some Go book.
func incr(s []int) {
s = append(s, 0)
for i := range s {
s[i]++
}
}
func main() {
s1 := []int{1, 2}
s2 := s1
s2 = append(s2, 3)
incr(s1)
incr(s2)
fmt.Print(s1, s2) // "[1, 2][2, 3, 4]"
}
I don't understand why the result is "[1, 2][2, 3, 4]".
Based on two points from https://golang.org/doc/effective_go#slices:
Slices hold references to an underlying array, and if you assign one slice to another, both refer to the same array
If a function takes a slice argument, changes it makes to the elements of the slice will be visible to the caller, analogous to passing a pointer to the underlying array
Here is what I imagined should happen:
At first, both s1 and s2's have the same underlying array [1, 2]
After appending 3 to s2, the underlying array changed to [1, 2, 3]. But s1 still only see [1, 2]
After incr(s1), s1 is appended 0 and all item incremented, resulting s1 into [2, 3, 1]. The appending also changed the underlying array, so s2 now see [2, 3, 1]
After incr(s2), s2 is appended 0 and all item incremented, resulting s2 into [3, 4, 2, 1]. The increment also affected underlying array, so now s1 see [3, 4, 2]
So the result printed should be [3, 4, 2][3, 4, 2, 1]
I obviously have huge mistake understanding slice in Go. Please tell me where I am wrong. It seems my reasoning is in accord with slice's behavior. (I know appending to an insufficient capacity slice will also reallocate an underlying array, but don't know how to fit it into this).
Let's analyze this program step by step:
s1 := []int{1, 2} // s1 -> [1,2]
s2 := s1 // s1, s2 -> [1,2]
The next operation is:
s2 = append(s2, 3)
This may allocate a new backing array if the capacity of the underlying array is not sufficient. In this case, it will do that to give:
s1 -> [1,2]
s2 -> [1,2,3]
Then incr(s1) will append a new element to s1, increment values, but the resulting slice will not be assigned to s1 in main, so still:
s1 -> [1,2]
s2 -> [1,2,3]
incr(s2) will do the same, but this time, the backing array has the capacity to hold the appended zero, so the increment operation increments the values, but the new slice is never assigned to s2 in main. So, s2 still has 3 elements:
s1 -> [1,2]
s2 -> [2,3,4]
The backing array of s2 has one more element in it, but s2 does not include that element.

The most concise way to concatenate 3 or more slices

I am looking for a way to concisely and efficiently concatenate 3 or more slices in Go.
Let's say I want to concatenate the following slices (all the code can be found here - https://play.golang.org/p/6682YiFF8qG):
a := []int{1, 2, 3}
b := []int{4, 5, 6}
c := []int{7, 8, 9}
My first attempt is by using the append method:
d1 := append(a, b...)
d1 = append(d1, c...) // [1 2 3 4 5 6 7 8 9]
However, this method is verbose and requires 2 append calls for concatenating three slices. So, for n slices, I will need n-1 calls to append, which is not only verbose, but also inefficient as it requires multiple allocations.
My next attempt is to create a variadic function to handle the concatenation with only one new slice allocation:
func concat(slicesOfSlices ...[]int) []int {
var totalLengthOfSlices int
for _, slice := range slicesOfSlices {
totalLengthOfSlices += len(slice)
}
arr := make([]int, 0, totalLengthOfSlices)
for _, slice := range slicesOfSlices {
arr = append(arr, slice...)
}
return arr
}
Then I can use it as follows:
d2 := concat(a, b, c) // [1 2 3 4 5 6 7 8 9]
To illustrate, I want to emulate the following convenient functionality of the spread operator in JavaScript, which I often use in the following way:
const a = [1, 2, 3];
const b = [4, 5, 6];
const c = [7, 8, 9];
const d = [...a, ...b, ...c]; // [1, 2, 3, 4, 5, 6, 7, 8, 9]
In other words, I am looking for a way to do something like d3 := append(a, b, c) or d3 := append(a, b..., c...) but with the standard Go library or using less code than I did.
Note on possible duplicates
I don't think this is a duplicate of the question "How to concatenate two slices" as my question is about concatenating 3 or more slices in the most concise and idiomatic way.
You could use your first method of using append like this:
a := []int{1, 2, 3, 4}
b := []int{9, 8, 7, 6}
c := []int{5, 4, 3, 2}
a = append(a, append(b, c...)...)
That being said, I think that your variadic concat function is cleaner and isn't very much code for a utility function.
(Go Playground Link)
Good luck!

append to a new slice affect original slice

I have the following code which causes a weird result. I cannot understand why:
func main() {
var s = []int{2, 3}
var s1 = append(s, 4)
var a = append(s1, 5)
var b = append(s1, 6)
fmt.Println(s)
fmt.Println(s1)
fmt.Println(a)
fmt.Println(b)
}
This then results in:
[2 3]
[2 3 4]
[2 3 4 6]
[2 3 4 6]
My question is: why a is not [2 3 4 5] but [2 3 4 6]? I know append to b changes a, but how. Is this a bug because I never changed a directly and obviously I don't want this happen?
Keep in mind that a slice is a structure of 3 fields.
a pointer to the underlying array
length of the slice
capacity of the slice
append() function may either modify its argument in-place or return a copy of its argument with an additional entry, depending on the size and capacity of its input. append() function creates a new slice, if the length the slice is greater than the length of the array pointed by the slice.

Golang: appending slices with or w/o allocation

Go's append() function only allocates new slice data, when the capacity of the given slice is not sufficient (see also: https://stackoverflow.com/a/28143457/802833). This can lead to unexpected behavior (at least for me as a golang newbie):
package main
import (
"fmt"
)
func main() {
a1 := make([][]int, 3)
a2 := make([][]int, 3)
b := [][]int{{1, 1, 1}, {2, 2, 2}, {3, 3, 3}}
common1 := make([]int, 0)
common2 := make([]int, 0, 12) // provide sufficient capacity
common1 = append(common1, []int{10, 20}...)
common2 = append(common2, []int{10, 20}...)
idx := 0
for _, k := range b {
a1[idx] = append(common1, k...) // new slice is allocated
a2[idx] = append(common2, k...) // no allocation
idx++
}
fmt.Println(a1)
fmt.Println(a2) // surprise!!!
}
output:
[[10 20 1 1 1] [10 20 2 2 2] [10 20 3 3 3]]
[[10 20 3 3 3] [10 20 3 3 3] [10 20 3 3 3]]
https://play.golang.org/p/8PEqFxAsMt
So, what ist the (idomatic) way in Go to force allocation of new slice data or more precisely to make sure that the slice argument to append() remains unchanged?
You might maintain a wrong idea of how slices work in Go.
When you append elements to a slice, the call to append() returns a new slice. If reallocation did not happen, both slice values — the one you called append() on and the one it returned back — share the same backing array but they will have different lengths; observe:
package main
import "fmt"
func main() {
a := make([]int, 0, 10)
b := append(a, 1, 2, 3)
c := append(a, 4, 3, 2)
fmt.Printf("a=%#v\nb=%#v\nc=%#v\n", a, b, c)
}
outputs:
a=[]int{}
b=[]int{4, 3, 2}
c=[]int{4, 3, 2}
So, len(a) == 0, len(b) == 3, len(c) == 3, and the second call to append() owerwrote what the first one did because all the slices share the same underlying array.
As to reallocation of the backing array, the spec is clear:
If the capacity of s is not large enough to fit the additional values, append allocates a new, sufficiently large underlying array that fits both the existing slice elements and the additional values. Otherwise, append re-uses the underlying array.
From this, it follows that:
append() never copies the underlying storage if the capacity of the slice being appeneded to is sufficient.
If there's not enough capacity, the array will be reallocated.
That is, given a slice s to which you want to append N elements, the reallocation won't be done iff cap(s) - len(s) ≥ N.
Hence I suspect your problem is not about unexpected reallocation results but rather about the concept of slices as implemented in Go. The code idea to absorb is that append() returns the resulting slice value, which you're supposed to be using after the call unless you fully understand the repercussions.
I recommend starting with this to fully understand them.
Thanx for your feedback.
So the solution to gain control of the memory allocation is to do it explicitely (which remembers me that Go is a more a system language than other (scripting) langs):
package main
import (
"fmt"
)
func main() {
a1 := make([][]int, 3)
a2 := make([][]int, 3)
b := [][]int{{1, 1, 1}, {2, 2, 2}, {3, 3, 3}}
common1 := make([]int, 0)
common2 := make([]int, 0, 12) // provide sufficient capacity
common1 = append(common1, []int{10, 20}...)
common2 = append(common2, []int{10, 20}...)
idx := 0
for _, k := range b {
a1[idx] = append(common1, k...) // new slice is allocated
a2[idx] = make([]int, len(common2), len(common2)+len(k))
copy(a2[idx], common2) // copy & append could probably be
a2[idx] = append(a2[idx], k...) // combined into a single copy step
idx++
}
fmt.Println(a1)
fmt.Println(a2)
}
output:
[[10 20 1 1 1] [10 20 2 2 2] [10 20 3 3 3]]
[[10 20 1 1 1] [10 20 2 2 2] [10 20 3 3 3]]
https://play.golang.org/p/Id_wSZwb84

Resources