Related
How can I implement RemoveRange method in golang? It is a method in C# as shown here
I want to implement RemoveRange method on my hashCode string array and return new modified array back if possible with those ranges remove.
func removeRange(hashCode []string, idx int, count int) []string {
var temp []string
for i, s := range hashCode {
fmt.Println(i, s)
// confuse here on what to do
}
return temp
}
Simply slice the slice up until idx, skip count elements and append the rest to the result of the first slicing:
func removeRange(hashCode []string, idx int, count int) []string {
return append(hashCode[:idx], hashCode[idx+count:]...)
}
Testing it:
s := []string{"0", "1", "2", "3", "4", "5"}
fmt.Println(s)
s = removeRange(s, 1, 2)
fmt.Println(s)
Which outputs (try it on the Go Playground):
[0 1 2 3 4 5]
[0 3 4 5]
Note: the above implementation does not check whether indices are valid (whether they are in range). If not, the code could panic. Add necessary checks if you need to.
Note #2: the above implementation modifies the elements of the passed slice, the returned slice will share the backing array of the parameter. If you want to avoid this, if you want to leave the input intact and allocate a new slice for the result, then do so:
func removeRange(hashCode []string, idx int, count int) []string {
result := make([]string, 0, len(hashCode)-count)
result = append(result, hashCode[:idx]...)
result = append(result, hashCode[idx+count:]...)
return result
}
Try this one on the Go Playground.
You don't need a method or function for this at all in golang. Go slices can be subsliced and appended in place, which is how you can quickly and easily remove subsets from any slice.
Say you want to remove 2 elements, starting at index 2, you'd simply write:
Sub := append(original [:2], original [4:]...)
Demo
How this works:
original[:2] creates a sub-slice starting at 0, with a length of 2 elements (so index 0 and 1)
append because to this first part, we want to add the rest of the slice, minus the range we want to skip/remove
original[4:] creates another sub-slice, this time starting at index 4, and ending wherever original ends. Just like we don't explicitly mention 0 as the starting point in the first sub-slice, by not specifying a number of elements here, golang will just include all of the remaining elements in the slice.
... because append is a variadic function (built-in, but you get the point), we need to pass in every element we want to append as a new argument. The ... operator expands the sub-slice and passes in every element as a separate argument.
Because we assigned the new slice to a new variable, original will remain unchanged, so if you want to overwrite the slice, you just assign it to the same variable.
Note I wrote this on my phone, so markup and code may not be quite right, but this should answer your question at least
I've explained the code using // comments and if not commented, code is self explanatory.
package main
import (
"fmt"
"os"
)
func RemoveRange(s []string, index, count int) []string {
sLen := len(s)
// Similar semantics to match (similar) the behavior of
// C# implementation
switch {
case index < 0, count < 0: // arguments are not valid
fmt.Fprintln(os.Stderr, "error: argument out of range error")
return s
case index+count-1 >= sLen: // range results in exceeding the limit
fmt.Fprintln(os.Stderr, "error: argument error")
return s
}
// Create a slice p and pre-allocate the size required
// to store the resultant slice after removing range.
// Result := s[:] -> s[:index] + s[index+count:]
// Remove := s[index:index+count-1]
p := make([]string, 0, sLen-count)
p = append(p, s[:index]...)
p = append(p, s[index+count:]...)
return p
}
func main() {
s := []string{"0", "1", "2", "3", "4", "5"}
fmt.Println(s)
r := RemoveRange(s, 1, 3)
fmt.Println(r)
}
Output:
[0 1 2 3 4 5]
[0 4 5]
Suppose I have a slice slice of type int. While declaring, I set the third argument to size, which I believe reserves memory for at least size ints by setting the cap parameter of the slice.
slice:=make([]int,0,size)
Now, suppose I have an integer variable value. To add it to the slice at the end, I use
slice=append(slice,value)
If the number of elements currently in the slice is less than size, then there will be no need to copy the entire underlying array to a new location in order to add the new element.
Further, if I want to prepend value to slice, as suggested here and here, I use
slice=append([]int{value},slice...)
My question is, what happens in this case? If the number of elements is still less than size, how are the elements stored in the memory? Assuming a contiguous allocation when the make() function was invoked, are all existing elements right shifted to free the first space for value? Or is memory reallocated and all elements copied?
The reason for asking is that I would like my program to be as fast as possible, and would like to know if this is a possible cause for slowing it down. If it is so, is there any alternative way of prepending that would be more time efficient?
With reslicing and copying
The builtin append() always appends elements to a slice. You cannot use it (alone) to prepend elements.
Having said that, if you have a slice capacity bigger than length (has "free" space after its elements) to which you want to prepend an element, you may reslice the original slice, copy all elements to an index one higher to make room for the new element, then set the element to the 0th index. This will require no new allocation. This is how it could look like:
func prepend(dest []int, value int) []int {
if cap(dest) > len(dest) {
dest = dest[:len(dest)+1]
copy(dest[1:], dest)
dest[0] = value
return dest
}
// No room, new slice need to be allocated:
// Use some extra space for future:
res := make([]int, len(dest)+1, len(dest)+5)
res[0] = value
copy(res[1:], dest)
return res
}
Testing it:
s := make([]int, 0, 5)
s = append(s, 1, 2, 3, 4)
fmt.Println(s)
s = prepend(s, 9)
fmt.Println(s)
s = prepend(s, 8)
fmt.Println(s)
Output (try it on the Go Playground):
[1 2 3 4]
[9 1 2 3 4]
[8 9 1 2 3 4]
Note: if no room for the new element, since performance does matter now, we didn't just do:
res := append([]int{value}, dest...)
Because it does more allocations and copying than needed: allocates a slice for the literal ([]int{value}), then append() allocates a new when appending dest to it.
Instead our solution allocates just one new array (by make(), even reserving some space for future growth), then just set value as the first element and copy dest (the previous elements).
With linked list
If you need to prepend many times, a normal slice may not be the right choice. A faster alternative would be to use a linked list, to which prepending an element requires no allocations of slices / arrays and copying, you just create a new node element, and you designate it to be the root by pointing it to the old root (first element).
The standard library provides a general implementation in the container/list package.
With manually managing a larger backing array
Sticking to normal slices and arrays, there is another solution.
If you're willing to manage a larger backing array (or slice) yourself, you can do so by leaving free space before the slice you use. When prepending, you can create a new slice value from the backing larger array or slice which starts at an index that leaves room for 1 element to be prepended.
Without completeness, just for demonstration:
var backing = make([]int, 15) // 15 elements
var start int
func prepend(dest []int, value int) []int {
if start == 0 {
// No more room for new value, must allocate bigger backing array:
newbacking := make([]int, len(backing)+5)
start = 5
copy(newbacking[5:], backing)
backing = newbacking
}
start--
dest = backing[start : start+len(dest)+1]
dest[0] = value
return dest
}
Testing / using it:
start = 5
s := backing[start:start] // empty slice, starting at idx=5
s = append(s, 1, 2, 3, 4)
fmt.Println(s)
s = prepend(s, 9)
fmt.Println(s)
s = prepend(s, 8)
fmt.Println(s)
// Prepend more to test reallocation:
for i := 10; i < 15; i++ {
s = prepend(s, i)
}
fmt.Println(s)
Output (try it on the Go Playground):
[1 2 3 4]
[9 1 2 3 4]
[8 9 1 2 3 4]
[14 13 12 11 10 8 9 1 2 3 4]
Analysis: this solution makes no allocations and no copying when there is room in the backing slice to prepend the value! All that happens is it creates a new slice from the backing slice that covers the destination +1 space for the value to be prepended, sets it and returns this slice value. You can't really get better than this.
If there is no room, then it allocates a larger backing slice, copies over the content of the old, and then does the "normal" prepending.
With tricky slice usage
Idea: imagine that you always store elements in a slice in backward order.
Storing your elements in backward order in a slice means a prepand becomes append!
So to "prepand" an element, you can simply use append(s, value). And that's all.
Yes, this has its limited uses (e.g. append to a slice stored in reverse order has the same issues and complexity as a "normal" slice and prepand operation), and you lose many conveniences (ability to list elements using for range just to name one), but performance wise nothing beats prepanding a value just by using append().
Note: iterating over the elements that stores elements in backward order has to use a downward loop, e.g.:
for i := len(s) - 1; i >= 0; i-- {
// do something with s[i]
}
Final note: all these solutions can easily be extended to prepend a slice instead of just a value. Generally the additional space when reslicing is not +1 but +len(values), and not simply setting dst[0] = value but instead a call to copy(dst, values).
The "prepend" call will need to allocate an array and copy all elements because a slice in Go is defined as a starting point, a size and an allocation (with the allocation being counted from the starting point).
There is no way a slice can know that the element before the first one can be used to extend the slice.
What will happen with
slice = append([]int{value}, slice...)
is
a new array of a single element value is allocated (probably on stack)
a slice is created to map this element (start=0, size=1, alloc=1)
the append call is done
append sees that there is not enough room to extend the single-element slice so allocates a new array and copies all the elements
a new slice object is created to refer to this array
If appending/removing at both ends of a large container is the common use case for your application then you need a deque container. It is unfortunately unavailable in Go and impossible to write efficiently for generic contained types while maintaining usability (because Go still lacks generics).
You can however implement a deque for your specific case and this is easy (for example if you have a large container with a known upper bound may be a circular buffer is all you need and that is just a couple of lines of code away).
I'm very new to Go, so may be the following is very bad Go code... but it's an attempt to implement a deque using a growing circular buffer (depending on the use case this may be or may be not a good solution)
type Deque struct {
buffer []interface{}
f, b, n int
}
func (d *Deque) resize() {
new_buffer := make([]interface{}, 2*(1+d.n))
j := d.f
for i := 0; i < d.n; i++ {
new_buffer[i] = d.buffer[j]
d.buffer[j] = nil
j++
if j == len(d.buffer) {
j = 0
}
}
d.f = 0
d.b = d.n
d.buffer = new_buffer
}
func (d *Deque) push_back(x interface{}) {
if d.n == len(d.buffer) {
d.resize()
}
d.buffer[d.b] = x
d.b++
if d.b == len(d.buffer) {
d.b = 0
}
d.n++
}
func (d *Deque) push_front(x interface{}) {
if d.n == len(d.buffer) {
d.resize()
}
if d.f == 0 {
d.f = len(d.buffer)
}
d.f--
d.buffer[d.f] = x
d.n++
}
func (d *Deque) pop_back() interface{} {
if d.n == 0 {
panic("Cannot pop from an empty deque")
}
if d.b == 0 {
d.b = len(d.buffer)
}
d.b--
x := d.buffer[d.b]
d.buffer[d.b] = nil
d.n--
return x
}
func (d *Deque) pop_front() interface{} {
if d.n == 0 {
panic("Cannot pop from an empty deque")
}
x := d.buffer[d.f]
d.buffer[d.f] = nil
d.f++
if d.f == len(d.buffer) {
d.f = 0
}
d.n--
return x
}
I encountered weird behaviour in go code today: when I append elements to slice in loop and then try to create new slices based on the result of the loop, last append overrides slices from previous appends.
In this particular example it means that sliceFromLoop j,g and h slice's last element are not 100,101 and 102 respectively, but...always 102!
Second example - sliceFromLiteral behaves as expected.
package main
import "fmt"
func create(iterations int) []int {
a := make([]int, 0)
for i := 0; i < iterations; i++ {
a = append(a, i)
}
return a
}
func main() {
sliceFromLoop()
sliceFromLiteral()
}
func sliceFromLoop() {
fmt.Printf("** NOT working as expected: **\n\n")
i := create(11)
fmt.Println("initial slice: ", i)
j := append(i, 100)
g := append(i, 101)
h := append(i, 102)
fmt.Printf("i: %v\nj: %v\ng: %v\nh:%v\n", i, j, g, h)
}
func sliceFromLiteral() {
fmt.Printf("\n\n** working as expected: **\n")
i := []int{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
fmt.Println("initial slice: ", i)
j := append(i, 100)
g := append(i, 101)
h := append(i, 102)
fmt.Printf("i: %v\nj: %v\ng: %v\nh:%v\n", i, j, g, h)
}
link to play.golang:
https://play.golang.org/p/INADVS3Ats
After some reading, digging and experimenting I found that this problem is originated in slices referencing the same underlaying array values and can be solved by copying slice to new one before appending anything, however it looks quite... hesitantly.
What's the idomatic way for creating many new slices based on old ones and not worrying about changing values of old slices?
Don't assign append to anything other than itself.
As you mention in the question, the confusion is due to the fact that append both changes the underlying array and returns a new slice (since the length might be changed). You'd imagine that it copies that backing array, but it doesn't, it just allocates a new slice object that points at it. Since i never changes, all those appends keep changing the value of backingArray[12] to a different number.
Contrast this to appending to an array, which allocates a new literal array every time.
So yes, you need to copy the slice before you can work on it.
func makeFromSlice(sl []int) []int {
result := make([]int, len(sl))
copy(result, sl)
return result
}
func main() {
i := make([]int, 0)
for ii:=0; ii<11; ii++ {
i = append(i, ii)
}
j := append(makeFromSlice(i), 100) // works fine
}
The slice literal behavior is explained because a new array is allocated if the append would exceed the cap of the backing array. This has nothing to do with slice literals and everything to do with the internals of how exceeding the cap works.
a := []int{1,2,3,4,5,6,7}
fmt.Printf("len(a) %d, cap(a) %d\n", a, len(a), cap(a))
// len(a) 7, cap(a) 7
b := make([]int, 0)
for i:=1; i<8, i++ {
b = append(b, i)
} // b := []int{1,2,3,4,5,6,7}
// len(b) 7, cap(b) 8
b = append(b, 1) // any number, just so it hits cap
i := append(b, 100)
j := append(b, 101)
k := append(b, 102) // these work as expected now
If you need a copy of a slice, there's no other way to do it other than, copying the slice. You should almost never assign the result of append to a variable other than the first argument of append. It leads to hard to find bugs, and will behave differently depending on whether the slice has the required capacity or not.
This isn't a commonly needed pattern, but as with all things of this nature if you need to repeate a few lines of code multiple times, then you can use a small helper function:
func copyAndAppend(i []int, vals ...int) []int {
j := make([]int, len(i), len(i)+len(vals))
copy(j, i)
return append(j, vals...)
}
https://play.golang.org/p/J99_xEbaWo
There is also a little bit simpler way to implement copyAndAppend function:
func copyAndAppend(source []string, items ...string) []string {
l := len(source)
return append(source[:l:l], items...)
}
Here we just make sure that source has no available capacity and so copying is forced.
I am using golang. Here is my code:
func main() {
quanPailie([]int{1, 2})
}
func quanPailie(nums []int) [][]int {
COUNT := len(nums)
//only one item
if COUNT == 1 {
return [][]int{nums}
}
insertItem(quanPailie(nums[:COUNT-1]), nums[COUNT-1])
return [][]int{}
}
func insertItem(res [][]int, insertNum int) {
fmt.Println("insertItem,res:", res, "insertNum", insertNum) //insertItem,res: [[1]] insertNum 2
for _, v := range res {
for i := 0; i < len(v); i++ {
fmt.Println("===before,v:", v)
c := append(v[:i], append([]int{insertNum}, v[i:]...)...)
fmt.Println("===after,v:", v)
fmt.Println("ccc", c)
}
}
}
What makes me very confused is the output:
===before,v: [1]
===after,v: [2]
Why did the value of v change? Hope someone can help me. Thanks a lot.
Go playground: https://play.golang.org/p/wITYsGpX7U
EDIT:
Thanks for icza's great help, I think I have understood this problem.
And, here is a simple code to show this issue.
func test1() {
nums := []int{1, 2, 3}
_ = append(nums[:2], 4)
fmt.Println("test1:", nums)
//nums changes because the cap is big enought, the original array is modified.
}
func test2() {
nums := []int{1, 2, 3}
c := append(nums[:2], []int{4, 5, 6}...)
fmt.Println("test2:", nums)
fmt.Println("cc:", c)
//nums dont't change because the cap isn't big enought.
//a new array is allocated while the nums still points to the old array.
//Of course, the return value of append points to the new array.
}
Go playground: https://play.golang.org/p/jBNFsCqUn3
This is the code in question:
fmt.Println("===before,v:", v)
c := append(v[:i], append([]int{insertNum}, v[i:]...)...)
fmt.Println("===after,v:", v)
You ask why v changes between the 2 Println() statements.
Because you are using the builtin append() function, quoting from its doc:
The append built-in function appends elements to the end of a slice. If it has sufficient capacity, the destination is resliced to accommodate the new elements. If it does not, a new underlying array will be allocated. Append returns the updated slice.
So if the slice you append to has enough room (capacity) to accomodate the elements you want to append, no new slice will be allocated, instead the destination slice will be re-sliced (which will use the same underlying array) and append will happen in that.
Let's check the capacity:
fmt.Println("===before,v:", v, cap(v))
c := append(v[:i], append([]int{insertNum}, v[i:]...)...)
fmt.Println("===after,v:", v, cap(v))
Output:
===before,v: [1] 2
===after,v: [2] 2
The v slice has a capacity of 2. When for loop starts, i=0, v[:i] is v[:0] which is an empty slice (but has capacity 2) and so appending 1 or 2 elements will not allocate a new array/slice, it will be done "in place". This "in place" is the 0th element of v, since v[:i] is shorthand for v[0:i]. Hence the elements will be appended starting from v[0] in the underlying array which is shared, so the element denoted by v[0] will change.
Note that slicing a slice results in a slice which shares its underlying backing array with the original slice (does not make a copy of the elements).
If you want to avoid this, use or allocate a new slice, copy original content and append to the new slice, e.g.:
src := []int{1, 2}
c := make([]int, len(src))
copy(c, src)
// Append something:
c = append(c, 3, 4)
fmt.Println(src) // [1 2] - src doesn't change
fmt.Println(c) // [1 2 3 4]
The code snippet below is the library implementation of the push methods for a priority queue. I am wondering why the line with the code a = a[0 : n+1] does not throw an out of bounds errors.
func (pq *PriorityQueue) Push(x interface{}) {
// Push and Pop use pointer receivers because they modify the slice's length,
// not just its contents.
// To simplify indexing expressions in these methods, we save a copy of the
// slice object. We could instead write (*pq)[i].
a := *pq
n := len(a)
a = a[0 : n+1]
item := x.(*Item)
item.index = n
a[n] = item
*pq = a
}
a slice is not an array; it is a view onto an existing array. The slice in question is backed by an array larger than itself. When you define a slice of an existing slice, you're actually slicing the underlying array, but the indexes referenced are relative to the source slice.
That's a mouthful. Let's prove this in the following way: we'll create a slice of zero length, but we'll force the underlying array to be larger. When creating a slice with make, the third parameter will set the size of the underlying array. The expression make([]int, 0, 2) will allocate an array of size 2, but it evaluates to a size-zero slice.
package main
import ("fmt")
func main() {
// create a zero-width slice over an initial array of size 2
a := make([]int, 0, 2)
fmt.Println(a)
// expand the slice. Since we're not beyond the size of the initial
// array, this isn't out of bounds.
a = a[0:len(a)+1]
a[0] = 1
fmt.Println(a)
fmt.Println(a[0:len(a)+1])
}
see here. You can use the cap keyword to reference the size of the array that backs a given slice.
The specific code that you asked about loops over cap(pq) in the calling context (container/heap/example_test.go line 90). If you modify the code at the call site and attempt to push another item into the queue, it will panic like you expect. I ... probably wouldn't suggest writing code like this. Although the code in the standard library executes, I would be very sour if I found that in my codebase. It's generally safer to use the append keyword.
Because it works in a specific example program. Here are the important parts from the original/full example source)
const nItem = 10
and
pq := make(PriorityQueue, 0, nItem)
and
for i := 0; i < cap(pq); i++ {
item := &Item{
value: values[i],
priority: priorities[i],
}
heap.Push(&pq, item)
}
Is it an example from container/heap? If yes, then it doesn't throws an exception because capacity is big enough (see how the Push method is used). If you change the example to Push more items then the capacity, then it'll throw.
It does in general; it doesn't in the container/heap example. Here's the general fix I already gave you some time ago.
func (pq *PriorityQueue) Push(x interface{}) {
a := *pq
n := len(a)
item := x.(*Item)
item.index = n
a = append(a, item)
*pq = a
}
Golang solution to Project Euler problem #81