How to remove items from a slice while ranging over it? - go

Question

What is the best way to remove items from a slice while ranging over it?
For example:
type MultiDataPoint []*DataPoint
func (m MultiDataPoint) Json() ([]byte, error) {
for i, d := range m {
err := d.clean()
if ( err != nil ) {
//Remove the DP from m
}
}
return json.Marshal(m)
}

Answer 1

As you have mentioned elsewhere, you can allocate new memory block and copy only valid elements to it. However, if you want to avoid the allocation, you can rewrite your slice in-place:
i := 0 // output index
for _, x := range s {
if isValid(x) {
// copy and increment index
s[i] = x
i++
}
}
// Prevent memory leak by erasing truncated values
// (not needed if values don't contain pointers, directly or indirectly)
for j := i; j < len(s); j++ {
s[j] = nil
}
s = s[:i]
Full example: http://play.golang.org/p/FNDFswPeDJ
Note this will leave old values after index i in the underlying array, so this will leak memory until the slice itself is garbage collected, if values are or contain pointers. You can solve this by setting all values to nil or the zero value from i until the end of the slice before truncating it.

Answer 2

I know its answered long time ago but i use something like this in other languages, but i don't know if it is the golang way.
Just iterate from back to front so you don't have to worry about indexes that are deleted. I am using the same example as Adam.
m = []int{3, 7, 2, 9, 4, 5}
for i := len(m)-1; i >= 0; i-- {
if m[i] < 5 {
m = append(m[:i], m[i+1:]...)
}
}

Answer 3

There might be better ways, but here's an example that deletes the even values from a slice:
m := []int{1,2,3,4,5,6}
deleted := 0
for i := range m {
j := i - deleted
if (m[j] & 1) == 0 {
m = m[:j+copy(m[j:], m[j+1:])]
deleted++
}
}
Note that I don't get the element using the i, d := range m syntax, since d would end up getting set to the wrong elements once you start deleting from the slice.

Answer 4

Here is a more idiomatic Go way to remove elements from slices.
temp := s[:0]
for _, x := range s {
if isValid(x) {
temp = append(temp, x)
}
}
s = temp
Playground link: https://play.golang.org/p/OH5Ymsat7s9
Note: The example and playground links are based upon #tomasz's answer https://stackoverflow.com/a/20551116/12003457

Answer 5

One other option is to use a normal for loop using the length of the slice and subtract 1 from the index each time a value is removed. See the following example:
m := []int{3, 7, 2, 9, 4, 5}
for i := 0; i < len(m); i++ {
if m[i] < 5 {
m = append(m[:i], m[i+1:]...)
i-- // -1 as the slice just got shorter
}
}
I don't know if len() uses enough resources to make any difference but you could also run it just once and subtract from the length value too:
m := []int{3, 7, 2, 9, 4, 5}
for i, s := 0, len(m); i < s; i++ {
if m[i] < 5 {
m = append(m[:i], m[i+1:]...)
s--
i--
}
}

Answer 6

Something like:
m = append(m[:i], m[i+1:]...)

Answer 7

You don't even need to count backwards but you do need to check that you're at the end of the array where the suggested append() will fail. Here's an example of removing duplicate positive integers from a sorted list:
// Remove repeating numbers
numbers := []int{1, 2, 3, 3, 4, 5, 5}
log.Println(numbers)
for i, numbersCount, prevNum := 0, len(numbers), -1; i < numbersCount; numbersCount = len(numbers) {
if numbers[i] == prevNum {
if i == numbersCount-1 {
numbers = numbers[:i]
} else {
numbers = append(numbers[:i], numbers[i+1:]...)
}
continue
}
prevNum = numbers[i]
i++
}
log.Println(numbers)
Playground: https://play.golang.org/p/v93MgtCQsaN

Answer 8

I just implement a method which removes all nil elements in slice.
And I used it to solve a leetcode problems, it works perfectly.
/**
* Definition for singly-linked list.
* type ListNode struct {
* Val int
* Next *ListNode
* }
*/
func removeNil(lists *[]*ListNode) {
for i := 0; i < len(*lists); i++ {
if (*lists)[i] == nil {
*lists = append((*lists)[:i], (*lists)[i+1:]...)
i--
}
}
}

Answer 9

You can avoid memory leaks, as suggested in #tomasz's answer, controlling the capacity of the underlying array with a full slice expression. Look at the following function that remove duplicates from a slice of integers:
package main
import "fmt"
func removeDuplicates(a []int) []int {
for i, j := 0, 1; i < len(a) && j < len(a); i, j = i+1, j+1 {
if a[i] == a[j] {
copy(a[j:], a[j+1:])
// resize the capacity of the underlying array using the "full slice expression"
// a[low : high : max]
a = a[: len(a)-1 : len(a)-1]
i--
j--
}
}
return a
}
func main() {
a := []int{2, 3, 3, 3, 6, 9, 9}
fmt.Println(a)
a = removeDuplicates(a)
fmt.Println(a)
}
// [2 3 3 3 6 9 9]
// [2 3 6 9]

Answer 10

For reasons #tomasz has explained, there are issues with removing in place. That's why it is practice in golang not to do that, but to reconstruct the slice. So several answers go beyond the answer of #tomasz.
If elements should be unique, it's practice to use the keys of a map for this. I like to contribute an example of deletion by use of a map.
What's nice, the boolean values are available for a second purpose. In this example I calculate Set a minus Set b. As Golang doesn't have a real set, I make sure the output is unique. I use the boolean values as well for the algorithm.
The map gets close to O(n). I don't know the implementation. append() should be O(n). So the runtime is similar fast as deletion in place. Real deletion in place would cause a shifting of the upper end to clean up. If not done in batch, the runtime should be worse.
In this special case, I also use the map as a register, to avoid a nested loop over Set a and Set b to keep the runtime close to O(n).
type Set []int
func differenceOfSets(a, b Set) (difference Set) {
m := map[int]bool{}
for _, element := range a {
m[element] = true
}
for _, element := range b {
if _, registered := m[element]; registered {
m[element] = false
}
}
for element, present := range m {
if present {
difference = append(difference, element)
}
}
return difference
}

Answer 11

Try Sort and Binary search.
Example:
package main
import (
"fmt"
"sort"
)
func main() {
// Our slice.
s := []int{3, 7, 2, 9, 4, 5}
// 1. Iterate over it.
for i, v := range s {
func(i, v int) {}(i, v)
}
// 2. Sort it. (by whatever condition of yours)
sort.Slice(s, func(i, j int) bool {
return s[i] < s[j]
})
// 3. Cut it only once.
i := sort.Search(len(s), func(i int) bool { return s[i] >= 5 })
s = s[i:]
// That's it!
fmt.Println(s) // [5 7 9]
}
https://play.golang.org/p/LnF6o0yMJGT