Shouldn't Go compiler capture for...range loop variables as a locally assigned closure variable?
Long Version:
This caused me some confusion in C# too and I was trying to understand it; that why it is fixed in C# 5.0 foreach (reason: the loop variable can not change inside the body of loop) and the reasoning for not fixing it in C# for loops (reason: the loop variable can change inside the body of loop).
Now (to me) for...range loops in Go seems pretty much like foreach loops in C#, but despite the fact that we can not alter those variables (like k and v in for k, v := range m { ... }); still we have to copy them to some local closures first, for them to behave as expected.
What is the reasoning behind this? (I suspect it's because Go treats any for loop the same way; but I'm not sure).
Here is some code to examine described behavior:
func main() {
lab1() // captured closure is not what is expected
fmt.Println(" ")
lab2() // captured closure is not what is expected
fmt.Println(" ")
lab3() // captured closure behaves ok
fmt.Println(" ")
}
func lab3() {
m := make(map[int32]int32)
var i int32
for i = 1; i <= 10; i++ {
m[i] = i
}
l := [](func() (int32, int32)){}
for k, v := range m {
kLocal, vLocal := k, v // (C) captures just the right values assigned to k and v
l = append(l, func() (int32, int32) {
return kLocal, vLocal
})
}
for _, x := range l {
k, v := x()
fmt.Println(k, v)
}
}
func lab2() {
m := make(map[int32]int32)
var i int32
for i = 1; i <= 10; i++ {
m[i] = i
}
l := [](func() (int32, int32)){}
for k, v := range m {
l = append(l, func() (int32, int32) {
kLocal, vLocal := k, v // (B) captures just the last values assigned to k and v from the range
return kLocal, vLocal
})
}
for _, x := range l {
k, v := x()
fmt.Println(k, v)
}
}
func lab1() {
m := make(map[int32]int32)
var i int32
for i = 1; i <= 10; i++ {
m[i] = i
}
l := [](func() (int32, int32)){}
for k, v := range m {
l = append(l, func() (int32, int32) { return k, v }) // (A) captures just the last values assigned to k and v from the range
}
for _, x := range l {
k, v := x()
fmt.Println(k, v)
}
}
As it is shown in lab1, at the comment // (A) we get just the last values from the range; the output is like printing 9,9 ten times instead of showing expected result like 1,1, 2,2, ... (and of-course maps are not necessarily sorted in Go so we may see 3,3 ten times as the last pair of values; instead of 10,10 ten times as the last pair of values). The same goes for code at comment // (B) at lab2, which was expected because we are trying to capture outer variables inside the inner scope (I put this one too just to try that). In lab3 at code at comment // (C) everything works fine and you will see ten pairs of numbers there like 1,1, 2,2, ....
I was trying to use closure+function as a replacement for tuples in Go.
Do you want the closure over the variable or the value? For example,
package main
import "fmt"
func VariableLoop() {
f := make([]func(), 3)
for i := 0; i < 3; i++ {
// closure over variable i
f[i] = func() {
fmt.Println(i)
}
}
fmt.Println("VariableLoop")
for _, f := range f {
f()
}
}
func ValueLoop() {
f := make([]func(), 3)
for i := 0; i < 3; i++ {
i := i
// closure over value of i
f[i] = func() {
fmt.Println(i)
}
}
fmt.Println("ValueLoop")
for _, f := range f {
f()
}
}
func VariableRange() {
f := make([]func(), 3)
for i := range f {
// closure over variable i
f[i] = func() {
fmt.Println(i)
}
}
fmt.Println("VariableRange")
for _, f := range f {
f()
}
}
func ValueRange() {
f := make([]func(), 3)
for i := range f {
i := i
// closure over value of i
f[i] = func() {
fmt.Println(i)
}
}
fmt.Println("ValueRange")
for _, f := range f {
f()
}
}
func main() {
VariableLoop()
ValueLoop()
VariableRange()
ValueRange()
}
Output:
VariableLoop
3
3
3
ValueLoop
0
1
2
VariableRange
2
2
2
ValueRange
0
1
2
References:
The Go Programming Language Specification
Function literals
Function literals are closures: they may refer to variables defined in
a surrounding function. Those variables are then shared between the
surrounding function and the function literal, and they survive as
long as they are accessible.
Go FAQ: What happens with closures running as goroutines?
To bind the current value of v to each closure as it is launched, one
must modify the inner loop to create a new variable each iteration.
One way is to pass the variable as an argument to the closure.
Even easier is just to create a new variable, using a declaration
style that may seem odd but works fine in Go.
Related
I want to call a number of function names stored in a slice. The code snippet below works so far but I need to return a value from those functions. Unfortunately I don't get it to work because I don't know to to call those functions and store the return value. Any ideas?
This is the code I'm currently working on:
package main
func A(x int) int {
return x + 1
}
func B(x int) int {
return x + 2
}
func C(x int) int {
return x + 3
}
func main() {
x := 10
type fs func(x int) int
f := []fs{A, B, C}
fns := make([]func(), 3)
for a, _ := range f {
a := a
fns[a] = func() {
f[a](x)
}
}
for _, f := range fns {
f()
}
}
Go Playground
You have call it...
for a, _ := range f {
a := a
fns[a] = func() {
f[a](x) // in this
}
}
here is the playground
I am just getting started learning the Golang language!
In for loop, I saw sometimes adding an underscore or without underscore.
Whatever add _ or not, I got the same result.
package main
import (
"fmt"
)
func main() {
doSomething()
sum := addValues(5, 8)
fmt.Println("The sum is", sum)
multiSum, multiCount := addAllValues(4, 7, 9)
fmt.Println("multisum", multiSum)
fmt.Println("multiCount", multiCount)
}
func doSomething() {
fmt.Println("Doing Something")
}
func addValues(value1 int, value2 int) int {
return value1 + value2
}
func addAllValues(values ...int) (int, int) {
total := 0
for _, v := range values {
total += v
}
return total, len(values)
}
func addAllValues(values ...int) (int, int) {
total := 0
for v := range values {
total += v
}
return total, len(values)
}
All I know is I don't care about the index. Is that all? or there is something more what I have to know??
I really appreciate your help!
For range over slices:
In for v := range values { the v is the index of the element in the slice.
In for _, v := range values { the v is the actual element value.
In for i, v := range values { the i is the index and the v is the element.
In for i, _ := range values { the i is the index of the element in the slice.
You can run this playground example to see the differences.
Range expression 1st value 2nd value
array or slice a [n]E, *[n]E, or []E index i int a[i] E
string s string type index i int see below rune
map m map[K]V key k K m[k] V
channel c chan E, <-chan E element e E
For more details see the spec.
If you don't want to use the variable that iterates in the loop, you can use _ to simply let Go ignore it:
mySlice := [int]{1,3,4,59,5}
for _,x := range mySlice {
fmt.Println(x)
}
By placing underscore you are telling the compiler this:
Ok, I'm aware that this function is returning something but I don't care! For example:
package main
import "fmt"
func main() {
mul1, add1 := test_function(2, 3)
fmt.Println(mul1, add1)
mul2, _ := test_function(4, 5)
fmt.Println(mul2)
_, add3 := test_function(7, 8)
fmt.Println(add3)
}
func test_function(a int, b int) (mul int, add int) {
return a * b, a + b
}
just to add to the amazing answer above:
I think one of the main benefits is to maintain readability in your program: if you replace the blank identifier with a variable then you have to use it or your program will not compile.
also this decrease memory allocation be neglecting one of the returned parameters...
I'm trying to write a function that returns the finds first character in a String that doesn't repeat, so far I have this:
package main
import (
"fmt"
"strings"
)
func check(s string) string {
ss := strings.Split(s, "")
smap := map[string]int{}
for i := 0; i < len(ss); i++ {
(smap[ss[i]])++
}
for k, v := range smap {
if v == 1 {
return k
}
}
return ""
}
func main() {
fmt.Println(check("nebuchadnezzer"))
}
Unfortunately in Go when you iterate a map there's no guarantee of the order so every time I run the code I get a different value, any pointers?
Using a map and 2 loops :
play
func check(s string) string {
m := make(map[rune]uint, len(s)) //preallocate the map size
for _, r := range s {
m[r]++
}
for _, r := range s {
if m[r] == 1 {
return string(r)
}
}
return ""
}
The benfit of this is using just 2 loops vs multiple loops if you're using strings.ContainsRune, strings.IndexRune (each function will have inner loops in them).
Efficient (in time and memory) algorithms for grabbing all or the first unique byte http://play.golang.org/p/ZGFepvEXFT:
func FirstUniqueByte(s string) (b byte, ok bool) {
occur := [256]byte{}
order := make([]byte, 0, 256)
for i := 0; i < len(s); i++ {
b = s[i]
switch occur[b] {
case 0:
occur[b] = 1
order = append(order, b)
case 1:
occur[b] = 2
}
}
for _, b = range order {
if occur[b] == 1 {
return b, true
}
}
return 0, false
}
As a bonus, the above function should never generate any garbage. Note that I changed your function signature to be a more idiomatic way to express what you're describing. If you need a func(string) string signature anyway, then the point is moot.
That can certainly be optimized, but one solution (which isn't using map) would be:
(playground example)
func check(s string) string {
unique := ""
for pos, c := range s {
if strings.ContainsRune(unique, c) {
unique = strings.Replace(unique, string(c), "", -1)
} else if strings.IndexRune(s, c) == pos {
unique = unique + string(c)
}
}
fmt.Println("All unique characters found: ", unique)
if len(unique) > 0 {
_, size := utf8.DecodeRuneInString(unique)
return unique[:size]
}
return ""
}
This is after the question "Find the first un-repeated character in a string"
krait suggested below that the function should:
return a string containing the first full rune, not just the first byte of the utf8 encoding of the first rune.
In Python, one can write code like this, to assign multiple values from a list:
(a, b, c, d) = [1,2,3,4]
Is there a similar set of Go library function for slices? That is, I can do this:
http://play.golang.org/p/DY1Bi5omm1
package main
func get3(s []interface{}) (
a interface{},
b interface{},
c interface{},
rest []interface{}) {
return s[0],s[1],s[2],s[4:]
}
func main() {
s := make([]interface{},5);
for i :=0 ; i < 5; i++ { s[i] = i}
a,b,c,_ := get3(s)
print(a.(int))
print(b.(int))
print(c.(int))
}
Is there a standard gophery way to do this?
And is there a way around the interface{} ugliness?
I don't think you can, not in an idiomatic/clean way at least. You CAN do multiple assignments, but you will have to pass individual values either directly or with a closure:
package main
import (
"fmt"
)
func valuesFromList(list[]int,startFrom int) func() int {
i:=startFrom
return func() int {
ret := list[i]
i++
return ret
}
}
func main () {
list := []int{0,1,2,3,4,5,6,7,8,9}
yield := valuesFromList(list,5)
//This works
a,b,c := yield(),yield(),yield()
fmt.Println(a)
fmt.Println(b)
fmt.Println(c)
//This also works
d,e,f := list[0],list[1],list[2]
fmt.Println(d)
fmt.Println(e)
fmt.Println(f)
//This won't work
//g,h,i:= list[7:9]
}
Not like that; you would need dynamic typing or parametric polymorphism, which are not available in Go. The closest I can think about is by fiddling with reflect, like this: http://play.golang.org/p/-K4jh2nZjq
// src is supposed to be []T.
// dst are supposed to be &T, except the last one, which must be a 'rest' &[]T (or nil for discarding).
// There must not be more dst vars than elements in src.
func extract(src interface{}, dst ...interface{}) {
srcV := reflect.ValueOf(src)
// Iterate over dst vars until we run out of them.
i := 0
for i = 0; i < len(dst)-1; i++ {
reflect.Indirect(reflect.ValueOf(dst[i])).Set(srcV.Index(i))
}
// Now, the rest.
restDst := dst[i]
if restDst == nil {
return
}
restV := reflect.ValueOf(restDst)
indirectRest := reflect.Indirect(restV)
l := srcV.Len() - i
indirectRest.Set(reflect.MakeSlice(restV.Type().Elem(), 0, l))
for ; i < srcV.Len(); i++ {
itemV := srcV.Index(i)
indirectRest.Set(reflect.Append(indirectRest, itemV))
}
return
}
Which then you call like:
sl := []int{1, 2, 3, 4, 5, 6} // int or any other type
var a, b, c int
var rest []int
extract(sl, &a, &b, &c, &rest)
So the ugliness doesn't get out the function.
But note that all that happens at runtime, so it's not safe nor efficient and definitely is not idiomatic Go.
How would you implement the deleteRecords function in the code below:
Example:
type Record struct {
id int
name string
}
type RecordList []*Record
func deleteRecords( l *RecordList, ids []int ) {
// Assume the RecordList can contain several 100 entries.
// and the number of the of the records to be removed is about 10.
// What is the fastest and cleanest ways to remove the records that match
// the id specified in the records list.
}
I did some micro-benchmarking on my machine, trying out most of the approaches given in the replies here, and this code comes out fastest when you've got up to about 40 elements in the ids list:
func deleteRecords(data []*Record, ids []int) []*Record {
w := 0 // write index
loop:
for _, x := range data {
for _, id := range ids {
if id == x.id {
continue loop
}
}
data[w] = x
w++
}
return data[:w]
}
You didn't say whether it's important to preserve the order of records in the list. If you don't then this function is faster than the above and still fairly clean.
func reorder(data []*Record, ids []int) []*Record {
n := len(data)
i := 0
loop:
for i < n {
r := data[i]
for _, id := range ids {
if id == r.id {
data[i] = data[n-1]
n--
continue loop
}
}
i++
}
return data[0:n]
}
As the number of ids rises, so does the cost of the linear search. At around 50 elements, using a map or doing a binary search to look up the id becomes more efficient, as long as you can avoid rebuilding the map (or resorting the list) every time. At several hundred ids, it becomes more efficient to use a map or a binary search even if you have to rebuild it every time.
If you wish to preserve original contents of the slice, something like this is more appropriate:
func deletePreserve(data []*Record, ids []int) []*Record {
wdata := make([]*Record, len(data))
w := 0
loop:
for _, x := range data {
for _, id := range ids {
if id == x.id {
continue loop
}
}
wdata[w] = x
w++
}
return wdata[0:w]
}
For a personal project, I did something like this:
func filter(sl []int, fn func(int) bool) []int {
result := make([]int, 0, len(sl))
last := 0
for i, v := range sl {
if fn(v) {
result = append(result, sl[last:i]...)
last = i + 1
}
}
return append(result, sl[last:]...)
}
It doesn't mutate the original, but should be relatively efficient.
It's probably better to just do:
func filter(sl []int, fn func(int) bool) (result []int) {
for _, v := range sl {
if !fn(v) {
result = append(result, v)
}
}
return
}
Simpler and cleaner.
If you want to do it in-place, you probably want something like:
func filter(sl []int, fn func(int) bool) []int {
outi := 0
res := sl
for _, v := range sl {
if !fn(v) {
res[outi] = v
outi++
}
}
return res[0:outi]
}
You can optimize this to use copy to copy ranges of elements, but that's twice
the code and probably not worth it.
So, in this specific case, I'd probably do something like:
func deleteRecords(l []*Record, ids []int) []*Record {
outi := 0
L:
for _, v := range l {
for _, id := range ids {
if v.id == id {
continue L
}
}
l[outi] = v
outi++
}
return l[0:outi]
}
(Note: untested.)
No allocations, nothing fancy, and assuming the rough size of the list of Records and the list of ids you presented, a simple linear search is likely to do as well as fancier things but without any overhead. I realize that my version mutates the slice and returns a new slice, but that's not un-idiomatic in Go, and it avoids forcing the slice at the callsite to be heap allocated.
For the case you described, where len(ids) is approximately 10 and len(*l) is in the several hundreds, this should be relatively fast, since it minimizes memory allocations by updating in place.
package main
import (
"fmt"
"strconv"
)
type Record struct {
id int
name string
}
type RecordList []*Record
func deleteRecords(l *RecordList, ids []int) {
rl := *l
for i := 0; i < len(rl); i++ {
rid := rl[i].id
for j := 0; j < len(ids); j++ {
if rid == ids[j] {
copy(rl[i:len(*l)-1], rl[i+1:])
rl[len(rl)-1] = nil
rl = rl[:len(rl)-1]
break
}
}
}
*l = rl
}
func main() {
l := make(RecordList, 777)
for i := range l {
l[i] = &Record{int(i), "name #" + strconv.Itoa(i)}
}
ids := []int{0, 1, 2, 4, 8, len(l) - 1, len(l)}
fmt.Println(ids, len(l), cap(l), *l[0], *l[1], *l[len(l)-1])
deleteRecords(&l, ids)
fmt.Println(ids, len(l), cap(l), *l[0], *l[1], *l[len(l)-1])
}
Output:
[0 1 2 4 8 776 777] 777 777 {0 name #0} {1 name #1} {776 name #776}
[0 1 2 4 8 776 777] 772 777 {1 name #1} {3 name #3} {775 name #775}
Instead of repeatedly searching ids, you could use a map. This code preallocates the full size of the map, and then just moves array elements in place. There are no other allocations.
func deleteRecords(l *RecordList, ids []int) {
m := make(map[int]bool, len(ids))
for _, id := range ids {
m[id] = true
}
s, x := *l, 0
for _, r := range s {
if !m[r.id] {
s[x] = r
x++
}
}
*l = s[0:x]
}
Use the vector package's Delete method as a guide, or just use a Vector instead of a slice.
Here is one option but I would hope there are cleaner/faster more functional looking ones:
func deleteRecords( l *RecordList, ids []int ) *RecordList {
var newList RecordList
for _, rec := range l {
toRemove := false
for _, id := range ids {
if rec.id == id {
toRemove = true
}
if !toRemove {
newList = append(newList, rec)
}
}
return newList
}
With large enough l and ids it will be more effective to Sort() both lists first and then do a single loop over them instead of two nested loops