Tricky case with value-to-references in for-range - go

Look to the code - what do you think the output would be? It's return "Third" instead of "Second" and took me a while to understand why.
Do you know a reason?
I get concept of pass-by-value & pass-by-reference quite well but this case is a bit tricky for people coming from languages like Python. So I decided it worth to share.
package main
import "fmt"
type Record struct {
Id int
Name string
}
var records = []Record{
Record{1, "First"},
Record{2, "Second"},
Record{3, "Third"},
}
func findRecod(id int) (foundRecord *Record) {
for _, record := range records {
if record.Id == id {
foundRecord = &record
// You think we can do a break here but imagine we need to do...
}
// ...something more here
}
return foundRecord
}
func main() {
foundRecord := findRecod(2)
if foundRecord == nil {
fmt.Println("Nothing found")
} else {
fmt.Println("Found: ", foundRecord.Name)
}
}
Run it online to check: https://play.golang.org/p/Y_iAl6m7Ms
I've spent some time figuring out what's going on.

You are returning pointer to a record variable which is reused by each iteration of the loop. Last iteration sets the pointer to the third structure.
The reuse of variable has an enormous advantage. It does not allocate memory in every iteration of the loop. This saves a lot of garbage collection time.
This is well know behaviour described in FAQ.
To fix it, return a pointer to an element of the slice. It is safe as slice elements are referencable in Go.
package main
import "fmt"
type Record struct {
Id int
Name string
}
var records = []Record{
Record{1, "First"},
Record{2, "Second"},
Record{3, "Third"},
}
func findRecod(id int) (foundRecord *Record) {
for i, record := range records {
if record.Id == id {
foundRecord = &records[i]
// You think we can do a break here but imagine we need to do...
}
// ...something more here
}
return foundRecord
}
func main() {
foundRecord := findRecod(2)
if foundRecord == nil {
fmt.Println("Nothing found")
} else {
fmt.Println("Found: ", foundRecord.Name)
}
}
Playground

Related

How to eliminate Data Race even with Mutex Locks around pointer variable

Some starter code goes here,
func (chm *ConcurrentHashMap) NFetchWorker() {
for {
key := <-NFetchWorkerPipe
chm.mu.RLock()
data := chm.data[string(key)]
chm.mu.RUnlock()
if data.IsUsingNFetch {
chm.mu.Lock()
*(chm.data[string(key)].NFetch)--
chm.mu.Unlock()
}
}
}
go NFetchWorker()
Struct ConcurrentHashMap looks like this,
type ConcurrentHashMap struct {
data map[string]DataBlock
mu sync.RWMutex
}
Struct DataBlock looks like this,
type DataBlock struct {
...
NFetch *int32
IsUsingNFetch bool
...
}
Now when I try to run tests with race flag enabled. I get,
Write at 0x00c00012e310 by goroutine 8:
(*ConcurrentHashMap).NFetchWorker()
The line numbers point to this line.
*(chm.data[string(key)].NFetch)--
However I don't face this issue when I do not use pointer in DataBlock.NFetch.
But if I do that I lose the ability to make changes to NFetch directly from map without reassigning a whole new struct object to that hash in map, which would be relatively computationally expensive. I want to change the value of NFetch without reassigning the whole struct again for one small change while being free from DATA RACE. Any solutions??
Fairly New to Golang, I could be doing something really stupid here, feel free to point it out.
UPDATE: Adding Read and Write Op function
func (chm *ConcurrentHashMap) Get(key Key) Value {
chm.mu.RLock()
fetch, ok := chm.data[string(key)]
chm.mu.RUnlock()
if ok {
if CheckValueValidity(&fetch) {
NFetchWorkerPipe <- key
return fetch.Value
} else {
chm.mu.Lock()
delete(chm.data, string(key))
chm.mu.Unlock()
}
}
return constants.NIL
}
Write Op
func (chm *ConcurrentHashMap) Insert(key Key, value Value, options Options) {
...
chm.mu.Lock()
chm.data[string(key)] = Block{
Value: value,
NFetch: nFetchOld,
Expiry: time.Now().Add(delay),
IsUsingNFetch: foundNFetch,
IsUsingExpiry: foundTTL,
mu: mutex,
}
chm.mu.Unlock()
}
The problem is in the CheckValueValidity where you are accessing NFetch without locking it. Pointers should never be accessed without locking them.
func CheckValueValidity(value *Block) bool {
if value.IsUsingExpiry && !value.Expiry.After(time.Now()) {
return false
}
if value.IsUsingNFetch && *(value.NFetch) <= 0 {
return false
}
return true
}
This code should work
func (chm *ConcurrentHashMap) Get(key Key) Value {
chm.mu.RLock()
fetch, ok := chm.data[string(key)]
isValid := CheckValueValidity(&fetch)
chm.mu.RUnlock()
if ok {
if isValid {
NFetchWorkerPipe <- key
return fetch.Value
} else {
chm.mu.Lock()
delete(chm.data, string(key))
chm.mu.Unlock()
}
}
return constants.NIL
}

Hashing reflect.Type

I'm trying to find a quick way of performing comparisons between two []reflect.Type. Right now I have the following:
func Equal(left, right []reflect.Type) bool {
if len(left) != len(right) {
return false
}
for i := 0; i < len(left); i++ {
if left[i] != right[i] {
return false
}
}
return true
}
Most of the slices don't change. So if I can find a way to hash them, I'd get a huge perf boost.
Background
I'm trying to (for fun) implement a form of function overloading in Go using the reflect package. The first thing I did is to convert each specialised/overloaded function into a signature type.
type Signature struct {
Variadic bool
In, Out []reflect.Type
}
The idea is that, when the overloaded function gets called, I'll convert the arguments into a slice of reflect.Type and then find a Signature where the In types match.
This works, but for each comparison, it's a linear scan which is pretty slow. If I could hash the slice of []reflect.Type I could stick that in a map and get constant time lookups.
I ended up abusing the built-in map to assign unique ids to each reflect.Type. Then I hash those using djb2.
type TypeCode struct {
seq int64
codes map[reflect.Type]int64
}
func (td *TypeCode) TypeID(t reflect.Type) int64 {
if code, ok := td.codes[t]; ok {
return code
}
td.seq++
td.codes[t] = td.seq
return td.seq
}
func (td *TypeCode) SliceTypeID(tt []reflect.Type) int64 {
id := int64(5381)
for _, t := range tt {
id = ((id << 5) + id) + td.TypeID(t)
}
return id
}
edit: I switched to a string based approach which is less efficient, but removes any potential for collisions.
type TypeCode struct {
seq int64
codes map[reflect.Type]string
}
func (td *TypeCode) TypeID(t reflect.Type) string {
if code, ok := td.codes[t]; ok {
return code
}
td.seq++
id := strconv.FormatInt(td.seq, 10)
td.codes[t] = id
return id
}
func (td *TypeCode) SliceTypeID(tt []reflect.Type) string {
ids := make([]string, len(tt))
for i := 0; i < len(tt); i++ {
ids[i] = td.TypeID(tt[i])
}
return strings.Join(ids, ".")
}

Making code more generic

I have a program where many functionalities are similar across different structures, however, I end up writing these functions again and again, esp because the variable that are being dealt inside are of different structures.
I have written a sample code here.
In Go Playgroud
package main
import "fmt"
func (a *Match) Add(v Match) {
a.Runs += v.Runs
a.Points += v.Points
}
type Match struct {
Runs uint64
Points uint64
}
func (a *Activity) Add(v Activity) {
a.Walk += v.Walk
a.Jog += v.Jog
}
type Activity struct {
Walk uint64
Jog uint64
}
func GetDailyMatches() map[string]Match {
var dailyMatches map[string]Match
Match1, Match2 := Match{5, 10}, Match{1, 2}
dailyMatches = make(map[string]Match)
dailyMatches["01"] = Match1
dailyMatches["02"] = Match2
dailyMatches["03"] = Match1
dailyMatches["04"] = Match2
return dailyMatches
}
func GetDailyActivities() map[string]Activity {
var dailyActivities map[string]Activity
Activity1, Activity2 := Activity{5, 10}, Activity{1, 2}
dailyActivities = make(map[string]Activity)
dailyActivities["01"] = Activity1
dailyActivities["02"] = Activity2
dailyActivities["03"] = Activity1
dailyActivities["04"] = Activity2
return dailyActivities
}
func main() {
fmt.Println(CalculateMatchSummary("01", "03"))
fmt.Println(CalculateActivitySummary("02", "04"))
fmt.Println(CalculateMatchSummary("01", "03"))
fmt.Println(CalculateActivitySummary("02", "04"))
}
func CalculateMatchSummary(start, end string) (total Match) {
dailyMatches := GetDailyMatches()
for day, value := range dailyMatches {
if day < start {
continue
} else if day > end {
continue
} else {
total.Add(value)
}
}
return
}
func CalculateActivitySummary(start, end string) (total Activity) {
dailyActivities := GetDailyActivities()
for day, value := range dailyActivities {
if day < start {
continue
} else if day > end {
continue
} else {
total.Add(value)
}
}
return
}
If you notice, both Match and Activity has the same functions and same structures, except that internally they are of different structures.
Is there a easy way to make the code more generic (Go generics, which is not there in Go??) in Golang itself.
Go has a pretty package "reflect". You can not do genericity strictly speaking but you can get unification of code for the same behavior.
I've changed your playground a bit : https://play.golang.org/p/bfqZsFOgVQ
The main part :
func AddTwo(a, b interface{}) interface{} {
va := reflect.ValueOf(a)
vb := reflect.ValueOf(b)
res := reflect.New(reflect.TypeOf(a)).Elem()
if va.Kind() != reflect.Struct && vb.Kind() != reflect.Struct {
return nil
}
na, nb := va.NumField(), vb.NumField()
if na != nb {
return nil
}
for i := 0; i < na; i++ {
// additional verification needed here
fa := va.Field(i).Uint()
fb := vb.Field(i).Uint()
fr := fa + fb
res.Field(i).SetUint(fr)
}
return res.Interface()
}
I use reflect to check the fields of the struct I am given. If both are uint64, I can add them reflectively. If your structs contains many uint64, it can add them all !
Note that you must convert the resulting interface to the type of the struct given after calling this function. That is why this is not strictly generic, because the returning type is a interface, and not a Match or Activity.
EDIT: No need even to return a new struct. You can simply update the field of the "a" struct by calling .SetUint() method.

Reference type confusing in Go language

I tried to make Trie data structures by Go Language, but somehow it stuck with References problem,
Here it is. http://play.golang.org/p/ASSGF5Oe9R
// Package main provides ...
package main
import "fmt"
type RootTrie []Trie
type Trie struct {
subtrie []Trie
index byte
}
func (trie *Trie) Insert(data string) *Trie {
if data != "" {
if trie.index == 0 {
trie.index = data[0]
}
if next := trie.containsIndex(data[1:]); next != nil {
//Problem Point
fmt.Println(string(data[1]), "found follwing", string(data[0]))
next.Insert(data[1:])
} else {
nt := &Trie{}
trie.subtrie = append(trie.subtrie, *nt.Insert(data[1:]))
}
}
return trie
}
func (trie *Trie) containsIndex(next string) *Trie {
if next != "" {
for _, st := range trie.subtrie {
if st.index == next[0] {
return &st
}
}
}
return nil
}
func main() {
t := &Trie{}
t = t.Insert("hanyang")
fmt.Println("result:", t)
t = t.Insert("hanyKk")
fmt.Println("result:", t)
t.Insert("hanyK")
}
The following problems happen in second "Insert",
the where I put, //Problem Point
I made containsIndex method for searching next linked trie, and it searched well actually.
But when I updated next property which containsIndex given, its not affected its mother struct trie though.
What I don't understand is I gave it reference type when returning containsIndex, but its still
act liked 'value copied', Why does it not affected its mother structure(trie)?
Thanks!
The problem is in method containsIndex. Golang range by default creates copy each element in slice and assigns copy of this value to st (in your example). Usually to preserve reference to element in slice you should use original slice and its index. In you case method containsIndex should look something like this:
func (trie *Trie) containsIndex(next string) *Trie {
if next != "" {
for i, st := range trie.subtrie {
if st.index == next[0] {
return &trie.subtrie[i]
}
}
}
return nil
}

Generic variadic argument in Go?

I know that Go doesn't support templates or overloaded functions, but I'm wondering if there's any way to do some kind of generic programming for variadic functions anyway?
I have many functions such as these:
func (this Document) GetString(name string, defaults ...string) string {
v, ok := this.GetValueFromDb(name)
if !ok {
if len(defaults) >= 1 {
return defaults[0]
} else {
return ""
}
}
return v.asString
}
func (this Document) GetInt(name string, defaults ...int) int {
v, ok := this.GetValueFromDb(name)
if !ok {
if len(defaults) >= 1 {
return defaults[0]
} else {
return 0
}
}
return v.asInt
}
// etc. for many different types
Is there any way to do this without having so much redundant code?
The most of what you can achieve is usage of interface{} type, something like this:
func (this Document) Get(name string, defaults ...interface{}) interface{} {
v, ok := this.GetValueFromDb(name)
if !ok {
if len(defaults) >= 1 {
return defaults[0]
} else {
return 0
}
}
return v
}
GetValueFromDb function should also be tweaked to return interface{} value and not some wrapper like now.
Then in the client code you can do the following:
value := document.Get("index", 1).(int) // Panics when the value is not int
or
value, ok := document.Get("index", 1).(int) // ok is false if the value is not int
This will yield some runtime overhead though. I'd better stick with separate functions and try to restructure the code somehow.
Here's a working example of how you could change your code.
package main
import (
"fmt"
)
type Document struct{
getSucceeds bool
}
func (d *Document) GetValueFromDb(name string) (interface{}, bool) {
return 1, d.getSucceeds
}
func (this Document) Get(name string, def ...int) interface{} {
v, ok := this.GetValueFromDb(name)
if !ok {
if len(def) >= 1 {
return def[0]
} else {
return 0
}
}
return v
}
func main() {
d1 := Document{true}
d2 := Document{false}
var int1, int2 int
int1 = d1.Get("foo", 2).(int)
int2 = d2.Get("foo", 2).(int)
fmt.Println(int1, int2)
}
Since you know what type you expect for the given name, you can write your Get method in a generic way, returning interface{}, and then assert the type at the call site. See the spec about type assertions.
There are different ways to emulate some aspects of generics in Go. There were lots of discussions on the mailing list. Often, there's a way to restructure code so it's less dependent on generics.
In the client code you can do like this :
res := GetValue("name", 1, 2, 3)
// or
// res := GetValue("name", "one", "two", "three")
if value, ok := res.(int); ok {
// process int return value
} else if value, ok := res.(string); ok {
// process string return value
}
// or
// res.(type) expression only work in switch statement
// and 'res' variable's type have to be interface type
switch value := res.(type) {
case int:
// process int return value
case string:
// process string return value
}

Resources