According the following function declarations from sync package:
Add -------> func (wg *WaitGroup) Add(delta int)
Done ------> func (wg *WaitGroup) Done()
Wait ------> func (wg *WaitGroup) Wait()
I understand that all 3 of them are called by a pointer to a WaitGroup, right?
If this is correct, I don't understand in the next pice of code, why Done function is called using a pointer variable, but Add and Wait functions are called using a variable (not a pointer):
package main
import (
"fmt"
"sync"
"time"
)
func worker(id int, wg *sync.WaitGroup) {
defer wg.Done()
fmt.Printf("Worker %d starting\n", id)
time.Sleep(time.Second)
fmt.Printf("Worker %d done\n", id)
}
func main() {
var wg sync.WaitGroup
for i := 1; i <= 5; i++ {
wg.Add(1)
go worker(i, &wg)
}
wg.Wait()
}
Thanks a lot for your help.
Done, Add and Wait are called on pointer. All functions refer to a pointer receiver *WaitGroup. The fact that you declare variable as value of WaitGroup doesn't mean much as all those methods will all access and modify the variable. The only problem happens when you want to pass your variable to worker - if you try to pass it as value you will make a copy and then Done will be referring to different pointer than Add and Wait - that's why you pass it's address with &.
I think here is best explanation I so far seen on the topic: https://github.com/golang/go/wiki/MethodSets#variables
In general, when you have a variable of a type, you can pretty much call whatever you want on it. When you combine the two rules above together, the following is valid:
type List []int
func (l List) Len() int { return len(l) }
func (l *List) Append(val int) { *l = append(*l, val) }
func main() {
// A bare value
var lst List
lst.Append(1)
fmt.Printf("%v (len: %d)\n", lst, lst.Len())
// A pointer value
plst := new(List)
plst.Append(2)
fmt.Printf("%v (len: %d)\n", plst, plst.Len())
}
Note that both pointer and value methods can both be called on both pointer and non-pointer values. To understand why, let's examine the method sets of both types, directly from the spec:
List
- Len() int
*List
- Len() int
- Append(int)
Notice that the method set for List does not actually contain Append(int) even though you can see from the above program that you can call the method without a problem. This is a result of the second spec section above. It implicitly translates the first line below into the second:
lst.Append(1)
(&lst).Append(1)
Now that the value before the dot is a *List, its method set includes Append, and the call is legal.
To make it easier to remember these rules, it may be helpful to simply consider the pointer- and value-receiver methods separately from the method set. It is legal to call a pointer-valued method on anything that is already a pointer or whose address can be taken (as is the case in the above example). It is legal to call a value method on anything which is a value or whose value can be dereferenced (as is the case with any pointer; this case is specified explicitly in the spec).
Related
I have a struct in Go which contains a mutex, and I want to ensure that that mutex is never nil. To that end, I have implemented a GetMutex() function, which checks if the mutex is nil, and if it is, then assigns it a value.
My question is: is the following code thread safe? If not, what would be an idiomatic way to ensure that mux is always initialized? The only thing I can think of is to have a global mutex in this package which is used within my GetMutex() function, but perhaps there is a different approach.
package main
import (
"sync"
)
type Counter struct {
mux *sync.Mutex
counter int
}
// Is this thread safe?
func (c *Counter) GetMux() *sync.Mutex {
if c.mux == nil {
c.mux = &sync.Mutex{}
}
return c.mux
}
func (c *Counter) Inc() {
c.GetMux().Lock()
c.counter++
c.GetMux().Unlock()
}
func main() {
c := &Counter{}
c.Inc()
}
No, it's not safe if Counter.GetMux() is called from multiple goroutines concurrently: GetMux() both reads and writes the Counter.mux field.
The general way is to use a "constructor" like function that takes care of the initialization, like this:
func NewCounter() *Counter {
return &Counter{
mux: &sync.Mutex{},
}
}
And of course always create counters with this NewCounter().
Another–limited–way would be to use a non-pointer mutex value:
type Counter struct {
mux sync.Mutex
counter int
}
So when you have a Counter struct value, it–by design–includes a mutex. But if you do this, then Counter should always be used as a pointer, and Counter struct values must not be copied (else the mutex field would also be copied, but as package doc of sync states: "Values containing the types defined in this package should not be copied.").
The obvious advantage of this is that the zero value of Counter is a valid and ready counter (something you should aim for with your custom types), and no constructor function is needed.
I have some code which is supposed to increment a count.
Here is the struct containing the count variable:
type PipelineData struct {
nodeData map[string]map[string]int
lastBurstResults map[string]map[string]string
burstReady map[string]bool
lastExecutionTime map[string]time.Time
currentNodeSize uint64
}
As you can see there is a member entitled currentNodeSize. This variable is intended to increment ever time the function addNodeData is called. You can see that the function addNodeData calls the function addCount which then uses an atomic incrementer.
func (p PipelineData) addNodeData(key string) {
nodeSlot := clusterScenario.GetNodeSlotByHashSlot(key)
i:=p.nodeData[nodeSlot][key]
i++
p.nodeData[nodeSlot][key]=i
p.addCount()
fmt.Println("Adding node count ",p.currentNodeSize)
}
func (p PipelineData) addCount(){
atomic.AddUint64(&p.currentNodeSize, 1)
}
Unfortunately, when I run this:
p.addNodeData("{pipelinetest}.key");
p.addNodeData("{pipelinetest}.key");
p.addNodeData("{pipelinetest}.key");
p.addNodeData("{pipelinetest}.key");
p.addNodeData("{pipelinetest}.key");
p.addNodeData("{pipelinetest}.key");
Dump(p.currentNodeSize)
The output is 0.
Here is the initialization of the PipelineData struct:
p = &PipelineData{
nodeData:make(map[string]map[string]int,0),
lastBurstResults:make(map[string]map[string]string,0),
burstReady:make(map[string]bool,0),
lastExecutionTime:make(map[string]time.Time,0),
currentNodeSize:0,
}
for i,_ := range clusterScenario.masterNodes{
p.nodeData[i]=make(map[string]int,0)
}
I understand the community has asked me to do my research. I'm not sure what to do at the point. Any help would be appreciated.
I also tried a general incrementor using a variable and that produced the same result which is why I tried an atomic incrementor
addCount takes a value receiver, not a pointer, so it's operating on a copy of the struct, incrementing the field of the copy and then discarding the copy. Change it instead to take a pointer:
func (p *PipelineData) addCount(){
This should resolve your issue.
From the Dave Cheney Blog, the following code apparently causes a race case that can be resolved merely by changing func (RPC) version() int to func (*RPC) version() int :
package main
import (
"fmt"
"time"
)
type RPC struct {
result int
done chan struct{}
}
func (rpc *RPC) compute() {
time.Sleep(time.Second) // strenuous computation intensifies
rpc.result = 42
close(rpc.done)
}
func (RPC) version() int {
return 1 // never going to need to change this
}
func main() {
rpc := &RPC{done: make(chan struct{})}
go rpc.compute() // kick off computation in the background
version := rpc.version() // grab some other information while we're waiting
<-rpc.done // wait for computation to finish
result := rpc.result
fmt.Printf("RPC computation complete, result: %d, version: %d\n", result, version)
}
After looking over the code a few times, I was having a hard time believing that the code had a race case. However, when running with --race, it claims that there was a write at rpc.result=42 and a previous read at version := rpc.version(). I understand the write, since the goroutine changes the value of rpc.result, but what about the read? Where in the version() method does the read occur? It does not touch any of the values of rpc, just returning 1.
I would like to understand the following:
1) Why is that particular line considered a read on the rpc struct?
2) Why would changing RPC to *RPC resolve the race case?
When you have a method with value receiver like this:
func (RPC) version() int {
return 1 // never going to need to change this
}
And you call this method:
version := rpc.version() // grab some other information while we're waiting
A copy has to be made from the value rpc, which will be passed to the method (used as the receiver value).
So while one goroutine go rpc.compute() is running and is modifying the rpc struct value (rpc.result = 42), the main goroutine is making a copy of the whole rpc struct value. There! It's a race.
When you modify the receiver type to pointer:
func (*RPC) version() int {
return 1 // never going to need to change this
}
And you call this method:
version := rpc.version() // grab some other information while we're waiting
This is a shorthand for
version := (&rpc).version()
This passes the address of the rpc value to RPC.version(), it uses only the pointer as the receiver, so no copy is made of the rpc struct value. And since nothing from the struct is used / read in RPC.version(), there is no race.
Note:
Note that if RPC.version() would read the RPC.result field, it would also be a race, as one goroutine modifies it while the main goroutine would read it:
func (rpc *RPC) version() int {
return rpc.result // RACE!
}
Note #2:
Also note that if RPC.version() would read another field of RPC which is not modified in RPC.compute(), that would not be a race, e.g.:
type RPC struct {
result int
done chan struct{}
dummy int
}
func (rpc *RPC) version() int {
return rpc.dummy // Not a race
}
I have the following structure
type Groups struct {
sync.Mutex
Names []string
}
and the following function
func NewGroups(names ...string) (Groups, error) {
// ...
return groups, nil
}
When I check for semantic errors with go vet, I am getting this warning:
NewGroups returns Lock by value: Groups
As go vet is shouting, it is not good. What problems can this code bring? How can I fix this?
You need to embed the sync.Mutex as a pointer:
type Groups struct {
*sync.Mutex
Names []strng
}
Addressing your comment on your question: In the article http://blog.golang.org/go-maps-in-action notice Gerrand is not returning the struct from a function but is using it right away, that is why he isn't using a pointer. In your case you are returning it, so you need a pointer so as not to make a copy of the Mutex.
Update: As #JimB points out, it may not be prudent to embed a pointer to sync.Mutex, it might be better to return a pointer to the outer struct and continue to embed the sync.Mutex as a value. Consider what you are trying to accomplish in your specific case.
Return a pointer *Groups instead.
Embedding the mutex pointer also works but has two disadvantages that require extra care from your side:
the zero value of the struct would have a nil mutex, so you must explicitly initialize it every time
func main() {
a, _ := NewGroups()
a.Lock() // panic: nil pointer dereference
}
func NewGroups(names ...string) (Groups, error) {
return Groups{/* whoops, mutex zero val is nil */ Names: names}, nil
}
assigning a struct value, or passing it as function arg, makes a copy so you also copy the mutex pointer, which then locks all copies. (This may be a legit use case in some particular circumstances, but most of the time it might not be what you want.)
func main() {
a, _ := NewGroups()
a.Lock()
lockShared(a)
fmt.Println("done")
}
func NewGroups(names ...string) (Groups, error) {
return Groups{Mutex: &sync.Mutex{}, Names: names}, nil
}
func lockShared(g Groups) {
g.Lock() // whoops, deadlock! the mutex pointer is the same
}
Keep your original struct and return pointers. You don't have to explicitly init the embedded mutex, and it's intuitive that the mutex is not shared with copies of your struct.
func NewGroups(names ...string) (*Groups, error) {
// ...
return &Groups{}, nil
}
Playground (with the failing examples): https://play.golang.org/p/CcdZYcrN4lm
The code is like the following:
package main
import (
"fmt"
"time"
)
type field struct {
name string
}
func (p *field) print() {
fmt.Println(p.name)
}
func main() {
data := []field{{"one"},{"two"},{"three"}}
for _,v := range data {
go v.print()
}
time.Sleep(3 * time.Second)
}
I know that the code is wrong,because the for loop variable is reused in the for-range loop.
When the goroutine has got the chance to launch,the value of v might has been modified. so the print result will be "three,three,three".
But when we modify the data variable into another declaration as:
data := []*field{{"one"},{"two"},{"three"}}
the print result will be "one ,two,three".
I didn't get the point of why. Does the pointer make any difference or any different mechanism is on this?
I read this from this article. But the poster didn't not tell why. Or it's just a incident the output is right.
In the first loop, v is the value of a field item. Because v is addressable, it is automatically referenced as the pointer receiver for the print() method. So v.print() is using the address of v itself, and the contents of that address is overwritten each iteration of the loop.
When you change the declaration to use a *field, v is now a pointer to a field value. When you call v.print() in this case, you are operating on the value that v points to, which is stored in data, and the overwriting of v has no effect.