I have a struct in Go which contains a mutex, and I want to ensure that that mutex is never nil. To that end, I have implemented a GetMutex() function, which checks if the mutex is nil, and if it is, then assigns it a value.
My question is: is the following code thread safe? If not, what would be an idiomatic way to ensure that mux is always initialized? The only thing I can think of is to have a global mutex in this package which is used within my GetMutex() function, but perhaps there is a different approach.
package main
import (
"sync"
)
type Counter struct {
mux *sync.Mutex
counter int
}
// Is this thread safe?
func (c *Counter) GetMux() *sync.Mutex {
if c.mux == nil {
c.mux = &sync.Mutex{}
}
return c.mux
}
func (c *Counter) Inc() {
c.GetMux().Lock()
c.counter++
c.GetMux().Unlock()
}
func main() {
c := &Counter{}
c.Inc()
}
No, it's not safe if Counter.GetMux() is called from multiple goroutines concurrently: GetMux() both reads and writes the Counter.mux field.
The general way is to use a "constructor" like function that takes care of the initialization, like this:
func NewCounter() *Counter {
return &Counter{
mux: &sync.Mutex{},
}
}
And of course always create counters with this NewCounter().
Another–limited–way would be to use a non-pointer mutex value:
type Counter struct {
mux sync.Mutex
counter int
}
So when you have a Counter struct value, it–by design–includes a mutex. But if you do this, then Counter should always be used as a pointer, and Counter struct values must not be copied (else the mutex field would also be copied, but as package doc of sync states: "Values containing the types defined in this package should not be copied.").
The obvious advantage of this is that the zero value of Counter is a valid and ready counter (something you should aim for with your custom types), and no constructor function is needed.
Related
According the following function declarations from sync package:
Add -------> func (wg *WaitGroup) Add(delta int)
Done ------> func (wg *WaitGroup) Done()
Wait ------> func (wg *WaitGroup) Wait()
I understand that all 3 of them are called by a pointer to a WaitGroup, right?
If this is correct, I don't understand in the next pice of code, why Done function is called using a pointer variable, but Add and Wait functions are called using a variable (not a pointer):
package main
import (
"fmt"
"sync"
"time"
)
func worker(id int, wg *sync.WaitGroup) {
defer wg.Done()
fmt.Printf("Worker %d starting\n", id)
time.Sleep(time.Second)
fmt.Printf("Worker %d done\n", id)
}
func main() {
var wg sync.WaitGroup
for i := 1; i <= 5; i++ {
wg.Add(1)
go worker(i, &wg)
}
wg.Wait()
}
Thanks a lot for your help.
Done, Add and Wait are called on pointer. All functions refer to a pointer receiver *WaitGroup. The fact that you declare variable as value of WaitGroup doesn't mean much as all those methods will all access and modify the variable. The only problem happens when you want to pass your variable to worker - if you try to pass it as value you will make a copy and then Done will be referring to different pointer than Add and Wait - that's why you pass it's address with &.
I think here is best explanation I so far seen on the topic: https://github.com/golang/go/wiki/MethodSets#variables
In general, when you have a variable of a type, you can pretty much call whatever you want on it. When you combine the two rules above together, the following is valid:
type List []int
func (l List) Len() int { return len(l) }
func (l *List) Append(val int) { *l = append(*l, val) }
func main() {
// A bare value
var lst List
lst.Append(1)
fmt.Printf("%v (len: %d)\n", lst, lst.Len())
// A pointer value
plst := new(List)
plst.Append(2)
fmt.Printf("%v (len: %d)\n", plst, plst.Len())
}
Note that both pointer and value methods can both be called on both pointer and non-pointer values. To understand why, let's examine the method sets of both types, directly from the spec:
List
- Len() int
*List
- Len() int
- Append(int)
Notice that the method set for List does not actually contain Append(int) even though you can see from the above program that you can call the method without a problem. This is a result of the second spec section above. It implicitly translates the first line below into the second:
lst.Append(1)
(&lst).Append(1)
Now that the value before the dot is a *List, its method set includes Append, and the call is legal.
To make it easier to remember these rules, it may be helpful to simply consider the pointer- and value-receiver methods separately from the method set. It is legal to call a pointer-valued method on anything that is already a pointer or whose address can be taken (as is the case in the above example). It is legal to call a value method on anything which is a value or whose value can be dereferenced (as is the case with any pointer; this case is specified explicitly in the spec).
Is this Get method buggy and prone to a theoretical data race?
type item struct {
val int
mutex sync.RWMutex
}
func (i *item) Set(val int) {
i.mutex.Lock()
defer i.mutex.Unlock()
i.val = val
}
func (i *item) Get() int {
i.mutex.RLock()
defer i.mutex.RUnlock()
return i.val
}
I ask because I saw a rare data race when running my tests with -race with the former code, but can't find any way of duplicating the effect.
Is it possible for i.val to be set to a different value between when the defer carries out the RUnlock, and when we read and return the value from the struct?
Must Get() be something like this instead?:
func (i *item) Get() int {
i.mutex.RLock()
defer i.mutex.RUnlock()
val := i.val
return val
}
Your code is safe, deferred functions are executed after the expression list of the return statement is evaluated. If you would have named result parameters, the return values would also be assigned to them before calling the deferred functions (and you could even modify the return values before "truly" returning from the enclosing function).
No need to create a local variable to store i.val.
Is there a way to obtain the total number of finalizers registered using runtime.SetFinalizer and which have not yet run?
We are considering adding a struct with a registered finalizer to some of our products to release memory allocated using malloc, and the object could potentially have a relatively high allocation rate. It would be nice if we could monitor the number of finalizers, to make sure that they do not pile up and trigger out-of-memory errors (like they tend to with other garbage collectors).
(I'm aware that explicit deallocation would avoid this problem, but we cannot change the existing code, which does not call a Close function or something like that.)
You can keep keep a count of these objects by incrementing and decrementing a unexported package variable when a new object is created and finalized, respectively.
For example:
package main
import (
"fmt"
"runtime"
"sync/atomic"
)
var totalObjects int32
func TotalObjects() int32 {
return atomic.LoadInt32(&totalObjects)
}
type Object struct {
p uintptr // C allocated pointer
}
func NewObject() *Object {
o := &Object{
}
// TODO: perform other initializations
atomic.AddInt32(&totalObjects, 1)
runtime.SetFinalizer(o, (*Object).finalizer)
return o
}
func (o *Object) finalizer() {
atomic.AddInt32(&totalObjects, -1)
// TODO: perform finalizations
}
func main() {
fmt.Println("Total objects:", TotalObjects())
for i := 0; i < 100; i++ {
_ = NewObject()
runtime.GC()
}
fmt.Println("Total objects:", TotalObjects())
}
https://play.golang.org/p/n35QABBIcj
It's possible to make a wrapper on runtime.SetFinalizer which does the counting for you. Of course, it's a question of using it everywhere where you use SetFinalizer.
In case this is problematic, you can also modify SetFinalizer source code directly, but that requires a modified Go compiler.
Atomic integers are used as SetFinalizer may be called on different threads, and otherwise a counter may not be accurate as without those a race condition could possibly occur. Golang guarantees that finalizers are called from a single goroutine, so it's not needed for inner function.
https://play.golang.org/p/KKCH2UwTFYw
package main
import (
"fmt"
"reflect"
"runtime"
"sync/atomic"
)
var finalizersCreated int64
var finalizersRan int64
func SetFinalizer(obj interface{}, finalizer interface{}) {
finType := reflect.TypeOf(finalizer)
funcType := reflect.FuncOf([]reflect.Type{finType.In(0)}, nil, false)
f := reflect.MakeFunc(funcType, func(args []reflect.Value) []reflect.Value {
finalizersRan++
return reflect.ValueOf(finalizer).Call([]reflect.Value{args[0]})
})
runtime.SetFinalizer(obj, f.Interface())
atomic.AddInt64(&finalizersCreated, 1)
}
func main() {
v := "a"
SetFinalizer(&v, func(a *string) {
fmt.Println("Finalizer ran")
})
fmt.Println(finalizersRan, finalizersCreated)
runtime.GC()
fmt.Println(finalizersRan, finalizersCreated)
}
From the Dave Cheney Blog, the following code apparently causes a race case that can be resolved merely by changing func (RPC) version() int to func (*RPC) version() int :
package main
import (
"fmt"
"time"
)
type RPC struct {
result int
done chan struct{}
}
func (rpc *RPC) compute() {
time.Sleep(time.Second) // strenuous computation intensifies
rpc.result = 42
close(rpc.done)
}
func (RPC) version() int {
return 1 // never going to need to change this
}
func main() {
rpc := &RPC{done: make(chan struct{})}
go rpc.compute() // kick off computation in the background
version := rpc.version() // grab some other information while we're waiting
<-rpc.done // wait for computation to finish
result := rpc.result
fmt.Printf("RPC computation complete, result: %d, version: %d\n", result, version)
}
After looking over the code a few times, I was having a hard time believing that the code had a race case. However, when running with --race, it claims that there was a write at rpc.result=42 and a previous read at version := rpc.version(). I understand the write, since the goroutine changes the value of rpc.result, but what about the read? Where in the version() method does the read occur? It does not touch any of the values of rpc, just returning 1.
I would like to understand the following:
1) Why is that particular line considered a read on the rpc struct?
2) Why would changing RPC to *RPC resolve the race case?
When you have a method with value receiver like this:
func (RPC) version() int {
return 1 // never going to need to change this
}
And you call this method:
version := rpc.version() // grab some other information while we're waiting
A copy has to be made from the value rpc, which will be passed to the method (used as the receiver value).
So while one goroutine go rpc.compute() is running and is modifying the rpc struct value (rpc.result = 42), the main goroutine is making a copy of the whole rpc struct value. There! It's a race.
When you modify the receiver type to pointer:
func (*RPC) version() int {
return 1 // never going to need to change this
}
And you call this method:
version := rpc.version() // grab some other information while we're waiting
This is a shorthand for
version := (&rpc).version()
This passes the address of the rpc value to RPC.version(), it uses only the pointer as the receiver, so no copy is made of the rpc struct value. And since nothing from the struct is used / read in RPC.version(), there is no race.
Note:
Note that if RPC.version() would read the RPC.result field, it would also be a race, as one goroutine modifies it while the main goroutine would read it:
func (rpc *RPC) version() int {
return rpc.result // RACE!
}
Note #2:
Also note that if RPC.version() would read another field of RPC which is not modified in RPC.compute(), that would not be a race, e.g.:
type RPC struct {
result int
done chan struct{}
dummy int
}
func (rpc *RPC) version() int {
return rpc.dummy // Not a race
}
In object oriented languages I use class variables to track how many instances are currently spawned by incrementing on construction and decrementing on destruction.
I try to implement similar behaviour in go:
package entity
type Entity struct {
Name string
}
func New(name string) Entity {
entity := Entity{name}
counter++
return entity
}
var counter int = 0
func (e *Entity) Count() int {
return counter
}
and that works half way as I can not decrement the counter via a destructor.
Can I somehow mimic object destruction?
How would I keep track of instance count correctly?
You can use runtime.SetFinalizer like this. See here for playground version.
package main
import (
"fmt"
"runtime"
)
type Entity struct {
Name string
}
var counter int = 0
func New(name string) Entity {
entity := Entity{name}
counter++
runtime.SetFinalizer(&entity, func(_ *Entity) {
counter--
})
return entity
}
func (e *Entity) Count() int {
return counter
}
func main() {
e := New("Sausage")
fmt.Println("Entities", counter, e)
e = New("Potato")
fmt.Println("Entities", counter, e)
runtime.GC()
fmt.Println("Entities", counter)
e = New("Leek")
fmt.Println("Entities", counter)
runtime.GC()
fmt.Println("Entities", counter)
}
This prints
Entities 1 {Sausage}
Entities 2 {Potato}
Entities 0
Entities 1
Entities 0
Note this from the docs for gotchas with Finalizers
The finalizer for x is scheduled to run at some arbitrary time after x
becomes unreachable. There is no guarantee that finalizers will run
before a program exits, so typically they are useful only for
releasing non-memory resources associated with an object during a
long-running program.
There was a discussion on golang-nuts about finalizers.
For now,
there is no finalizer function (edit : no reliable finalizer function, as was shown to me by Nick)
the GC doesn't use and doesn't maintain any reference count
So you have to manage your instance count yourself.
Usually, you don't have instances living on themselves, so for many practical uses (not including the profiling of a complex and hard to understand program), you can use defer to track the end of life of your variables. I won't pretend this really replaces finalizers but it's simple and often sufficient.