Counter that rotates that is safe for concurrent use - go

I want to write something like this:
type RoundRobinList struct {
lst []string
idx uint32
}
// rr_list is a RoundRobinList
func loop(rr_list) {
start = rr_list.idx
rr_list.idx = (rr_list.idx + 1)%len(rr_list.lst)
print(rr_list.lst[idx])
}
If rr_list.lst = ["a", "b", "c"], and loop is called over and over, I would expect the following printed:
"a"
"b"
"c"
"a"
"b"
"c" ...
Is this safe? rr_list.idx = (rr_list.idx + 1)%len(rr_list.lst)

Whenever you have a read and write of a value that's not protected in some way (using a mutex, or some of the other things in the sync package), you have a race, and that means you can't use it safely in a concurrent setting.
Here, you're reading and writing the idx field of your RoundRobinList structure without any protection, so you'll have a race if you use it concurrently.
As a first line of defense you should understand the rules for memory safety and follow them carefully, and not write code unless you're pretty sure it's safe. As a second line of defense, you can use the race detector to find a lot of problems with lack of safety of concurrent access.
Here's a simple test case, in file a_test.go. I had to also fix some bugs in the code to get it to compile. It starts two goroutines which call loop 1000 times each on a shared RoundRobinList.
package main
import (
"sync"
"testing"
)
type RoundRobinList struct {
lst []string
idx uint32
}
func loop(rr *RoundRobinList) {
rr.idx = (rr.idx + 1) % uint32(len(rr.lst))
}
func TestLoop(t *testing.T) {
rr := RoundRobinList{[]string{"a", "b", "c"}, 0}
var wg sync.WaitGroup
wg.Add(2)
for n := 0; n < 2; n++ {
go func() {
for i := 0; i < 1000; i++ {
loop(&rr)
}
wg.Done()
}()
}
wg.Wait()
}
Running with go test -race ./a_test.go results in:
==================
WARNING: DATA RACE
Read at 0x00c0000b0078 by goroutine 9:
command-line-arguments.loop()
/mnt/c/Users/paul/Desktop/a_test.go:14 +0xa1
command-line-arguments.TestLoop.func1()
/mnt/c/Users/paul/Desktop/a_test.go:24 +0x78
Previous write at 0x00c0000b0078 by goroutine 8:
command-line-arguments.loop()
/mnt/c/Users/paul/Desktop/a_test.go:14 +0x4e
command-line-arguments.TestLoop.func1()
/mnt/c/Users/paul/Desktop/a_test.go:24 +0x78
Goroutine 9 (running) created at:
command-line-arguments.TestLoop()
/mnt/c/Users/paul/Desktop/a_test.go:22 +0x1cc
testing.tRunner()
/usr/local/go/src/testing/testing.go:992 +0x1eb
Goroutine 8 (finished) created at:
command-line-arguments.TestLoop()
/mnt/c/Users/paul/Desktop/a_test.go:22 +0x1cc
testing.tRunner()
/usr/local/go/src/testing/testing.go:992 +0x1eb
==================
--- FAIL: TestLoop (0.00s)
testing.go:906: race detected during execution of test
FAIL
FAIL command-line-arguments 0.006s
FAIL

Related

unclear on reasons why there is a race condition

The question concerns the following code:
package main
import "fmt"
func main() {
var counters = map[int]int{}
for i := 0; i < 5; i++ {
go func(counters map[int]int, th int) {
for j := 0; j < 5; j++ {
counters[th*10+j]++
}
}(counters, i)
}
fmt.Scanln()
fmt.Println("counters result", counters)
}
Here is the output I get when I run this code with go run -race race.go
$ go run -race race.go
==================
WARNING: DATA RACE
Read at 0x00c000092150 by goroutine 8:
runtime.mapaccess1_fast64()
/usr/lib/go-1.13/src/runtime/map_fast64.go:12 +0x0
main.main.func1()
/tmp/race.go:10 +0x6b
Previous write at 0x00c000092150 by goroutine 7:
runtime.mapassign_fast64()
/usr/lib/go-1.13/src/runtime/map_fast64.go:92 +0x0
main.main.func1()
/tmp/race.go:10 +0xaf
Goroutine 8 (running) created at:
main.main()
/tmp/race.go:8 +0x67
Goroutine 7 (finished) created at:
main.main()
/tmp/race.go:8 +0x67
==================
==================
WARNING: DATA RACE
Read at 0x00c0000aa188 by main goroutine:
reflect.typedmemmove()
/usr/lib/go-1.13/src/runtime/mbarrier.go:177 +0x0
reflect.copyVal()
/usr/lib/go-1.13/src/reflect/value.go:1297 +0x7b
reflect.(*MapIter).Value()
/usr/lib/go-1.13/src/reflect/value.go:1251 +0x15e
internal/fmtsort.Sort()
/usr/lib/go-1.13/src/internal/fmtsort/sort.go:61 +0x259
fmt.(*pp).printValue()
/usr/lib/go-1.13/src/fmt/print.go:773 +0x146f
fmt.(*pp).printArg()
/usr/lib/go-1.13/src/fmt/print.go:716 +0x2ee
fmt.(*pp).doPrintln()
/usr/lib/go-1.13/src/fmt/print.go:1173 +0xad
fmt.Fprintln()
/usr/lib/go-1.13/src/fmt/print.go:264 +0x65
main.main()
/usr/lib/go-1.13/src/fmt/print.go:274 +0x13c
Previous write at 0x00c0000aa188 by goroutine 10:
main.main.func1()
/tmp/race.go:10 +0xc4
Goroutine 10 (finished) created at:
main.main()
/tmp/race.go:8 +0x67
==================
counters result map[0:1 1:1 2:1 3:1 4:1 10:1 11:1 12:1 13:1 14:1 20:1 21:1 22:1 23:1 24:1 30:1 31:1 32:1 33:1 34:1 40:1 41:1 42:1 43:1 44:1]
Found 2 data race(s)
exit status 66
Here is what I can't understand. Why there a race condition at all? Aren't we reading/writing values only one go routine can access? For example routine 0 will modify values only in counter[0] through counters[4], routine 1 will modify values only in counters[10] through counters[14], routine 2 will only modify values in counters[20] through counters[24] and so on. I'm not seeing a race condition here. Feels like I'm missing something. Will someone be able to shed some light on this?
Just an FYI I'm new to go. If you could dumb down the explanation (if it is possible) I would appreciate it.
That would be true for an array (or a slice), but a map is a complicated data structure which, among others, have the following properties:
It's free to relocate the elements stored in it in memory at any time it sees fit.
A map is initially empty, and placing an element in it (what appears as assignment in your case) involves a lot of operations on the map's internals.
Additionally, in a case like yours — incrementing an integer stored in a map — is really a map lookup, increment, and a map store.
The first and the last operations involve lookup by key.
Now consider what happens if one goroutine performs lookup at the same time another goroutine modifies the map's internal state when performing map store.
You might want to read up a bit on what is an associative array, and how it's typically implemented.
Aren't we reading/writing values only one go routine can access?
You already got a great answer from #kostix on that matter: the internals of the map are modified when you add elements to it, so it's not accurate to think that routine 0 will modify values only in counter[0] through counters[4].
But that's not all.
There's yet another data race issue in your code that's a bit more subtle and might be very difficult to catch even in tests.
To explore it, let's get rid of the "map internals" issue that #kostix mentioned, by imagining that your code is almost exactly the same, but with one tiny change: instead of using a map[int]int, imagine that you're using a []int, initialized to have at least length 56. Something like this:
// THERE'S ANOTHER RACE CONDITION HERE.
// var counters = map[int]int{}
var counters = make([]int, 56)
for i := 0; i < 5; i++ {
// go func(counters map[int]int, th int) {
go func(counters []int, th int) {
for j := 0; j < 5; j++ {
counters[th*10+j]++
}
}(counters, i)
}
fmt.Scanln()
fmt.Println("counters result", counters)
This is nearly equivalent, but gets rid of the "map internals" issue. The goal is to shift the focus away from "map internals" to show you the second issue.
There's still a race condition there. By the way, it's also similar to a race condition that exists in the first attempted solution in another answer you got, that uses a sync.Mutex but in a way that is still wrong.
The problem here is that there's no happens before relationship between the operations that change the counters and the operation that reads from it.
The fmt.Scanln() doesn't help: even though it allows you to introduce an arbitrary time delay between the code right before it (i.e., when the for loop launches the goroutines) and the code right after it (i.e., the fmt.Println()) — so that you could think "Ok, I'm just gonna wait 'a reasonably long amount of time' before pressing Enter", that doesn't eliminate the race condition.
The race condition here arises from the fact that "passage of time" (i.e., you waiting to hit Enter) does not establish a happens-before relationship between the writes to counters and the reads from it.
This notion of happens-before is absolutely fundamental for avoiding data races: you can only guarantee the absence of a data race if you can guarantee the existence of a happens-before relationship between 2 operations.
Like I mentioned, "passage of time" doesn't establish a "happens before". To establish it, you could use one of many alternatives, including primitives in the sync or atomic packages, or channels, etc.
While I'd probably suggest focusing on studying channels, and then the sync package (sync.Mutex, sync.WaitGroup, etc), and maybe only after all that the atomic package, if you do want to read more about this idea of happens before from the authoritative source, here's the link: https://golang.org/ref/mem . But be warned that it's a nasty can of worms.
Hopefully these comments here help you see why it's absolutely fundamental to follow the standard patterns for concurrency in Go. Things can be way more subtle than at first sight.
And to conclude, a quote from The Go Memory Model link I shared above:
If you must read the rest of this document to understand the behavior of your program, you are being too clever.
Don't be clever.
EDIT: for completion, here's how you could solve the problem.
There are 2 parts to the solution: (1) make sure that there's no concurrent modifications to the map; (2) make sure that there's a happens-before between all the changes to the map and the read.
For (1), you can use a sync.Mutex. Lock it before writing, unlock it after the write.
For (2), you need to ensure that the main goroutine can only get to the fmt.Println() after all the modifications are done. And remember: here, after doesn't mean "at a later point in time", but it specifically means that a happens-before relationship must be established. The 2 common patterns to solve this are to use a channel or a sync.WaitGroup. The WaitGroup solution is probably easier to reason about here, so that's what I'd use.
var mu sync.Mutex // (A)
var wg sync.WaitGroup // (A)
var counters = map[int]int{}
wg.Add(5) // (B)
for i := 0; i < 5; i++ {
go func(counters map[int]int, th int) {
for j := 0; j < 5; j++ {
mu.Lock() // (C)
counters[th*10+j]++
mu.Unlock() // (C)
}
wg.Done() // (D)
}(counters, i)
}
wg.Wait() // (E)
fmt.Scanln()
fmt.Println("counters result", counters)
(A) You don't need to initialize either the Mutex nor the WaitGroup, since their zero values are ready to use. Also, you don't need to make them pointers to anything.
(B) You .Add(5) to the WaitGroup's counter, meaning that it will have to wait for 5 .Done() signals before proceeding if you .Wait() on it. The number 5 here is because you're launching 5 goroutines, and you need to establish happens-before relationships between the changes made on all of them and the main goroutine's fmt.Println().
(C) You .Lock() and .Unlock() the Mutex around modifications to the map, to ensure that they are not done concurrently.
(D) Just before each goroutine terminates, you call wg.Done(), which decrements the WaitGroup's internal counter.
(E) Finally, you wg.Wait(). This function blocks until the wg's counter reaches 0. And here's the super important piece: the WaitGroup establishes a happens-before relationship between the calls to wg.Done() and the return of the wg.Wait() call. In other words, from a memory consistency perspective, the main goroutine is guaranteed to see all the changes performed to the map by all the goroutines!
AND FINALLY you can run that code with -race and be happy!
For you to explore further: instead of map + sync.Mutex, you could replace that with just sync.Map. But the sync.WaitGroup would still be necessary. Try to write a solution using that, it might be a nice exercise.
In addition to #kostix answer. You've to know that multiple goroutines should not access (write/read) to the same ressource at a given time.
So, in your implementation you may easly be in the case that multiple goroutines are updating (reading/writing) concurrently the same ressource (which is your map) at the same time.
What should happen ? Which value should be in this given map key ? This a what called race condition
Here is some potential fixes to your code:
Using Mutex:
package main
import (
"fmt"
"sync"
)
func main() {
var counters = map[int]int{}
var mutex = &sync.Mutex{}
for i := 0; i < 3; i++ {
go func(counters map[int]int, th int) {
for j := 0; j < 3; j++ {
mutex.Lock() // Lock the access to the map
counters[th*10+j]++
mutex.Unlock() // Release the access
}
}(counters, i)
}
fmt.Scanln()
fmt.Println("counters result", counters)
}
Output:
counters result map[0:1 1:1 2:1 10:1 11:1 12:1 20:1 21:1 22:1]
Using sync.Map:
package main
import (
"fmt"
"sync"
)
func main() {
var counters sync.Map
for i := 0; i < 3; i++ {
go func(th int) {
for j := 0; j < 3; j++ {
if result, ok := counters.Load(th*10 + j); ok {
value := result.(int) + 1
counters.Store(th*10+j, value+1)
} else {
counters.Store(th*10+j, 1)
}
}
}(i)
}
fmt.Scanln()
counters.Range(func(k, v interface{}) bool {
fmt.Println("key:", k, ", value:", v)
return true
})
}
Output:
key: 21 , value: 1
key: 10 , value: 1
key: 11 , value: 1
key: 0 , value: 1
key: 1 , value: 1
key: 20 , value: 1
key: 2 , value: 1
key: 22 , value: 1
key: 12 , value: 1

What is causing this data race?

Why does this code cause data race?
I have already used atomic add.
package main
import (
"sync/atomic"
"time"
)
var a int64
func main() {
for {
if a < 100 {
atomic.AddInt64(&a, 1)
go run()
}
}
}
func run() {
<-time.After(5 * time.Second)
atomic.AddInt64(&a, -1)
}
I run command go run --race with this code and get:
==================
WARNING: DATA RACE
Write at 0x000001150f30 by goroutine 8:
sync/atomic.AddInt64()
/usr/local/Cellar/go/1.11.2/libexec/src/runtime/race_amd64.s:276 +0xb
main.run()
/Users/flask/test.go:22 +0x6d
Previous read at 0x000001150f30 by main goroutine:
main.main()
/Users/flask/test.go:12 +0x3a
Goroutine 8 (running) created at:
main.main()
/Users/flask/test.go:15 +0x75
==================
Could you help me explain this?
And how to fix this warning?
Thanks!
You didn't use the atomic package at all places where you accessed the variable. All access must be synchronized to variables that are accessed from multiple goroutines concurrently, including reads:
for {
if value := atomic.LoadInt64(&a); value < 100 {
atomic.AddInt64(&a, 1)
go run()
}
}
With that change, the race condition goes away.
If you just want to inspect the value, you don't even need to store it in a variable, so you may simply do:
for {
if atomic.LoadInt64(&a) < 100 {
atomic.AddInt64(&a, 1)
go run()
}
}

Most simple Go race-condition example?

I need a simple Go code sample which will definitely run the program into an race-condition.
Any ideas?
The original question:
I need a simple Go code sample which will definitely run the program
into an race-condition.
For example,
racer.go:
package main
import (
"time"
)
var count int
func race() {
count++
}
func main() {
go race()
go race()
time.Sleep(1 * time.Second)
}
Output:
$ go run -race racer.go
==================
WARNING: DATA RACE
Read at 0x00000052ccf8 by goroutine 6:
main.race()
/home/peter/gopath/src/racer.go:10 +0x3a
Previous write at 0x00000052ccf8 by goroutine 5:
main.race()
/home/peter/gopath/src/racer.go:10 +0x56
Goroutine 6 (running) created at:
main.main()
/home/peter/gopath/src/racer.go:15 +0x5a
Goroutine 5 (finished) created at:
main.main()
/home/peter/gopath/src/racer.go:14 +0x42
==================
Found 1 data race(s)
exit status 66
$
package main
import (
"fmt"
)
func main() {
i := 0
// Run forever to make it to show race condition
for {
var x, y int
// Set x to 60
go func(v *int) {
*v = 60
}(&x)
// Set y to 3
go func(v *int) {
*v = 3
}(&y)
/*
A race condition is when multiple threads are trying to access and manipulate the same variable.
the code below is all accessing and changing the value.
Divide x (60) by y (3) and assign to z (42)...
Except if y is not assigned 3 before x is assigned 60,
y's initialized value of 0 is used,
Due to the uncertainty of the Goroutine scheduling mechanism, the results of the following program is unpredictable,
which causes a divide by zero exception.
*/
go func(v1 int, v2 int) {
fmt.Println(v1 / v2)
}(x, y)
i += 1
fmt.Printf("%d\n", i)
}
}
Run code using: go run -race .go

Race condition even when using sync.Mutex in golang

Complete code is here: https://play.golang.org/p/ggUoxtcv5m
go run -race main.go says there is a race condition there which I fail to explain.
The program outputs correct final result, though.
The essence:
type SafeCounter struct {
c int
sync.Mutex
}
func (c *SafeCounter) Add() {
c.Lock()
c.c++
c.Unlock()
}
var counter *SafeCounter = &SafeCounter{} // global
use *SafeCounter in incrementor:
func incrementor(s string) {
for i := 0; i < 20; i++ {
x := counter
x.Add()
counter = x
}
}
The incrementor method is spawned twice in main:
func main() {
go incrementor()
go incrementor()
// some other non-really-related stuff like
// using waitGroup is ommited here for problem showcase
}
So, as I said, go run -race main.go will always say there is a race cond found.
Also, the final result is always correct (at least I've run this program for a number of times and it always say final counter is 40, which is correct).
BUT, the program prints incorrect values in the beginning so you can get something like:
Incrementor1: 0 Counter: 2
Incrementor2: 0 Counter: 3
Incrementor2: 1 Counter: 4
// ang the rest is ok
so, printing out 1 is missing there.
Can somebody explain why there is a race condition there is my code?
You have a number of race conditions, all pointed out specifically by the race detector:
x := counter // this reads the counter value without a lock
fmt.Println(&x.c)
x.Add()
counter = x // this writes the counter value without a lock
time.Sleep(time.Duration(rand.Intn(3)) * time.Millisecond)
fmt.Println(s, i, "Counter:", x.c) // this reads the c field without a lock
race #1 is between the read and the write of the counter value in incrementor
race #2 is between the concurrent writes to the counter value in incrementor
race #3 is between the read of the x.c field in fmt.Println, and the increment to x.c in the Add method.
The two lines that read and write the counter pointer are not protected by the mutex and are done concurrently from multiple goroutines.
func incrementor(s string) {
for i := 0; i < 20; i++ {
x := counter // <-- this pointer read
x.Add()
counter = x // <-- races with this pointer write
}
}

Data race occurs with xml.Marshal

I'm getting a strange data race reported when I use xml.Marshal on a struct. I am 99% sure that I am not sending the same variable or something like that - instead I get intermittent errors in which the marshal function believes that it is causing a data race.
Here's the code (simplified a bit but all functional elements are there):
// this is run prior to any calls being sent to the below functions
func Setup() (descriptor *serviceDescriptor) {
descriptor = new(serviceDescriptor)
wpChan := make(chan *Call)
for i := 1; i < 100; i++ {
go serviceWorker(wpChan)
}
descriptor.wpChan = wpChan
return
}
// called externally to initiate a call
func (s *serviceDescriptor) Add(c *Call) {
go s.makeCall(c)
}
// sends a call to the worker pool set up in the Setup() function
func (s *serviceDescriptor) makeCall(c *Call) {
cw := new(callwrapper)
cw.internal = new(etFullCall)
cw.internal.Calldata = c
/// this is a channel that the next function listens to
s.wpChan <- ct
// the result is sent back on a channel and processed here
}
// this is a function
func worker(wpChan chan *callwrapper) {
for cw := range wpChan {
v := new(Result)
// this is where the data race occurs.
byt, err := xml.Marshal(cw.internal)
// do stuff with the result down here
}
}
Here's the error:
WARNING: DATA RACE
Write by goroutine 115:
runtime.copy()
/usr/lib/go/src/pkg/runtime/slice.c:120 +0x0
encoding/xml.(*parentStack).push()
/usr/lib/go/src/pkg/encoding/xml/marshal.go:901 +0x2bd
encoding/xml.(*printer).marshalStruct()
/usr/lib/go/src/pkg/encoding/xml/marshal.go:819 +0x58c
encoding/xml.(*printer).marshalValue()
/usr/lib/go/src/pkg/encoding/xml/marshal.go:524 +0x12a8
encoding/xml.(*Encoder).Encode()
/usr/lib/go/src/pkg/encoding/xml/marshal.go:153 +0x83
encoding/xml.Marshal()
/usr/lib/go/src/pkg/encoding/xml/marshal.go:72 +0x9d
exacttarget.serviceWorker()
/service.go:94 +0x20e
Previous write by goroutine 114:
runtime.copy()
/usr/lib/go/src/pkg/runtime/slice.c:120 +0x0
encoding/xml.(*parentStack).push()
/usr/lib/go/src/pkg/encoding/xml/marshal.go:901 +0x2bd
encoding/xml.(*printer).marshalStruct()
/usr/lib/go/src/pkg/encoding/xml/marshal.go:819 +0x58c
encoding/xml.(*printer).marshalValue()
/usr/lib/go/src/pkg/encoding/xml/marshal.go:524 +0x12a8
encoding/xml.(*Encoder).Encode()
/usr/lib/go/src/pkg/encoding/xml/marshal.go:153 +0x83
encoding/xml.Marshal()
/usr/lib/go/src/pkg/encoding/xml/marshal.go:72 +0x9d
exacttarget.serviceWorker()
/service.go:94 +0x20e
Goroutine 115 (running) created at:
service.Setup()
/service.go:39 +0x112
/src.processSingleUser()
/main_test.go:405 +0xdf
/src.testAddLists()
/main_test.go:306 +0x1f2
Goroutine 114 (running) created at:
service.Setup()
/service.go:39 +0x112
/src.processSingleUser()
/main_test.go:405 +0xdf
src.testAddLists()
/main_test.go:306 +0x1f2

Resources