Why the result is not as expected with flag "-race" ?
It expected the same result: 1000000 - with flag "-race" and without this
https://gist.github.com/romanitalian/f403ceb6e492eaf6ba953cf67d5a22ff
package main
import (
"fmt"
"runtime"
"sync/atomic"
"time"
)
//$ go run -race main_atomic.go
//954203
//
//$ go run main_atomic.go
//1000000
type atomicCounter struct {
val int64
}
func (c *atomicCounter) Add(x int64) {
atomic.AddInt64(&c.val, x)
runtime.Gosched()
}
func (c *atomicCounter) Value() int64 {
return atomic.LoadInt64(&c.val)
}
func main() {
counter := atomicCounter{}
for i := 0; i < 100; i++ {
go func(no int) {
for i := 0; i < 10000; i++ {
counter.Add(1)
}
}(i)
}
time.Sleep(time.Second)
fmt.Println(counter.Value())
}
The reason why the result is not the same is because time.Sleep(time.Second) does not guarantee that all of your goroutines are going to be executed in the timespan of one second. Even if you execute go run main.go, it's not guaranteed that you will get the same result every time. You can test this out if you put time.Milisecond instead of time.Second, you will see much more inconsistent results.
Whatever value you put in the time.Sleep method, it does not guarantee that all of your goroutines will be executed, it just means that it's less likely that all of your goroutines won't finish in time.
For consistent results, you would want to synchronise your goroutines a bit. You can use WaitGroup or channels.
With WaitGroup:
//rest of the code above is the same
func main() {
counter := atomicCounter{}
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func(no int) {
for i := 0; i < 10000; i++ {
counter.Add(1)
}
wg.Done()
}(i)
}
wg.Wait()
fmt.Println(counter.Value())
}
With channels:
func main() {
valStream := make(chan int)
doneStream := make(chan int)
result := 0
for i := 0; i < 100; i++ {
go func() {
for i := 0; i < 10000; i++ {
valStream <- 1
}
doneStream <- 1
}()
}
go func() {
counter := 0
for count := range doneStream {
counter += count
if counter == 100 {
close(doneStream)
}
}
close(valStream)
}()
for val := range valStream {
result += val
}
fmt.Println(result)
}
Related
This question already has answers here:
Why does adding concurrency slow down this golang code?
(4 answers)
Closed 3 months ago.
I wrote some code to execute Monte Carlo simulations. The first thing I wrote was this sequential version:
func simulationSequential(experiment func() bool, numTrials int) float64 {
ocurrencesEvent := 0
for trial := 0; trial < numTrials; trial++ {
eventHappend := experiment()
if eventHappend {
ocurrencesEvent++
}
}
return float64(ocurrencesEvent) / float64(numTrials)
}
Then, I figured I could run some of the experiments concurrently and get a result faster using my laptop's multiple cores. So, I wrote the following version:
func simulationConcurrent(experiment func() bool, numTrials, nGoroutines int) float64 {
ch := make(chan int)
var wg sync.WaitGroup
// Launch work in multiple goroutines
for i := 0; i < nGoroutines; i++ {
wg.Add(1)
go func() {
localOcurrences := 0
for j := 0; j < numTrials/nGoroutines; j++ {
eventHappend := experiment()
if eventHappend {
localOcurrences++
}
}
ch <- localOcurrences
wg.Done()
}()
}
// Close the channel when all the goroutines are done
go func() {
wg.Wait()
close(ch)
}()
// Acummulate the results of each experiment
ocurrencesEvent := 0
for localOcurrences := range ch {
ocurrencesEvent += localOcurrences
}
return float64(ocurrencesEvent) / float64(numTrials)
}
To my surprise, when I run benchmarks on the two versions, I get that the sequential is faster than the concurrent one, with the concurrent version getting better as I decrease the number of goroutines. Why does this happen? I thought the concurrent version will be faster since this is a highly parallelizable problem.
Here is my benchmark's code:
func tossEqualToSix() bool {
// Simulate the toss of a six-sided die
roll := rand.Intn(6) + 1
if roll != 6 {
return false
}
return true
}
const (
numsSimBenchmark = 1_000_000
numGoroutinesBenckmark = 10
)
func BenchmarkSimulationSequential(b *testing.B) {
for i := 0; i < b.N; i++ {
simulationSequential(tossEqualToSix, numsSimBenchmark)
}
}
func BenchmarkSimulationConcurrent(b *testing.B) {
for i := 0; i < b.N; i++ {
simulationConcurrent(tossEqualToSix, numsSimBenchmark, numGoroutinesBenckmark)
}
}
And the results:
goos: linux
goarch: amd64
pkg: github.com/jpuriol/montecarlo
cpu: Intel(R) Core(TM) i7-10510U CPU # 1.80GHz
BenchmarkSimulationSequential-8 36 30453588 ns/op
BenchmarkSimulationConcurrent-8 9 117462720 ns/op
PASS
ok github.com/jpuriol/montecarlo 2.478s
You can download my code from Github.
I thought I would elaborate on my comment and post it with code and benchmark result.
Examine function uses package level rand functions from rand package. These functions under the hood uses globalRand instance of rand.Rand. For example func Intn(n int) int { return globalRand.Intn(n) }. As the random number generator is not thread safe, the globalRand is instantiated in following way:
/*
* Top-level convenience functions
*/
var globalRand = New(&lockedSource{src: NewSource(1).(*rngSource)})
type lockedSource struct {
lk sync.Mutex
src *rngSource
}
func (r *lockedSource) Int63() (n int64) {
r.lk.Lock()
n = r.src.Int63()
r.lk.Unlock()
return
}
...
This means that all invocations of rand.Intn are guarded by the global lock. The consequence is that examine function "works sequentially", because of the lock. More specifically, each call to rand.Intn will not start generating a random number before the previous call completes.
Here is redesigned code. Each examine function has its own random generator. The assumption is that single examine function is used by one goroutine, so it does not require lock protection.
package main
import (
"math/rand"
"sync"
"testing"
"time"
)
func simulationSequential(experimentFuncFactory func() func() bool, numTrials int) float64 {
experiment := experimentFuncFactory()
ocurrencesEvent := 0
for trial := 0; trial < numTrials; trial++ {
eventHappend := experiment()
if eventHappend {
ocurrencesEvent++
}
}
return float64(ocurrencesEvent) / float64(numTrials)
}
func simulationConcurrent(experimentFuncFactory func() func() bool, numTrials, nGoroutines int) float64 {
ch := make(chan int)
var wg sync.WaitGroup
// Launch work in multiple goroutines
for i := 0; i < nGoroutines; i++ {
wg.Add(1)
go func() {
experiment := experimentFuncFactory()
localOcurrences := 0
for j := 0; j < numTrials/nGoroutines; j++ {
eventHappend := experiment()
if eventHappend {
localOcurrences++
}
}
ch <- localOcurrences
wg.Done()
}()
}
// Close the channel when all the goroutines are done
go func() {
wg.Wait()
close(ch)
}()
// Acummulate the results of each experiment
ocurrencesEvent := 0
for localOcurrences := range ch {
ocurrencesEvent += localOcurrences
}
return float64(ocurrencesEvent) / float64(numTrials)
}
func tossEqualToSix() func() bool {
prng := rand.New(rand.NewSource(time.Now().UnixNano()))
return func() bool {
// Simulate the toss of a six-sided die
roll := prng.Intn(6) + 1
if roll != 6 {
return false
}
return true
}
}
const (
numsSimBenchmark = 5_000_000
numGoroutinesBenchmark = 10
)
func BenchmarkSimulationSequential(b *testing.B) {
for i := 0; i < b.N; i++ {
simulationSequential(tossEqualToSix, numsSimBenchmark)
}
}
func BenchmarkSimulationConcurrent(b *testing.B) {
for i := 0; i < b.N; i++ {
simulationConcurrent(tossEqualToSix, numsSimBenchmark, numGoroutinesBenchmark)
}
}
Benchmark result are as follows:
goos: darwin
goarch: arm64
pkg: scratchpad
BenchmarkSimulationSequential-8 20 55142896 ns/op
BenchmarkSimulationConcurrent-8 82 12944360 ns/op
in below example, the race detector will trigger an error. I am fine with it, though, as it does not change keys, the map header (if i might say), i struggle figuring out what is the reason of the race. I simply dont understand what is going on under hood so that a race detection is emitted.
package main
import (
"fmt"
"sync"
)
// scores holds values incremented by multiple goroutines.
var scores = make(map[string]int)
func main() {
var wg sync.WaitGroup
wg.Add(2)
scores["A"] = 0
scores["B"] = 0
go func() {
for i := 0; i < 1000; i++ {
// if _, ok := scores["A"]; !ok {
// scores["A"] = 1
// } else {
scores["A"]++
// }
}
wg.Done()
}()
go func() {
for i := 0; i < 1000; i++ {
scores["B"]++ // Line 28
}
wg.Done()
}()
wg.Wait()
fmt.Println("Final scores:", scores)
}
Map values are not addressable, so incrementing the integer values requires writing them back to the map itself.
The line
scores["A"]++
is equivalent to
tmp := scores["A"]
scores["A"] = tmp + 1
If you use a pointer to make the integer values addressable, and assign all the keys before the goroutines are dispatched, you can see there is no longer a race on the map itself:
var scores = make(map[string]*int)
func main() {
var wg sync.WaitGroup
wg.Add(2)
scores["A"] = new(int)
scores["B"] = new(int)
go func() {
for i := 0; i < 1000; i++ {
(*scores["A"])++
}
wg.Done()
}()
go func() {
for i := 0; i < 1000; i++ {
(*scores["B"])++
}
wg.Done()
}()
wg.Wait()
fmt.Println("Final scores:", scores)
}
I'm trying launch goroutines in an infinite loop until I get the response I'm looking for but the select is unreachable if I change for i := 0; i < 10; i++ {} to for {}. What's a pattern for solving this?
package main
import (
"fmt"
"math/rand"
"time"
)
func myFunc() float64 {
c := make(chan float64)
for i := 0; i < 10; i++ {
go func() {
var value float64
value = someOp()
if value > .9 {
c <- value
}
}()
}
// unreachable code the if the for loop above is infinite
for {
select {
case x := <-c:
return x
default:
}
}
}
func someOp() float64 {
rand.Seed(time.Now().UnixNano())
return rand.Float64()
}
func main() {
fmt.Println(myFunc())
}
Starting an unlimited number of goroutines is usually not a good idea. A better approach is to start a fixed number of goroutines that loop looking for an answer. Return from these goroutines when an answer is found.
func myFunc() float64 {
c := make(chan float64, 1) // Size 1 prevents race between main goroutine and workers
done := make(chan struct{})
defer close(done)
// Start a fixed number of goroutines
for i := 0; i < 10; i++ {
go func() {
for {
select {
case <-done:
// myfunc exited with result, return from this goroutine
return
default:
var value float64
value = someOp()
if value > .9 {
select {
case c <- value:
// This is first goroutine to send a value
default:
// Another goroutine sent a value
}
return
}
}
}
}()
}
return <-c
}
https://play.golang.org/p/SRpeT8k34eA
I am new to golang, and I am puzzled with this deadlock (run here)
package main
import (
"fmt"
"runtime"
"time"
)
func main() {
c := make(chan string)
work := make(chan int, 1)
clvl := runtime.NumCPU()
count := 0
for i := 0; i < clvl; i++ {
go func(i int) {
for jdId := range work {
time.Sleep(time.Second * 1)
c <- fmt.Sprintf("done %d", jdId)
}
}(i)
}
go func() {
for i := 0; i < 10; i++ {
work <- i
}
close(work)
}()
for resp := range c {
fmt.Println(resp, count)
count += 1
}
}
You never close c, so your for range loop waits forever. Close it like this:
var wg sync.WaitGroup
for i := 0; i < clvl; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
for jdId := range work {
time.Sleep(time.Second * 1)
c <- fmt.Sprintf("done %d", jdId)
}
}(i)
}
go func() {
for i := 0; i < 10; i++ {
work <- i
}
close(work)
wg.Wait()
close(c)
}()
EDIT: Fixed the panic, thanks Crast
Alright, Go "experts". How would you write this code in idiomatic Go, aka without a mutex in next?
package main
import (
"fmt"
)
func main() {
done := make(chan int)
x := 0
for i := 0; i < 10; i++ {
go func() {
y := next(&x)
fmt.Println(y)
done <- 0
}()
}
for i := 0; i < 10; i++ {
<-done
}
fmt.Println(x)
}
var mutex = make(chan int, 1)
func next(p *int) int {
mutex <- 0
// critical section BEGIN
x := *p
*p++
// critical section END
<-mutex
return x
}
Assume you can't have two goroutines in the critical section at the same time, or else bad things will happen.
My first guess is to have a separate goroutine to handle the state, but I can't figure out a way to match up inputs / outputs.
You would use an actual sync.Mutex:
var mutex sync.Mutex
func next(p *int) int {
mutex.Lock()
defer mutex.Unlock()
x := *p
*p++
return x
}
Though you would probably also group the next functionality, state and sync.Mutex into a single struct.
Though there's no reason to do so in this case, since a Mutex is better suited for mutual exclusion around a single resource, you can use goroutines and channels to achieve the same effect
http://play.golang.org/p/RR4TQXf2ct
x := 0
var wg sync.WaitGroup
send := make(chan *int)
recv := make(chan int)
go func() {
for i := range send {
x := *i
*i++
recv <- x
}
}()
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
send <- &x
fmt.Println(<-recv)
}()
}
wg.Wait()
fmt.Println(x)
As #favoretti mentioned, sync/atomic is a way to do it.
But, you have to use int32 or int64 rather than int (since int can be different sizes on different platforms).
Here's an example on Playground
package main
import (
"fmt"
"sync/atomic"
)
func main() {
done := make(chan int)
x := int64(0)
for i := 0; i < 10; i++ {
go func() {
y := next(&x)
fmt.Println(y)
done <- 0
}()
}
for i := 0; i < 10; i++ {
<-done
}
fmt.Println(x)
}
func next(p *int64) int64 {
return atomic.AddInt64(p, 1) - 1
}