I'm facing a problem in golang
var a = 0
func main() {
go func() {
for {
a = a + 1
}
}()
time.Sleep(time.Second)
fmt.Printf("result=%d\n", a)
}
expected: result=(a big int number)
result: result=0
You have a race condition,
run your program with -race flag
go run -race main.go
==================
WARNING: DATA RACE
Read at 0x0000005e9600 by main goroutine:
main.main()
/home/jack/Project/GoProject/src/gitlab.com/hooshyar/GoNetworkLab/StackOVerflow/race/main.go:17 +0x6c
Previous write at 0x0000005e9600 by goroutine 6:
main.main.func1()
/home/jack/Project/GoProject/src/gitlab.com/hooshyar/GoNetworkLab/StackOVerflow/race/main.go:13 +0x56
Goroutine 6 (running) created at:
main.main()
/home/jack/Project/GoProject/src/gitlab.com/hooshyar/GoNetworkLab/StackOVerflow/race/main.go:11 +0x46
==================
result=119657339
Found 1 data race(s)
exit status 66
what is solution?
There is some solution, A solution is using a mutex:
var a = 0
func main() {
var mu sync.Mutex
go func() {
for {
mu.Lock()
a = a + 1
mu.Unlock()
}
}()
time.Sleep(3*time.Second)
mu.Lock()
fmt.Printf("result=%d\n", a)
mu.Unlock()
}
before any read and write lock the mutex and then unlock it, now you don not have any race and resault will bi big int at the end.
For more information read this topic.
Data races in Go(Golang) and how to fix them
and this
Golang concurrency - data races
As other writers have mentioned, you have a data race, but if you are comparing this behavior to, say, a program written in C using pthreads, you are missing some important data. Your problem is not just about timing, it's about the very language definition. Because concurrency primitives are baked into the language itself, the Go language memory model (https://golang.org/ref/mem) describes exactly when and how changes in one goroutine -- think of goroutines as "super-lightweight user-space threads" and you won't be too far off -- are guaranteed to be visible to code running in another goroutine.
Without any synchronizing actions, like channel sends/receives or sync.Mutex locks/unlocks, the Go memory model says that any changes you make to 'a' inside that goroutine don't ever have to be visible to the main goroutine. And, since the compiler knows that, it is free to optimize away pretty much everything in your for loop. Or not.
It's a similar situation to when you have, say, a local int variable in C set to 1, and maybe you have a while loop reading that variable in a loop waiting for it to be set to 0 by an ISR, but then your compiler gets too clever and decides to optimize away the test for zero because it thinks your variable can't ever change within the loop and you really just wanted an infinite loop, and so you have to declare the variable as volatile to fix the 'bug'.
If you are going to be working in Go, (my current favorite language, FWIW,) take time to read and thoroughly grok the Go memory model linked above, and it will really pay off in the future.
Your program is running into race condition. go can detect such scenarios.
Try running your program using go run -race main.go assuming your file name is main.go. It will show how race occured ,
attempted write inside the goroutine ,
simultaneous read by the main goroutine.
It will also print a random int number as you expected.
Related
This is my entire Go code! What confused me is that case balances <- balance: did't occurs.I dont know why?
package main
import (
"fmt"
)
func main() {
done := make(chan int)
var balance int
balances := make(chan int)
balance = 1
go func() {
fmt.Println(<-balances)
done <- 1
}()
select {
case balances <- balance:
fmt.Println("done case")
default:
fmt.Println("default case")
}
<-done
}
default case
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
/tmp/sandbox575832950/prog.go:29 +0x13d
goroutine 18 [chan receive]:
main.main.func1()
/tmp/sandbox575832950/prog.go:17 +0x38
created by main.main
/tmp/sandbox575832950/prog.go:16 +0x97
The main goroutine executes the select before the anonymous goroutine function executes the receive from balances. The main goroutine executes the default clause in the select because there is no ready receiver on balances. The main goroutine continues on to receive on done.
The goroutine blocks on receive from balances because there is no sender. Main continued past the send by taking the default clause.
The main goroutine blocks on receive from done because there is no sender. The goroutine is blocked on receive from balances.
Fix by replacing the select statement with balances <- balance. The default clause causes the problem. When the the default class is removed, all that remains in the select is send to balances.
Because of concurrency, there's no guarantee that the goroutine will execute before the select. We can see this by adding a print to the goroutine.
go func() {
fmt.Println("Here")
fmt.Println(<-balances)
done <- 1
}()
$ go run test.go
default case
Here
fatal error: all goroutines are asleep - deadlock!
...
If the select runs first, balances <- balance would block; balances has no buffer and nothing is trying to read from it. case balances <- balance would block so select skips it and executes its default.
Then the goroutine runs and blocks reading balances. Meanwhile the main code blocks reading done. Deadlock.
You can solve this by either removing the default case from the select and allowing it to block until balances is ready to be written to.
select {
case balances <- balance:
fmt.Println("done case")
}
Or you can add a buffer to balances so it can be written to before it is read from. Then case balances <- balance does not block.
balances := make(chan int, 1)
What confused me is that case balances <- balance: did't occurs
To be specific: it's because of select with a default case.
Whenever you create a new goroutine with go ...(), there is no guarantee about whether the invoking goroutine, or the invoked goroutine, will run next.
In practice it's likely that the next statements in the invoking goroutine will execute next (there being no particularly good reason to stop it). Of course, we should write programs that function correctly all the time, not just some, most, or even almost all the time! Concurrent programming with go ...() is all about synchronizing the goroutines so that the intended behavior must occur. Channels can do that, if used properly.
I think the balances channel can receive data
It's an unbuffered channel, so it can receive data if someone is reading from it. Otherwise, that write to the channel will block. Which brings us back to select.
Since you provided a default case, it's quite likely that the goroutine that invoked go ...() will continue to execute, and select that can't immediately choose a different case, will choose default. So it would be very unlikely for the invoked goroutine to be ready to read from balances before the main goroutine had already proceeded to try to write to it, failed, and gone on to the default case.
You can solve this by either removing the default case from the select and allowing it to block until balances is ready to be written to.
You sure can, as #Schwern points out. But it's important that you understand you don't necessarily need to use select to use channels. Instead of a select with just one case, you could instead just write
balances <- balance
fmt.Println("done")
select is not required in this case, default is working against you, and there's just one case otherwise, so there's no need for select. You want the main function to block on that channel.
you can add a buffer to balances so it can be written to before it is read from.
Sure. But again, important to understand that the fact that a channel might block both sender and receiver until they're both ready to communicate , is a valid, useful, and common use of channels. Unbuffered channels are not the cause of your problem - providing a default case for your select, and thus a path for unintended behavior, is the cause.
Trying to understand the flow of goroutines so i wrote this code only one thing which i am not able to understand is that how routine-end runs between the other go routines and complete a single go routines and print the output from the channel at the end.
import(
"fmt"
)
func add(dataArr []int,dataChannel chan int,i int ){
var sum int
fmt.Println("GOROUTINE",i+1)
for i:=0;i<len(dataArr);i++{
sum += dataArr[i]
}
fmt.Println("wRITING TO CHANNEL.....")
dataChannel <- sum
fmt.Println("routine-end")
}
func main(){
fmt.Println("main() started")
dataChannel := make(chan int)
dataArr := []int{1,2,3,4,5,6,7,8,9}
for i:=0;i<len(dataArr);i+=3{
go add(dataArr[i:i+3],dataChannel,i)
}
fmt.Println("came to blocking statement ..........")
fmt.Println(<-dataChannel)
fmt.Println("main() end")
}
output
main() started
came to blocking statement ..........
GOROUTINE 1
wRITING TO CHANNEL.....
routine-end
GOROUTINE 4
wRITING TO CHANNEL.....
6
main() end
Your for loop launches 3 goroutines that invoke the add function.
In addition, main itself runs in a separate "main" goroutine.
Since goroutines execute concurrently, the order of their run is typically unpredictable and depends on timing, how busy your machine is, etc. Results may differ between runs and between machines. Inserting time.Sleep calls in various places may help visualize it. For example, inserting time.Sleep for 100ms before "came to blocking statement" shows that all add goroutines launch.
What you may see in your run typically is that one add goroutine launches, adds up its slice to its sum and writes sum to dataChannel. Since main launches a few goroutines and immediately reads from the channel, this read gets the sum written by add and then the program exists -- because by default main won't wait for all goroutines to finish.
Moreover, since the dataChannel channel is unbuffered and main only reads one value, the other add goroutines will block on the channel indefinitely while writing.
I do recommend going over some introductory resources for goroutines and channels. They build up the concepts from simple principles. Some good links for you:
Golang tour
https://gobyexample.com/ -- start with the Goroutines example and do the next several ones.
The output of the code given bellow and is somewhat confusing, Please help me understand the behaviour of the channels and goroutines and how
does the execution actually takes place.
I have tried to understand the flow of the program but the statement after the "call of goroutine" gets executed, even though the goroutine is called,
later on the statements in goroutines are executed,
on second "call of goroutine" the behaviour is different and the sequence of printing/flow of program changes.
Following is the code:
package main
import "fmt"
func main() {
fmt.Println("1")
done := make(chan string)
go test(done)
fmt.Println("7")
fmt.Println(<-done)
fmt.Println("8")
fmt.Println(<-done)
fmt.Println("9")
fmt.Println(<-done)
}
func test(done chan string) {
fmt.Println("2")
done <- "3"
done <- "10"
fmt.Println("4")
done <- "5"
fmt.Println("6")
}
The result of the above code:
1
7
2
3
8
10
9
4
6
5
Please help me understand why and how this result comes out.
Concept 1: Channels
Visualize a channel as a tube where data goes in one end and out the other. The first data in is the first data that comes out the other side. There are buffered channels and non-buffered channels but for your example you only need to understand the default channel, which is unbuffered. Unbuffered channels only allow one value in the channel at a time.
Writing to an Unbuffered Channel
Code that looks like this writes data into one end of the channel.
ch <- value
Now, this code actually waits to be done executing until something reads the value out of the channel. An unbuffered channel only allows for one value at a time to be within it, and doesn't continue executing until it is read. We'll see later how this affects the ordering of how your code is executed.
Reading from an Unbuffered Channel
To read from an unbuffered channel (visualize taking a value out of the channel), the code to do this looks like
[value :=] <-ch
when you read code documentation [things in] square brackets indicate that what's within them is optional. Above, without the [value :=] you'll just take a value out of the channel and don't use it for anything.
Now when there's a value in the channel, this code has two side effects. One, it reads the value out of a channel in whatever routine we are in now, and proceeds with the value. The other effect it has is to allow the goroutine which put the value into the channel to continue. This is the critical bit that's necessary to understand your example program.
In the event there is NO value in the channel yet, it will wait for a value to be written into the channel before continuing. In other words, the thread blocks until the channel has a value to read.
Concept 2: Goroutines
A goroutine allows your code to continue executing two pieces of code concurrently. This can be used to allow your code to execute faster, or attend to multiple problems at the same time (think of a server where multiple users are loading pages from it at the same time).
Your question arises when you try to figure out the ordering that code is executed when you have multiple routines executing concurrently. This is a good question and others have correctly stated that it depends. When you spawn two goroutines, the ordering of which lines of code are executed is arbitrary.
The code below with a goroutine may print executing a() or end main() first. This is due to the fact that spawning a gorouting means there are two concurrent streams (threads) of execution happening at the same time. In this case, one thread stays in main() and the other starts executing the first line in a(). How the runtime decides to choose which to run first is arbitrary.
func main() {
fmt.Println("start main()")
go a()
fmt.Println("end main()")
}
func a() {
fmt.Println("executing a()")
}
Goroutines + Channels
Now let's use a channel to control the ordering of what get's executed, when.
The only difference now is we create a channel, pass it into the goroutine, and wait for it's value to be written before continuing in main. From earlier, we discussed how the routine reading the value from a channel needs to wait until there's a value in the channel before continuing. Since executing a() is always printed before the channel is written to, we will always wait to read the value put into the channel until executing a() has printed. Since we read from the channel (which happens after the channel is written) before printing end main(), executing a() will always print before end main(). I made this playground so you can run it for yourself.
func main() {
fmt.Println("start main()")
ch := make(chan int)
go a(ch)
<-ch
fmt.Println("end main()")
}
func a(ch chan int) {
fmt.Println("executing a()")
ch <- 0
}
Your Example
I think at this point you could figure out what happens when, and what might happen in a different order. My own first attempt was wrong when I went through it in my head (see edit history). You have to be careful! I'll not give the right answer, upon editing, since I realized this may be a homework assignment.
EDIT: more semantics about <-done
On my first go through, I forgot to mention that fmt.Println(<-done) is conceptually the same as the following.
value := <-done
fmt.Println(value)
This is important because it helps you see that when the main() thread reads from the done channel, it doesn't print it at the same time. These are two separate steps to the runtime.
I'm just wondering if there is potential for corruption as a result of writing the same value to a global variable at the same time. My brain is telling me there is nothing wrong with this because its just a location in memory, but I figure I should probably double check this assumption.
I have concurrent processes writing to a global map var linksToVisit map[string]bool. The map is actually tracking what links on a website need to be further crawled.
However it can be the case that concurrent processes may have the same link on their respective pages and therefore each will mark that same link as true concurrently. There's nothing wrong with NOT using locks in this case right? NOTE: I never change the value back to false so either the key exists and it's value is true or it doesn't exist.
I.e.
var linksToVisit = map[string]bool{}
...
// somewhere later a goroutine finds a link and marks it as true
// it is never marked as false anywhere
linksToVisit[someLink] = true
What happens if concurrent processes write to a global variable the
same value?
The results of a data race are undefined.
Run the Go data race detector.
References:
Wikipedia: Race condition
Benign Data Races: What Could Possibly Go Wrong?
The Go Blog: Introducing the Go Race Detector
Go: Data Race Detector
Go 1.8 Release Notes
Concurrent Map Misuse
In Go 1.6, the runtime added lightweight, best-effort detection of
concurrent misuse of maps. This release improves that detector with
support for detecting programs that concurrently write to and iterate
over a map.
As always, if one goroutine is writing to a map, no other goroutine
should be reading (which includes iterating) or writing the map
concurrently. If the runtime detects this condition, it prints a
diagnosis and crashes the program. The best way to find out more about
the problem is to run the program under the race detector, which will
more reliably identify the race and give more detail.
For example,
package main
import "time"
var linksToVisit = map[string]bool{}
func main() {
someLink := "someLink"
go func() {
for {
linksToVisit[someLink] = true
}
}()
go func() {
for {
linksToVisit[someLink] = true
}
}()
time.Sleep(100 * time.Millisecond)
}
Output:
$ go run racer.go
fatal error: concurrent map writes
$
$ go run -race racer.go
==================
WARNING: DATA RACE
Write at 0x00c000078060 by goroutine 6:
runtime.mapassign_faststr()
/home/peter/go/src/runtime/map_faststr.go:190 +0x0
main.main.func2()
/home/peter/gopath/src/racer.go:16 +0x6a
Previous write at 0x00c000078060 by goroutine 5:
runtime.mapassign_faststr()
/home/peter/go/src/runtime/map_faststr.go:190 +0x0
main.main.func1()
/home/peter/gopath/src/racer.go:11 +0x6a
Goroutine 6 (running) created at:
main.main()
/home/peter/gopath/src/racer.go:14 +0x88
Goroutine 5 (running) created at:
main.main()
/home/peter/gopath/src/racer.go:9 +0x5b
==================
fatal error: concurrent map writes
$
It is better to use locks if you are changing the same value concurrently using multiple go routines. Since mutex and locks are used whenever it comes to secure the value from accessing when another function is changing the same just like writing to database table while accessing the same table.
For your question on using maps with different keys it is not preferable in Go as:
The typical use of maps did not require safe access from multiple
goroutines, and in those cases where it did, the map was probably part
of some larger data structure or computation that was already
synchronized. Therefore requiring that all map operations grab a mutex
would slow down most programs and add safety to few.
Map access is unsafe only when updates are occurring. As long as all
goroutines are only reading—looking up elements in the map, including
iterating through it using a for range loop—and not changing the map
by assigning to elements or doing deletions, it is safe for them to
access the map concurrently without synchronization.
So In case of update of maps it is not recommended. For more information Check FAQ on why maps operations not defined atomic.
Also it is noticed that if you realy wants to go for there should be a way to synchronize them.
Maps are not safe for concurrent use: it's not defined what happens
when you read and write to them simultaneously. If you need to read
from and write to a map from concurrently executing goroutines, the
accesses must be mediated by some kind of synchronization mechanism.
One common way to protect maps is with sync.RWMutex.
Concurrent map write is not ok, so you will most likely get a fatal error. So I think a lock should be used
As of Go 1.6, simultaneous map writes will cause a panic. Use a sync.Map to synchronize access.
See the map value assign implementation:
https://github.com/golang/go/blob/fe8a0d12b14108cbe2408b417afcaab722b0727c/src/runtime/hashmap.go#L519
I have a different output for println and fmt.Println in race detector which I couldn't explain. I expected both to be race, or at least both to be no race.
package main
var a int
func f() {
a = 1
}
func main() {
go f()
println(a)
}
And, it finds race condition as expected.
0
==================
WARNING: DATA RACE
Write by goroutine 5:
main.f()
/home/felmas/test.go:6 +0x30
Previous read by main goroutine:
main.main()
/home/felmas/test.go:11 +0x4d
Goroutine 5 (running) created at:
main.main()
/home/felmas/test.go:10 +0x38
==================
Found 1 data race(s)
However, this one runs without any detected race.
package main
import "fmt"
var a int
func f() {
a = 1
}
func main() {
go f()
fmt.Println(a)
}
To my knowledge, no race is detected doesn't mean there is no race so is this one of these deficiencies or is there a deeper explanation since println is builtin and quite special?
The race detector is a dynamic testing tool and no static analysis. In order to get reliable results from the race detector, you should strife for a high test coverage of your program, preferable by writing lots of benchmarks using multiple processes (by setting GOMAXPROCS > 1, GOMAXPROCS=NumCPU is the default for Go 1.5) and use a continuous integration tool that executes those tests regularly.
The race detector does not report any false positives so you should take every output serious. On the other hand it might not detect every race on every run, depending on the order goroutines and processes are scheduled.
In your example, wrapping everything in a tight loop and re-executing the tests reports the race correctly in both cases.