In some circumstance, i would copy some content to different piece of the slice.
Like this
a := make([]int, 10)
for i := 0; i < 10; i++ {
b := []int{i}
go func(i int) {
copy(a[i:i+1], b)
}(i)
}
time.Sleep(time.Second)
fmt.Println(a)
It leads DATA RACE. But it always behave right in product environment.
So my question is:
Any data race cloud be undefined behavior?
Can i always get right result in such a practice?
To avoid the data race, which will have undefined results, synchronize. For example,
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
a := make([]int, 10)
for i := 0; i < 10; i++ {
b := []int{i}
wg.Add(1)
go func(i int) {
defer wg.Done()
copy(a[i:i+1], b)
}(i)
}
wg.Wait()
fmt.Println(a)
}
Playground: https://play.golang.org/p/rYCBMV7wuNn
Output:
$ go run -race norace.go
[0 1 2 3 4 5 6 7 8 9]
$
Package sync
import "sync"
type WaitGroup
A WaitGroup waits for a collection of goroutines to finish. The main
goroutine calls Add to set the number of goroutines to wait for. Then
each of the goroutines runs and calls Done when finished. At the same
time, Wait can be used to block until all goroutines have finished.
A WaitGroup must not be copied after first use.
type WaitGroup struct {
// contains filtered or unexported fields
}
Related
I just started to learn Go so please bear with me, I've tried to play around with Go routines and channels but are getting a deadlock somehow.
Here's the example
package main
import (
"fmt"
"sync"
)
func main() {
total := 2
var wg sync.WaitGroup
wg.Add(total)
ch := make(chan int)
for idx := 0; idx < total; idx++ {
fmt.Printf("Processing idx %d\n", idx)
go func(idx int) {
defer wg.Done()
ch <- idx
}(idx)
}
for val := range ch {
fmt.Println(val)
}
fmt.Println("Wait")
wg.Wait()
}
which throws the error
Processing idx 0
Processing idx 1
1
0
fatal error: all goroutines are asleep - deadlock!
range ch reads from the channel until it is closed.
How many times do you call close(ch)? When will the for val := range ch loop terminate?
When should you close the channel? You have a lot of options here, but one way to do it is to add another goroutine:
go func() {
wg.Wait()
close(ch)
}()
e.g., after spinning off all routines that will write-to-channel-then-call-wg.Done(), so that the channel is closed once all the writers are done writing. (You can run this goroutine as soon as you've increased the wg count to account for all writers.)
When writing data into same channel using multiple go routines with waitgroup after waiting wg.Wait() getting exception saying all go routines are asleep or deedlock.
package main
import (
"fmt"
"runtime"
"sync"
)
var wg sync.WaitGroup
func CreateMultipleRoutines() {
ch := make(chan int)
for i := 0; i < 10; i++ { // creates 10 go routines and adds to waitgroup
wg.Add(1)
go func() {
for j := 0; j < 10; j++ {
ch <- j
}
wg.Done() // indication of go routine is done to main routine
}()
}
fmt.Println(runtime.NumGoroutine())
wg.Wait() //wait for all go routines to complete
close(ch) // closing channel after completion of wait fo go routines
for v := range ch { // range can be used since channel is closed
fmt.Println(v)
}
fmt.Println("About to exit program ...")
}
When tried to implement this without waitgroup I am able to read data from channel by looping exact number of times data pushed to channel but i cant range since there will be panic when we close channel. here is the example code
package main
import (
"fmt"
"runtime"
)
func main() {
ch := make(chan int)
for i := 0; i < 10; i++ { // creates 10 go routines and adds to waitgroup
go func(i int) {
for j := 0; j < 10; j++ {
ch <- j * i
}
}(i)
}
fmt.Println(runtime.NumGoroutine())
for v := 0; v < 100; v++ {
fmt.Println(<-ch)
}
fmt.Println("About to exit program ...")
}
I want to understand why waitgroup in wait state is still waiting even though all go routines are signalled Done() which inturn makes number of go routines to zero
I think your original code has some problems.
You are closing the channel before reading from it.
You are not getting the advantage of using 10 goroutines because of your channel is 1 "sized". So one goroutine is producing one result per once.
My solution would be to spawn a new goroutine to monitor if the 10 goroutines finished its jobs. There you will use your WaitGroup.
Then the code would be like:
package main
import (
"fmt"
"runtime"
"sync"
)
var wg sync.WaitGroup
func main() {
ch := make(chan int, 10)
for i := 0; i < 10; i++ { // creates 10 go routines and adds to waitgroup
wg.Add(1)
go func() {
for j := 0; j < 10; j++ {
ch <- j
}
wg.Done() // indication of go routine is done to main routine
}()
}
go func(){
wg.Wait()
close(ch)
}()
fmt.Println(runtime.NumGoroutine())
for v := range ch { // range can be used since channel is closed
fmt.Println(v)
}
fmt.Println("About to exit program ...")
}
By default a chan holds no items, so all go routines are blocked on sending, until something reads from it. They never actually reach the wg.Done() statement.
A solution would be to close the channel in it's own go routine. Wrap your wg.Wait() and close(ch) lines like this:
go func() {
wg.Wait() //wait for all go routines to complete
close(ch) // closing channel after completion of wait fo go routines
}()
Then you can range over the channel, which will only close after all of the sending go routines have finished (and implicitly all values have been received).
Here is my code at Go Playground
package main
import (
"fmt"
)
func sum_up(my_int int, cs chan int) {
my_sum := 0
for i := 0; i < my_int; i++ {
my_sum += i
}
cs <- my_sum
}
func main() {
my_channel := make(chan int)
for i := 2; i < 5; i++ {
go sum_up(i, my_channel)
}
for ele := range my_channel {
fmt.Println(ele)
}
//fatal error: all goroutines are asleep - deadlock!
fmt.Println("Done")
}
Which results in:
1
3
6
fatal error: all goroutines are asleep - deadlock!
And I don't understand what causes the error. My understanding is that in my function sum_up I am adding new values to my_channel. Why does the problem occur after I try to print out the values? Since I see 1,3,6 are printed it means that all goroutines have successfully finished.
Moreover, if the block that attempts to print the values of the channel
for ele := range my_channel {
fmt.Println(ele)
}
Is removed, then I don't get the error. So it is including the block that causes the error, but why?
A for-range on a empty channel will block until there are elements to read from the channel or until the channel is closed.
Here is a version that uses sync.WaitGroup to account for how many goroutines remain active. After all goroutines have finished, the channel is closed and the for-range loop exists.
https://play.golang.org/p/ZnLYxLMNdF
package main
import (
"fmt"
"sync"
)
func sum_up(my_int int, cs chan int, wg *sync.WaitGroup) {
my_sum := 0
for i := 0; i < my_int; i++ {
my_sum += i
}
cs <- my_sum
wg.Done()
}
func main() {
wg := &sync.WaitGroup{}
my_channel := make(chan int)
for i := 2; i < 5; i++ {
wg.Add(1)
go sum_up(i, my_channel, wg)
}
// Run a goroutine that will monitor how many sum_up are running.
go func(cs chan int, wg *sync.WaitGroup) {
wg.Wait()
close(cs)
}(my_channel, wg)
for ele := range my_channel {
fmt.Println(ele)
}
//fatal error: all goroutines are asleep - deadlock!
fmt.Println("Done")
}
When you use range on a channel, it will wait for values forever or until the channel is closed. It's deadlocking because when the last value is written to my_channel, it will wait forever for a value that will never come.
Here's a slightly modified variant that shows how to cleanly leave the range: https://play.golang.org/p/YDlM8EcRnx
for range chan exit when chan receive close signal. You must close(my_channel) somewhere, or loop will wait forever.
I want to create slice and add values returned from channel.
Below is the code I tried but could not able to solve it.
I have to send address of the slice, but I am not able to figure out how :(
package main
import "fmt"
import "time"
func sendvalues(cs chan int){
for i:=0;i<10;i++{
cs<-i
}
}
func appendInt(cs chan int, aINt []int)[]*int{
for {
select {
case i := <-cs:
aINt = append(aINt,i)//append returns new type right ?
fmt.Println("slice",aINt)
}
}
}
func main() {
cs := make(chan int)
intSlice := make([]int, 0,10)
fmt.Println("Before",intSlice)
go sendvalues(cs)
go appendInt(cs,intSlice)// I have to pass address here
time.Sleep(999*999999)
fmt.Println("After",intSlice)
}
Your code won't work for two (in fact three) reasons:
append returns a new slice as soon as the capacity is reached.
Thus, the assignment in appendInt will do nothing.
appendInt runs concurrently, therefore:
As long as appendInt does not message main that it is finished,
main does not know when the intSlice has all the values you want.
You have to wait for all goroutines to return at the end of main
Problem 1: Modifying slices in functions
You may know that in Go every value you pass to a function is copied. Reference values,
such as slices, are copied too, but have pointers internally which then point to the original memory location. That means you can modify the elements of a slice in a function. What you
can't do is reassigning this value with a new slice as the internal pointer would point to somewhere different. You need pointers for that. Example (Play):
func modify(s *[]int) {
for i:=0; i < 10; i++ {
*s = append(*s, i)
}
}
func main() {
s := []int{1,2,3}
modify(&s)
fmt.Println(s)
}
Problem 2: Synchronizing goroutines
To wait for started goroutines, you can use a sync.WaitGroup. Example (Play):
func modify(wg *sync.WaitGroup, s *[]int) {
defer wg.Done()
for i:=0; i < 10; i++ {
*s = append(*s, i)
}
}
func main() {
wg := &sync.WaitGroup{}
s := []int{1,2,3}
wg.Add(1)
go modify(wg, &s)
wg.Wait()
fmt.Println(s)
}
The example above waits (using wg.Wait()) for modify to finish
(modify calls wg.Done() when finished). If you remove the wg.Wait() call, you will
see why not synchronizing is a problem. Comparison of outputs:
With wg.Wait(): [1 2 3 0 1 2 3 4 5 6 7 8 9]
Without wg.Wait(): [1 2 3]
The main goroutine returns earlier than the modify goroutine which is why you will never
see the modified results. Therefore synchronizing is absolutely necessary.
A good way to communicate the new slice would be to use a channel. You would not need to
use pointers and you would have synchronization. Example (Play):
func modify(res chan []int) {
s := []int{}
for i:=0; i < 10; i++ {
s = append(s, i)
}
res <- s
}
func main() {
c := make(chan []int)
go modify(c)
s := <-c
fmt.Println(s)
}
Playing around with go, I threw together this code:
package main
import "fmt"
const N = 10
func main() {
ch := make(chan int, N)
done := make(chan bool)
for i := 0; i < N; i++ {
go (func(n int, ch chan int, done chan bool) {
for i := 0; i < N; i++ {
ch <- n*N + i
}
done <- true
})(i, ch, done)
}
numDone := 0
for numDone < N {
select {
case i := <-ch:
fmt.Println(i)
case <-done:
numDone++
}
}
for {
select {
case i := <-ch:
fmt.Println(i)
default:
return
}
}
}
Basically I have N channels doing some work and reporting it on the same channel -- I want to know when all the channels are done. So I have this other done channel that each worker goroutine sends a message on (message doesn't matter), and this causes main to count that thread as done. When the count gets to N, we're actually done.
Is this "good" go? Is there a more go-idiomatic way of doing this?
edit: To clarify a bit, I'm doubtful because the done channel seems to be doing a job that channel closing seems to be for, but of course I can't actually close the channel in any goroutine because all the routines share the same channel. So I'm using done to simulate a channel that does some kind of "buffered closing".
edit2: Original code wasn't really working since sometimes the done signal from a routine was read before the int it just put on ch. Needs a "cleanup" loop.
Here is an idiomatic use of sync.WaitGroup for you to study
(playground link)
package main
import (
"fmt"
"sync"
)
const N = 10
func main() {
ch := make(chan int, N)
var wg sync.WaitGroup
for i := 0; i < N; i++ {
wg.Add(1)
go func(n int) {
defer wg.Done()
for i := 0; i < N; i++ {
ch <- n*N + i
}
}(i)
}
go func() {
wg.Wait()
close(ch)
}()
for i := range ch {
fmt.Println(i)
}
}
Note the use of closures in the two go routine definitions and note the second go statement to wait for all the routines to finish, then close the channel, so range can be used.
looks like you want a sync.WaitGroup (http://golang.org/pkg/sync/#WaitGroup)
Just use a WaitGroup! They are the built-in primitive that essentially let you wait for stuff in different goroutines to finish up.
http://golang.org/pkg/sync/#WaitGroup
As for your doubts, The way to thing about is that being done by closing a channel (done permanently) and being done with work (temporarily) are different.
In the first approximation the code seems more or less okay to me.
Wrt the details, the 'ch' should be buffered. Also the 'done' channel goroutine "accounting" might be possibly replaced with sync.WaitGroup.
If you're iterating over values generated from goroutines, you can iterate directly over the
communication channel:
for value := range ch {
println(value)
}
The only thing necessary for this is, that the channel ch is closed later on, or else the
loop would wait for new values forever.
This would effectively replace your for numDone < N when used in combination with sync.WaitGroup.
I was dealing with the same issue in some code of mine and found this to be a more than adequate solution.
The answer provides Go's idiom for handling multiple goroutines all sending across a single channel.