I am trying to implement a stop to a loop in Go.
The inspiration from the code I have right now is from here:
how to kill goroutine
However, I've not been able to get my code to behave as expected. The simplified version of my code from a complex code base is this:
package main
import (
"fmt"
"time"
)
var quit chan struct{}
var send chan int
func quitroutine() {
for {
select {
case cnt := <-send:
fmt.Println(cnt)
if cnt == 5 {
quit <- struct{}{}
}
}
}
}
func runLoop() {
cnt := 0
for {
select {
case <-quit:
fmt.Println("quit!")
default:
fmt.Println("default")
}
fmt.Println("inloop")
time.Sleep(1 * time.Second)
cnt++
send <- cnt
}
}
func main() {
quit = make(chan struct{})
send = make(chan int)
go quitroutine()
runLoop()
fmt.Println("terminated")
}
This code crashes:
default
inloop
5
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.runLoop()
/tmp/t.go:37 +0x1a6
main.main()
/tmp/t.go:45 +0xa4
goroutine 5 [chan send]:
main.quitroutine()
/tmp/t.go:18 +0x10e
created by main.main
/tmp/t.go:44 +0x9f
exit status 2
Questions:
Why does it crash at all after cnt is 5? quitroutine only writes to the quitchannel if cnt == 5, but doesn't terminate itself. While runLoop, if it receives on the quit channel, should just print "quit!" (which it doesn't), but not terminate itself.
Why don't I get the "quit!" output? Do I even get the quit channel?
How does this need to be implemented correctly
Just as Adrian has said, one of your goroutines is trying to send on quit, while the other is trying to send on send. To answer your questions:
When cnt == 5 the quitroutine begins attempting to send on quit. Because quit <- struct{}{} is not a select case, the goroutine will block until another tries to read from quit. The other goroutine is similarly stuck trying to do send <- cnt (when cnt = 6).
You never get the "quit!" output because that goroutine is stuck trying to do send <-cnt.
The simplest solution I see would be to adjust runLoop() so that the send <- cnt is a case in the select.
I'd change runLoop() to look like this:
func runLoop() {
cnt := 0
for {
select {
case <-quit:
fmt.Println("quit!")
case send <- cnt: // moved stuff here
fmt.Println("inloop")
time.Sleep(1 * time.Second)
cnt++
default:
fmt.Println("default")
}
// stuff used to be here
}
}
This gives me output (until I killed the program):
default
inloop
0
inloop
1
inloop
2
inloop
3
inloop
4
inloop
5
quit!
default
inloop
6
inloop
7
Which seems to mostly be what you were after.
I'd also like to note that the select block in quitroutine() is unnecessary because it only has one case. Cleaning that up may make it more clear that that goroutine is stuck trying to send, and never takes the input from the send channel.
When you try to send on the quit channel, the quitroutine blocks until something reads from it.
At the same time, the main routine, in runloop, is trying to send the next number on the send channel. This also blocks, because the routine that would be reading from it is currently blocked trying to send on the quit channel.
Both routines are blocked, which is a deadlock, so the program crashes.
This could be fixed by either putting one or both channel send in a select, or making one or both channels buffered (even a buffer length of 1 would suffice).
Related
I'm learning channel and below is a test I tried, but deadlock happens
func main() {
ch := make(chan int)
go func() {
select {
case ch <- 1:
fmt.Println("send suc")
default: // if comment this line, it will run smoothly
fmt.Println("default")
}
}()
time.Sleep(2) // do some time consuming thing...
fmt.Printf("receive val: %d", <-ch)
}
I expected there is no deadlock, but result is:
default
fatal error: all goroutines are asleep - deadlock!
But if I remove default or time.Sleep(2), code will run smoothly, result:
send suc
receive val: 1
Can someone explain why deadlock happen?
You have a select with default, that means if none of the communication ops are ready, select will not wait, will not block, but execute default immediately. The channel is unbuffered, so the send on it is not ready (because there are no receivers ready – main is sleeping).
So once the sleep is over in main, it tries to receive from the channel, but by then there is nobody trying to send on it. Deadlock.
package main
import (
"fmt"
)
func write(ch chan int) {
for i := 0; i < 5; i++ {
fmt.Println("avaliable", i)
ch <- i
fmt.Println("successfully wrote", i, "to ch")
}
close(ch)
}
func main() {
ch := make(chan int)
go write(ch)
for v := range ch {
fmt.Println("read value", v, "from ch")
}
}
Output
avaliable 0
successfully wrote 0 to ch
avaliable 1
read value 0 from ch
read value 1 from ch
successfully wrote 1 to ch
avaliable 2
successfully wrote 2 to ch
avaliable 3
read value 2 from ch
read value 3 from ch
successfully wrote 3 to ch
avaliable 4
successfully wrote 4 to ch
read value 4 from ch
Since this is a unbuffered channel it should have blocked as soon as data is written into it until another goroutine reads from the same channel. But it accepts data beyond it's capacity.
Intended Behavior
package main
import (
"fmt"
"time"
)
func write(ch chan int) {
for i := 0; i < 5; i++ {
fmt.Println("avaliable", i)
ch <- i
fmt.Println("successfully wrote", i, "to ch")
}
close(ch)
}
func main() {
ch := make(chan int)
go write(ch)
time.Sleep(time.Second)
for v := range ch {
fmt.Println("read value", v, "from ch")
time.Sleep(time.Second)
}
}
Output
avaliable 0
read value 0 from ch
successfully wrote 0 to ch
avaliable 1
read value 1 from ch
successfully wrote 1 to ch
avaliable 2
read value 2 from ch
successfully wrote 2 to ch
avaliable 3
read value 3 from ch
successfully wrote 3 to ch
avaliable 4
read value 4 from ch
successfully wrote 4 to ch
If some timers are placed throughout the code so that the main goroutine is blocked before each iteration it works as expected.
You can't infer anything from your output, because there is no ordering guarantee between the "read value" and "successfully wrote" Printfs executing. (There is one between the "available" Printf, which occurs before the channel send, and the "read value" Printf, which occurs after the corresponding channel receive, and you can see that that ordering is never violated in your output).
The channel isn't buffering anything, because it has no buffer; it's just that the two different goroutines run in an indeterminate order after the channel send completes. Sometimes the sending side gets to go first and prints the "successfully wrote" message, and sometimes the receiving side gets to go first and prints the "read value" message. Neither one ever "gets ahead" by more than one value, because they're still fully synchronized on the channel send; they're just racing to print their status messages immediately after.
When you add the Sleep calls to main, it just so happens to make it so that the goroutine running write will always be blocked on waiting to send the next value, while the one running main blocks on the call to Sleep. When the timer expires, the main goroutine wakes up, and immediately finds that it has something waiting for it on the channel, grabs it, and goes back to sleep, and then the write goroutine gets woken up. By slowing things down you've gotten the scheduler to run things in a consistent order (although it's still partly a matter of luck); without the sleeps, with everything running as fast as it can, the result is apparently random.
If that were the case, the output of the "sleeping" version would be even more chaotic.
You cannot expect a coherent output from two independent threads even if they are synchronized at some point.
I was trying to exit multiple Goroutines Simultaneously.
According to https://www.godesignpatterns.com/2014/04/exiting-multiple-goroutines-simultaneously.html
there is a well defined way to do it.
Another approach i see is the following
package main
import (
"fmt"
"time"
)
func main() {
var inCh chan int = make(chan int, 100)
var exit chan bool = make(chan bool)
for i := 0; i < 20; i++ {
go func(instance int) {
fmt.Println("In go routine ", instance)
for {
select {
case <-exit:
fmt.Println("Exit received from ", instance)
exit <- true
return
case value := <-inCh:
fmt.Println("Value=", value)
}
}
}(i)
}
time.Sleep(1 * time.Second)
exit <- true
<-exit // Final exit
fmt.Println("Final exit")
}
But I am confused and i really don't get it why the unbuffered channel at the end is executed as a last statement.
In reality I have 20 go routines listening for exit channel. Randomly one will receive it and send it to another.
Why always the reception within go routines is taking place and only when all of them are finished the channel reception with comment "// Final Exit" will execute ?
I will really appreciate if someone can give me an explanation.
Use close() for cancelation as shown in the linked article.
The code in question is not guaranteed to work. Here's a scenario where it fails:
One goroutine is ready to receive from exit. All other goroutines busy somewhere else.
The value sent by main is received by the ready goroutine.
That goroutine sends a value to exit that is received by main().
The other goroutines do not exit because no more values are sent to exit. See this playground example that uses time.Seep to induce the problem scenario.
Why always the reception within go routines is taking place and only when all of them are finished the channel reception with comment "// Final Exit" will execute ?
The program executes as if the channel maintains an ordered queue of waiting goroutines, but there's nothing in the specification that guarantees that behavior. Even if the channel has an ordered queue, the program can encounter the scenario above if a goroutine is doing something other than waiting on receive from exit.
If you notice the output of you program
In go routine 6
In go routine 0
In go routine 7
.
.
Exit received from 6
Exit received from 0
Exit received from 7
.
.
Final exit
They get called in the same( or almost the same) order of how they were started. If none of your Go routines are busy the first one registered will be used. This is just an implementation of the runtime and I would not count on this behavior.
Your final exit was the last channel to be listened on so it is used last.
If you remove the time.Sleep after your loop your final exit will get called almost immediately and most of your go routines will not receive the exit signal
Output with out time.Sleep (will very between runs)
In go routine 0
Exit received from 0
In go routine 1
In go routine 2
In go routine 3
In go routine 4
In go routine 5
In go routine 6
In go routine 7
In go routine 14
In go routine 15
In go routine 16
In go routine 17
In go routine 18
In go routine 19
Final exit
Consider this slight modification.
package main
import (
"fmt"
)
func main() {
var exit chan int = make(chan int)
var workers = 20
for i := 0; i < workers; i++ {
go func(instance int) {
fmt.Println("In go routine ", instance)
for {
select {
case i := <-exit:
fmt.Println("Exit", i, "received from ", instance)
exit <- i-1
return
}
}
}(i)
}
exit <- workers
fmt.Println("Final exit:", <-exit)
}
Here, I've done 3 things: First, I removed the unused channel, for brevity. Second, I removed the sleep. third, I changed the exit channel to an int channel that is decremented by every pass. If I pass the number of workers in, any value other than 0 from the "Final" message indicates dropped workers.
Here's one example run:
% go run t.go
In go routine 8
In go routine 5
In go routine 0
In go routine 2
Exit 20 received from 8
Exit 19 received from 5
Final exit: 18
In go routine 13
When main calls time.Sleep it doesn't get scheduled until the sleep is over. The other goroutines all have this time to set up their channel readers. I can only assume, because I can't find it written anywhere, that channel readers are likely to be queued in roughly chronological error - thus, the sleep guarantees that main's reader is the last.
If this is consistent behavior, it certainly isn't reliable.
See Multiple goroutines listening on one channel for many more thoughts on this.
I have a snippet of code that I am trying to understand based on how I put the close call and the location
func main() {
ch := make(chan int, 2)
go func(ch chan int) {
for i := 1; i <= 5; i++ {
ch <- i
fmt.Println("Func goroutine sends data: ", i)
}
//Pos1 - Works perfectly
//close(ch)
}(ch)
fmt.Println("Main goroutine sleeps 2 seconds")
time.Sleep(time.Second * 2)
fmt.Println("Main goroutine begins receiving data")
//Pos2 - Puts in only 2 ints 1 and 2 and then prints only that
//close(ch)
for d := range ch {
fmt.Println("Main goroutine received data:", d)
}
//Pos3 - Throws fatal error
close(ch)
}
I have been trying to understand and read blogs on this but not able to understand somethings still
When I place the close at Pos1, it works fine. But I am not sure why
it works. The buffer cannot hold more than 2 elements at any given
time, so when 2 elements are written, the loop will block until the
main routing does a read. But I thought to do a range over a
buffered channel, the range function has to know beforehand how many
elements to iterate over and for that channel must be closed. Why
does close work at this position?
When I put it as position 2, it prints only 2 elements which kind of makes sense but why is the for loop not throwing an exception when it is trying to write more elements to the channel that is closed?
When I close at Pos3, I get an exception fatal error: all goroutines are asleep - deadlock! though all 5 ints are printed. Why is that?
But I thought to do a range over a buffered channel, the range function has to know beforehand how many elements to iterate over and for that channel must be closed.
That assumption is wrong and the root of all misunderstandings.
The behavior of ranging over a channel is described in the Go Spec: https://golang.org/ref/spec#For_statements
For channels, the iteration values produced are the successive values sent on the channel until the channel is closed. If the channel is nil, the range expression blocks forever.
The channel does not need to be closed when the for statement is evaluated and the statement does not need to know number of elements.
So, in your code, when you put close in Pos1, it is indeed the right way to do it. When you put it in Pos3, the for loop waits the channel to be closed, which can only happen after the for loop itself, so it is a deadlock.
Putting close in Pos2 is buggy and the behavior is a little tricky. It is possible to raise an error, yet it is possible to just output two numbers. This is because when the channel is closed before the for loop, the loop can run without block and then main() returns. And when main() returns, the Go program ends. Whether it raises an error solely depends on if the scheduler switch to goroutine between the process, which is not guaranteed.
Modifying the code snippets with some peppered print statements to add on to the excellent answer by #leaf bebop, to clarify what's happening.
Pos 2, but make main wait a bit
func main() {
ch := make(chan int, 2)
go func(ch chan int) {
for i := 1; i <= 5; i++ {
ch <- i
fmt.Println("Func goroutine sends data: ", i)
}
fmt.Println("goR work is done")
}(ch)
fmt.Println("Main goroutine sleeps 2 seconds")
time.Sleep(time.Second * 2)
fmt.Println("Main goroutine begins receiving data")
close(ch)
for d := range ch {
fmt.Println("Main goroutine received data:", d)
}
fmt.Println("Main waiting, will the other goR panic?")
time.Sleep(time.Second * 10)
}
Main goroutine sleeps 2 seconds
Func goroutine sends data: 1
Func goroutine sends data: 2
Main goroutine begins receiving data
Main goroutine received data: 1
panic: send on closed channel
goroutine 6 [running]:
main.main.func1(0x0?)
/tmp/sandbox3746599466/prog.go:15 +0x3b
created by main.main
/tmp/sandbox3746599466/prog.go:13 +0x85
Program exited.
So, given enough time (main routine didn't exit), the scheduler went back to the go routine, where we wrote to a closed (closed by main) channel - Panic!
Pos 2, but only send 2 values, i.e not writing to a closed channel
func main() {
ch := make(chan int, 2)
go func(ch chan int) {
// Send only 2 values
for i := 1; i <= 2; i++ {
ch <- i
fmt.Println("Func goroutine sends data: ", i)
}
fmt.Println("goR work is done")
}(ch)
fmt.Println("Main goroutine sleeps 2 seconds")
time.Sleep(time.Second * 2)
fmt.Println("Main goroutine begins receiving data")
close(ch)
for d := range ch {
fmt.Println("Main goroutine received data:", d)
}
fmt.Println("Main waiting, will the other goR panic?")
time.Sleep(time.Second * 10)
fmt.Println("Main exiting")
}
Main goroutine sleeps 2 seconds
Func goroutine sends data: 1
Func goroutine sends data: 2
goR work is done
Main goroutine begins receiving data
Main goroutine received data: 1
Main goroutine received data: 2
Main waiting, will the other goR panic?
Main exiting
Program exited.
No panic, no spinning for ever. Main routine just exits.
Pos 3, check if main got unblocked?
func main() {
ch := make(chan int, 2)
go func(ch chan int) {
for i := 1; i <= 5; i++ {
ch <- i
fmt.Println("Func goroutine sends data: ", i)
}
fmt.Println("Routine done with work")
}(ch)
fmt.Println("Main goroutine sleeps 2 seconds")
time.Sleep(time.Second * 2)
fmt.Println("Main goroutine begins receiving data")
for d := range ch {
fmt.Println("Main goroutine received data:", d)
}
fmt.Println("Have I closed this channel?")
close(ch)
fmt.Println("I have")
}
Main goroutine sleeps 2 seconds
Func goroutine sends data: 1
Func goroutine sends data: 2
Main goroutine begins receiving data
Main goroutine received data: 1
Main goroutine received data: 2
Main goroutine received data: 3
Func goroutine sends data: 3
Func goroutine sends data: 4
Func goroutine sends data: 5
Routine done with work
Main goroutine received data: 4
Main goroutine received data: 5
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
/tmp/sandbox3301646152/prog.go:26 +0x194
Program exited.
Main never got even to close the channel, it was blocked in the range loop. It'll stay here forever, until it gets a value, or someone closes the channel. We don't have any other routine to do that, so deadlock.
This question already has answers here:
Go channels and deadlock
(3 answers)
Closed 7 years ago.
I am trying to have two separate consumer go routines, that would filter out even and odd numbers from the input channel. This is just a toy example in order to see if it is possible to have consumers do something with the message read from the input channel if it matches certain condition, otherwise put back onto the input channel.
My current code is as follows:
package main
func filterOdd(ch chan int, out chan int) {
val := <- ch
if val % 2 == 0 {
ch <- val
} else {
out <- val
}
}
func filterEven(ch chan int, out chan int) {
val := <- ch
if val % 2 != 0 {
ch <- val
} else {
out <- val
}
}
func main() {
even := make(chan int)
odd := make(chan int)
input := make(chan int)
go filterOdd(input, odd)
go filterEven(input, even)
for i:=1; i <= 10; i++ {
input <- i
}
println("Even...")
for i := range even {
println(i)
}
println("Odd...")
for i := range odd {
println(i)
}
}
However, this produces the following output:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/tmp/sandbox594577124/main.go:27 +0x140
goroutine 4 [chan send]:
main.filterOdd(0x10336100, 0x103360c0)
/tmp/sandbox594577124/main.go:8 +0xc0
created by main.main
/tmp/sandbox594577124/main.go:24 +0xc0
Link to the Go Playground: https://play.golang.org/p/9RIvFsGKI-
You have a deadlock because your even and odd goroutines are blocked on sending to out because nothing is reading from it. Why is nothing reading out? Because the main goroutine is blocked on sending to input because nothing is reading from it. Why is nothing reading from input? Because the two goroutines that would read from it are blocked.
Also, both filterEven and filterOdd will only run once unless you wrap their contents in for { } (but then they'll never stop until you break). On the other hand, range even will block (and range odd never happens) when nothing is left to write to even , because range over a channel only stops when the channel is closed or break is called.
In general, these aren't hard problems to solve as long as you know when you can close a channel. With what you're describing, that gets more difficult. None of the goroutines know when it's OK to close input, because all three write to it and two also read from it. You can use a sync.WaitGroup to make sure that everything you've put into the input channel has been processed before closing it. Once it's closed, the two other goroutines can use that as a signal to close their own channels and break or return to finish running.
However, writes to the in and out channels will still block until there is a corresponding read, because they are unbuffered. However, if you buffer them by specifying a size as the second argument to make, writes won't block until the channel is full. Since you know neither even or odd will have more written to them than what main send to input, you can use that as a safe buffer capacity.
Here's an example of using a WaitGroup with buffered channels for your code: https://play.golang.org/p/VXqfwUwRcx
If you don't want buffered channels, you can also use another pair of goroutines to capture the values and send them back to main as slices once finished. This way writes on the even and odd channels don't block: https://play.golang.org/p/i5vLDcsK1v
Otherwise, if don't need to print the contents of each channel all at once, you can use those two extra goroutines to read from the channels and print right away: https://play.golang.org/p/OCaUTcJkKB