Why does this cause a deadlock in Go? - go

This is not a question about how to better write this. It's a question specifically about why Go is causing a deadlock in this scenario.
package main
import "fmt"
func main() {
chan1 := make(chan bool)
chan2 := make(chan bool)
go func() {
for {
<-chan1
fmt.Printf("chan1\n")
chan2 <- true
}
}()
go func() {
for {
<-chan2
fmt.Printf("chan2\n")
chan1 <- true
}
}()
for {
chan1 <- true
}
}
Outputs:
chan1
chan2
chan1
chan2
chan1
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
goroutine 5 [chan send]:
goroutine 6 [chan send]:
exit status 2
Why does this not cause an infinite loop? How come it does two full "ping-pings" (instead of just one) before giving up?

From the runtime perspective you get a deadlock because all routines try to send onto a channel and there's no routine waiting to receive anything.
But why is it happening? I will give you a story as I like visualising what my routines are doing when I encounter a deadlock.
You have two players (routines) and one ball (true value). Every player waits for a ball and once they get it they pass it back to the other player (through a channel). This is what your two routines are really doing and this would indeed produce an infinite loop.
The problem is the third player introduced in your main loop. He's hiding behind the second player and once he sees the first player has empty hands, he throws another ball at him. So we end up with both players holding a ball, couldn't pass it to another player because the other one has (the first) ball in his hands already. The hidden, evil player is also trying to pass yet another ball. Everyone is confused, because there're three balls, three players and no empty hands.
In other words, you have introduced the third player who is breaking the game. He should be an arbiter passing the very first ball at the beginning of the game, watching it, but stop producing balls! It means, instead of having a loop in you main routine, there should be simply chan1 <- true (and some condition to wait, so we don't exit the program).
If you enable logging in the loop of main routine, you will see the deadlock occurs always on the third iteration. The number of times the other routines are executed depends on the scheduler. Bringing back the story: first iteration is a kick-off of the first ball; next iteration is a mysterious second ball, but this can be handled. The third iteration is a deadlock – it brings to life the third ball which can't be handled by anybody.

It looks complicated but the answer is easy.
It'll deadlock when:
First routine is trying to write to chan2
Second route is trying to write to chan1.
Main is trying to write to chan1.
How can that happen? Example:
Main writes chan1. Blocks on another write.
Routine 1: chan1 receives from Main. Prints. Blocks on write chan2.
Routine 2: chan2 receives. Prints. Blocks on write chan1.
Routine 1: chan1 receives from Routine 2. Prints. Blocks on write chan2.
Routine 2: chan2 receives. Prints. Blocks on write chan1.
Main writes chan1. Blocks on another write.
Routine 1: chan1 receives from Main. Prints. Blocks on write chan2.
Main writes chan1. Blocks on another write.
Currently all routines are blocked. i.e.:
Routine 1 cannot write to chan2 because Routine 2 is not receiving but is actually blocked trying to write to chan1. But no one is listening on chan1.
As #HectorJ said, it all depends on the scheduler. But in this setup, a deadlock is inevitable.

goroutine 1 [chan send]:
goroutine 5 [chan send]:
goroutine 6 [chan send]:
This tells it all: all your goroutines are blocked trying to send on a channel with no one to receive on the other end.
So your first goroutine blocks on chan2 <- true, your second blocks on chan1 <- true and your main goroutine blocks on its own chan1 <- true.
As to why it does two "full ping-pings" like you say, it depends on scheduling and from which sender <-chan1 decides to receive first.
On my computer, I get more and it varies each time I run it:
chan1
chan2
chan1
chan2
chan1
chan2
chan1
chan2
chan1
chan2
chan1
chan2
chan1
fatal error: all goroutines are asleep - deadlock!

Related

When my simple Go program run ,Why the result is deadlock?

This is my entire Go code! What confused me is that case balances <- balance: did't occurs.I dont know why?
package main
import (
"fmt"
)
func main() {
done := make(chan int)
var balance int
balances := make(chan int)
balance = 1
go func() {
fmt.Println(<-balances)
done <- 1
}()
select {
case balances <- balance:
fmt.Println("done case")
default:
fmt.Println("default case")
}
<-done
}
default case
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
/tmp/sandbox575832950/prog.go:29 +0x13d
goroutine 18 [chan receive]:
main.main.func1()
/tmp/sandbox575832950/prog.go:17 +0x38
created by main.main
/tmp/sandbox575832950/prog.go:16 +0x97
The main goroutine executes the select before the anonymous goroutine function executes the receive from balances. The main goroutine executes the default clause in the select because there is no ready receiver on balances. The main goroutine continues on to receive on done.
The goroutine blocks on receive from balances because there is no sender. Main continued past the send by taking the default clause.
The main goroutine blocks on receive from done because there is no sender. The goroutine is blocked on receive from balances.
Fix by replacing the select statement with balances <- balance. The default clause causes the problem. When the the default class is removed, all that remains in the select is send to balances.
Because of concurrency, there's no guarantee that the goroutine will execute before the select. We can see this by adding a print to the goroutine.
go func() {
fmt.Println("Here")
fmt.Println(<-balances)
done <- 1
}()
$ go run test.go
default case
Here
fatal error: all goroutines are asleep - deadlock!
...
If the select runs first, balances <- balance would block; balances has no buffer and nothing is trying to read from it. case balances <- balance would block so select skips it and executes its default.
Then the goroutine runs and blocks reading balances. Meanwhile the main code blocks reading done. Deadlock.
You can solve this by either removing the default case from the select and allowing it to block until balances is ready to be written to.
select {
case balances <- balance:
fmt.Println("done case")
}
Or you can add a buffer to balances so it can be written to before it is read from. Then case balances <- balance does not block.
balances := make(chan int, 1)
What confused me is that case balances <- balance: did't occurs
To be specific: it's because of select with a default case.
Whenever you create a new goroutine with go ...(), there is no guarantee about whether the invoking goroutine, or the invoked goroutine, will run next.
In practice it's likely that the next statements in the invoking goroutine will execute next (there being no particularly good reason to stop it). Of course, we should write programs that function correctly all the time, not just some, most, or even almost all the time! Concurrent programming with go ...() is all about synchronizing the goroutines so that the intended behavior must occur. Channels can do that, if used properly.
I think the balances channel can receive data
It's an unbuffered channel, so it can receive data if someone is reading from it. Otherwise, that write to the channel will block. Which brings us back to select.
Since you provided a default case, it's quite likely that the goroutine that invoked go ...() will continue to execute, and select that can't immediately choose a different case, will choose default. So it would be very unlikely for the invoked goroutine to be ready to read from balances before the main goroutine had already proceeded to try to write to it, failed, and gone on to the default case.
You can solve this by either removing the default case from the select and allowing it to block until balances is ready to be written to.
You sure can, as #Schwern points out. But it's important that you understand you don't necessarily need to use select to use channels. Instead of a select with just one case, you could instead just write
balances <- balance
fmt.Println("done")
select is not required in this case, default is working against you, and there's just one case otherwise, so there's no need for select. You want the main function to block on that channel.
you can add a buffer to balances so it can be written to before it is read from.
Sure. But again, important to understand that the fact that a channel might block both sender and receiver until they're both ready to communicate , is a valid, useful, and common use of channels. Unbuffered channels are not the cause of your problem - providing a default case for your select, and thus a path for unintended behavior, is the cause.

Why a go-routine block on channel is considered as deadlock?

As per the definition here, deadlock is related to resource contention.
In an operating system, a deadlock occurs when a process or thread enters a waiting state because a requested system resource is held by another waiting process, which in turn is waiting for another resource held by another waiting process. If a process is unable to change its state indefinitely because the resources requested by it are being used by another waiting process, then the system is said to be in a deadlock.
In the below code:
package main
import "fmt"
func main() {
c := make(chan string)
c <- "John"
fmt.Println("main() stopped")
}
main() go-routine blocks until any other go-routine(no such) reads the same data from that channel.
but the output shows:
$ bin/cs61a
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/home/user/../myhub/cs61a/Main.go:8 +0x54
$
edit:
For the point: "the main goroutine–which is blocked, hence all goroutines are blocked, hence it's a deadlock." in the below code, non-main goroutine is also blocked on channel, aren't all goroutines supposed to get blocked?
package main
import (
"fmt"
"time"
)
func makeRandom(randoms chan int) {
var ch chan int
fmt.Printf("print 1\n")
<-ch
fmt.Printf("print 2\n")
}
func main() {
randoms := make(chan int)
go makeRandom(randoms)
}
Edit 2:
For your point in the answer: "not all your goroutines are blocked so it's not a deadlock". In the below code, only main() goroutine is blocked, but not worker():
package main
import (
"fmt"
)
func worker() {
fmt.Printf("some work\n")
}
func main() {
ch := make(chan int)
go worker()
<-ch
}
and the output says deadlock:
$ bin/cs61a
some work
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
/home/user/code/src/github.com/myhub/cs61a/Main.go:18 +0x6f
$
Ideally main() should not exit, because channel resource is used by any one go-routine.
Why a go-routine block on channel considered as deadlock?
In Go a deadlock is when all existing goroutines are blocked.
Your example has a single goroutine–the main goroutine–which is blocked, hence all goroutines are blocked, hence it's a deadlock.
Note: since all goroutines are blocked, new goroutines will not (cannot) be launched (because they can only be launched from running goroutines). And if all goroutines are blocked and cannot do anything, there is no point in waiting forever for nothing. So the runtime exits.
Edit:
Your edited code where you use a sleep in main is a duplicate of this: Go channel deadlock is not happening. Basically a sleep is not a blocking forever operation (the sleep duration is finite), so a goroutine sleeping is not considered in deadlock detection.
Edit #2:
Since then you removed the sleep() but it doesn't change anything. You have 2 goroutines: the main and the one executing makeRandom(). makeRandom() is blocked and main() isn't. So not all your goroutines are blocked so it's not a deadlock.
Edit #3:
In your last example when the runtime detects the deadlock, then there is only a single goroutine still running: the main(). It's true that you launch a goroutine executing worker(), but that only prints a text and terminates. "Past" goroutines do not count, terminated goroutines also can't do anything to change the blocked state of existing goroutines. Only existing goroutines count.
Check out this article to understand exactly why a go-routine block on channel considered as deadlock:
http://dmitryvorobev.blogspot.com/2016/08/golang-channels-implementation.html
In your example above, the main goroutine gets added to the waiting queue(sendq) and cannot be released until Go runs some goroutine that receives a value from the channel.

another golang channels questions on understanding how it processes

I'm walking through this blog post to understand channels and I have a question on the 2nd example. I modified it a bit in the playground to this, where I'm putting more items in the channel like this:
package main
import (
"fmt"
)
func main() {
n := 3
in := make(chan int)
out := make(chan int)
// We now supply 2 channels to the `multiplyByTwo` function
// One for sending data and one for receiving
go multiplyByTwo(in, out)
// We then send it data through the channel and wait for the result
in <- n
in <- 3
in <- 6
in <- 10
fmt.Println(<-out)
}
func multiplyByTwo(in <-chan int, out chan<- int) {
// This line is just to illustrate that there is code that is
// executed before we have to wait on the `in` channel
fmt.Println("Initializing goroutine...")
// The goroutine does not proceed until data is received on the `in` channel
num := <-in
// The rest is unchanged
result := num * 2
out <- result
}
but this throws an error:
Initializing goroutine...
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/tmp/sandbox639017164/prog.go:18 +0xe0
goroutine 6 [chan send]:
main.multiplyByTwo(0x430080, 0x4300c0)
/tmp/sandbox639017164/prog.go:34 +0xe0
created by main.main
/tmp/sandbox639017164/prog.go:14 +0xa0
my interpretation of this is that the channels should process data that comes in, so why would it throw an error if I'm just simply adding more to the channel? I'd assume that it would pass in the other numbers too and run those through the function as well.
if I run it like this without an out channel:
package main
import (
"fmt"
)
func main() {
n := 3
in := make(chan int)
//out := make(chan int)
// We now supply 2 channels to the `multiplyByTwo` function
// One for sending data and one for receiving
go multiplyByTwo(in)
// We then send it data through the channel and wait for the result
in <- n
in <- 3
in <- 6
in <- 10
}
func multiplyByTwo(in <-chan int) {
// This line is just to illustrate that there is code that is
// executed before we have to wait on the `in` channel
fmt.Println("Initializing goroutine...")
// The goroutine does not proceed until data is received on the `in` channel
num := <-in
// The rest is unchanged
result := num * 2
fmt.Println(result)
}
it process the first input into the channel but then errors out again. fatal error: all goroutines are asleep - deadlock!
The goroutine processes one value, and then terminates. You can only send the first value to your goroutine, after that, the goroutine is gone, and there's nothing listening to your channel. That's why you get deadlock, you're trying to send data to a channel where there are no listeners.
Your channels are unbuffered. That means, data exchange through the channel happens only when there is at least one listener reading from the channel, and some other goroutine writes to it. If you create buffered channels, you can keep adding to them until the buffer is full. Otherwise, for the write operation to succeed, there must be a matching read operation.
This would work:
func multiplyByTwo(in <-chan int) {
for num:=range in {
// process num
}
// If here, then channel in is closed
}
in <- n
in <- 3
in <- 6
in <- 10
close(in)
// Wait for the goroutine to finish
You can, if you like, think of a channel as a sort of mailbox (perhaps with special teleportation abilities, like a portal from the game Portal).
An unbuffered channel is a mailbox that has no room at all for any packages. For someone to mail a package (send a value), they must wait until the receiver's hand pokes out of the mailbox. They can then drop the package into the hand, which will withdraw back into the mailbox, taking the package with it. If someone else is in line, you must get in line behind the someone-else.
A buffered channel is a mailbox that can hold one or more packages. To send a package, get into the line if there is one. When you reach the head of the line, you may look at the box. If there is room for your package, you put it in and go on about your business. If not, you can wait until there is room, then put the package in and go on about your business.
So there is a general pattern to send:
Get in line if you have to.
When you reach the head of the queue, put your package in if there is room, otherwise wait for room—or, for an unbuffered channel, for someone to come to the other (receive) side and put their hand in to receive.
Meanwhile, if you want to receive from a channel, you queue up if needed, just as for sending. Once you're at the head of the line, you can take a package out of the box, or—for an unbuffered channel—wait with your hand sticking out of the other side of the box-with-no-room for someone to come along and put something in it.
Each goroutine is, in this analogy, like a person, or a Go gopher. It (or he or she or whatever pronoun you prefer) can queue up if needed, and put things into, or take them out of, one of these channels. Your program starts with one goroutine, which invokes main.
In your code, you spin off a second goroutine, which begins at multiplyByTwo. This one goroutine waits—once—for a number to show up in the channel, or in this case, for someone to be waiting to send a number since the channel is unbuffered. It then doubles the (single) number it got, prints the result, and quits / dies / gets buried, never to exist again.
Meanwhile your main waits for someone to be receiving—that would be your second goroutine—until it's ready to take the number 3 that's in n. That part succeeds. Then your main waits for another receive so that it can send the constant 3.
While your main is waiting, your other goroutine is doing its work—or maybe has finished its work—and exits. Now there is only one "person" (or gopher or whatever) in the whole system, waiting for a second person—who does not exist and will not ever be born—to come along to take the number. The underlying Go system can tell that this event won't ever happen, and that's when you get the message:
fatal error: all goroutines are asleep - deadlock!
(this also terminates the program).
Burak Serdar's answer shows how you can have your second goroutine keep reading numbers from the channel. This introduces a new problem: how do you tell the second goroutine that no more numbers are coming? The answer is that you can close the channel, with close.
If we stick with the mailbox analogy, you can think of closing the channel as putting a special sticker or label on the send side of the channel. This prevents anyone from doing any further putting-values-in. Any packages that are in the channel already are safe—they stay there until someone receives them—but no new packages can go in. On the receiver side, it's easy to tell the difference between a package and this special sticker: so when you encounter the "closed" sticker, you know no more values will ever come through. If the channel is unbuffered, you can see this sticker immediately. If it's buffered, you'll have to take out all the existing packages first, before you can see it.
In general, the sender should close the channel so that receivers know they will not get anything more from it. (In many specific cases, you can get away without closing the channel. In particular, if the goroutine running main returns from its call to main, all the other goroutines die more or less immediately.)
Note that once closed, no sender can close the channel again, so this means that if you have a single channel that you share across multiple senders, only one of them can close the channel! Making that work right is tricky, so it's more common to avoid sharing a channel across more than one writing-goroutine like this.

Golang: forever channel

Just have a question, what is happening here?
forever := make(chan bool)
log.Printf(" [*] Waiting for messages. To exit press CTRL+C")
<-forever
That code creates an unbuffered channel, and attempts to receive from it.
And since no one ever sends anything on it, it's essentially a blocking forever operation.
The purpose of this is to keep the goroutine from ending / returning, most likely because there are other goroutines which do some work concurrently or they wait for certain events or incoming messages (like your log message says).
And the need for this is that without this, the application might quit without waiting for other goroutines. Namely, if the main goroutine ends, the program ends as well. Quoting from Spec: Program execution:
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
Check out this answer for similar and more techniques: Go project's main goroutine sleep forever?
For an introduction about channels, see What are channels used for?
The code in the question is probably coming from the RabbitMQ golang tutorial here.
Here's a more extended chunk of it with some commends of my own:
...
// create an unbuffered channel for bool types.
// Type is not important but we have to give one anyway.
forever := make(chan bool)
// fire up a goroutine that hooks onto msgs channel and reads
// anything that pops into it. This essentially is a thread of
// execution within the main thread. msgs is a channel constructed by
// previous code.
go func() {
for d := range msgs {
log.Printf("Received a message: %s", d.Body)
}
}()
log.Printf(" [*] Waiting for messages. To exit press CTRL+C")
// We need to block the main thread so that the above thread stays
// on reading from msgs channel. To do that just try to read in from
// the forever channel. As long as no one writes to it we will wait here.
// Since we are the only ones that know of it it is guaranteed that
// nothing gets written in it. We could also do a busy wait here but
// that would waste CPU cycles for no good reason.
<-forever

Why does this code generate an error?

The below piece of code generates an error why?
func main() {
messages := make(chan string)
messages <- "test" //line 16
fmt.Println(<-messages)
}
Generates the below error.
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/tmp/sandbox994400718/main.go:16 +0x80
A value is sent to the channel, and in the next line it's being received. Technically it should work.
Channels can be buffered or unbuffered. A buffered channel can store a number of items “inside” it, but when you add something to a buffered channel the goroutine adding the item can only continue when another goroutine removes the item. There is no place to “leave” the item, it must be passed directly to the other goroutine, and the first goroutine will wait until another one take the item from it.
This is what is happening in your code. When you create a channel with make, if you don’t specify a capacity as the second argument you get an unbuffered channel. To create a buffered channel pass a second argument to make, e.g.
messages := make(chan string, 1) // could be larger than 1 if you want
This allows the goroutine to add the item (a string in this case) to the channel, where it will be available when another goroutine tries to get an item from the channel in the future, and the original goroutine can then continue processing.
I have learnt a lot about channels now, and now I'm able to answer the question.
In line 16 when the message "test" is sent to the channel by the main thread(goroutine) the execution pauses and the runtime looks for other goroutines which are ready to receive the value from the channel message. Since there are no other channels the runtime raises a panic with the deadlock message. This a classic example for deadlock.
To fix this there are two things that can be done.
1) Use buffered channels as Matt suggested(one of the answers).
2) Else have the statement that sends to channel or receives from channel in a go routine.
func main() {
messages := make(chan string)
go func() {
messages <- "test" //line 16
}()
fmt.Println(<-messages)
}
So the essential take aways from this is that,
1) Channels can be used only to communicate between goroutines i.e when you send to channel in one goroutine you can receive it only in another goroutine not the same.
2) When data is sent to a channel in a goroutine the flow/execution of that goroutine is paused until the data is received from the same channel in another goroutine.

Resources