When having the code:
package main
import (
"os"
"os/signal"
)
func main() {
sig := make(chan os.Signal, 1)
signal.Notify(sig)
<-sig
}
Runs without problem, of course, blocking until you send a signal that interrupts the program.
But:
package main
func main() {
sig := make(chan int, 1)
<-sig
}
throws this error:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
/home/user/project/src/main.go:5 +0x4d
exit status 2
While I understand why reading from the int channel causes a deadlock, I have only a suspicion
that the os.Signal doesn't because its channel can suffer writes from "the outside" as, well,
it handles signals and they come from outside the program.
Is my suspicion somewhat correct? If so, how the runtime handles this differently from other channel types?
Thank you!
You have a deadlock because try to receive message from channel but no other goroutine running that is no sender exists. In the same time call to signal.Notify starts watchSignalLoop() goroutine in background and you can verify implementation details here https://golang.org/src/os/signal/signal.go.
Channels don't care about element type unless your element type is larger than 64kB (strictly speaking, there are other nuances, please check the implementation).
Don't guess about how runtime works, make researches about it. For example, you can check what happens when you call make(chan int). You can do go tool compile -S main.go | grep main.go:line of make chan and check which function is called from runtime package. Then just jump to this file and invest your time to understand the implementation. You will see that implementation of channels is thin and straightforward comparing to other things
Hope it helps!
Related
Page 253 of The Go Programming Language states:
... if instead of returning from main in the event of cancellation, we execute a call to panic, then the runtime will dump the stack of every goroutine in the program.
This code deliberately leaks a goroutine by waiting on a channel that never has anything to receive:
package main
import (
"fmt"
"time"
)
func main() {
never := make(chan struct{})
go func() {
defer fmt.Println("End of child")
<-never
}()
time.Sleep(10 * time.Second)
panic("End of main")
}
However, the runtime only lists the main goroutine when panic is called:
panic: End of main
goroutine 1 [running]:
main.main()
/home/simon/panic/main.go:15 +0x7f
exit status 2
If I press Ctrl-\ to send SIGQUIT during the ten seconds before main panics, I do see the child goroutine listed in the output:
goroutine 1 [sleep]:
time.Sleep(0x2540be400)
/usr/lib/go-1.17/src/runtime/time.go:193 +0x12e
main.main()
/home/simon/panic/main.go:14 +0x6c
goroutine 18 [chan receive]:
main.main.func1()
/home/simon/panic/main.go:12 +0x76
created by main.main
/home/simon/panic/main.go:10 +0x5d
I thought maybe the channel was getting closed as panic runs (which still wouldn't guarantee the deferred fmt.Println had time to execute), but I get the same behaviour if the child goroutine does a time.Sleep instead of waiting on a channel.
I know there are ways to dump goroutine stacktraces myself, but my question is why doesn't panic behave as described in the book? The language spec only says that a panic will terminate the program, so is the book simply describing implementation-dependent behaviour?
Thanks to kostix for pointing me to the GOTRACEBACK runtime environment variable. Setting this to all instead of leaving it at the default of single restores the behaviour described in TGPL. Note that this variable is significant to the runtime, but you can't manipulate it with go env.
The default to only list the panicking goroutine is a change in go 1.6 - my edition of the book is copyrighted 2016 and gives go 1.5 as the prequisite for its example code, so it must predate the change. It's interesting reading the change discussion that there was concern about hiding useful information (as the recipient of many an incomplete error report, I can sympathise with this), but nobody called out the issue of scaling to large production systems that kostix mentioned.
As per the definition here, deadlock is related to resource contention.
In an operating system, a deadlock occurs when a process or thread enters a waiting state because a requested system resource is held by another waiting process, which in turn is waiting for another resource held by another waiting process. If a process is unable to change its state indefinitely because the resources requested by it are being used by another waiting process, then the system is said to be in a deadlock.
In the below code:
package main
import "fmt"
func main() {
c := make(chan string)
c <- "John"
fmt.Println("main() stopped")
}
main() go-routine blocks until any other go-routine(no such) reads the same data from that channel.
but the output shows:
$ bin/cs61a
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/home/user/../myhub/cs61a/Main.go:8 +0x54
$
edit:
For the point: "the main goroutine–which is blocked, hence all goroutines are blocked, hence it's a deadlock." in the below code, non-main goroutine is also blocked on channel, aren't all goroutines supposed to get blocked?
package main
import (
"fmt"
"time"
)
func makeRandom(randoms chan int) {
var ch chan int
fmt.Printf("print 1\n")
<-ch
fmt.Printf("print 2\n")
}
func main() {
randoms := make(chan int)
go makeRandom(randoms)
}
Edit 2:
For your point in the answer: "not all your goroutines are blocked so it's not a deadlock". In the below code, only main() goroutine is blocked, but not worker():
package main
import (
"fmt"
)
func worker() {
fmt.Printf("some work\n")
}
func main() {
ch := make(chan int)
go worker()
<-ch
}
and the output says deadlock:
$ bin/cs61a
some work
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
/home/user/code/src/github.com/myhub/cs61a/Main.go:18 +0x6f
$
Ideally main() should not exit, because channel resource is used by any one go-routine.
Why a go-routine block on channel considered as deadlock?
In Go a deadlock is when all existing goroutines are blocked.
Your example has a single goroutine–the main goroutine–which is blocked, hence all goroutines are blocked, hence it's a deadlock.
Note: since all goroutines are blocked, new goroutines will not (cannot) be launched (because they can only be launched from running goroutines). And if all goroutines are blocked and cannot do anything, there is no point in waiting forever for nothing. So the runtime exits.
Edit:
Your edited code where you use a sleep in main is a duplicate of this: Go channel deadlock is not happening. Basically a sleep is not a blocking forever operation (the sleep duration is finite), so a goroutine sleeping is not considered in deadlock detection.
Edit #2:
Since then you removed the sleep() but it doesn't change anything. You have 2 goroutines: the main and the one executing makeRandom(). makeRandom() is blocked and main() isn't. So not all your goroutines are blocked so it's not a deadlock.
Edit #3:
In your last example when the runtime detects the deadlock, then there is only a single goroutine still running: the main(). It's true that you launch a goroutine executing worker(), but that only prints a text and terminates. "Past" goroutines do not count, terminated goroutines also can't do anything to change the blocked state of existing goroutines. Only existing goroutines count.
Check out this article to understand exactly why a go-routine block on channel considered as deadlock:
http://dmitryvorobev.blogspot.com/2016/08/golang-channels-implementation.html
In your example above, the main goroutine gets added to the waiting queue(sendq) and cannot be released until Go runs some goroutine that receives a value from the channel.
Trying to understand the flow of goroutines so i wrote this code only one thing which i am not able to understand is that how routine-end runs between the other go routines and complete a single go routines and print the output from the channel at the end.
import(
"fmt"
)
func add(dataArr []int,dataChannel chan int,i int ){
var sum int
fmt.Println("GOROUTINE",i+1)
for i:=0;i<len(dataArr);i++{
sum += dataArr[i]
}
fmt.Println("wRITING TO CHANNEL.....")
dataChannel <- sum
fmt.Println("routine-end")
}
func main(){
fmt.Println("main() started")
dataChannel := make(chan int)
dataArr := []int{1,2,3,4,5,6,7,8,9}
for i:=0;i<len(dataArr);i+=3{
go add(dataArr[i:i+3],dataChannel,i)
}
fmt.Println("came to blocking statement ..........")
fmt.Println(<-dataChannel)
fmt.Println("main() end")
}
output
main() started
came to blocking statement ..........
GOROUTINE 1
wRITING TO CHANNEL.....
routine-end
GOROUTINE 4
wRITING TO CHANNEL.....
6
main() end
Your for loop launches 3 goroutines that invoke the add function.
In addition, main itself runs in a separate "main" goroutine.
Since goroutines execute concurrently, the order of their run is typically unpredictable and depends on timing, how busy your machine is, etc. Results may differ between runs and between machines. Inserting time.Sleep calls in various places may help visualize it. For example, inserting time.Sleep for 100ms before "came to blocking statement" shows that all add goroutines launch.
What you may see in your run typically is that one add goroutine launches, adds up its slice to its sum and writes sum to dataChannel. Since main launches a few goroutines and immediately reads from the channel, this read gets the sum written by add and then the program exists -- because by default main won't wait for all goroutines to finish.
Moreover, since the dataChannel channel is unbuffered and main only reads one value, the other add goroutines will block on the channel indefinitely while writing.
I do recommend going over some introductory resources for goroutines and channels. They build up the concepts from simple principles. Some good links for you:
Golang tour
https://gobyexample.com/ -- start with the Goroutines example and do the next several ones.
I'm walking through this blog post to understand channels and I have a question on the 2nd example. I modified it a bit in the playground to this, where I'm putting more items in the channel like this:
package main
import (
"fmt"
)
func main() {
n := 3
in := make(chan int)
out := make(chan int)
// We now supply 2 channels to the `multiplyByTwo` function
// One for sending data and one for receiving
go multiplyByTwo(in, out)
// We then send it data through the channel and wait for the result
in <- n
in <- 3
in <- 6
in <- 10
fmt.Println(<-out)
}
func multiplyByTwo(in <-chan int, out chan<- int) {
// This line is just to illustrate that there is code that is
// executed before we have to wait on the `in` channel
fmt.Println("Initializing goroutine...")
// The goroutine does not proceed until data is received on the `in` channel
num := <-in
// The rest is unchanged
result := num * 2
out <- result
}
but this throws an error:
Initializing goroutine...
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/tmp/sandbox639017164/prog.go:18 +0xe0
goroutine 6 [chan send]:
main.multiplyByTwo(0x430080, 0x4300c0)
/tmp/sandbox639017164/prog.go:34 +0xe0
created by main.main
/tmp/sandbox639017164/prog.go:14 +0xa0
my interpretation of this is that the channels should process data that comes in, so why would it throw an error if I'm just simply adding more to the channel? I'd assume that it would pass in the other numbers too and run those through the function as well.
if I run it like this without an out channel:
package main
import (
"fmt"
)
func main() {
n := 3
in := make(chan int)
//out := make(chan int)
// We now supply 2 channels to the `multiplyByTwo` function
// One for sending data and one for receiving
go multiplyByTwo(in)
// We then send it data through the channel and wait for the result
in <- n
in <- 3
in <- 6
in <- 10
}
func multiplyByTwo(in <-chan int) {
// This line is just to illustrate that there is code that is
// executed before we have to wait on the `in` channel
fmt.Println("Initializing goroutine...")
// The goroutine does not proceed until data is received on the `in` channel
num := <-in
// The rest is unchanged
result := num * 2
fmt.Println(result)
}
it process the first input into the channel but then errors out again. fatal error: all goroutines are asleep - deadlock!
The goroutine processes one value, and then terminates. You can only send the first value to your goroutine, after that, the goroutine is gone, and there's nothing listening to your channel. That's why you get deadlock, you're trying to send data to a channel where there are no listeners.
Your channels are unbuffered. That means, data exchange through the channel happens only when there is at least one listener reading from the channel, and some other goroutine writes to it. If you create buffered channels, you can keep adding to them until the buffer is full. Otherwise, for the write operation to succeed, there must be a matching read operation.
This would work:
func multiplyByTwo(in <-chan int) {
for num:=range in {
// process num
}
// If here, then channel in is closed
}
in <- n
in <- 3
in <- 6
in <- 10
close(in)
// Wait for the goroutine to finish
You can, if you like, think of a channel as a sort of mailbox (perhaps with special teleportation abilities, like a portal from the game Portal).
An unbuffered channel is a mailbox that has no room at all for any packages. For someone to mail a package (send a value), they must wait until the receiver's hand pokes out of the mailbox. They can then drop the package into the hand, which will withdraw back into the mailbox, taking the package with it. If someone else is in line, you must get in line behind the someone-else.
A buffered channel is a mailbox that can hold one or more packages. To send a package, get into the line if there is one. When you reach the head of the line, you may look at the box. If there is room for your package, you put it in and go on about your business. If not, you can wait until there is room, then put the package in and go on about your business.
So there is a general pattern to send:
Get in line if you have to.
When you reach the head of the queue, put your package in if there is room, otherwise wait for room—or, for an unbuffered channel, for someone to come to the other (receive) side and put their hand in to receive.
Meanwhile, if you want to receive from a channel, you queue up if needed, just as for sending. Once you're at the head of the line, you can take a package out of the box, or—for an unbuffered channel—wait with your hand sticking out of the other side of the box-with-no-room for someone to come along and put something in it.
Each goroutine is, in this analogy, like a person, or a Go gopher. It (or he or she or whatever pronoun you prefer) can queue up if needed, and put things into, or take them out of, one of these channels. Your program starts with one goroutine, which invokes main.
In your code, you spin off a second goroutine, which begins at multiplyByTwo. This one goroutine waits—once—for a number to show up in the channel, or in this case, for someone to be waiting to send a number since the channel is unbuffered. It then doubles the (single) number it got, prints the result, and quits / dies / gets buried, never to exist again.
Meanwhile your main waits for someone to be receiving—that would be your second goroutine—until it's ready to take the number 3 that's in n. That part succeeds. Then your main waits for another receive so that it can send the constant 3.
While your main is waiting, your other goroutine is doing its work—or maybe has finished its work—and exits. Now there is only one "person" (or gopher or whatever) in the whole system, waiting for a second person—who does not exist and will not ever be born—to come along to take the number. The underlying Go system can tell that this event won't ever happen, and that's when you get the message:
fatal error: all goroutines are asleep - deadlock!
(this also terminates the program).
Burak Serdar's answer shows how you can have your second goroutine keep reading numbers from the channel. This introduces a new problem: how do you tell the second goroutine that no more numbers are coming? The answer is that you can close the channel, with close.
If we stick with the mailbox analogy, you can think of closing the channel as putting a special sticker or label on the send side of the channel. This prevents anyone from doing any further putting-values-in. Any packages that are in the channel already are safe—they stay there until someone receives them—but no new packages can go in. On the receiver side, it's easy to tell the difference between a package and this special sticker: so when you encounter the "closed" sticker, you know no more values will ever come through. If the channel is unbuffered, you can see this sticker immediately. If it's buffered, you'll have to take out all the existing packages first, before you can see it.
In general, the sender should close the channel so that receivers know they will not get anything more from it. (In many specific cases, you can get away without closing the channel. In particular, if the goroutine running main returns from its call to main, all the other goroutines die more or less immediately.)
Note that once closed, no sender can close the channel again, so this means that if you have a single channel that you share across multiple senders, only one of them can close the channel! Making that work right is tricky, so it's more common to avoid sharing a channel across more than one writing-goroutine like this.
I have been looking into Golang, and seeing how good its concurrency is with its enforcement of a coroutine-channel-only model through its innovative goroutines construct.
One thing that I immediately find troubling is the use of the Wait() method, used to wait until multiple outstanding goroutines spawned inside a parent goroutine have finished. To quote the Golang docs
Wait can be used to block until all goroutines have finished
The fact that many go developers prescribe Wait() as the preferred way to implement concurrency seems antithetical to Golang's mission of enabling developers to write efficient software, because blocking is inefficient, and truly asynchronous code never blocks.
A process [or thread] that is blocked is one that is waiting for some event, such as a resource becoming available or the completion of an I/O operation.
In other words, a blocked thread will spend CPU cycles doing nothing useful, just checking repeatedly to see if its currently running task can stop waiting and continue its execution.
In truly asynchronous code, when a coroutine encounters a situation where it cannot continue until a result arrives, it must yield its execution to the scheduler instead of blocking, by switching its state from running to waiting, so the scheduler can begin executing the next-in-line coroutine from the runnable queue. The waiting coroutine should have its state changed from waiting to runnable only once the result it needs has arrived.
Therefore, since Wait() blocks until x number of goroutines have invoked Done(), the goroutine which calls Wait() will always remain in either a runnable or running state, wasting CPU cycles and relying on the scheduler to preempt the long-running goroutine only to change its state from running to runnable, instead of changing it to waiting as it should be.
If all this is true, and I'm understanding how Wait() works correctly, then why aren't people using the built-in Go channels for the task of waiting for sub-goroutines to complete? If I understand correctly, sending to a buffered channel, and reading from any channel are both asynchronous operations, meaning that invoking them will put the goroutine into a waiting state, so why aren't they the preferred method?
The article I referenced gives a few examples. Here's what the author calls the "Old School" way:
package main
import (
"fmt"
"time"
)
func main() {
messages := make(chan int)
go func() {
time.Sleep(time.Second * 3)
messages <- 1
}()
go func() {
time.Sleep(time.Second * 2)
messages <- 2
}()
go func() {
time.Sleep(time.Second * 1)
messages <- 3
}()
for i := 0; i < 3; i++ {
fmt.Println(<-messages)
}
}
and here is the preferred, "Canonical" way:
package main
import (
"fmt"
"sync"
"time"
)
func main() {
messages := make(chan int)
var wg sync.WaitGroup
wg.Add(3)
go func() {
defer wg.Done()
time.Sleep(time.Second * 3)
messages <- 1
}()
go func() {
defer wg.Done()
time.Sleep(time.Second * 2)
messages <- 2
}()
go func() {
defer wg.Done()
time.Sleep(time.Second * 1)
messages <- 3
}()
wg.Wait()
for i := range messages {
fmt.Println(i)
}
}
I can understand that the second might be easier to understand than the first, but the first is asynchronous where no coroutines block, and the second has one coroutine which blocks: the one running the main function. Here is another example of Wait() being the generally accepted approach.
Why isn't Wait() considered an anti-pattern by the Go community if it creates an inefficient blocked thread? Why aren't channels preferred by most in this situation, since they can by used to keep all the code asynchronous and the thread optimized?
Your understanding of "blocking" is incorrect. Blocking operations such as WaitGroup.Wait() or a channel receive (when there is no value to receive) only block the execution of the goroutine, they do not (necessarily) block the OS thread which is used to execute the (statements of the) goroutine.
Whenever a blocking operation (such as the above mentioned) is encountered, the goroutine scheduler may (and it will) switch to another goroutine that may continue to run. There are no (significant) CPU cycles lost during a WaitGroup.Wait() call, if there are other goroutines that may continue to run, they will.
Please check related question: Number of threads used by Go runtime