Just have a question, what is happening here?
forever := make(chan bool)
log.Printf(" [*] Waiting for messages. To exit press CTRL+C")
<-forever
That code creates an unbuffered channel, and attempts to receive from it.
And since no one ever sends anything on it, it's essentially a blocking forever operation.
The purpose of this is to keep the goroutine from ending / returning, most likely because there are other goroutines which do some work concurrently or they wait for certain events or incoming messages (like your log message says).
And the need for this is that without this, the application might quit without waiting for other goroutines. Namely, if the main goroutine ends, the program ends as well. Quoting from Spec: Program execution:
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
Check out this answer for similar and more techniques: Go project's main goroutine sleep forever?
For an introduction about channels, see What are channels used for?
The code in the question is probably coming from the RabbitMQ golang tutorial here.
Here's a more extended chunk of it with some commends of my own:
...
// create an unbuffered channel for bool types.
// Type is not important but we have to give one anyway.
forever := make(chan bool)
// fire up a goroutine that hooks onto msgs channel and reads
// anything that pops into it. This essentially is a thread of
// execution within the main thread. msgs is a channel constructed by
// previous code.
go func() {
for d := range msgs {
log.Printf("Received a message: %s", d.Body)
}
}()
log.Printf(" [*] Waiting for messages. To exit press CTRL+C")
// We need to block the main thread so that the above thread stays
// on reading from msgs channel. To do that just try to read in from
// the forever channel. As long as no one writes to it we will wait here.
// Since we are the only ones that know of it it is guaranteed that
// nothing gets written in it. We could also do a busy wait here but
// that would waste CPU cycles for no good reason.
<-forever
Related
I'm walking through this blog post to understand channels and I have a question on the 2nd example. I modified it a bit in the playground to this, where I'm putting more items in the channel like this:
package main
import (
"fmt"
)
func main() {
n := 3
in := make(chan int)
out := make(chan int)
// We now supply 2 channels to the `multiplyByTwo` function
// One for sending data and one for receiving
go multiplyByTwo(in, out)
// We then send it data through the channel and wait for the result
in <- n
in <- 3
in <- 6
in <- 10
fmt.Println(<-out)
}
func multiplyByTwo(in <-chan int, out chan<- int) {
// This line is just to illustrate that there is code that is
// executed before we have to wait on the `in` channel
fmt.Println("Initializing goroutine...")
// The goroutine does not proceed until data is received on the `in` channel
num := <-in
// The rest is unchanged
result := num * 2
out <- result
}
but this throws an error:
Initializing goroutine...
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/tmp/sandbox639017164/prog.go:18 +0xe0
goroutine 6 [chan send]:
main.multiplyByTwo(0x430080, 0x4300c0)
/tmp/sandbox639017164/prog.go:34 +0xe0
created by main.main
/tmp/sandbox639017164/prog.go:14 +0xa0
my interpretation of this is that the channels should process data that comes in, so why would it throw an error if I'm just simply adding more to the channel? I'd assume that it would pass in the other numbers too and run those through the function as well.
if I run it like this without an out channel:
package main
import (
"fmt"
)
func main() {
n := 3
in := make(chan int)
//out := make(chan int)
// We now supply 2 channels to the `multiplyByTwo` function
// One for sending data and one for receiving
go multiplyByTwo(in)
// We then send it data through the channel and wait for the result
in <- n
in <- 3
in <- 6
in <- 10
}
func multiplyByTwo(in <-chan int) {
// This line is just to illustrate that there is code that is
// executed before we have to wait on the `in` channel
fmt.Println("Initializing goroutine...")
// The goroutine does not proceed until data is received on the `in` channel
num := <-in
// The rest is unchanged
result := num * 2
fmt.Println(result)
}
it process the first input into the channel but then errors out again. fatal error: all goroutines are asleep - deadlock!
The goroutine processes one value, and then terminates. You can only send the first value to your goroutine, after that, the goroutine is gone, and there's nothing listening to your channel. That's why you get deadlock, you're trying to send data to a channel where there are no listeners.
Your channels are unbuffered. That means, data exchange through the channel happens only when there is at least one listener reading from the channel, and some other goroutine writes to it. If you create buffered channels, you can keep adding to them until the buffer is full. Otherwise, for the write operation to succeed, there must be a matching read operation.
This would work:
func multiplyByTwo(in <-chan int) {
for num:=range in {
// process num
}
// If here, then channel in is closed
}
in <- n
in <- 3
in <- 6
in <- 10
close(in)
// Wait for the goroutine to finish
You can, if you like, think of a channel as a sort of mailbox (perhaps with special teleportation abilities, like a portal from the game Portal).
An unbuffered channel is a mailbox that has no room at all for any packages. For someone to mail a package (send a value), they must wait until the receiver's hand pokes out of the mailbox. They can then drop the package into the hand, which will withdraw back into the mailbox, taking the package with it. If someone else is in line, you must get in line behind the someone-else.
A buffered channel is a mailbox that can hold one or more packages. To send a package, get into the line if there is one. When you reach the head of the line, you may look at the box. If there is room for your package, you put it in and go on about your business. If not, you can wait until there is room, then put the package in and go on about your business.
So there is a general pattern to send:
Get in line if you have to.
When you reach the head of the queue, put your package in if there is room, otherwise wait for room—or, for an unbuffered channel, for someone to come to the other (receive) side and put their hand in to receive.
Meanwhile, if you want to receive from a channel, you queue up if needed, just as for sending. Once you're at the head of the line, you can take a package out of the box, or—for an unbuffered channel—wait with your hand sticking out of the other side of the box-with-no-room for someone to come along and put something in it.
Each goroutine is, in this analogy, like a person, or a Go gopher. It (or he or she or whatever pronoun you prefer) can queue up if needed, and put things into, or take them out of, one of these channels. Your program starts with one goroutine, which invokes main.
In your code, you spin off a second goroutine, which begins at multiplyByTwo. This one goroutine waits—once—for a number to show up in the channel, or in this case, for someone to be waiting to send a number since the channel is unbuffered. It then doubles the (single) number it got, prints the result, and quits / dies / gets buried, never to exist again.
Meanwhile your main waits for someone to be receiving—that would be your second goroutine—until it's ready to take the number 3 that's in n. That part succeeds. Then your main waits for another receive so that it can send the constant 3.
While your main is waiting, your other goroutine is doing its work—or maybe has finished its work—and exits. Now there is only one "person" (or gopher or whatever) in the whole system, waiting for a second person—who does not exist and will not ever be born—to come along to take the number. The underlying Go system can tell that this event won't ever happen, and that's when you get the message:
fatal error: all goroutines are asleep - deadlock!
(this also terminates the program).
Burak Serdar's answer shows how you can have your second goroutine keep reading numbers from the channel. This introduces a new problem: how do you tell the second goroutine that no more numbers are coming? The answer is that you can close the channel, with close.
If we stick with the mailbox analogy, you can think of closing the channel as putting a special sticker or label on the send side of the channel. This prevents anyone from doing any further putting-values-in. Any packages that are in the channel already are safe—they stay there until someone receives them—but no new packages can go in. On the receiver side, it's easy to tell the difference between a package and this special sticker: so when you encounter the "closed" sticker, you know no more values will ever come through. If the channel is unbuffered, you can see this sticker immediately. If it's buffered, you'll have to take out all the existing packages first, before you can see it.
In general, the sender should close the channel so that receivers know they will not get anything more from it. (In many specific cases, you can get away without closing the channel. In particular, if the goroutine running main returns from its call to main, all the other goroutines die more or less immediately.)
Note that once closed, no sender can close the channel again, so this means that if you have a single channel that you share across multiple senders, only one of them can close the channel! Making that work right is tricky, so it's more common to avoid sharing a channel across more than one writing-goroutine like this.
I was going through tutorials on gobyexample. I noticed the author has shown example of gochannel using go routine but in buggered go channel he is directly sending messages to channel .
I tried on my local system to run unbuffered channel without go routine but it is throwing "fatal error: all goroutines are asleep - deadlock!"
but buffered channels are working fine without go routine
func channelDemo() {
message := make(chan string)
// go func() {
// message <- "Hello"
// }()
message <- "Hello"
msg := <-message
fmt.Println("msg", msg)
}
func channelBufferingDemo() {
messages := make(chan string, 3)
messages <- "Buffered"
messages <- "channel"
fmt.Println(<-messages)
fmt.Println(<-messages)
}
A channel send will succeed only if the channel can accept the input, that is, either there's a listener for that channel, or there is available buffer in the channel. Otherwise goroutine will go to sleep until one of those become true: either someone starts listening, or someone reads from the channel and there is now buffer space.
With a channel with no buffer, the only way you can write to it is if someone is listening to it. If there is only one goroutine and if you write to the channel, all goroutines will be asleep.
With a channel with buffer size 3 and one goroutine, you can write to if 3 times without reading from it. The 4th write will put all goroutines to sleep.
buffered channels are working fine without go routine
because they won't block until their buffer is full.
Which is what happens in channelBufferingDemo. It has a length of 3, you write twice on it, it is not full, thus the program keeps going on, you read twice on it, it dequeues and let the program finish.
In channelDemo the channel does not have a buffer, so the first time you read or write, it blocks until another routine performs the opposite operations.
Because in your example you are running both operations sequentially within the same routine, it inevitably block, ending with this fatal error.
The output of the code given bellow and is somewhat confusing, Please help me understand the behaviour of the channels and goroutines and how
does the execution actually takes place.
I have tried to understand the flow of the program but the statement after the "call of goroutine" gets executed, even though the goroutine is called,
later on the statements in goroutines are executed,
on second "call of goroutine" the behaviour is different and the sequence of printing/flow of program changes.
Following is the code:
package main
import "fmt"
func main() {
fmt.Println("1")
done := make(chan string)
go test(done)
fmt.Println("7")
fmt.Println(<-done)
fmt.Println("8")
fmt.Println(<-done)
fmt.Println("9")
fmt.Println(<-done)
}
func test(done chan string) {
fmt.Println("2")
done <- "3"
done <- "10"
fmt.Println("4")
done <- "5"
fmt.Println("6")
}
The result of the above code:
1
7
2
3
8
10
9
4
6
5
Please help me understand why and how this result comes out.
Concept 1: Channels
Visualize a channel as a tube where data goes in one end and out the other. The first data in is the first data that comes out the other side. There are buffered channels and non-buffered channels but for your example you only need to understand the default channel, which is unbuffered. Unbuffered channels only allow one value in the channel at a time.
Writing to an Unbuffered Channel
Code that looks like this writes data into one end of the channel.
ch <- value
Now, this code actually waits to be done executing until something reads the value out of the channel. An unbuffered channel only allows for one value at a time to be within it, and doesn't continue executing until it is read. We'll see later how this affects the ordering of how your code is executed.
Reading from an Unbuffered Channel
To read from an unbuffered channel (visualize taking a value out of the channel), the code to do this looks like
[value :=] <-ch
when you read code documentation [things in] square brackets indicate that what's within them is optional. Above, without the [value :=] you'll just take a value out of the channel and don't use it for anything.
Now when there's a value in the channel, this code has two side effects. One, it reads the value out of a channel in whatever routine we are in now, and proceeds with the value. The other effect it has is to allow the goroutine which put the value into the channel to continue. This is the critical bit that's necessary to understand your example program.
In the event there is NO value in the channel yet, it will wait for a value to be written into the channel before continuing. In other words, the thread blocks until the channel has a value to read.
Concept 2: Goroutines
A goroutine allows your code to continue executing two pieces of code concurrently. This can be used to allow your code to execute faster, or attend to multiple problems at the same time (think of a server where multiple users are loading pages from it at the same time).
Your question arises when you try to figure out the ordering that code is executed when you have multiple routines executing concurrently. This is a good question and others have correctly stated that it depends. When you spawn two goroutines, the ordering of which lines of code are executed is arbitrary.
The code below with a goroutine may print executing a() or end main() first. This is due to the fact that spawning a gorouting means there are two concurrent streams (threads) of execution happening at the same time. In this case, one thread stays in main() and the other starts executing the first line in a(). How the runtime decides to choose which to run first is arbitrary.
func main() {
fmt.Println("start main()")
go a()
fmt.Println("end main()")
}
func a() {
fmt.Println("executing a()")
}
Goroutines + Channels
Now let's use a channel to control the ordering of what get's executed, when.
The only difference now is we create a channel, pass it into the goroutine, and wait for it's value to be written before continuing in main. From earlier, we discussed how the routine reading the value from a channel needs to wait until there's a value in the channel before continuing. Since executing a() is always printed before the channel is written to, we will always wait to read the value put into the channel until executing a() has printed. Since we read from the channel (which happens after the channel is written) before printing end main(), executing a() will always print before end main(). I made this playground so you can run it for yourself.
func main() {
fmt.Println("start main()")
ch := make(chan int)
go a(ch)
<-ch
fmt.Println("end main()")
}
func a(ch chan int) {
fmt.Println("executing a()")
ch <- 0
}
Your Example
I think at this point you could figure out what happens when, and what might happen in a different order. My own first attempt was wrong when I went through it in my head (see edit history). You have to be careful! I'll not give the right answer, upon editing, since I realized this may be a homework assignment.
EDIT: more semantics about <-done
On my first go through, I forgot to mention that fmt.Println(<-done) is conceptually the same as the following.
value := <-done
fmt.Println(value)
This is important because it helps you see that when the main() thread reads from the done channel, it doesn't print it at the same time. These are two separate steps to the runtime.
I try to explore go channel, i create channel buffer max 10, with gomaxprocess is 2, but i wonder why this code won't receive message
runtime.GOMAXPROCS(2)
messages := make(chan int, 9)
go func() {
for {
i := <-messages
fmt.Println("Receive data:", i)
}
}()
for i := 0; i <= 9; i++ {
fmt.Println("Send data ", i)
messages <- i
}
Your case works like this, though it may appear to work certain times, but it's not guaranteed to always.
Just to add some context, in an unbuffered channel, the sending go routine is blocked as it tries to send a value and a receive is guaranteed to occur before the sending go routine is awakened (in this case the main), so it may seem like a viable option in such cases. But the sending go routine may still exit before the print statement in the receiving go routine is executed. So basically you need to use a synchronization mechanism such that the sending go routine exits only after the work in the receiver is completed.
Here's how you can use a synchronization mechanism, have annotated it so that you can make better sense out of it. This will work for both buffered and unbuffered channels. Another option is to have the receive in the main thread itself so that it doesn't exit before receive processing is done, this way you won't need a separate synchronization mechanism. Hope this helps.
You created a channel which has 9 buffer space, which means main routine (r1) will not blocked until the 10th element was ready to send to messages.
In your go func() (r2), it most probably starts running when r1 almost finished for r2 is a new routine and system takes time to create stacks etc.
so, r2 doesn't print anything, for r1 is done and program exits while r2 has just begin running.
Take the following code snippet:
func main() {
ch := make(chan int)
quit := make(chan int)
go func() {
for {
ch <- querySomePeriodicThing()
}
}()
// ...
loop:
for {
select {
case <-ch: handlePeriodicThing()
case <-quit: break loop
}
}
}
The goroutine should run for the duration of execution. When the select statement receives something from the quit channel, it breaks out of the loop and the program ends, without any attempt to stop the goroutine.
My question: will this have any intermittent adverse effects that are not obvious from running it once or twice? I know that in other languages threads should be cleaned up (i.e., exited) before the program ends, but is go different? Assume that querySomePeriodicThing() does not open file descriptors or sockets or anything that would be bad to leave open.
As mentioned in the spec, your program will exit when the main function completes:
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
So the fact you have other goroutines still running is not a problem from a language point of view. It may still be a problem depending on what your program is doing.
If the goroutine has created some resources that should be cleaned up before program exit, then having execution stop mid-way through might be a problem: in this case, you should make your main function wait for them to complete first. There is no equivalent to pthread_join, so you will need to code this yourself (e.g. by using a channel or sync.WaitGroup).
Note that for some resources are automatically cleaned up by the operating system on process exit (e.g. open files, file locks, etc), so in some cases no special clean up will be necessary
Goroutines aren't threads, they are very lightweight and the runtime automatically cleans them up when they are no longer running, or if the program exits.