I'm following a quick intro to Go and one of the examples is:
package main
import (
"fmt"
"time"
)
func worker(done chan bool) {
fmt.Print("working...")
time.Sleep(time.Second)
fmt.Println("done")
done <- true
}
func main() {
done := make(chan bool, 1)
go worker(done)
<-done
}
I understand whats occuring but I guess I'm not grasping the sequence of events or the limitations?
A channel is created called done with a buffer size of 1.
The channel is passed into a function
After the timer is complete it adds a true boolean to the channel
I'm not sure what the final <-done is doing though
from: https://gobyexample.com/channel-synchronization
Receiver operator <- followed by channel name (done in this case) is used to wait for a value written to channel from worker goroutine. (i.e this read operation will be blocking. If you omit <-done, main goroutine will exit immediately even before worker's goroutine start and you won't be able to see results)
You can do whatever you want with <-done as value: assign it to another variable, pass it as a parameter to another function or just ignore it as in your case... etc.
Related
I'm new to Golang and I have hard time figuring out why exactly the following code produces deadlock. Also, how can I fix it so it works?
package main
import "fmt"
func main() {
m := make(map[int]chan string)
go func() {
m[0] = make(chan string)
m[0] <- "abab"
}()
fmt.Println(<-m[0])
}
EDIT:
Thanks for your answers! Unfortunately, initializing m[0] with
m[0] = make(chan string)
before launching a new goroutine is not exactly what I want. My problem is: is there a way to create channels "dynamically"? E.g. I have a map m of type map[int]chan string and I receive requests that contain something like id of type int. I would like to send a message via channel map[id], but initializing channels for every int would be too costly. How do I solve/work around this?
So, in other words, I would like to have a separate job queue for every id and initialize each queue lazily.
Updated answer after OP updated the question
You can just loop on all the keys in your map, maybe have another goroutine that keeps looping on all the keys. Obviously if a key hasnt been initialized, then it wont come up in the for range loop. For each key, you can then start a goroutine that listens so it doesnt block, or you can use a buffered channels so they wont block up to the buffer limit. You can also preferably use a waitGroup, rather than the time.Sleep(), these are only for this trivial example.
package main
import (
"fmt"
"time"
)
func main() {
m := make(map[int]chan string)
go func() {
m[0] = make(chan string)
m[0] <- "abab"
}()
time.Sleep(time.Second * 1) //sleep so the above goroutine initializes the key 0 channel
for key := range m{ //loop on all non-nil keys
fmt.Println(key)
go func(k int){ // goroutine to listen on this channel
fmt.Println(<- m[k])
}(key)
}
time.Sleep(time.Second * 1) //sleep so u can see the effects of the channel recievers
}
Old answer
This is how the flow is. The main goroutine starts. The map is created. The main goroutine encounters another goroutine. It spawns said goroutine and goes on with its life. Then it meets this line, fmt.Println(<-m[0]), which is a problem, since the map is indeed initialized, but the channel in the map itself isnt initialized! By the time the main goroutine has reached fmt.Println(<-m[0]), the other goroutine hadn't yet initialized the channel! So its a simple fix, just initialize the channel before spawning the goroutine and you're good to go!
package main
import "fmt"
func main() {
m := make(map[int]chan string)
m[0] = make(chan string)
go func() {
m[0] <- "abab"
}()
fmt.Println(<-m[0])
}
Edit: Note that fmt.Println(<-m[0]) is blocking, which means that if in that other goroutine, you dont send on the channel, you will also go into a deadlock, since you are trying to recieve on the channel when no one is actually sending.
You need to synchronize the creation of a channel.
As it stands, your main thread arrives at <-m[0] while m[0] is still an uninitialized channel, and receiving on an uninitialized channel blocks forever.
Your go routine creates a new channel and places it in m[0], but the main go routine is already listening on the prior zero value. Sending on this new channel also blocks forever, as there is nothing reading from it, so all go routines block.
To fix this, move m[0] = make(chan string) above your go routine, so it happens synchronously.
I read it on (https://www.geeksforgeeks.org/channel-in-golang/) that:
"In the channel, the send and receive operation block until another side is not ready by default.
It allows goroutine to synchronize with each other without explicit locks or condition variables."
To test above statement, I have written a sample program mentioned below:
Program:
package main
import (
"fmt"
"sync"
"time"
)
func myFunc(ch chan int) {
fmt.Println("Inside goroutine:: myFunc()")
fmt.Println(10 + <-ch) //<-- According to rule, control will be blocked here until 'ch' sends some data so that it will be received in our myFunc() go routine.
}
func main() {
fmt.Println("Start Main method")
// Creating a channel
ch := make(chan int)
go myFunc(ch) //<-- This go routine started in a new thread
time.Sleep(2 * time.Second) //<--- introduced a Sleep of 2 seconds to ensure that myFunc() go routine executes before main thread
ch <- 10
fmt.Println("End Main method")
}
I was expecting below output:
Start Main method
Inside goroutine:: myFunc()
20
End Main method
But, Actual output received is:
Start Main method
Inside goroutine:: myFunc()
End Main method
Why the value sent through channel is not printed?
I think, it is because main thread finished its execution first and hence, all other goroutine also terminated.
If that is the case, then, why does the rule said - It allows goroutine to synchronize with each other without explicit locks or condition variables.
Because, to get the expected output, I have to use sync.WaitGroup to tell the main thread to wait for the other goroutine to finish. Isn't it violating the above rule as I am using locks in form of waitgroup?
PS: I am learning golang. So please forgive if I get the concept totally wrong.
The main goroutine exists before the myFunc goroutine is able to print the output. Here is an implementation which ensures that myFunc goroutine finishes before the main goroutine exits.
package main
import (
"fmt"
"sync"
"time"
)
func myFunc(ch chan int, wg *sync.WaitGroup) {
defer wg.Done()
fmt.Println("Inside goroutine:: myFunc()")
fmt.Println(10 + <-ch) //<-- According to rule, control will be blocked here until 'ch' sends some data so that it will be received in our myFunc() go routine.
}
func main() {
fmt.Println("Start Main method")
// Creating a channel
ch := make(chan int)
wg := sync.WaitGroup{}
wg.Add(1)
go myFunc(ch, &wg) //<-- This go routine started in a new thread
time.Sleep(2 * time.Second) //<--- introduced a Sleep of 2 seconds to ensure that myFunc() go routine executes before main thread
ch <- 10
wg.Wait()
fmt.Println("End Main method")
}
The channels are used here for synchronization and it works as described in documentation. It does not mean that the code starting from this point in the code will be executed at the same speed. It only means that main goroutine will not continue if myFunc goroutine is not reading from channel. And myFunc will wait for main goroutine to push data to channel. After this happen both goroutines will continue it execution independently.
Try this, used your code as basis
package main
import (
"fmt"
"time"
)
func myFunc(ch chan int, done chan struct{}) {
defer close(done) // channel will be closed in the function exit phase
fmt.Println("Inside goroutine:: myFunc()")
fmt.Println(10 + <-ch) //<-- According to rule, control will be blocked here until 'ch' sends some data so that it will be received in our myFunc() go routine.
}
func main() {
fmt.Println("Start Main method")
// Creating a channel
ch := make(chan int)
done := make(chan struct{}) // signal channel
go myFunc(ch, done) //<-- This go routine started in a new thread
time.Sleep(2 * time.Second) //<--- introduced a Sleep of 2 seconds to ensure that myFunc() go routine executes before main thread
ch <- 10
<-done // waiting for function complete
fmt.Println("End Main method")
}
Or use Jaroslaw's suggestion.
Because go is so fast... https://play.golang.org/p/LNyDAA3mGYY
After you send to channel scheduler isn't fast enoght... and program exists. I have introduced an additional context switcher for scheduler to show effect.
Yes, you are right
I think, it is because main thread finished its execution first and hence, all other goroutine also terminated.
If you check the above program execution. The sleep is before main thread writes to the channel. Now even though which goroutine() will have CPU time is completely arbitary, but in the above case if the main sleeps before the explicit sleep logic. myFunc will be blocked as there is no data in ch
Here I made a slight change to the above code to make main sleep after writing data into Channel. It gives the expected output, Without using waitgroup or quit channels.
package main
import (
"fmt"
"time"
)
func myFunc(ch chan int) {
fmt.Println("Inside goroutine:: myFunc()")
fmt.Println(10 + <-ch) //<-- According to rule, control will be blocked here until 'ch' sends some data so that it will be received in our myFunc() go routine.
}
func main() {
fmt.Println("Start Main method")
// Creating a channel
ch := make(chan int)
go myFunc(ch) //<-- This go routine started in a new thread
ch <- 10
time.Sleep(2 * time.Second) //<--- introduced a Sleep of 2 seconds to ensure that myFunc() go routine executes before main thread
fmt.Println("End Main method")
}
It is currently a race condition between the myFunc being able to print and your main function exiting.
If we look at the spec for program execution at
https://golang.org/ref/spec#Program_execution
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
It is still your job to make sure that all spawned goroutines will complete before your main goroutine exists.
In your case, you could use a waitgroup as you mentioned or you could use a done channel.
https://play.golang.org/p/RVr0HXuUMgn
Given your code you could also close the channel you use to send the integer over since you are passing it to the function as bidirectional but it's not strictly idiomatic.
https://play.golang.org/p/wGvexC5ZgIi
I am trying to understand channels in Go. I have read that by default sends and receives block until both the sender and receiver are ready. But how do we figure out readyness of sender and receiver.
For example in the following code
package main
import "fmt"
func main() {
ch := make(chan int)
ch <- 1
fmt.Println(<-ch)
}
The program will get stuck on the channel send operation waiting forever for someone to read the value. Even though we have a receive operation in println statement it ends up in a deadlock.
But for the following program
package main
import "fmt"
func main() {
ch := make(chan int)
go func () {
ch <- 1
}()
fmt.Println(<-ch)
}
The integer is passed successfully from go routine to main program. What made this program work? Why second works but first do not? Is go routine causing some difference?
Let's step through the first program:
// My notes here
ch := make(chan int) // make a new int channel
ch <- 1 // block until we can send to that channel
// keep blocking
// keep blocking
// still waiting for a receiver
// no reason to stop blocking yet...
// this line is never reached, because it blocks above forever.
fmt.Println(<-ch)
The second program splits the send off into its own line of execution, so now we have:
ch := make(chan int) // make a new int channel
go func () { // start a new line of execution
ch <- 1 // block this second execution thread until we can send to that channel
}()
fmt.Println(<-ch) // block the main line of execution until we can read from that channel
Since those two lines of execution can work independently, the main line can get down to fmt.Println and try and receive from the channel. The second thread will wait to send until it has.
The go routine absolutely makes a difference. The go routine that writes to the channel will be blocked until your main function is ready to read from the channel in the print statement. Having two concurrent threads, one that reads and one that writes fulfills the readiness on both sides.
In your first example, the single thread gets blocked by the channel write statement and will never reach the channel read.
You need to have a concurrent go routine to read from a channel whenever you write to it. Concurrency goes hand-in-hand with channel usage.
I’m writing the Walk function in the go tutorial that basically traverses a tree in-order. What I have works:
package main
import (
"fmt"
"code.google.com/p/go-tour/tree"
)
// Walk walks the tree t sending all values
// from the tree to the channel ch.
func Walk__helper(t *tree.Tree, ch chan int) {
if (t == nil) {
return
}
Walk__helper(t.Left, ch)
ch <- t.Value
Walk__helper(t.Right, ch)
}
func Walk(t *tree.Tree, ch chan int) {
Walk__helper(t, ch)
close(ch)
}
func main() {
ch := make(chan int)
go Walk(tree.New(1), ch)
for v := range ch {
fmt.Println(v)
}
}
Why must I use go Walk(tree.New(1), ch) instead of just Walk(tree.New(1), ch)?
I was under the impression that the go keyword basically spawns a new thread. In that case, we’d run into issues since the for loop might run before the subroutine completes.
Strangely, when I take out the go keyword, I get a deadlock. This is rather counterintuitive to me. What exactly is the go keyword doing here?
The key point here is range when coupled with a channel.
When you range over a channel (in this case, ch), it will wait for items to be sent on the channel before iterating through the loop. This is a safe, "blocking" action, that will not deadlock while it waits for the channel to receive an item.
The deadlock occurs when not using a goroutine because your channel isn't buffered. If you don't use a goroutine, then the method call is synchronous, Walk puts something on the channel.. and it blocks until that is popped off. It never gets popped off.. because the method call was synchronous.
I was under the impression that the go keyword basically spawns a new thread
..that is incorrect. There are many more important implementation details required to understand what goes on there. You should separate your thought process of a goroutine from a thread.. and just think of a goroutine as a concurrently executing piece of code, without a "thread".
I'm trying to implement an Observer Pattern suggested here; Observer pattern in Go language
(the code listed above doesn't compile and is incomplete). Here, is a complete code that compiles but I get deadlock error.
package main
import (
"fmt"
)
type Publisher struct{
listeners []chan int
}
type Subscriber struct{
Channel chan int
Name string
}
func (p *Publisher) Sub(c chan int){
p.listeners = append(p.listeners, c)
}
func (p *Publisher) Pub(m int, quit chan int){
for _, c := range p.listeners{
c <- m
}
quit <- 0
}
func (s *Subscriber) ListenOnChannel(){
data := <-s.Channel
fmt.Printf("Name: %v; Data: %v\n", s.Name, data)
}
func main() {
quit := make(chan int)
p := &Publisher{}
subscribers := []*Subscriber{&Subscriber{Channel: make(chan int), Name: "1"}, &Subscriber{Channel: make(chan int), Name: "2"}, &Subscriber{Channel: make(chan int), Name: "3"}}
for _, v := range subscribers{
p.Sub(v.Channel)
go v.ListenOnChannel()
}
p.Pub(2, quit)
<-quit
}
Also, if I get rid of 'quit' completely, I get no error but it only prints first record.
The problem is that you're sending to quit on the same goroutine that's receiving from quit.
quit has a buffer size of 0, which means that in order to proceed there has to be a sender on one side and a receiver on the other at the same time. You're sending, but no one's on the other end, so you wait forever. In this particular case the Go runtime is able to detect the problem and panic.
The reason only the first value is printed when you remove quit is that your main goroutine is exiting before your remaining two are able to print.
Do not just increase channel buffer sizes to get rid of problems like this. It can help (although in this case it doesn't), but it only covers up the problem and doesn't truly fix the underlying cause. Increasing a channel's buffer size is strictly an optimization. In fact, it's usually better to develop with no buffer because it makes concurrency problems more obvious.
There are two ways to fix the problem:
Keep quit, but send 0 on it in each goroutine inside ListenOnChannel. In main, make sure you receive a value from each goroutine before moving on. (In this case, you'll wait for three values.)
Use a WaitGroup. There's a good example of how it works in the documentation.
In general this looks good, but there is one problem. Remember that channels are either buffered or unbuffered (synchronous or asynchronous). When you send to an unbuffered channel or to a channel with a full buffer the sender will block until the data has been removed from the channel by a receiver.
So with that, I'll ask a question or two of my own:
Is the quit channel synchronous or asynchronous?
What happens in Pub when execution hits quit<-0?
One solution that fixes your problem and allows the code to run is to change the second-to-last code line to be go p.Pub(2, quit). But there is another solution. Can you see what it is?
I don't actually get the same behavior you do if I remove <-quit from the original code. And this should not affect the output because as it is written that line is never executed.