I am trying to understand how channels work in golang. The code I have is very simple but the output is surprising.
As the documentation states: reading and writing from/to a channel is blocking the current goroutine, so I thought writing to a channel would block the channel until the main routine yields.
package main
func rtn(messages chan<- string) {
defer close(messages)
println("p1")
messages <- "ping1"
//for i := 0; i < 10000000; i++ { }
println("p2")
messages <- "ping2"
}
func main() {
messages := make(chan string)
go rtn(messages)
for msg := range messages {
println(msg)
}
}
I thought it would print
p1
ping1
p2
ping2
but it actually prints
p1
p2
ping1
ping2
You are using an unbuffered channels, that works as a point of synchronization between the main and second goroutines.
In this case, you only know that when the second goroutine is here messages <- "ping1" the main one is at the line for msg := range messages. Thus, there is no guarantee that main loop reaches println(msg) immediately. I.e., in the meantime the second goroutine could have moved on and reached lines println("p2") and messages <- "ping2".
As a counterexample, I am adding a channel just to enforce the complete synchronization between prints.
package main
func rtn(messages chan<- string, syncChan chan struct{}) {
defer close(messages)
println("p1")
messages <- "ping1"
//Wait for main goroutine to print its message
<-syncChan
//for i := 0; i < 10000000; i++ { }
println("p2")
messages <- "ping2"
//Wait for main goroutine to print its message
<-syncChan
}
func main() {
messages := make(chan string)
syncChan := make(chan struct{})
go rtn(messages, syncChan)
for msg := range messages {
println(msg)
//Notify the second goroutine that is free to go
syncChan <- struct{}{}
}
}
Which prints the output you expected:
p1
ping1
p2
ping2
Here is another example that produce the output you are looking for. In this case the main goroutine is forcefully blocked by the time.Sleep(). This will make the second goroutine be ready to send before the receiver is ready to receive. Therefore, the sender will actually block on the send operation.
package main
import (
"time"
)
func rtn(messages chan<- string) {
defer close(messages)
println("p1")
messages <- "ping1"
//for i := 0; i < 10000000; i++ { }
println("p2")
messages <- "ping2"
}
func main() {
messages := make(chan string)
go rtn(messages)
//Put main goroutine to sleep. This will make the
//sender goroutine ready before the receiver.
//Therefore it will have to actually block!
time.Sleep(time.Millisecond * 500)
for msg := range messages {
println(msg)
}
}
Related
In this example, we have a worker. The idea here is simulate clean shutdown of all go routines based on a condition.
In this case, go routines get spun - based on workers count. Each go routine reads the channel, does some work and sends output to the outputChannel.
The main go routine reads this output and prints it. To simulate a stop condition, the doneChannel is closed. Expected outcome is that select inside each go routine will pick this up and execute return which in turn will call the defer println. The actual output is that it never gets called and main exits.
Not sure what's the reason behind this.
package main
import (
"log"
"time"
)
const jobs = 100
const workers = 1
var timeout = time.After(5 * time.Second)
func main() {
doneChannel := make(chan interface{})
outputChannel := make(chan int)
numberStream := generator()
for i := 1; i <= workers; i++ {
go worker(doneChannel, numberStream, outputChannel)
}
// listen for output
loop:
for {
select {
case i := <-outputChannel:
log.Println(i)
case <-timeout:
// before you timeout cleanup go routines
break loop
}
}
close(doneChannel)
time.Sleep(5 * time.Second)
log.Println("main exited")
}
func generator() <-chan int {
defer log.Println("generator completed !")
c := make(chan int)
go func() {
for i := 1; i <= jobs; i++ {
c <- i
}
defer close(c)
}()
return c
}
func worker(done <-chan interface{}, c <-chan int, output chan<- int) {
// this will be a go routine
// Do some work and send results to output Channel.
// Incase if the done channel is called kill the go routine.
defer log.Println("go routines exited")
for {
select {
case <-done:
log.Println("here")
return
case i := <-c:
time.Sleep(1 * time.Second) // worker delay
output <- i * 100
}
}
}
When your main loop finishes during the timeout, you continue your program and
Close done channel
Print message
Exit
There is no reason to wait for any goroutine to process the signal of this channel.
If you add a small sleep you will see some messages
In real scenarios we use a waitgroup to be sure all goroutine finish properly
Im not 100% clear on why my code doesnt work
package main
import (
"fmt"
"sync"
)
//var wg sync.WaitGroup
func main() {
c := make(chan int)
send(c)
receive(c)
}
func send(c chan<- int) {
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
for j := 0; j < 10; j++ {
c <- j
}
wg.Done()
}()
}
wg.Wait()
close(c)
}
func receive(c <-chan int) {
for v := range c {
fmt.Println(v)
}
}
If I put Go before send it works fine. But i thought having the wait in the send would mean the program would wait there until the go routines created in that function completed and would only go to the receive function once they were all done and so the channel was populated?
The channel is not buffered. A write to an unbuffered channel will only complete if there is a read from that channel in another goroutine. So, all the goroutines created in the send are stuck waiting to write to the channel, never reaching wg.Done, and thus, wg.Wait waits indefinitely. When you put the send in a separate goroutine, receive can run, thus reading from the channel, releasing the waiting goroutines.
I am trying to wrap my mind around go. I want to make a simple program that basically
starts a bunch of go routines
process messages
sends the processed result to a channel
have the main thread collect these results
shut down.
Seems simple. I started with no logic at all. I just send a number and try to get that number back.
issue: I'm deadlocking and I'm not sure why. I think I might be misusing wait groups with channels, because they work individually, but I'm not sure how to get the main thread to block on an arbitrary number of initiated go routines.
package main
import (
"fmt"
"sync"
"time"
)
func main() {
queue := make(chan int)
start := time.Now()
var wg sync.WaitGroup
for i := 0; i < 10; i += 1 {
wg.Add(1)
go count(i, queue, &wg)
}
wg.Wait()
for value := range queue {
println(value)
}
close(queue)
fmt.Println(time.Now().Sub(start))
// fmt.Println(summation)
}
func count(number int, queue chan int, wg *sync.WaitGroup) {
defer wg.Done()
fmt.Println("Starting ", number)
queue <- number
fmt.Println("ending")
}
Your goroutines block on queue <- number because queue is an unbuffered channel and nobody is reading from it, as main blocks on wg.Wait.
Declare queue as a buffered channel instead. For example: queue := make(chan int, 10)
From the Go Tour (concurrency) and subsequent page:
By default, sends and receives block until the other side is ready. This allows goroutines to synchronize without explicit locks or condition variables.
Sends to a buffered channel block only when the buffer is full. Receives block when the buffer is empty.
Alternatively, move wg.Wait after the for v := range queue loop.
This should help.
package main
import (
"fmt"
"sync"
"time"
)
type event struct {
data chan string
numWorker int
}
func (e event) Send() {
var wg sync.WaitGroup
// Spaw numWorker goroutines that sends message to
// the same channel.
for i := 0; i < e.numWorker; i++ {
wg.Add(1)
go func(id int) {
// Do some fake work
time.Sleep(1 * time.Second)
e.data <- fmt.Sprintf("message from go #%d", id)
wg.Done()
}(i)
}
// Wait for goroutines to finish their work.
wg.Wait()
// Close the channel to signal Recv to stop ranging
// over the channel.
close(e.data)
}
func (e event) Recv() {
// Range over the data channel to receive message(s).
for msg := range e.data {
fmt.Println(msg)
}
}
func main() {
e := event{
numWorker: 10, // Number of worker goroutine(s)
data: make(chan string, 5 /* Buffer Size */),
}
// Spawn a goroutine for Send
go e.Send()
// Recv receives data from Send
e.Recv()
}
To avoid deadlocking you can manage the channel and wait groups in separate goroutine. Try change that:
wg.Wait()
for value := range queue {
println(value)
}
close(queue)
with this:
go func() {
wg.Wait()
close(queue)
}()
for value := range queue {
println(value)
}
go version go1.11.4 darwin/amd64
A new channel and goroutine were created, and the content of the old channel was transferred to the new channel through goroutine. It should not block, but after testing, it was found to be blocked.
thanks.
type waiter struct {
ch1 chan struct{}
ch2 <-chan time.Time
limit int
count int
}
func (w *waiter) recv1Block() chan struct{} {
ch := make(chan struct{})
go func() {
for m := range w.ch1 {
ch <- m
}
}()
return ch
}
func (w *waiter) runBlock(wg *sync.WaitGroup) {
defer wg.Done()
i := 0
for i < w.limit {
select {
case <-w.recv1Block(): **// why block here?**
i++
case <-w.recv2():
}
}
w.count = i
}
why recv1Block will be block.
Every time you call recv1Block(), it creates a new channel and launches a background goroutine that tries to read all of the data from it. Since you're calling it in a loop, you will have many things all trying to consume the data from the channel; since the channel never closes, all of the goroutines will run forever.
As an exercise, you might try changing your code to pass around a chan int instead of a chan struct{}, and write a series of sequential numbers, and print them out as they're ultimately received. A sequence like this is valid:
run on goroutine #1 calls recv1Block().
recv1Block() on GR#1 spawns GR#2, and returns channel#2.
run on GR#1 blocks receiving on channel#2.
recv1Block() on GR#2 reads 0 from w.c1.
recv1Block() on GR#2 writes 0 to channel#2 (where run on GR#1 is ready to read).
recv1Block() on GR#2 reads 1 from w.c1.
recv1Block() on GR#2 wants to write 0 to channel#2 but blocks.
run on GR#1 wakes up, and receives the 0.
run on GR#1 calls recv1Block().
recv1Block() on GR#1 spawns GR#3, and returns channel #3.
recv1Block() on GR#3 reads 2 from w.c1.
...
Notice that the value 1 in this sequence will never be written anywhere, and in fact there is nothing left that could read it.
The easy solution here is to not call the channel-creating function in a loop:
func (w *waiter) runBlock(wg *sync.WaitGroup) {
defer wg.Done()
ch1 := w.recv1Block()
ch2 := w.recv2()
for {
select {
case _, ok := <-ch1:
if !ok {
return
}
w.count++
case <-ch2:
}
}
It's also standard practice to close channels when you're done with them. This will terminate a for ... range ch loop, and it will appear as readable to a select statement. In your top-level generator function:
for i := 0; i < w.limit; i++ {
w.ch1 <- struct{}{}
}
close(w.ch1)
And in your "copy the channel" function:
func (w *waiter) recv1Block() chan struct{} {
ch := make(chan struct{})
go func() {
for m := range w.ch1 {
ch <- m
}
close(ch)
}()
return ch
}
This also means that you don't need to run the main loop by "dead reckoning", expecting it to produce exactly 100 items then stop; you can stop whenever its channel exits. The consumer loop I show above does this.
I want to send a value in a channel to go routines from main function. What happens is which go routine will receive the value from the channel first.
package main
import (
"fmt"
"math/rand"
//"runtime"
"strconv"
"time"
)
func main() {
var ch chan int
ch = make(chan int)
ch <- 1
receive(ch)
}
func receive(ch chan int){
for i := 0; i < 4; i++ {
// Create some threads
go func(i int) {
time.Sleep(time.Duration(rand.Intn(1000)) * time.Millisecond)
fmt.Println(<-ch)
}(i)
}
}
My current implementation is giving an error.
fatal error: all goroutines are asleep - deadlock!
How can I know that which go routine will receive the value from the channel first. And what happens to other go routine If those will run or throw an error since there is no channel to receive the value. As it is already received by one of them.
If a create a buffered channel my code works. So I don't get it what has happened behind the scene which is making it work when creating a buffered channel like below:
func main() {
var ch chan int
ch = make(chan int, 10)
ch <- 1
receive(ch)
}
If we look at below code. I can see that we can send values through channels directly there is no need of creating a go routine to send a value thorugh a channel to another go routines.
package main
import "fmt"
func main() {
// We'll iterate over 2 values in the `queue` channel.
queue := make(chan string, 2)
queue <- "one"
queue <- "two"
close(queue)
for elem := range queue {
fmt.Println(elem)
}
}
Then what is wrong with my code. Why is it creating a deadlock.
If all you need is to start several workers and send a task to any of them, then you'd better run workers before sending a value to a channel, because as #mkopriva said above, writing to a channel is a blocking operation. You always have to have a consumer, or the execution will freeze.
func main() {
var ch chan int
ch = make(chan int)
receive(ch)
ch <- 1
}
func receive(ch chan int) {
for i := 0; i < 4; i++ {
// Create some threads
go func(i int) {
time.Sleep(time.Duration(rand.Intn(1000)) * time.Millisecond)
fmt.Printf("Worker no %d is processing the value %d\n", i, <-ch)
}(i)
}
}
Short answer for the question "Which go routine will receive it?" - Whatever. :) Any of them, you can't say for sure.
However I have no idea what is time.Sleep(...) for there, kept it as is.
An unbuffered channel (without a length) blocks until the value has been received. This means the program that wrote to the channel will stop after writing to the channel until it has been read from. If that happens in the main thread, before your call to receive, it causes a deadlock.
There are two more issues: you need to use a WaitGroup to pause the completion until finished, and a channel behaves like a concurrent queue. In particular, it has push and pop operations, which are both performed using <-. For example:
//Push to channel, channel contains 1 unless other things were there
c <- 1
//Pop from channel, channel is empty
x := <-c
Here is a working example:
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
func main() {
var ch chan int
ch = make(chan int)
go func() {
ch <- 1
ch <- 1
ch <- 1
ch <- 1
}()
receive(ch)
}
func receive(ch chan int) {
wg := &sync.WaitGroup{}
for i := 0; i < 4; i++ {
// Create some threads
wg.Add(1)
go func(i int) {
time.Sleep(time.Duration(rand.Intn(1000)) * time.Millisecond)
fmt.Println(<-ch)
wg.Done()
}(i)
}
wg.Wait()
fmt.Println("done waiting")
}
Playground Link
As you can see the WaitGroup is quite simple as well. You declare it at a higher scope. It's essentially a fancy counter, with three primary methods. When you call wg.Add(1) the counter is increased, when you call wg.Done() the counter is decreased, and when you call wg.Wait(), the execution is halted until the counter reaches 0.