In the following code
What happens to goroutine1?(at the end of program we have three goroutine goroutine1 without any functionality)
What happens to channel?(when we make a channel in loop it release the previous channel memory? close it? or something else?)
func main() {
for i := 1; i <= 3; i += 1 {
ch := make(chan int)
// gorutine1
go func() {
time.Sleep(3 * time.Second)
ch <- i
fmt.Println("gorutine1 end")
}()
// gorutine2
go func() {
time.Sleep(1 * time.Second)
ch <- i+1000
fmt.Println("gorutine2 end")
}()
fmt.Println("loop", <-ch)
}
time.Sleep(10 * time.Second)
fmt.Println("main end")
}
Run above code here
For i=1, the loop creates two goroutines, and starts waiting to read from the channel. goroutine2 writes first and terminates. The channel is read, and then i becomes 2. goroutine1 will wait forever, because nobody will read from the channel again. You create a new channel, and do the same thing. When everything is said and done, you have three instances of goroutine1 waiting to write to three different channels.
Related
I'm trying to build some short of semaphore in Go. Although when when the channel receives the signal it just sleeps forever.
I've tried changing the way to sleep and the duration to sleep, but it stills just stopping forever.
Here a representation of what I've tried:
func main() {
backOffChan := make(chan struct{})
go func() {
time.Sleep(2)
backOffChan <- struct{}{}
}()
for {
select {
case <-backOffChan:
d := time.Duration(5 * time.Second)
log.Println("reconnecting in %s", d)
select {
case <-time.After(d):
log.Println("reconnected after %s", d)
return
}
default:
}
}
}
I expect that it just returns after printing the log message and returning.
Thanks!
This code has a number of problems, mainly a tight loop using for/select that may not allow the other goroutine to ever get to send on the channel. Since the default case is empty and the select has only one case, the whole select is unnecessary. The following code works correctly:
backOffChan := make(chan struct{})
go func() {
time.Sleep(1 * time.Millisecond)
backOffChan <- struct{}{}
}()
for range backOffChan {
d := time.Duration(10 * time.Millisecond)
log.Printf("reconnecting in %s", d)
select {
case <-time.After(d):
log.Printf("reconnected after %s", d)
return
}
}
This will wait until the backOffChan gets a message without burning a tight loop.
(Note that this code also addresses issues using log.Println with formatting directives - these were corrected to log.Printf).
See it in action here: https://play.golang.org/p/ksAzOq5ekrm
I was just experimenting with Go channels on my Ubuntu 64 bit environment and got confused with the output the following program produced.
I got the output:
0
1
2
3
Exit
The output when I uncommented the two commented lines:
0
1
2
3
4
Exit
Please explain the behavior.
TIA.
package main
import (
"fmt"
//"time"
)
func main() {
ch := make(chan int)
done := make(chan bool)
go func() {
for i := 0; i < 5; i++ {
ch <- i
}
//time.Sleep(1 * time.Second)
done <- false
}()
go func() {
for {
select {
case message := <-ch:
fmt.Println(message)
case <-done:
return
}
}
}()
<-done
fmt.Println("Exit")
}
Your main thread is waiting on done, then exiting. Meanwhile your first go function pipes 5 values into ch, then sends to done.
The value in done then gets read from the main thread and happens to occur before the second go function reads the last value from ch. When it does so, it exits the program.
Note that if your second thread did happen to both read from ch and done, then your program would deadlock since the main thread would never receive on done and all running go threads would be blocked waiting to receive on channels.
You're not waiting for both goroutines, and only sending a single value over done to 2 receivers, which will deadlock if the second receiver happens to be main.
Using a WaitGroup simplifies the code, and allows you to easily wait for as many goroutines as needed. https://play.golang.org/p/MWknv_9AFKp
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
defer close(ch)
for i := 0; i < 5; i++ {
ch <- i
}
}()
wg.Add(1)
go func() {
defer wg.Done()
for message := range ch {
fmt.Println(message)
}
}()
wg.Wait()
fmt.Println("Exit")
You have two go routines running in parallel. One inserts 5 numbers into the channel and then signals the main thread to exit, and another one reads the numbers from the channel.
Note that once the go routine that is responsible for enqueuing the numbers into the channel finishes, it signals the main thread to exit, regardless of whether the go routine that reads the numbers finished or not. So you could end up with a case where the enqueueing go routine finished before the dequeuing finished, and the main thread exited.
By adding the sleep, you make the enqueueing go routine live a bit longer and give a chance to the dequeueing go routine to read and print all the numbers before the enqueueing go routine signals the main thread to exit.
To solve that, you could just run the dequeuing code in the main thread. No need to run it in a go routine in this case.
go version go1.11.4 darwin/amd64
A new channel and goroutine were created, and the content of the old channel was transferred to the new channel through goroutine. It should not block, but after testing, it was found to be blocked.
thanks.
type waiter struct {
ch1 chan struct{}
ch2 <-chan time.Time
limit int
count int
}
func (w *waiter) recv1Block() chan struct{} {
ch := make(chan struct{})
go func() {
for m := range w.ch1 {
ch <- m
}
}()
return ch
}
func (w *waiter) runBlock(wg *sync.WaitGroup) {
defer wg.Done()
i := 0
for i < w.limit {
select {
case <-w.recv1Block(): **// why block here?**
i++
case <-w.recv2():
}
}
w.count = i
}
why recv1Block will be block.
Every time you call recv1Block(), it creates a new channel and launches a background goroutine that tries to read all of the data from it. Since you're calling it in a loop, you will have many things all trying to consume the data from the channel; since the channel never closes, all of the goroutines will run forever.
As an exercise, you might try changing your code to pass around a chan int instead of a chan struct{}, and write a series of sequential numbers, and print them out as they're ultimately received. A sequence like this is valid:
run on goroutine #1 calls recv1Block().
recv1Block() on GR#1 spawns GR#2, and returns channel#2.
run on GR#1 blocks receiving on channel#2.
recv1Block() on GR#2 reads 0 from w.c1.
recv1Block() on GR#2 writes 0 to channel#2 (where run on GR#1 is ready to read).
recv1Block() on GR#2 reads 1 from w.c1.
recv1Block() on GR#2 wants to write 0 to channel#2 but blocks.
run on GR#1 wakes up, and receives the 0.
run on GR#1 calls recv1Block().
recv1Block() on GR#1 spawns GR#3, and returns channel #3.
recv1Block() on GR#3 reads 2 from w.c1.
...
Notice that the value 1 in this sequence will never be written anywhere, and in fact there is nothing left that could read it.
The easy solution here is to not call the channel-creating function in a loop:
func (w *waiter) runBlock(wg *sync.WaitGroup) {
defer wg.Done()
ch1 := w.recv1Block()
ch2 := w.recv2()
for {
select {
case _, ok := <-ch1:
if !ok {
return
}
w.count++
case <-ch2:
}
}
It's also standard practice to close channels when you're done with them. This will terminate a for ... range ch loop, and it will appear as readable to a select statement. In your top-level generator function:
for i := 0; i < w.limit; i++ {
w.ch1 <- struct{}{}
}
close(w.ch1)
And in your "copy the channel" function:
func (w *waiter) recv1Block() chan struct{} {
ch := make(chan struct{})
go func() {
for m := range w.ch1 {
ch <- m
}
close(ch)
}()
return ch
}
This also means that you don't need to run the main loop by "dead reckoning", expecting it to produce exactly 100 items then stop; you can stop whenever its channel exits. The consumer loop I show above does this.
Here is my program which is producing deadlock, how do I avoid it and what is the recommended pattern to handle this kind of situation.
The problem is after timeout how do I detect that there is no reader on my channel ?
var wg sync.WaitGroup
func main() {
wg.Add(1)
c := make(chan int)
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
c <- 10
wg.Wait()
}
func readFromChannel(c chan int, ti <-chan time.Time) {
select {
case x := <-c:
fmt.Println("Read", x)
case <-ti:
fmt.Println("TIMED OUT")
}
wg.Done()
}
So, lets look at what's really going on in your source. You have two goroutines (there's more than two, but we're going to focus on the explicit ones), main and readFromChannel.
Lets look at what readFromChannel does:
if channel `c` is not empty before `ti` has expired, print its contents and return, after signalling its completion to wait group.
if `ti` has expired before `c` is not empty, print "TIMED OUT" and return, after signalling its completion to wait group.
now Main:
adds to waitgroup
make a channel `c`
start a goroutine `readFromChannel`
sleep for 5 seconds
send 10 to channel `c`
call wait for waitgroup
Now, lets go through the flow of execution for your code, concurrently (your code may/ may not execute in this order every time, keep that in mind)
1) wg.Add(1)
2) c := make(chan int)
3) go readFromChannel(c, time.After(time.Duration(2)*time.Second))
#timer ti starts#
4) time.Sleep(time.Duration(5) * time.Second)
#MAIN Goroutine begins sleep
#timer ti expires#
5) case <-ti:
6) fmt.Println("TIMED OUT")
7) wg.Done()
# readFromChannel Goroutine returns #
#MAIN Goroutine exits sleep#
8) c<-10
9) ......#DEADLOCK#
Now you can guess why you got a deadlock. In go, non buffered channels will block until something happens on the other end of the channel, regardless of whether you're sending or receiving. So c <- 10 will block until something reads from the other end of c, but the goroutine you had for that has dropped out of the picture 2 seconds ago. Therefore, c blocks forever, and since main is the last goroutine left, you get a Deadlock.
How to prevent it? When using channels, ensure that there's always a receive at the other end of the channel for every send.
Using a buffered channel in this scenario can serve as a quick fix, but may fuel potential gotchas in larger repositories. For example, assuming you wrote more data to c afterward and ran go readFromChannel(c, time.After(time.Duration(2)*time.Second)) a second time. You might see:
Read D1
Read D2
or
TIMED OUT
Read D1
solely based on chance. That's probably not the behavior you'd want.
Here's how I'd resolve the deadlock:
func main() {
wg.Add(1)
c := make(chan int)
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
c <- 10
wg.Wait()
}
func readFromChannel(c chan int, ti <-chan time.Time) {
// the forloop will run forever
loop: // **
for {
select {
case x := <-c:
fmt.Println("Read", x)
break loop // breaks out of the for loop and the select **
case <-ti:
fmt.Println("TIMED OUT")
}
}
wg.Done()
}
** see this answer for details
You have an unbuffered channel. According to the docs:
If the channel is unbuffered, the sender blocks until the receiver has
received the value. If the channel has a buffer, the sender blocks
only until the value has been copied to the buffer
By changing the channel to being buffered, we can avoid deadlock.
c := make(chan int, 10) // holds 10 ints
I also suggest reading https://golang.org/doc/effective_go.html#channels, it's got some good stuff in there related to channels.
Your problem is that you are using select statement but you are not using inside a goroutine.
go func() {
for {
select {
case x := <-c:
fmt.Println("Read", x)
case <-ti:
fmt.Println("TIMED OUT")
}
}
}()
Getting the values out of different concurrently executing goroutines can be accomplished with the select keyword, which closely resembles the switch control statement and is sometimes called the communications switch.
Using a send operation in a select statement with a default case guarantees that the send will be non-blocking! If there are no cases, the select blocks execution forever.
https://play.golang.org/p/Ai1ggveb4s
This is an older question, but i'm diving deep into learning channels myself and found this here.
I think you just need to close the channel after your done sending on it?
Code:
func main() {
wg.Add(1)
c := make(chan int)
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
c <- 10
close(c) // <- CLOSE IT HERE
wg.Wait()
}
To avoid deadlock here is some general advice from Deadlocks: the dark side of concurrency
Channel deadlock, Don’t send and receive to the same channel in the same goroutine
Read lock, Don’t take a read lock twice in the same goroutine
Release the lock as soon as possible
Objects which contain locks that call each other methods are a deadlock waiting to happen
Care total lock ordering
Access only read variables outside the lock
You can add a buffer to the c chan, like this:
var wg sync.WaitGroup
func main() {
wg.Add(1)
c := make(chan int, 1)
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
c <- 10 // never blocks, because of the buffer
wg.Wait()
}
func readFromChannel(c chan int, ti <-chan time.Time) {
select {
case x := <-c:
fmt.Println("Read", x)
case <-ti:
fmt.Println("TIMED OUT")
}
wg.Done()
}
a write operation(c <- 10) to an unbuffered channel will be blocked until it there is a read operation(case x := <-c:).
Your timeout(2s) is smaller than sleep duration(5s) and because of this channel c will never be read in the select block.
I would recommend you to read chapter 8 & 9 of this book
func main() {
wg.Add(1)
c := make(chan int)
go func() {
c <- 10
}()
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
wg.Wait()
}//This will read without deadlock
func main() {
wg.Add(1)
c := make(chan int)
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
go func() {
c <- 10
}()
wg.Wait()
}//Time out without deadlock
This question already has answers here:
Timeout for WaitGroup.Wait()
(10 answers)
Closed 7 months ago.
I have come across a situation that i want to trace some goroutine to sync on a specific point, for example when all the urls are fetched. Then, we can put them all and show them in specific order.
I think this is the barrier comes in. It is in go with sync.WaitGroup. However, in real situation that we can not make sure that all the fetch operation will succeed in a short time. So, i want to introduce a timeout when wait for the fetch operations.
I am a newbie to Golang, so can someone give me some advice?
What i am looking for is like this:
wg := &sync.WaigGroup{}
select {
case <-wg.Wait():
// All done!
case <-time.After(500 * time.Millisecond):
// Hit timeout.
}
I know Wait do not support Channel.
If all you want is your neat select, you can easily convert blocking function to a channel by spawning a routine which calls a method and closes/sends on channel once done.
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
// All done!
case <-time.After(500 * time.Millisecond):
// Hit timeout.
}
Send your results to a buffered channel enough to take all results, without blocking, and read them in for-select loop in the main thread:
func work(msg string, d time.Duration, ret chan<- string) {
time.Sleep(d) // Work emulation.
select {
case ret <- msg:
default:
}
}
// ...
const N = 2
ch := make(chan string, N)
go work("printed", 100*time.Millisecond, ch)
go work("not printed", 1000*time.Millisecond, ch)
timeout := time.After(500 * time.Millisecond)
loop:
for received := 0; received < N; received++ {
select {
case msg := <-ch:
fmt.Println(msg)
case <-timeout:
fmt.Println("timeout!")
break loop
}
}
Playground: http://play.golang.org/p/PxeEEJo2dz.
See also: Go Concurrency Patterns: Timing out, moving on.
Another way to do it would be to monitor it internally, your question is limited but I'm going to assume you're starting your goroutines through a loop even if you're not you can refactor this to work for you but you could do one of these 2 examples, the first one will timeout each request to timeout individually and the second one will timeout the entire batch of requests and move on if too much time has passed
var wg sync.WaitGroup
wg.Add(1)
go func() {
success := make(chan struct{}, 1)
go func() {
// send your request and wait for a response
// pretend response was received
time.Sleep(5 * time.Second)
success <- struct{}{}
// goroutine will close gracefully after return
fmt.Println("Returned Gracefully")
}()
select {
case <-success:
break
case <-time.After(1 * time.Second):
break
}
wg.Done()
// everything should be garbage collected and no longer take up space
}()
wg.Wait()
// do whatever with what you got
fmt.Println("Done")
time.Sleep(10 * time.Second)
fmt.Println("Checking to make sure nothing throws errors after limbo goroutine is done")
Or if you just want a general easy way to timeout ALL requests you could do something like
var wg sync.WaitGroup
waiter := make(chan int)
wg.Add(1)
go func() {
success := make(chan struct{}, 1)
go func() {
// send your request and wait for a response
// pretend response was received
time.Sleep(5 * time.Second)
success <- struct{}{}
// goroutine will close gracefully after return
fmt.Println("Returned Gracefully")
}()
select {
case <-success:
break
case <-time.After(1 * time.Second):
// control the timeouts for each request individually to make sure that wg.Done gets called and will let the goroutine holding the .Wait close
break
}
wg.Done()
// everything should be garbage collected and no longer take up space
}()
completed := false
go func(completed *bool) {
// Unblock with either wait
wg.Wait()
if !*completed {
waiter <- 1
*completed = true
}
fmt.Println("Returned Two")
}(&completed)
go func(completed *bool) {
// wait however long
time.Sleep(time.Second * 5)
if !*completed {
waiter <- 1
*completed = true
}
fmt.Println("Returned One")
}(&completed)
// block until it either times out or .Wait stops blocking
<-waiter
// do whatever with what you got
fmt.Println("Done")
time.Sleep(10 * time.Second)
fmt.Println("Checking to make sure nothing throws errors after limbo goroutine is done")
This way your WaitGroup will stay in sync and you won't have any goroutines left in limbo
http://play.golang.org/p/g0J_qJ1BUT try it here you can change the variables around to see it work differently
Edit: I'm on mobile If anybody could fix the formatting that would be great thanks.
If you would like to avoid mixing concurrency logic with business logic, I wrote this library https://github.com/shomali11/parallelizer to help you with that. It encapsulates the concurrency logic so you do not have to worry about it.
So in your example:
package main
import (
"github.com/shomali11/parallelizer"
"fmt"
)
func main() {
urls := []string{ ... }
results = make([]*HttpResponse, len(urls)
options := &Options{ Timeout: time.Second }
group := parallelizer.NewGroup(options)
for index, url := range urls {
group.Add(func(index int, url string, results *[]*HttpResponse) {
return func () {
...
results[index] = &HttpResponse{url, response, err}
}
}(index, url, &results))
}
err := group.Run()
fmt.Println("Done")
fmt.Println(fmt.Sprintf("Results: %v", results))
fmt.Printf("Error: %v", err) // nil if it completed, err if timed out
}