I'm trying to build some short of semaphore in Go. Although when when the channel receives the signal it just sleeps forever.
I've tried changing the way to sleep and the duration to sleep, but it stills just stopping forever.
Here a representation of what I've tried:
func main() {
backOffChan := make(chan struct{})
go func() {
time.Sleep(2)
backOffChan <- struct{}{}
}()
for {
select {
case <-backOffChan:
d := time.Duration(5 * time.Second)
log.Println("reconnecting in %s", d)
select {
case <-time.After(d):
log.Println("reconnected after %s", d)
return
}
default:
}
}
}
I expect that it just returns after printing the log message and returning.
Thanks!
This code has a number of problems, mainly a tight loop using for/select that may not allow the other goroutine to ever get to send on the channel. Since the default case is empty and the select has only one case, the whole select is unnecessary. The following code works correctly:
backOffChan := make(chan struct{})
go func() {
time.Sleep(1 * time.Millisecond)
backOffChan <- struct{}{}
}()
for range backOffChan {
d := time.Duration(10 * time.Millisecond)
log.Printf("reconnecting in %s", d)
select {
case <-time.After(d):
log.Printf("reconnected after %s", d)
return
}
}
This will wait until the backOffChan gets a message without burning a tight loop.
(Note that this code also addresses issues using log.Println with formatting directives - these were corrected to log.Printf).
See it in action here: https://play.golang.org/p/ksAzOq5ekrm
Related
I'm trying to make my own custom "Channel Timeout". More precisely, the time.After function inside it. In other words, I'm trying to implement this:
select {
case v := <-c:
fmt.Println("Value v: ", v)
case <-time.After(1 * time.Second):
fmt.Println("Timeout")
}
But unfortunately I ran into a problem.
My implementation is:
func waitFun(wait int) chan int {
time.Sleep(time.Duration(wait) * time.Second)
c := make(chan int)
c <- wait
return c
}
func main() {
c := make(chan int)
go func() {
time.Sleep(3 * time.Second)
c <- 10
}()
select {
case v := <-c:
fmt.Println("Value v: ", v)
case <-waitFun(1):
fmt.Println("Timeout")
}
time.Sleep(4 * time.Second)
}
For some reason this doesn't work. The error is: all goroutines are asleep - deadlock!. I understand that at some point all goroutines (main and the one introduced with an anonymous function) will go to sleep, but is this the reason for the error or something else? I mean, it's not "infinite sleep" or "infinite wait for something", so it's not a dead lock, right? Additionally, using time.After also sleeps goroutines, right? What do I need to change to make my program work correctly?
select statement will evaluate all cases when it is run, so this code will actually wait for the waitFun to return before it starts listening to any channels. You have to change the waitFun to return the channel immediately:
func waitFun(wait int) chan int {
c := make(chan int)
go func() {
time.Sleep(time.Duration(wait) * time.Second)
c <- wait
}()
return c
}
In the following code
What happens to goroutine1?(at the end of program we have three goroutine goroutine1 without any functionality)
What happens to channel?(when we make a channel in loop it release the previous channel memory? close it? or something else?)
func main() {
for i := 1; i <= 3; i += 1 {
ch := make(chan int)
// gorutine1
go func() {
time.Sleep(3 * time.Second)
ch <- i
fmt.Println("gorutine1 end")
}()
// gorutine2
go func() {
time.Sleep(1 * time.Second)
ch <- i+1000
fmt.Println("gorutine2 end")
}()
fmt.Println("loop", <-ch)
}
time.Sleep(10 * time.Second)
fmt.Println("main end")
}
Run above code here
For i=1, the loop creates two goroutines, and starts waiting to read from the channel. goroutine2 writes first and terminates. The channel is read, and then i becomes 2. goroutine1 will wait forever, because nobody will read from the channel again. You create a new channel, and do the same thing. When everything is said and done, you have three instances of goroutine1 waiting to write to three different channels.
I need to read data from the Go channel for a certain period of time (say 5 seconds). The select statement with timeout doesn't work for me, as I need to read as many values as available and stop exactly after 5 seconds. So far, I've come up with a solution using an extra time channel https://play.golang.org/p/yev9CcvzRIL
package main
import "time"
import "fmt"
func main() {
// I have no control over dataChan
dataChan := make(chan string)
// this is a stub to demonstrate some data coming from dataChan
go func() {
for {
dataChan <- "some data"
time.Sleep(time.Second)
}
}()
// the following is the code I'm asking about
timeChan := time.NewTimer(time.Second * 5).C
for {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-timeChan:
fmt.Println("Time's up!")
return
}
}
}
I'm wondering is there any better or more idiomatic way for solving this problem?
That's pretty much it. But if you don't need to stop or reset the timer, simply use time.After() instead of time.NewTimer(). time.After() is "equivalent to NewTimer(d).C".
afterCh := time.After(5 * time.Second)
for {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-afterCh:
fmt.Println("Time's up!")
return
}
}
Alternatively (to your liking), you may declare the after channel in the for statement like this:
for afterCh := time.After(5 * time.Second); ; {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-afterCh:
fmt.Println("Time's up!")
return
}
}
Also I know this is just an example, but always think how a goroutine you start will properly end, as the goroutine producing data in your case will never terminate.
Also don't forget that if multiple cases may be executed without blocking, one is chosen randomly. So if dataChan is ready to receive from "non-stop", there is no guarantee that the loop will terminate immediately after the timeout. In practice this is usually not a problem (starting with that even the timeout is not a guarantee, see details at Golang Timers with 0 length), but you should not forget about it in "mission-critial" applications. For details, see related questions:
force priority of go select statement
golang: How the select worked when multiple channel involved?
It looks like context with deadline would be good fit, something like
func main() {
dataChan := make(chan string)
ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(5*time.Second))
defer cancel()
go func(ctx context.Context) {
for {
select {
case dataChan <- "some data":
time.Sleep(time.Second)
case <-ctx.Done():
fmt.Println(ctx.Err())
close(dataChan)
return
}
}
}(ctx)
for d := range dataChan {
fmt.Println("Got:", d)
}
}
I'm curious about the behaviour of channels and how they work in relation to loops. Suppose I have the following code:
Consumer
tick := time.Tick(time.Duration(2) * time.Second)
for {
select {
case <-tick:
p.channel <-true
}
}
And I have a goroutine that has the following:
Processor
for {
select {
case canProcess := <-p.channel:
// synchronous process that takes longer than 2 seconds
case <-p.stop:
return
}
}
What happens when the Consumer pushes to the channel faster than the Processor can complete its synchronous process?
Do they pile up waiting for the Processor to complete or do they skip a "beat" as such?
If they pile up, is there potential for memory leaking?
I know I can put the synchronous process in a goroutine instead, but this is really to understand how channels behave. (i.e. my example has a 2-second tick, but it doesn't have to).
select will only invoke next time.Tick if previous case corresponding code was completed,
you can still have some lever by specifying size of channel i.e channel := make(chan bool,10)
See below:
func main() {
channel := make(chan bool, 10)
go func() {
tick := time.Tick(time.Duration(1) * time.Second)
for {
select {
case <-tick:
fmt.Printf("Producer: TICK %v\n", time.Now())
channel <- true
}
}
}()
for {
select {
case canProcess := <-channel:
time.Sleep(3* time.Second)
fmt.Printf("Consumer: Completed : %v\n")
fmt.Printf("%v\n", canProcess)
}
}
}
This question already has answers here:
Timeout for WaitGroup.Wait()
(10 answers)
Closed 7 months ago.
I have come across a situation that i want to trace some goroutine to sync on a specific point, for example when all the urls are fetched. Then, we can put them all and show them in specific order.
I think this is the barrier comes in. It is in go with sync.WaitGroup. However, in real situation that we can not make sure that all the fetch operation will succeed in a short time. So, i want to introduce a timeout when wait for the fetch operations.
I am a newbie to Golang, so can someone give me some advice?
What i am looking for is like this:
wg := &sync.WaigGroup{}
select {
case <-wg.Wait():
// All done!
case <-time.After(500 * time.Millisecond):
// Hit timeout.
}
I know Wait do not support Channel.
If all you want is your neat select, you can easily convert blocking function to a channel by spawning a routine which calls a method and closes/sends on channel once done.
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
// All done!
case <-time.After(500 * time.Millisecond):
// Hit timeout.
}
Send your results to a buffered channel enough to take all results, without blocking, and read them in for-select loop in the main thread:
func work(msg string, d time.Duration, ret chan<- string) {
time.Sleep(d) // Work emulation.
select {
case ret <- msg:
default:
}
}
// ...
const N = 2
ch := make(chan string, N)
go work("printed", 100*time.Millisecond, ch)
go work("not printed", 1000*time.Millisecond, ch)
timeout := time.After(500 * time.Millisecond)
loop:
for received := 0; received < N; received++ {
select {
case msg := <-ch:
fmt.Println(msg)
case <-timeout:
fmt.Println("timeout!")
break loop
}
}
Playground: http://play.golang.org/p/PxeEEJo2dz.
See also: Go Concurrency Patterns: Timing out, moving on.
Another way to do it would be to monitor it internally, your question is limited but I'm going to assume you're starting your goroutines through a loop even if you're not you can refactor this to work for you but you could do one of these 2 examples, the first one will timeout each request to timeout individually and the second one will timeout the entire batch of requests and move on if too much time has passed
var wg sync.WaitGroup
wg.Add(1)
go func() {
success := make(chan struct{}, 1)
go func() {
// send your request and wait for a response
// pretend response was received
time.Sleep(5 * time.Second)
success <- struct{}{}
// goroutine will close gracefully after return
fmt.Println("Returned Gracefully")
}()
select {
case <-success:
break
case <-time.After(1 * time.Second):
break
}
wg.Done()
// everything should be garbage collected and no longer take up space
}()
wg.Wait()
// do whatever with what you got
fmt.Println("Done")
time.Sleep(10 * time.Second)
fmt.Println("Checking to make sure nothing throws errors after limbo goroutine is done")
Or if you just want a general easy way to timeout ALL requests you could do something like
var wg sync.WaitGroup
waiter := make(chan int)
wg.Add(1)
go func() {
success := make(chan struct{}, 1)
go func() {
// send your request and wait for a response
// pretend response was received
time.Sleep(5 * time.Second)
success <- struct{}{}
// goroutine will close gracefully after return
fmt.Println("Returned Gracefully")
}()
select {
case <-success:
break
case <-time.After(1 * time.Second):
// control the timeouts for each request individually to make sure that wg.Done gets called and will let the goroutine holding the .Wait close
break
}
wg.Done()
// everything should be garbage collected and no longer take up space
}()
completed := false
go func(completed *bool) {
// Unblock with either wait
wg.Wait()
if !*completed {
waiter <- 1
*completed = true
}
fmt.Println("Returned Two")
}(&completed)
go func(completed *bool) {
// wait however long
time.Sleep(time.Second * 5)
if !*completed {
waiter <- 1
*completed = true
}
fmt.Println("Returned One")
}(&completed)
// block until it either times out or .Wait stops blocking
<-waiter
// do whatever with what you got
fmt.Println("Done")
time.Sleep(10 * time.Second)
fmt.Println("Checking to make sure nothing throws errors after limbo goroutine is done")
This way your WaitGroup will stay in sync and you won't have any goroutines left in limbo
http://play.golang.org/p/g0J_qJ1BUT try it here you can change the variables around to see it work differently
Edit: I'm on mobile If anybody could fix the formatting that would be great thanks.
If you would like to avoid mixing concurrency logic with business logic, I wrote this library https://github.com/shomali11/parallelizer to help you with that. It encapsulates the concurrency logic so you do not have to worry about it.
So in your example:
package main
import (
"github.com/shomali11/parallelizer"
"fmt"
)
func main() {
urls := []string{ ... }
results = make([]*HttpResponse, len(urls)
options := &Options{ Timeout: time.Second }
group := parallelizer.NewGroup(options)
for index, url := range urls {
group.Add(func(index int, url string, results *[]*HttpResponse) {
return func () {
...
results[index] = &HttpResponse{url, response, err}
}
}(index, url, &results))
}
err := group.Run()
fmt.Println("Done")
fmt.Println(fmt.Sprintf("Results: %v", results))
fmt.Printf("Error: %v", err) // nil if it completed, err if timed out
}