func main() {
// Start a long-running process, capture stdout and stderr
findCmd := cmd.NewCmd("find", "/", "--name", "needle")
statusChan := findCmd.Start() // non-blocking
ticker := time.NewTicker(2 * time.Second)
// Print last line of stdout every 2s
go func() { ------------ this keeps running in background, how do I stop this goroutine
for range ticker.C {
status := findCmd.Status()
n := len(status.Stdout)
if len > 10 {
findCmd.Stop() ------- at this point I want to stop this goroutine
}
fmt.Println(status.Stdout[n-1])
}
}()
// Stop command after 1 hour
go func() {
<-time.After(1 * time.Hour)
findCmd.Stop()
}()
// Check if command is done
select {
case finalStatus := <-statusChan:
// done
default:
// no, still running
}
// Block waiting for command to exit, be stopped, or be killed
finalStatus := <-statusChan
}
Goroutine is used to print last line of stdout every 2s, tried few things to stop the goroutine but none worked.
How can I stop that goroutine using channels
This is where select, case is used. You can do something like this:
ticker := time.NewTicker(2 * time.Second)
done := make(chan struct{})
// The go routine that you want to stop when needed
go func() {
select {
case <-ticker.C:
// do stuff
case <-done:
ticker.Stop() // this is optional based on your purpose
}
}()
// do stuff
// stop above go routine
close(done) // when there are many go routine you want to stop as reading from closed channel return the channel zero value
// OR
done <- struct{}{} // when there is only 1 go routine you want to stop
One of the options is to pass the context to such a function/feature. Whenever you pass a context you can catch the cancel signal and use it in the select statements.
func main() {
appCtx := context.Background()
// ...
ticker := time.NewTicker(2 * time.Second)
tickerCtx, tickerCancel := context.WithCancel(appCtx) // create context for ticker, derive from root context
// Print last line of stdout every 2s
go func(ctx context.Context) {
for {
select {
case <-ticker.C: // tick!
// ...
// if you need to perform any operation requiring the context pass `ctx`.
// If case the ctx is canceled all operations should be terminated as well.
case <-ctx.Done() // context has been cancelled
return
}
}
}(tickerCtx)
// Stop command after 1 hour
go func() {
<-time.After(1 * time.Hour)
tickerCancel()
}()
// ...
}
So, this way we are using context, and we have control over all operations where a context is needed (we can cancel them at any time).
In your case there could be another improvement, to terminate the ctx-based operation after some time (without the need to use another goroutine with a timer):
func main() {
appCtx := context.Background()
// ...
ticker := time.NewTicker(2 * time.Second)
tickerCtx, tickerCancel := context.WithTimeout(appCtx, time.Duration(1) * time.Hour) // create a context with a timeout
// Print last line of stdout every 2s
go func(ctx context.Context) {
for {
select {
case <-ticker.C: // tick!
// ...
case <-ctx.Done() // context has been cancelled
return
}
}
}(tickerCtx)
// We don't need to run a new goroutine terminate the ctx. The timeout we set will do the job.
// ...
}
Related
I have a basic question about scheduling "cancellable" goroutines.
I want to schedule a function execution, every 3 seconds.
The function can take up to 5 seconds.
In case it takes more than 2999ms I want to stop/terminate it, to avoid overlapping w/ the next one.
I'm doing it wrong:
func main() {
fmt.Println("startProcessing")
go startProcessing()
time.Sleep(time.Second * 60)
fmt.Println("endProcessing after 60s")
}
func startProcessing() {
ticker := time.NewTicker(3 * time.Second)
for _ = range ticker.C {
ctx, _ := context.WithTimeout(context.Background(), (time.Second*3)-time.Millisecond)
fmt.Println("start doSomething")
doSomething(ctx)
}
}
func doSomething(ctx context.Context) {
executionTime := time.Duration(rand.Intn(5)+1) * time.Second
for {
select {
case <-ctx.Done():
fmt.Printf("timed out after %s\n", executionTime)
return
default:
time.Sleep(executionTime)
fmt.Printf("did something in %s\n", executionTime)
return
}
}
}
This is my output now:
startProcessing
start doSomething
did something in 2s
start doSomething
did something in 3s
start doSomething
did something in 3s
start doSomething
did something in 5s
start doSomething
did something in 2s
...
I want to read timed out after 5s instead of did something in 5s.
You just need to put the time.Sleep(executionTime) outside the select and there is no need for the for loop. I think this is somehow what you want but beware that it's not good practice. So take a look at the warning below.
func doSomething(ctx context.Context) {
executionTime := time.Duration(rand.Intn(5)+1) * time.Second
processed := make(chan int)
go func() {
time.Sleep(executionTime)
processed <- 1
}()
select {
case <-ctx.Done():
fmt.Printf("timed out after %s\n", executionTime)
case <-processed:
fmt.Printf("did something in %s\n", executionTime)
}
}
Obs: I changed the original answer a bit. We can not interrupt a goroutine in the middle of its execution. We could delegate the long-running task to another goroutine and receive the result through a dedicated channel.
Warning: I wouldn't recommend that if you expect the processing time to exceed the deadline because now you will have a leaking goroutine.
In this example, we have a worker. The idea here is simulate clean shutdown of all go routines based on a condition.
In this case, go routines get spun - based on workers count. Each go routine reads the channel, does some work and sends output to the outputChannel.
The main go routine reads this output and prints it. To simulate a stop condition, the doneChannel is closed. Expected outcome is that select inside each go routine will pick this up and execute return which in turn will call the defer println. The actual output is that it never gets called and main exits.
Not sure what's the reason behind this.
package main
import (
"log"
"time"
)
const jobs = 100
const workers = 1
var timeout = time.After(5 * time.Second)
func main() {
doneChannel := make(chan interface{})
outputChannel := make(chan int)
numberStream := generator()
for i := 1; i <= workers; i++ {
go worker(doneChannel, numberStream, outputChannel)
}
// listen for output
loop:
for {
select {
case i := <-outputChannel:
log.Println(i)
case <-timeout:
// before you timeout cleanup go routines
break loop
}
}
close(doneChannel)
time.Sleep(5 * time.Second)
log.Println("main exited")
}
func generator() <-chan int {
defer log.Println("generator completed !")
c := make(chan int)
go func() {
for i := 1; i <= jobs; i++ {
c <- i
}
defer close(c)
}()
return c
}
func worker(done <-chan interface{}, c <-chan int, output chan<- int) {
// this will be a go routine
// Do some work and send results to output Channel.
// Incase if the done channel is called kill the go routine.
defer log.Println("go routines exited")
for {
select {
case <-done:
log.Println("here")
return
case i := <-c:
time.Sleep(1 * time.Second) // worker delay
output <- i * 100
}
}
}
When your main loop finishes during the timeout, you continue your program and
Close done channel
Print message
Exit
There is no reason to wait for any goroutine to process the signal of this channel.
If you add a small sleep you will see some messages
In real scenarios we use a waitgroup to be sure all goroutine finish properly
I have some code, there are 3 timers,
GracefulExecutionTimeout - total running time
WaitTimeout - time to initial wait for first message
IdleTimeout - time to wait for subsequent messages
If any of the timers are reached the app should cleanly exit. I have it working below
msgs := make(chan string)
go func() {
time.Sleep(time.Second)
msgs <- "test"
}()
// graceful max execution time
gracefulMaxTimeout := time.Second * time.Duration(10)
gracefulMaxTimer := time.NewTimer(gracefulMaxTimeout)
// idleTimeout
idleTimeout := time.Second * time.Duration(5)
idleTimer := time.NewTimer(idleTimeout)
// waitTimeout
waitTimeout := time.Second * time.Duration(2)
waitTimer := time.NewTimer(waitTimeout)
for {
select {
case <-gracefulMaxTimer.C:
fmt.Println("GracefulMaxExecutionTimeout Reached")
// graceful exit
os.Exit(0)
case <-idleTimer.C:
fmt.Println("IdleTimeout Reached")
// graceful exit
os.Exit(0)
case <-waitTimer.C:
fmt.Println("WaitTimeout Reached")
// graceful exit
os.Exit(0)
case msg := <-msgs:
// stop wait timer
waitTimer.Stop()
fmt.Println(msg)
// Reset idle timer
if !idleTimer.Stop() {
<-idleTimer.C
}
fmt.Println("IdleIimeout Reset")
idleTimer.Reset(idleTimeout)
}
}
Go Playground
I want to make the WaitTimeout optional but not sure how to approach it. If i surround the construction of the waitTimer with an if statement then it wont work as the waitTimer isnt defined for the select statement ... How can i make the WaitTimeout conditional ?
I can just .Stop() the timer after its created but that seems a little dirty ...
You may declare the wait timer and its channel outside the if statement, and only initialize them if wait timer is needed. If not, the channel may remain its zero value–which is nil–because receiving from a nil channel blocks forever, so this case will never be ready (for details, see How does a non initialized channel behave?).
useWaitTimer := true
var (
waitTimer *time.Timer
waitTimerC <-chan time.Time
)
if useWaitTimer {
waitTimeout := time.Millisecond * time.Duration(500)
waitTimer = time.NewTimer(waitTimeout)
waitTimerC = waitTimer.C
}
// ...
for {
select {
// ...
case <-waitTimerC:
fmt.Println("WaitTimeout Reached")
// graceful exit
os.Exit(0)
// ...
}
}
Then of course you can only reset the wait timer if it exists, this must also be checked (and don't forget to drain the channel if it returns false):
// stop wait timer if exists
if waitTimer != nil && !waitTimer.Stop() {
<-waitTimerC
}
Try it on the Go Playground.
I would like to pool for a token in on a timely base. The Token itself got also information about when it expires.
This should run forever until the user enters ctrl+c.
I tried the same with
span := timeLeft(*expDate)
timer := time.NewTimer(span).C
ticker := time.NewTicker(time.Second * 5).C
which also does not work (the application hangs after count down). So I decided to try it with <- time.After(...)
This is my code that does not work. You will see the count down but it never breaks on expiration.
This is is a small extract with the polling logic for simplicity sake in a main.go:
func refreshToken() (time.Time, error) {
//This should simulate a http request and returns the new target date for the next refresh
time.Sleep(2 * time.Second)
return time.Now().Add(10 * time.Second), nil
}
func timeLeft(d time.Time) time.Duration {
exactLeft := d.Sub(time.Now())
floorSeconds := math.Floor(exactLeft.Seconds())
return time.Duration(floorSeconds) * time.Second
}
func poller(expDate *time.Time) {
exp := timeLeft(*expDate)
done := make(chan bool)
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
for {
select {
// print time left on the screen
case <-time.After(3 * time.Second):
go func() {
fmt.Printf("\rNext Token refresh will be in: %v", timeLeft(*expDate))
}()
// mark as done when date is due
case <-time.After(exp):
fmt.Println("Refresh token now!")
done <- true
// exit app
case <-c:
os.Exit(0)
break
// exit function when done
case <-done:
break
}
}
}
func main() {
var expiration time.Time
expiration = time.Now().Add(10 * time.Second)
// loop and refresh token as long as the app does not exit
for {
poller(&expiration)
ex, err := refreshToken()
expiration = ex
if err != nil {
panic(err)
}
fmt.Println("next round poller")
}
}
I am also not sure if I need the done channel at all?
What is required to listen to two timers and call itself until someone hits ctrl+c?
Found a solution. While #ain was right wit the buffered done channel, it is not really required in the code now. It worked without it.
The trick did have the timeroutside of the for loop and the ticker within it. Reason is the time.Afteris a functhat return a new channel on every iteration. This seams perfectly fine for the ticker, but not for the timer.
With the following changes it worked =) ...
func poller(expDate *time.Time) {
exp := timeLeft(*expDate)
timer := time.After(exp)
fmt.Printf("Next Token refresh will be in: %v\n", exp)
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
for {
select {
// print time left on the screen
case <-time.After(3 * time.Second):
go func() {
fmt.Printf("\r ")
fmt.Printf("\r%v", timeLeft(*expDate))
}()
// mark as done when date is due
case <-timer:
fmt.Println("Refresh token now!")
return
// exit app
case <-c:
os.Exit(0)
break
}
}
}
This question already has answers here:
Timeout for WaitGroup.Wait()
(10 answers)
Closed 7 months ago.
I have come across a situation that i want to trace some goroutine to sync on a specific point, for example when all the urls are fetched. Then, we can put them all and show them in specific order.
I think this is the barrier comes in. It is in go with sync.WaitGroup. However, in real situation that we can not make sure that all the fetch operation will succeed in a short time. So, i want to introduce a timeout when wait for the fetch operations.
I am a newbie to Golang, so can someone give me some advice?
What i am looking for is like this:
wg := &sync.WaigGroup{}
select {
case <-wg.Wait():
// All done!
case <-time.After(500 * time.Millisecond):
// Hit timeout.
}
I know Wait do not support Channel.
If all you want is your neat select, you can easily convert blocking function to a channel by spawning a routine which calls a method and closes/sends on channel once done.
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
// All done!
case <-time.After(500 * time.Millisecond):
// Hit timeout.
}
Send your results to a buffered channel enough to take all results, without blocking, and read them in for-select loop in the main thread:
func work(msg string, d time.Duration, ret chan<- string) {
time.Sleep(d) // Work emulation.
select {
case ret <- msg:
default:
}
}
// ...
const N = 2
ch := make(chan string, N)
go work("printed", 100*time.Millisecond, ch)
go work("not printed", 1000*time.Millisecond, ch)
timeout := time.After(500 * time.Millisecond)
loop:
for received := 0; received < N; received++ {
select {
case msg := <-ch:
fmt.Println(msg)
case <-timeout:
fmt.Println("timeout!")
break loop
}
}
Playground: http://play.golang.org/p/PxeEEJo2dz.
See also: Go Concurrency Patterns: Timing out, moving on.
Another way to do it would be to monitor it internally, your question is limited but I'm going to assume you're starting your goroutines through a loop even if you're not you can refactor this to work for you but you could do one of these 2 examples, the first one will timeout each request to timeout individually and the second one will timeout the entire batch of requests and move on if too much time has passed
var wg sync.WaitGroup
wg.Add(1)
go func() {
success := make(chan struct{}, 1)
go func() {
// send your request and wait for a response
// pretend response was received
time.Sleep(5 * time.Second)
success <- struct{}{}
// goroutine will close gracefully after return
fmt.Println("Returned Gracefully")
}()
select {
case <-success:
break
case <-time.After(1 * time.Second):
break
}
wg.Done()
// everything should be garbage collected and no longer take up space
}()
wg.Wait()
// do whatever with what you got
fmt.Println("Done")
time.Sleep(10 * time.Second)
fmt.Println("Checking to make sure nothing throws errors after limbo goroutine is done")
Or if you just want a general easy way to timeout ALL requests you could do something like
var wg sync.WaitGroup
waiter := make(chan int)
wg.Add(1)
go func() {
success := make(chan struct{}, 1)
go func() {
// send your request and wait for a response
// pretend response was received
time.Sleep(5 * time.Second)
success <- struct{}{}
// goroutine will close gracefully after return
fmt.Println("Returned Gracefully")
}()
select {
case <-success:
break
case <-time.After(1 * time.Second):
// control the timeouts for each request individually to make sure that wg.Done gets called and will let the goroutine holding the .Wait close
break
}
wg.Done()
// everything should be garbage collected and no longer take up space
}()
completed := false
go func(completed *bool) {
// Unblock with either wait
wg.Wait()
if !*completed {
waiter <- 1
*completed = true
}
fmt.Println("Returned Two")
}(&completed)
go func(completed *bool) {
// wait however long
time.Sleep(time.Second * 5)
if !*completed {
waiter <- 1
*completed = true
}
fmt.Println("Returned One")
}(&completed)
// block until it either times out or .Wait stops blocking
<-waiter
// do whatever with what you got
fmt.Println("Done")
time.Sleep(10 * time.Second)
fmt.Println("Checking to make sure nothing throws errors after limbo goroutine is done")
This way your WaitGroup will stay in sync and you won't have any goroutines left in limbo
http://play.golang.org/p/g0J_qJ1BUT try it here you can change the variables around to see it work differently
Edit: I'm on mobile If anybody could fix the formatting that would be great thanks.
If you would like to avoid mixing concurrency logic with business logic, I wrote this library https://github.com/shomali11/parallelizer to help you with that. It encapsulates the concurrency logic so you do not have to worry about it.
So in your example:
package main
import (
"github.com/shomali11/parallelizer"
"fmt"
)
func main() {
urls := []string{ ... }
results = make([]*HttpResponse, len(urls)
options := &Options{ Timeout: time.Second }
group := parallelizer.NewGroup(options)
for index, url := range urls {
group.Add(func(index int, url string, results *[]*HttpResponse) {
return func () {
...
results[index] = &HttpResponse{url, response, err}
}
}(index, url, &results))
}
err := group.Run()
fmt.Println("Done")
fmt.Println(fmt.Sprintf("Results: %v", results))
fmt.Printf("Error: %v", err) // nil if it completed, err if timed out
}