I am trying to set up a service routine that runs every hour on the hour. It seems to me that either of those two are easy. To run my routine on the hour I can use time.AfterFunc(), first calculating the remaining time until the top of the hour. And to run my routine every hour I can use time.NewTicker().
However, I'm struggling to figure out how to start the NewTicker only after the function passed to AfterFunc() has fired.
My main() function looks something like this:
func main() {
fmt.Println("starting up")
// Here I'm setting up all kinds of HTTP listeners and gRPC listeners, none
// of which is important, save to mention that the app has more happening
// than just this service routine.
// Calculate duration until next hour and call time.AfterFunc()
// For the purposes of this exercise I'm just using 5 seconds so as not to
// have to wait increments of hours to see the results
time.AfterFunc(time.Second * 5, func() {
fmt.Println("AfterFunc")
})
// Set up a ticker to run every hour. Again, for the purposes of this
// exercise I'm ticking every 2 seconds just to see some results
t := time.NewTicker(time.Second * 2)
defer t.Stop()
go func() {
for now := range t.C {
fmt.Println("Ticker")
}
}()
// Block until termination signal is received
osSignals := make(chan os.Signal, 1)
signal.Notify(osSignals, syscall.SIGINT, syscall.SIGTERM, os.Interrupt, os.Kill)
<-osSignals
fmt.Println("exiting gracefully")
}
Of course time.AfterFunc() is blocking and the payload of my Ticker is deliberately put in a go routine so it also won't be blocking. This is so that my HTTP and gRPC listeners can continue to listen but also to allow the block of code at the end of main() to exit gracefully upon a termination signal from the OS. But the obvious downside now is that the Ticker kicks off pretty much immediately and fires twice (2 second intervals) before the function passed to AfterFunc() fires. The output looks like this:
Ticker
Ticker
AfterFunc
Ticker
Ticker
Ticker
etc.
What I wanted of course is:
AfterFunc
Ticker
Ticker
Ticker
Ticker
Ticker
etc.
The following also doesn't work and I'm not exactly sure why. It prints AfterFunc but the Ticker never fires.
time.AfterFunc(time.Second * 5, func() {
fmt.Println("AfterFunc")
t := time.NewTicker(time.Second * 2)
defer t.Stop()
go func() {
for now := range t.C {
fmt.Println("Ticker")
}
}()
})
The Go Programming Language Specification
Program execution
Program execution begins by initializing the main package and then
invoking the function main. When that function invocation returns, the
program exits. It does not wait for other (non-main) goroutines to
complete.
time.AfterFunc(time.Second * 5, func() {
fmt.Println("AfterFunc")
t := time.NewTicker(time.Second * 2)
defer t.Stop()
go func() {
for now := range t.C {
fmt.Println("Ticker")
}
}()
})
defer t.Stop() stops the ticker.
You are not waiting for the goroutine to run.
Gosh, I figured it out not long after posting, though I still think my solution lacks some elegance.
As #peterSO pointed out, the function passed to AfterFunc() executes and stops the Ticker moments after creating it with defer t.Stop()
The solution for me was to define the t variable before the call to AfterFunc() so that it has scope outside of the AfterFunc() payload function, and then stop it at the end of my main() func. Here is the new main() func:
func main() {
fmt.Println("starting up")
var t *time.Ticker
time.AfterFunc(time.Second * 5, func() {
fmt.Println("AfterFunc")
t = time.NewTicker(time.Second * 2)
go func() {
for now := range t.C {
fmt.Println("Ticker")
}
}()
})
//defer t.Stop()
// Block until termination signal is received
osSignals := make(chan os.Signal, 1)
signal.Notify(osSignals, syscall.SIGINT, syscall.SIGTERM, os.Interrupt, os.Kill)
<-osSignals
t.Stop()
logger.Info("exiting gracefully")
}
Strangely though that commented out defer t.Stop() causes a panic (invalid memory address or nil pointer dereference) when the application closes upon a termination signal. If I stop it as in the uncommented t.Stop() at the very end of the code, it works as expected. Not sure why that is.
time.AfterFunc(time.Second*5, func() {
fmt.Println("AfterFunc")
t := time.NewTicker(time.Second * 2)
defer t.Stop()
for range t.C {
fmt.Println("Ticker")
}
})
It produces the output you need:
starting up
AfterFunc
Ticker
Ticker
Ticker
Ticker
Ticker
Related
Since I am not experinced Go developer, I didn't understand way of working with Ticker. I have following scenario:
A go web service running on specific port 8080, it is getting data from another applications and processing the data. So far so good, but I have a another sendData function in this web service which loop through the some files and send them to another extern service. I am trying to call the sendData() function every 1 minute. Here is how main function looks without Tickers:
func main() {
http.HandleFunc("/data", headers) //line 1
log.Printf("Ready for data ...%s\n", 8080) //line 2
http.ListenAndServe(":8080", nil) //line 3
}
If I add the Ticker after line 2 it's keeping loop infinitively.
If I add after line 3, the programm is not invoking the Ticker.
Any idea how to handle this?
The Ticker part
ticker := schedule(sendData, time.Second, done)
time.Sleep(60 * time.Second)
close(done)
ticker.Stop()
and the schedule from
func schedule(f func(), interval time.Duration, done <-chan bool) *time.Ticker {
ticker := time.NewTicker(interval)
go func() {
for {
select {
case <-ticker.C:
f()
case <-done:
return
}
}
}()
return ticker
So basically I want to sendData evert minute or hour etc. Could someone explain how internally Ticker works?
http.ListenAndServe(":8080", nil) runs an infinite for loop listening for inbound connections, that's why the ticker is not invoked if you call it afterwards.
And then here
ticker := schedule(sendData, time.Second, done)
time.Sleep(60 * time.Second)
close(done)
ticker.Stop()
you're exiting the loop inside schedule() after 60 seconds, so your ticker will run only once or won't run at all (depending on whether the done channel receives the value before or later that the ticker ticks, as they are concurrent we cannot determine their order)
So what you want is the following
func main() {
http.HandleFunc("/data", headers)
ticker := time.NewTicker(time.Minute)
go schedule(ticker)
log.Printf("Ready for data ...%s\n", 8080)
http.ListenAndServe(":8080", nil)
}
func schedule(ticker *time.Ticker) {
for {
// This blocks until a value is received, the ticker
// sends a value to it every one minute (or the interval specified)
<-ticker.C
fmt.Println("Tick")
}
}
As you may have noticed, once the server connection is interrupted the program will terminate so there's no point on having a done channel to exit the loop.
Try it here
You are on the right track - you just need to wrap the ticker declaration in a self executing function and then run it as a goroutine. ListenAndServe and Schedule are both blocking tasks, so they need to run on separate go routines. Luckily go makes this really simple to achieve.
Note - this sample code is meant to stay as close to your example as possible. I would recommend separating the declaration of the ticker from the schedule func.
func main() {
http.HandleFunc("/data", func(w http.ResponseWriter, req *http.Request) {}) //line 1
var done chan bool
go func() {
ticker := schedule(func() { fmt.Println("Tick") }, time.Second, done)
time.Sleep(60 * time.Second)
close(done)
ticker.Stop()
}()
fmt.Printf("Ready for data ...%v\n", 8080) //line 2
http.ListenAndServe(":8080", nil) //line 3
}
I'm trying to build some short of semaphore in Go. Although when when the channel receives the signal it just sleeps forever.
I've tried changing the way to sleep and the duration to sleep, but it stills just stopping forever.
Here a representation of what I've tried:
func main() {
backOffChan := make(chan struct{})
go func() {
time.Sleep(2)
backOffChan <- struct{}{}
}()
for {
select {
case <-backOffChan:
d := time.Duration(5 * time.Second)
log.Println("reconnecting in %s", d)
select {
case <-time.After(d):
log.Println("reconnected after %s", d)
return
}
default:
}
}
}
I expect that it just returns after printing the log message and returning.
Thanks!
This code has a number of problems, mainly a tight loop using for/select that may not allow the other goroutine to ever get to send on the channel. Since the default case is empty and the select has only one case, the whole select is unnecessary. The following code works correctly:
backOffChan := make(chan struct{})
go func() {
time.Sleep(1 * time.Millisecond)
backOffChan <- struct{}{}
}()
for range backOffChan {
d := time.Duration(10 * time.Millisecond)
log.Printf("reconnecting in %s", d)
select {
case <-time.After(d):
log.Printf("reconnected after %s", d)
return
}
}
This will wait until the backOffChan gets a message without burning a tight loop.
(Note that this code also addresses issues using log.Println with formatting directives - these were corrected to log.Printf).
See it in action here: https://play.golang.org/p/ksAzOq5ekrm
I am trying to understand context in golang. I copied an example from https://golang.org/pkg/context/#example_WithCancel and changed it a bit:
Playgroud: https://play.golang.org/p/Aczc2CqcVZR
package main
import (
"context"
"fmt"
"time"
)
func main() {
// gen generates integers in a separate goroutine and
// sends them to the returned channel.
// The callers of gen need to cancel the context once
// they are done consuming generated integers not to leak
// the internal goroutine started by gen.
gen := func(ctx context.Context) <-chan int {
dst := make(chan int)
n := 1
go func() {
for {
select {
case <-ctx.Done():
fmt.Println("DONE")
return // returning not to leak the goroutine
case dst <- n:
n++
}
}
fmt.Println("END")
}()
return dst
}
ctx, cancel := context.WithCancel(context.Background())
defer time.Sleep(1 * time.Second)
defer fmt.Println("Before cancel")
defer cancel() // cancel when we are finished consuming integers
defer fmt.Println("After cancel")
channel := gen(ctx)
for n := range channel {
fmt.Println(n)
if n == 5 {
break
}
}
fmt.Println( <-channel)
}
When commenting out the
defer time.Sleep(1 * time.Second)
the "DONE" never gets printed. Playgroud: (https://play.golang.org/p/K0OcyZaj_xK)
I would expect the go routine which was started in the anonymous function still to be active. Once cancel() is called due to being deferred, the select should no longer block as
case <-ctx.Done():
should be available. However it seems to just end, unless I wait for 1 second and give it time. This behavior seems very wrong.
This behavior seems very wrong.
It's not. That's how program execution is specified. After main and its deferred functions return, the program exits.
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
https://golang.org/ref/spec#Program_execution
func main() {
c := make(chan os.Signal, 1)
signal.Notify(c)
ticker := time.NewTicker(time.Second)
stop := make(chan bool)
go func() {
defer func() { stop <- true }()
for {
select {
case <-ticker.C:
fmt.Println("Tick")
case <-stop:
fmt.Println("Goroutine closing")
return
}
}
}()
<-c
ticker.Stop()
stop <- true
<-stop
fmt.Println("Application stopped")
}
No matter how many times I run the code above, I got the same result. That is, "Goroutine closing" is always printed before "Application stopped" after I press Ctrl+C.
I think, theoretically, there is a chance that "Goroutine closing" won't be printed at all. Am I right? Unfortunately, I never get this theoretical result.
BTW: I know reading and writing a channel in one routine should be avoided. Ignore that temporarily.
In your case, Goroutine closing will always be executed and it will always be printed before Application stopped, because your stop channel is not buffered. This means that the sending will block until the result is received.
In your code, the stop <- true in your main will block until the goroutine has received the value, causing the channel to be empty again. Then the <-stop in your main will block until another value is sent to the channel, which happens when your goroutine returns after printing Goroutine closing.
If you would initialize your channel in a buffered fashion
stop := make(chan bool, 1)
then Goroutine closing might not be executed. To see this, you can add a time.Sleep right after printing Tick, as this makes this case more likely (it will occur everytime you press Ctrl+C during the sleep).
Using a sync.WaitGroup to wait for goroutines to finish is a good alternative, especially if you have to wait for more than one goroutine. You can also use a context.Context to stop goroutines. Reworking your code to use these two methods could look something like this:
func main() {
c := make(chan os.Signal, 1)
signal.Notify(c)
ticker := time.NewTicker(time.Second)
ctx, cancel := context.WithCancel(context.Background())
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer func() { wg.Done() }()
for {
select {
case <-ctx.Done():
fmt.Println("Goroutine closing")
return
case <-ticker.C:
fmt.Println("Tick")
time.Sleep(time.Second)
}
}
}()
<-c
ticker.Stop()
cancel()
wg.Wait()
fmt.Println("Application stopped")
}
This question already has answers here:
Timeout for WaitGroup.Wait()
(10 answers)
Closed 7 months ago.
I have come across a situation that i want to trace some goroutine to sync on a specific point, for example when all the urls are fetched. Then, we can put them all and show them in specific order.
I think this is the barrier comes in. It is in go with sync.WaitGroup. However, in real situation that we can not make sure that all the fetch operation will succeed in a short time. So, i want to introduce a timeout when wait for the fetch operations.
I am a newbie to Golang, so can someone give me some advice?
What i am looking for is like this:
wg := &sync.WaigGroup{}
select {
case <-wg.Wait():
// All done!
case <-time.After(500 * time.Millisecond):
// Hit timeout.
}
I know Wait do not support Channel.
If all you want is your neat select, you can easily convert blocking function to a channel by spawning a routine which calls a method and closes/sends on channel once done.
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
// All done!
case <-time.After(500 * time.Millisecond):
// Hit timeout.
}
Send your results to a buffered channel enough to take all results, without blocking, and read them in for-select loop in the main thread:
func work(msg string, d time.Duration, ret chan<- string) {
time.Sleep(d) // Work emulation.
select {
case ret <- msg:
default:
}
}
// ...
const N = 2
ch := make(chan string, N)
go work("printed", 100*time.Millisecond, ch)
go work("not printed", 1000*time.Millisecond, ch)
timeout := time.After(500 * time.Millisecond)
loop:
for received := 0; received < N; received++ {
select {
case msg := <-ch:
fmt.Println(msg)
case <-timeout:
fmt.Println("timeout!")
break loop
}
}
Playground: http://play.golang.org/p/PxeEEJo2dz.
See also: Go Concurrency Patterns: Timing out, moving on.
Another way to do it would be to monitor it internally, your question is limited but I'm going to assume you're starting your goroutines through a loop even if you're not you can refactor this to work for you but you could do one of these 2 examples, the first one will timeout each request to timeout individually and the second one will timeout the entire batch of requests and move on if too much time has passed
var wg sync.WaitGroup
wg.Add(1)
go func() {
success := make(chan struct{}, 1)
go func() {
// send your request and wait for a response
// pretend response was received
time.Sleep(5 * time.Second)
success <- struct{}{}
// goroutine will close gracefully after return
fmt.Println("Returned Gracefully")
}()
select {
case <-success:
break
case <-time.After(1 * time.Second):
break
}
wg.Done()
// everything should be garbage collected and no longer take up space
}()
wg.Wait()
// do whatever with what you got
fmt.Println("Done")
time.Sleep(10 * time.Second)
fmt.Println("Checking to make sure nothing throws errors after limbo goroutine is done")
Or if you just want a general easy way to timeout ALL requests you could do something like
var wg sync.WaitGroup
waiter := make(chan int)
wg.Add(1)
go func() {
success := make(chan struct{}, 1)
go func() {
// send your request and wait for a response
// pretend response was received
time.Sleep(5 * time.Second)
success <- struct{}{}
// goroutine will close gracefully after return
fmt.Println("Returned Gracefully")
}()
select {
case <-success:
break
case <-time.After(1 * time.Second):
// control the timeouts for each request individually to make sure that wg.Done gets called and will let the goroutine holding the .Wait close
break
}
wg.Done()
// everything should be garbage collected and no longer take up space
}()
completed := false
go func(completed *bool) {
// Unblock with either wait
wg.Wait()
if !*completed {
waiter <- 1
*completed = true
}
fmt.Println("Returned Two")
}(&completed)
go func(completed *bool) {
// wait however long
time.Sleep(time.Second * 5)
if !*completed {
waiter <- 1
*completed = true
}
fmt.Println("Returned One")
}(&completed)
// block until it either times out or .Wait stops blocking
<-waiter
// do whatever with what you got
fmt.Println("Done")
time.Sleep(10 * time.Second)
fmt.Println("Checking to make sure nothing throws errors after limbo goroutine is done")
This way your WaitGroup will stay in sync and you won't have any goroutines left in limbo
http://play.golang.org/p/g0J_qJ1BUT try it here you can change the variables around to see it work differently
Edit: I'm on mobile If anybody could fix the formatting that would be great thanks.
If you would like to avoid mixing concurrency logic with business logic, I wrote this library https://github.com/shomali11/parallelizer to help you with that. It encapsulates the concurrency logic so you do not have to worry about it.
So in your example:
package main
import (
"github.com/shomali11/parallelizer"
"fmt"
)
func main() {
urls := []string{ ... }
results = make([]*HttpResponse, len(urls)
options := &Options{ Timeout: time.Second }
group := parallelizer.NewGroup(options)
for index, url := range urls {
group.Add(func(index int, url string, results *[]*HttpResponse) {
return func () {
...
results[index] = &HttpResponse{url, response, err}
}
}(index, url, &results))
}
err := group.Run()
fmt.Println("Done")
fmt.Println(fmt.Sprintf("Results: %v", results))
fmt.Printf("Error: %v", err) // nil if it completed, err if timed out
}