I've written a code snipped that creates a timer with a 0 length time, and it does not immediately expire (which is what I expected). A very short sleep call does make it expire, but I'm confused as to why.
The reason I care is that the code using this idea has a snippet that returns 0 on a low probability error, with the idea that the timer should be set to immediately expire, and retry a function. I do not believe that the nanosecond sleep needed here will affect my implementation, but it bothers me.
Did I make a mistake, is this expected behaviour?
Thanks!
package main
import (
"fmt"
"time"
)
func main() {
testTimer := time.NewTimer(time.Duration(0) * time.Millisecond)
fmt.Println(Expired(testTimer))
time.Sleep(time.Nanosecond)
fmt.Println(Expired(testTimer))
}
func Expired(T *time.Timer) bool {
select {
case <-T.C:
return true
default:
return false
}
}
Playground link: https://play.golang.org/p/xLLHoR8aKq
Prints
false
true
time.NewTimer() does not guarantee maximum wait time. It only guarantees a minimum wait time. Quoting from its doc:
NewTimer creates a new Timer that will send the current time on its channel after at least duration d.
So passing a zero duration to time.NewTimer(), it's not a surprise the returned time.Timer is not "expired" immediately.
The returned timer could be "expired" immediately if the implementation would check if the passed duration is zero, and would send a value on the timer's channel before returning it, but it does not. Instead it starts an internal timer normally as it does for any given duration, which will take care of sending a value on its channel, but only some time in the future.
Note that with multiple CPU cores and with runtime.GOMAXPROCS() being greater than 1 there is a slight chance that another goroutine (internal to the time package) sends a value on the timer's channel before NewTimer() returns, but this is a very small chance... Also since this is implementation detail, a future version might add this "optimization" to check for 0 passed duration, and act as you expected it, but as with all implementation details, don't count on it. Count on what's documented, and expect no more.
Go's timer functions guarantee to sleep at least the specified time. See the docs for Sleep and NewTimer respectively:
Sleep pauses the current goroutine for at least the duration d. A negative or zero duration causes Sleep to return immediately.
NewTimer creates a new Timer that will send the current time on its channel after at least duration d.
(emphasis added)
In your situation, you should probably just not use a timer in the situation that you don't want to sleep at all.
This is due to the internal time it takes to set up the timer object. If you'll note in the playground link below the timer does expire at the proper time, but the internal go routine that sets it up and starts it takes longer than your Expire function does to check it.
When the Timer expires, the current time will be sent on C (the channel)
So you'll notice that after it expires, it still sends the original time, because it has expired even before the nanosecond Sleep finished.
https://play.golang.org/p/Ghwq9kJq3J
package main
import (
"fmt"
"time"
)
func main() {
testTimer := time.NewTimer(0 * time.Millisecond)
Expired(testTimer)
time.Sleep(time.Nanosecond)
Expired(testTimer)
n := time.Now()
fmt.Printf("after waiting: %d\n", n.UnixNano())
}
func Expired(T *time.Timer) bool {
select {
case t:= <-T.C:
fmt.Printf("expired %d\n", t.UnixNano())
return true
default:
n := time.Now()
fmt.Printf("not expired: %d\n", n.UnixNano())
return false
}
}
Related
Here is the code I ran:
package main
import (
"fmt"
"time"
)
const delay = 9 * time.Millisecond
func main() {
n := 0
go func() {
time.Sleep(delay)
n++
}()
fmt.Println(n)
}
Here is the command I used:
go run -race data_race_demo.go
Here is the behavior I noticed:
With delay set to 9ms or lower, data race is always detected (program throws Found 1 data race(s))
With delay set to 12ms or higher, data race is never detected (program simply prints 0)
With delay set to 10 to 11ms, data race occurs intermittently (that is, sometimes prints 0 and sometimes throws Found 1 data race(s))
Why does this happen at around 10-11ms?
I'm using Go 1.16.3 on darwin/amd64, if that matters.
You have 2 goroutines: the main and the one you launch. They access the n variable (and one is a write) without synchronization: that's a data race.
Whether this race is detected depends on whether this racy access occurs. When the main() function ends, your app ends as well, it does not wait for other non-main goroutines to finish.
If you increase the sleep delay, main() will end sooner than the end of sleep and will not wait for the n++ racy write to occur, so nothing is detected. If sleep is short, shorter than the fmt.Prinln() execution time, the racy write occurs and is detected.
There's nothing special about the 10ms. That's just the approximated time it takes to execute fmt.Println() and terminate your app in your environment. If you do other "lengthy" task before the Println() statement, such as this:
for i := 0; i < 5_000_000_000; i++ {
}
fmt.Println(n)
The race will be detected even with 50ms sleep too (because that loop will take some time to execute, allowing the racy write to occur before n is read for the fmt.Println() call and the app terminated). (A simple time.Sleep() would also do, I just didn't want anyone to draw the wrong conclusion that they "interact" with each other somehow.)
I have a scenario in which I'm processing events on a channel, and one of those events is a heartbeat which needs to occur within a certain timeframe. Events which are not heartbeats will continue consuming the timer, however whenever the heartbeat is received I want to reset the timer. The obvious way to do this would be by using a time.NewTimer.
For example:
func main() {
to := time.NewTimer(3200 * time.Millisecond)
for {
select {
case event, ok := <-c:
if !ok {
return
} else if event.Msg == "heartbeat" {
to.Reset(3200 * time.Millisecond)
}
case remediate := <-to.C:
fmt.Println("do some stuff ...")
return
}
}
}
Note that a time.Ticker won't work here as the remediation should only be triggered if the heartbeat hasn't been received, not every time.
The above solution works in the handful of low volume tests I've tried it on, however I came across a Github issue indicating that resetting a Timer which has not fired is a no-no. Additionally the documentation states:
Reset should be invoked only on stopped or expired timers with drained channels. If a program has already received a value from t.C, the timer is known to have expired and the channel drained, so t.Reset can be used directly. If a program has not yet received a value from t.C, however, the timer must be stopped and—if Stop reports that the timer expired before being stopped—the channel explicitly drained:
if !t.Stop() {
<-t.C
}
t.Reset(d)
This gives me pause, as it seems to describe exactly what I'm attempting to do. I'm resetting the Timer whenever the heartbeat is received, prior to it having fired. I'm not experienced enough with Go yet to digest the whole post, but it certainly seems like I may be headed down a dangerous path.
One other solution I thought of is to simply replace the Timer with a new one whenever the heartbeat occurs, e.g:
else if event.Msg == "heartbeat" {
to = time.NewTimer(3200 * time.Millisecond)
}
At first I was worried that the rebinding to = time.NewTimer(3200 * time.Millisecond) wouldn't be visible within the select:
For all the cases in the statement, the channel operands of receive operations and the channel and right-hand-side expressions of send statements are evaluated exactly once, in source order, upon entering the "select" statement. The result is a set of channels to receive from or send to, and the corresponding values to send.
But in this particular case since we are inside a loop, I would expect that upon each iteration we re-enter select and therefore the new binding should be visible. Is that a fair assumption?
I realize there are similar questions out there, and I've tried to read the relevant posts/documentation, but I am new to Go just want to be sure I'm understanding things correctly here.
So my questions are:
Is my use of timer.Reset() unsafe, or are the cases mentioned in the Github issue highlighting other problems which are not applicable here? Is the explanation in the docs cryptic or do I just need more experience with Go?
If it is unsafe, is my second proposed solution acceptable (rebinding the timer on each iteration).
ADDENDUM
Upon further reading, most of the pitfalls outlined in the issues are describing scenarios in which the timer has already fired (placing a result on the channel), and subsequent to that firing some other process attempts to Reset it. For this narrow case, I understand the need to test with !t.Stop() since a false return of Stop would indicate the timer has already fired, and as such must be drained prior to calling Reset.
What I still do not understand, is why it is necessary to call t.Stop() prior to t.Reset(), when the Timer has yet to fire. None of the examples go into that as far as I can tell.
What I still do not understand, is why it is necessary to call t.Stop() prior to t.Reset(), when the Timer has yet to fire.
The "when the Timer has yet to fire" bit is critical here. The timer fires within a separate go routine (part of the runtime) and this can happen at any time. You have no way of knowing whether the timer has fired at the time you call to.Reset(3200 * time.Millisecond) (it may even fire while that function is running!).
Here is an example that demonstrates this and is somewhat similar to what you are attempting (based on this):
func main() {
eventC := make(chan struct{}, 1)
go keepaliveLoop(eventC )
// Reset the timer 1000 (approx) times; once every millisecond (approx)
// This should prevent the timer from firing (because that only happens after 2 ms)
for i := 0; i < 1000; i++ {
time.Sleep(time.Millisecond)
// Don't block if there is already a reset request
select {
case eventC <- struct{}{}:
default:
}
}
}
func keepaliveLoop(eventC chan struct{}) {
to := time.NewTimer(2 * time.Millisecond)
for {
select {
case <-eventC:
//if event.Msg == "heartbeat"...
time.Sleep(3 * time.Millisecond) // Simulate reset work (delay could be partly dur to whatever is triggering the
to.Reset(2 * time.Millisecond)
case <-to.C:
panic("this should never happen")
}
}
}
Try it in the playground.
This may appear contrived due to the time.Sleep(3 * time.Millisecond) but that is just included to consistently demonstrate the issue. Your code may work 99.9% of the time but there is always the possibility that both the event and timer channels will fire before the select is run (in which a random case will run) or while the code in the case event, ok := <-c: block is running (including while Reset() is in progress). The result of this happening would be unexpected calls of the remediate code (which may not be a big issue).
Fortunately solving the issue is relatively easy (following the advice in the documentation):
time.Sleep(3 * time.Millisecond) // Simulate reset work (delay could be partly dur to whatever is triggering the
if !to.Stop() {
<-to.C
}
to.Reset(2 * time.Millisecond)
Try this in the playground.
This works because to.Stop returns "true if the call stops the timer, false if the timer has already expired or been stopped". Note that things get a more complicated if the timer is used in multiple go-routines "This cannot be done concurrent to other receives from the Timer's channel or other calls to the Timer's Stop method" but this is not the case in your use-case.
Is my use of timer.Reset() unsafe, or are the cases mentioned in the Github issue highlighting other problems which are not applicable here?
Yes - it is unsafe. However the impact is fairly low. The event arriving and timer triggering would need to happen almost concurrently and, in that case, running the remediate code might not be a big issue. Note that the fix is fairly simple (as per the docs)
If it is unsafe, is my second proposed solution acceptable (rebinding the timer on each iteration).
Your second proposed solution also works (but note that the garbage collector cannot free the timer until after it has fired, or been stopped, which may cause issues if you are creating timers rapidly).
Note: Re the suggestion from #JotaSantos
Another thing that could be done is to add a select when draining <-to.C (on the Stop "if") with a default clause. That would prevent the pause.
See this comment for details of why this may not be a good approach (it's also unnecessary in your situation).
I've faced a similar issue. After reading a lot of information, I came up with a solution that goes along these lines:
package main
import (
"fmt"
"time"
)
func main() {
const timeout = 2 * time.Second
// Prepare a timer that is stopped and ready to be reset.
// Stop will never return false, because an hour is too long
// for timer to fire. Thus there's no need to drain timer.C.
timer := time.NewTimer(timeout)
timer.Stop()
// Make sure to stop the timer when we return.
defer timer.Stop()
// This variable is needed because we need to track if we can safely reset the timer
// in a loop. Calling timer.Stop() will return false on every iteration, but we can only
// drain the timer.C once, otherwise it will deadlock.
var timerSet bool
c := make(chan time.Time)
// Simulate events that come in every second
// and every 5th event delays so that timer can fire.
go func() {
var i int
ticker := time.NewTicker(1 * time.Second)
defer ticker.Stop()
for t := range ticker.C {
i++
if i%5 == 0 {
fmt.Println("Sleeping")
time.Sleep(3 * time.Second)
}
c <- t
if i == 20 {
break
}
}
close(c)
}()
for {
select {
case t, ok := <-c:
if !ok {
fmt.Println("Closed channel")
return
}
fmt.Println("Got event", t, timerSet)
// We got an event, and timer was already set.
// We need to stop the timer and drain the channel if needed,
// so that we can safely reset it later.
if timerSet {
if !timer.Stop() {
<-timer.C
}
timerSet = false
}
// If timer was not set, or it was stopped before, it's safe to reset it.
if !timerSet {
timerSet = true
timer.Reset(timeout)
}
case remediate := <-timer.C:
fmt.Println("Timeout", remediate)
// It's important to store that timer is not set anymore.
timerSet = false
}
}
}
Link to playground: https://play.golang.org/p/0QlujZngEGg
I am trying to reuse timers by stopping and resetting them. I am following the pattern provided by the documentation. Here is a simple example which can be run in go playground that demonstrates the issue I am experiencing.
Is there a correct way to stop and reset a timer that doesn't involve deadlock or race conditions? I am aware that using a select with default involves a race condition on channel message delivery timing and cannot be depended on.
package main
import (
"fmt"
"time"
"sync"
)
func main() {
fmt.Println("Hello, playground")
timer := time.NewTimer(1 * time.Second)
wg := &sync.WaitGroup{}
wg.Add(1)
go func(_wg *sync.WaitGroup) {
<- timer.C
fmt.Println("Timer done")
_wg.Done()
}(wg)
wg.Wait()
fmt.Println("Checking timer")
if !timer.Stop() {
<- timer.C
}
fmt.Println("Done")
}
According to the timer.Stop docs, there is a caveat for draining the channel:
assuming the program has not received from t.C already ...
This cannot be done concurrent to other receives from the Timer's
channel.
Since the channel has already been drained - and will never fire again, the second <-timer.C will block forever.
The question asks, in the first place, why the timer hangs. That's a good question, because even in the absence of bugs in the user program, there is... at least some weird ambiguity in how this thing, called time.Timer, works in Go. The spec says, specifically this:
Stop prevents the Timer from firing. It returns true if the call stops
the timer, false if the timer has already expired or been stopped.
Stop does not close the channel, to prevent a read from the channel
succeeding incorrectly.
To ensure the channel is empty after a call to Stop, check the return
value and drain the channel. For example, assuming the program has not
received from t.C already:
if !t.Stop() {
<-t.C
}
This cannot be done concurrent to other receives from the Timer's
channel or other calls to the Timer's Stop method.
There are very short and precise words, but it may be not that easy to understand them (at least for me). I tried to use the Timer repeatedly in a piece of code, and reset it each time before the next use. Each time I do so, I may want to Stop() it - just for sure. The spec above implies how you should do that, and provides an example - and it may not work! It depends, it depends where you try to apply the Stop idiom. In case you do it after you already in a select-case on this very timer, then it will hang the program.
Specifically, I do not have any concurrent receivers, only a single goroutine. So let's make a simple test program, and try to experiment with it (https://play.golang.org/p/d7BlNReE9Jz):
package main
import (
"fmt"
"time"
)
func main() {
i := 0
d2s := time.Second * 1
i++; fmt.Println(i)
t := time.NewTimer(d2s)
<-t.C
i++; fmt.Println(i)
t.Reset(d2s)
<-t.C
i++; fmt.Println(i)
// if !t.Stop() { <-t.C }
// if !t.Stop() { select { case <-t.C: default: } }
t.Reset(d2s)
<-t.C
i++; fmt.Println(i)
}
This code WORKS. It prints 1,2,3,4, delayed by 1 sec, and that's what it is expected to print. So far so good.
Now, try to un-comment the first commented line. Now the thing: according to spec, it is 100% right (is it?), and must work, but it does not, and hangs. Why? Because, according to spec, it must hang! I already read the channel, and the timer is stopped, so the if fires, and the channel drain op hangs.
Is this a bug? No. Is the spec wrong? No, it's correct. But, it's contrary to what a typical timer user would want. (Maybe a subject for proposal to Go?). All we need, is something like:
t.SafeStopDrain()
Which would do this right, and never hang. But, sadly, it is non-existent.
Here's the life-hack, the workaround, to make this work, is the second commented line. Un-comment it, and that will both work, and do what you wanted - make sure the timer is stopped, channel drained, and the whole thing is fresh anew for re-use.
EDIT: My question is different from How to write my own Sleep function using just time.After? It has a different variant of the code that's not working for a separate reason and I needed explanation as to why.
I'm trying to solve the homework problem here: https://www.golang-book.com/books/intro/10 (Write your own Sleep function using time.After).
Here's my attempt so far based on the examples discussed in that chapter:
package main
import (
"fmt"
"time"
)
func myOwnSleep(duration int) {
for {
select {
case <-time.After(time.Second * time.Duration(duration)):
fmt.Println("slept!")
default:
fmt.Println("Waiting")
}
}
}
func main() {
go myOwnSleep(3)
var input string
fmt.Scanln(&input)
}
http://play.golang.org/p/fb3i9KY3DD
My thought process is that the infinite for will keep executing the select statement's default until the time.After function's returned channel talks. Problem with the current code being, the latter does not happen, while the default statement is called infinitely.
What am I doing wrong?
In each iteration of your for loop the select statement is executed which involves evaluating the channel operands.
In each iteration time.After() will be called and a new channel will be created!
And if duration is more than 0, this channel is not ready to receive from, so the default case will be executed. This channel will not be tested/checked again, the next iteration creates a new channel which will again not be ready to receive from, so the default case is chosen again - as always.
The solution is really simple though as can be seen in this answer:
func Sleep(sec int) {
<-time.After(time.Second* time.Duration(sec))
}
Fixing your variant:
If you want to make your variant work, you have to create one channel only (using time.After()), store the returned channel value, and always check this channel. And if the channel "kicks in" (a value is received from it), you must return from your function because more values will not be received from it and so your loop will remain endless!
func myOwnSleep(duration int) {
ch := time.After(time.Second * time.Duration(duration))
for {
select {
case <-ch:
fmt.Println("slept!")
return // MUST RETURN, else endless loop!
default:
fmt.Println("Waiting")
}
}
}
Note that though until a value is received from the channel, this function will not "rest" and just execute code relentlessly - loading one CPU core. This might even give you trouble if only 1 CPU core is available (runtime.GOMAXPROCS()), other goroutines (including the one that will (or would) send the value on the channel) might get blocked and never executed. A sleep (e.g. time.Sleep(time.Millisecond)) could release the CPU core from doing endless work (and allow other goroutines to run).
I'm trying to understand a bit more about what happens under the surface during various blocking/waiting types of operations in Go. Take the following example:
otherChan = make(chan int)
t = time.NewTicker(time.Second)
for {
doThings()
// OPTION A: Sleep
time.Sleep(time.Second)
// OPTION B: Blocking ticker
<- t.C
// OPTION C: Select multiple
select {
case <- otherChan:
case <- t.C:
}
}
From a low level view (system calls, cpu scheduling) what is the difference between these while waiting?
My understanding is that time.Sleep leaves the CPU free to perform other tasks until the specified time has elapsed. Does the blocking ticker <- t.C do the same? Is the processor polling the channel or is there an interrupt involved? Does having multiple channels in a select change anything?
In other words, assuming that otherChan never had anything put into it, would these three options execute in an identical way, or would one be less resource intensive than the others?
That's a very interesting question, so I did cd into my Go source to start looking.
time.Sleep
time.Sleep is defined like this:
// src/time/sleep.go
// Sleep pauses the current goroutine for at least the duration d.
// A negative or zero duration causes Sleep to return immediately.
func Sleep(d Duration)
No body, no definition in an OS-specific time_unix.go!?! A little search and the answer is because time.Sleep is actually defined in the runtime:
// src/runtime/time.go
// timeSleep puts the current goroutine to sleep for at least ns nanoseconds.
//go:linkname timeSleep time.Sleep
func timeSleep(ns int64) {
// ...
}
Which in retrospect makes a lot of sense, as it has to interact with the goroutine scheduler. It ends up calling goparkunlock, which "puts the goroutine into a waiting state". time.Sleep creates a runtime.timer with a callback function that is called when the timer expires - that callback function wakes up the goroutine by calling goready. See next section for more details on the runtime.timer.
time.NewTicker
time.NewTicker creates a *Ticker (and time.Tick is a helper function that does the same thing but directly returns *Ticker.C, the ticker's receive channel, instead of *Ticker, so you could've written your code with it instead) has similar hooks into the runtime: a ticker is a struct that holds a runtimeTimer and a channel on which to signal the ticks.
runtimeTimer is defined in the time package but it must be kept in sync with timer in src/runtime/time.go, so it is effectively a runtime.timer. Remember that in time.Sleep, the timer had a callback function to wake up the sleeping goroutine? In the case of *Ticker, the timer's callback function sends the current time on the ticker's channel.
Then, the real waiting/scheduling happens on the receive from the channel, which is essentially the same as the select statement unless otherChan sends something before the tick, so let's look at what happens on a blocking receive.
<- chan
Channels are implemented (now in Go!) in src/runtime/chan.go, by the hchan struct. Channel operations have matching functions, and a receive is implemented by chanrecv:
// chanrecv receives on channel c and writes the received data to ep.
// ep may be nil, in which case received data is ignored.
// If block == false and no elements are available, returns (false, false).
// Otherwise, if c is closed, zeros *ep and returns (true, false).
// Otherwise, fills in *ep with an element and returns (true, true).
func chanrecv(t *chantype, c *hchan, ep unsafe.Pointer, block bool) (selected, received bool) {
// ...
}
This part has a lot of different cases, but in your example, it is a blocking receive from an asynchronous channel (time.NewTicker creates a channel with a buffer of 1), but anyway it ends up calling... goparkunlock, again to allow other goroutines to proceed while this one is stuck waiting.
So...
In all cases, the goroutine ends up being parked (which is not really shocking - it can't make progress, so it has to leave its thread available for a different goroutine if there's any available). A glance at the code seems to suggest that the channel has a bit more overhead than a straight-up time.Sleep. However, it allows far more powerful patterns, such as the last one in your example: the goroutine can be woken up by another channel, whichever comes first.
To answer your other questions, regarding polling, the timers are managed by a goroutine that sleeps until the next timer in its queue, so it's working only when it knows a timer has to be triggered. When the next timer has expired, it wakes up the goroutine that called time.Sleep (or sends the value on the ticker's channel, it does whatever the callback function does).
There's no polling in channels, the receive is unlocked when a send is made on the channel, in chansend of the chan.go file:
// wake up a waiting receiver
sg := c.recvq.dequeue()
if sg != nil {
recvg := sg.g
unlock(&c.lock)
if sg.releasetime != 0 {
sg.releasetime = cputicks()
}
goready(recvg, 3)
} else {
unlock(&c.lock)
}
That was an interesting dive into Go's source code, very interesting question! Hope I answered at least part of it!