I'm learning Golang, and after reading this post on Go's blog, I have the following question.
I start with the following code (from the post):
select {
case <-ch:
// a read from ch has occurred
case <-timeout:
// the read from ch has timed out
}
And based on what A Tour of Go states:
... It chooses one at random if multiple are ready.
As I understand, it is possible to have my result ready and have a timeout at the same time. My question is whether it is worth it (or correct) to double-check for this inside the default case.
Something like the following:
select {
case <-ch:
// a read from ch has occurred
case <-timeout:
// the read from ch has timed out
// So check ch one last time
select {
case <-ch:
// a read from ch has occurred at same time of a timeout,
// so ignore the timeout
default:
// definitive timeout
}
}
If one of the channels is a timeout, odds of your work being done and the timeout firing at exactly the same time are so small they make no sense to consider.
The statement "... It chooses one at random if multiple are ready." is applicable when you actually have a viable reason for this to happen - when you have a select case on multiple job channels that you're processing with a single goroutine, for instance.
Related
I have the following piece of code:
func sendRegularHeartbeats(ctx context.Context) {
for {
select {
case <-ctx.Done():
return
case <-time.After(1 * time.Second):
sendHeartbeat()
}
}
}
This function is executed in a dedicated go-routine and sends a heartbeat-message every second. The whole process should stop immediately when the context is canceled.
Now consider the following scenario:
ctx, cancel := context.WithCancel(context.Background())
cancel()
go sendRegularHeartbeats(ctx)
This starts the heartbeat-routine with a closed context. In such a case, I don't want any heartbeats to be transmitted. So the first case block in the select should be entered immediately.
However, it seems that the order in which case blocks are evaluated is not guaranteed, and that the code sometimes sends a heartbeat message, even though the context is already canceled.
What is the correct way to implement such a behaviour?
I could add a "isContextclosed"-check in the second case, but that looks more like an ugly workaround for the problem.
The accepted answer has a wrong suggestion:
func sendRegularHeartbeats(ctx context.Context) {
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
//first select
select {
case <-ctx.Done():
return
default:
}
//second select
select {
case <-ctx.Done():
return
case <-ticker.C:
sendHeartbeat()
}
}
}
This doesn't help, because of the following scenario:
both channels are empty
first select runs
both channels get a message concurrently
you are in the same probability game as if you haven't done anything in the first select
An alternative but still imperfect way is to guard against concurrent Done() events (the "wrong select") after consuming the ticker event i.e.
func sendRegularHeartbeats(ctx context.Context) {
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
//select as usual
select {
case <-ctx.Done():
return
case <-ticker.C:
//give priority to a possible concurrent Done() event non-blocking way
select {
case <-ctx.Done():
return
default:
}
sendHeartbeat()
}
}
}
Caveat: the problem with this one is that it allows for "close enough" events to be confused - e.g. even though a ticker event arrived earlier, the Done event came soon enough to preempt the heartbeat. There is no perfect solution as of now.
Note beforehand:
Your example will work as you intend it to, as if the context is already cancelled when sendRegularHeartbeats() is called, the case <-ctx.Done() communication will be the only one ready to proceed and therefore chosen. The other case <-time.After(1 * time.Second) will only be ready to proceed after 1 second, so it will not be chosen at first. But to explicitly handle priorities when multiple cases might be ready, read on.
Unlike the case branches of a switch statement (where the evaluation order is the order they are listed), there is no priority or any order guaranteed in the case branches of a select statement.
Quoting from Spec: Select statements:
If one or more of the communications can proceed, a single one that can proceed is chosen via a uniform pseudo-random selection. Otherwise, if there is a default case, that case is chosen. If there is no default case, the "select" statement blocks until at least one of the communications can proceed.
If more communications can proceed, one is chosen randomly. Period.
If you want to maintain priority, you have to do that yourself (manually). You may do it using multiple select statements (subsequent, not nested), listing ones with higher priority in an earlier select, also be sure to add a default branch to avoid blocking if those are not ready to proceed. Your example requires 2 select statements, first one checking <-ctx.Done() as that is the one you want higher priority for.
I also recommend using a single time.Ticker instead of calling time.After() in each iteration (time.After() also uses a time.Ticker under the hood, but it doesn't reuse it just "throws it away" and creates a new one on the next call).
Here's an example implementation:
func sendRegularHeartbeats(ctx context.Context) {
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
default:
}
select {
case <-ctx.Done():
return
case <-ticker.C:
sendHeartbeat()
}
}
}
This will send no heartbeat if the context is already cancelled when sendRegularHeartbeats() is called, as you can check / verify it on the Go Playground.
If you delay the cancel() call for 2.5 seconds, then exactly 2 heartbeats will be sent:
ctx, cancel := context.WithCancel(context.Background())
go sendRegularHeartbeats(ctx)
time.Sleep(time.Millisecond * 2500)
cancel()
time.Sleep(time.Second * 2)
Try this one on the Go Playground.
If it is absolutely critical to maintain that priority of operations, you can:
Consume from each channel in a separate goroutine
Have each of those goroutines write a message to a shared third channel indicating its type
Have a third goroutine consume from that channel, reading the messages it receives to determine if it is a tick and should sendHeartbeat or if it is a cancel and it should exit
This way, messages received on the other channels will (probably - you can't guarantee order of execution of concurrent routines) come in on the third channel in the order they're triggered, allowing you to handle them appropriately.
However, it's worth noting that this is probably not necessary. A select does not guarantee which case will execute if multiple cases succeed simultaneously. That is probably a rare event; the cancel and ticker would both have to fire before either was handled by the select. The vast majority of the time, only one or the other will fire at any given loop iteration, so it will behave exactly as expected. If you can tolerate rare occurrences of firing one additional heartbeat after a cancellation, you're better off keeping the code you have, as it is more efficient and more readable.
I have a loop that reads from a channel (indirectly receiving from a socket) but must also send a ping regularly if there is no other traffic. So I create a time.Timer on every loop iteration like this:
var timer *time.Timer
for {
timer = time.NewTimer(pingFrequency)
select {
case message := <-ch:
....
case <-timer.C:
ping()
}
_ = timer.Stop()
}
The doco for the Stop method implies you should drain the channel if it returns false but is that really necessary since the previous value of timer is no longer used and will be released by the GC. Ie even if the old timer.C has an unread value it does not matter since nothing is using the channel any longer.
The doc. for t.Stop() says:
To ensure the channel is empty after a call to Stop,
check the return value and drain the channel.
My question is: Why would you need to ensure the channel is empty since nothing is using it and it will be freed by the GC?
Please note that I am aware of the proper way to stop a timer in the general case.
if !timer.Stop() {
select {
case <-timer.C:
default:
}
}
This specific question has not been answered on SO or elsewhere (apart from the comment from #JimB below).
I've been working with examples trying to get my first "go routine" running and while I got it running, it won't work as prescribed by the go documentation with the timer.Reset() function.
In my case I believe that the way I am doing it is just fine because I don't actually care what's in the chan buffer, if anything. All as this is meant to do is trigger case <-tmr.C: if anything happened on case _, ok := <-watcher.Events: and then all goes quiet for at least one second. The reason for this is that case _, ok := <-watcher.Events: can get from one to dozens of events microseconds apart and I only care once they are all done and things have settled down again.
However I'm concerned that doing it the way that the documentation says you "must do" doesn't work. If I knew go better I would say the documentation is flawed because it assumes there is something in the buffer when there may not be but I don't know go well enough to have confidence in making that determination so I'm hoping some experts out there can enlighten me.
Below is the code. I haven't put this up on playground because I would have to do some cleaning up (remove calls to other parts of the program) and I'm not sure how I would make it react to filesystem changes for showing it working.
I've clearly marked in the code which alternative works and which doesn't.
func (pm *PluginManager) LoadAndWatchPlugins() error {
// DOING OTHER STUFF HERE
fmt.Println(`m1`)
done := make(chan interface{})
terminated := make(chan interface{})
go pm.watchDir(done, terminated, nil)
fmt.Println(`m2.pre-10`)
time.Sleep(10 * time.Second)
fmt.Println(`m3-post-10`)
go pm.cancelWatchDir(done)
fmt.Println(`m4`)
<-terminated
fmt.Println(`m5`)
os.Exit(0) // Temporary for testing
return Err
}
func (pm *PluginManager) cancelWatchDir(done chan interface{}) {
fmt.Println(`t1`)
time.Sleep(5 * time.Second)
fmt.Println()
fmt.Println(`t2`)
close(done)
}
func (pm *PluginManager) watchDir(done <-chan interface{}, terminated chan interface{}, strings <-chan string) {
watcher, err := fsnotify.NewWatcher()
if err != nil {
Logger("watchDir::"+err.Error(), `plugins`, Error)
}
//err = watcher.Add(pm.pluginDir)
err = watcher.Add(`/srv/plugins/`)
if err != nil {
Logger("watchDir::"+err.Error(), `plugins`, Error)
}
var tmr = time.NewTimer(time.Second)
tmr.Stop()
defer close(terminated)
defer watcher.Close()
defer tmr.Stop()
for {
select {
case <-tmr.C:
fmt.Println(`UPDATE FIRED`)
tmr.Stop()
case _, ok := <-watcher.Events:
if !ok {
return
}
fmt.Println(`Ticker: STOP`)
/*
* START OF ALTERNATIVES
*
* THIS IS BY EXAMPLE AND STATED THAT IT "MUST BE" AT:
* https://golang.org/pkg/time/#Timer.Reset
*
* BUT DOESN'T WORK
*/
if !tmr.Stop() {
fmt.Println(`Ticker: CHAN DRAIN`)
<-tmr.C // STOPS HERE AND GOES NO FURTHER
}
/*
* BUT IF I JUST DO THIS IT WORKS
*/
tmr.Stop()
/*
* END OF ALTERNATIVES
*/
fmt.Println(`Ticker: RESET`)
tmr.Reset(time.Second)
case <-done:
fmt.Println(`DONE TRIGGERED`)
return
}
}
}
Besides what icza said (q.v.), note that the documentation says:
For example, assuming the program has not received from t.C already:
if !t.Stop() {
<-t.C
}
This cannot be done concurrent to other receives from the Timer's channel.
One could argue that this is not a great example since it assumes that the timer was running at the time you called t.Stop. But it does go on to mention that this is a bad idea if there's already some existing goroutine that is or may be reading from t.C.
(The Reset documentation repeats all of this, and kind of in the wrong order because Reset sorts before Stop.)
Essentially, the whole area is a bit fraught. There's no good general answer, because there are at least three possible situations during the return from t.Stop back to your call:
No one is listening to the channel, and no timer-tick is in the channel now. This is often the case if the timer was already stopped before the call to t.Stop. If the timer was already stopped, t.Stop always returns false.
No one is listening to the channel, and a timer-tick is in the channel now. This is always the case when the timer was running but t.Stop was unable to stop it from firing. In this case, t.Stop returns false. It's also the case when the timer was running but fired before you even called t.Stop, and had therefore stopped on its own, so that t.Stop was not able to stop it and returned false.
Someone else is listening to the channel.
In the last situation, you should do nothing. In the first situation, you should do nothing. In the second situation, you probably want to receive from the channel so as to clear it out. That's what their example is for.
One could argue that:
if !t.Stop() {
select {
case <-t.C:
default:
}
}
is a better example. It does one non-blocking attempt that will consume the timer-tick if present, and does nothing if there is no timer-tick. This works whether or not the timer was not actually running when you called t.Stop. Indeed, it even works if t.Stop returns true, though in that case, t.Stop stopped the timer, so the timer never managed to put a timer-tick into the channel. (Thus, if there is a datum in the channel, it must necessarily be left over from a previous failure to clear the channel. If there are no such bugs, the attempt to receive was in turn unnecessary.)
But, if someone else—some other goroutine—is or may be reading the channel, you should not do any of this at all. There is no way to know who (you or them) will get any timer tick that might be in the channel despite the call to Stop.
Meanwhile, if you're not going to use the timer any further, it's relatively harmless just to leave a timer-tick, if there is one, in the channel. It will be garbage-collected when the channel itself is garbage-collected. Of course, whether this is sensible depends on what you are doing with the timer, but in these cases it suffices to just call t.Stop and ignore its return value.
You create a timer and you stop it immediately:
var tmr = time.NewTimer(time.Second)
tmr.Stop()
This doesn't make any sense, I assume this is just an "accident" from your part.
But going further, inside your loop:
case _, ok := <-watcher.Events:
When this happens, you claim this doesn't work:
if !tmr.Stop() {
fmt.Println(`Ticker: CHAN DRAIN`)
<-tmr.C // STOPS HERE AND GOES NO FURTHER
}
Timer.Stop() documents that it returns true if this call stops the timer, and false if the timer has already been stopped (or expired). But your timer was already stopped, right after its creation, so tmr.Stop() returns false properly, so you go inside the if and try to receive from tmr.C, but since the timer was "long" stopped, nothing will be sent on its channel, so this is a blocking (forever) operation.
If you're the one stopping the timer explicitly with timer.Stop(), the recommended "pattern" to drain its channel doesn't make any sense and doesn't work for the 2nd Timer.Stop() call.
I am a Go rookie.
I'm looking at this construct:
for {
select {
case <-resyncCh:
case <-stopCh:
return
case <-cancelCh:
return
}
if r.ShouldResync == nil || r.ShouldResync() {
// do stuff
}
resyncCh = r.resyncChan()
}
I understand that the for loop runs forever.
I understand that break is implicit in Go.
I understand that channel operations in a select statement are blocking if there's no default clause (which there isn't here).
Suppose resyncCh does not have a message on it.
Are all the cases evaluated (blocked on) in parallel? Or is there another path through this I'm not seeing?
I read this as:
Block on the resyncCh, the stopCh and the cancelCh chans in parallel waiting for messages
If a message is received on resyncCh, we effectively fall through to the r.ShouldResync stuff, but the other blocks on the other chans remain.
If a message is received at any point on either the stopCh or the cancelCh chan, return, effectively "disconnecting" from all chans here.
Is that correct?
In direct answer to your questions:
Block on the resyncCh, the stopCh and the cancelCh chans in parallel waiting for messages. YES.
If a message is received on resyncCh, we effectively fall through to the r.ShouldResync stuff, but the other blocks on the other chans remain. No, they don't remain, you are past the select However, since this loops, you will block again. You could also use the fallthrough keyword to make them block after passing the initial one.
If a message is received at any point on either the stopCh or the cancelCh chan, return, effectively "disconnecting" from all chans here. Correct - they would return from this function.
Also, bear in mind what you can do with a default --> https://gobyexample.com/non-blocking-channel-operations
for {
select {
case <-resyncCh:
case <-stopCh:
return
case <-cancelCh:
return
default:
fmt.Printf("will keep printing\n")
}
if r.ShouldResync == nil || r.ShouldResync() {
// do stuff
}
resyncCh = r.resyncChan()
}
update: Another useful pattern, I'm using right now, which takes advantage of this:
select {
case m := <-c:
handle(m)
case <-time.After(5 * time.Minute):
fmt.Println("timed out")
}
Here you can wait, blocking, on a channel, but eventually timeout, just using the golang time package. Very succinct and easy to read. Compare that to poll() with timespec values.
https://golang.org/pkg/time/#After
select takes first not blocked action and goes to next operation.
Block on the resyncCh, the stopCh and the cancelCh chans in parallel waiting for messages
Yes, waiting for first of them.
If a message is received on resyncCh, we effectively fall through to the r.ShouldResync stuff, but the other blocks on the other chans remain.
Unlike in some other languages fallthrough is explicit in go - you should state it.
If a message is received at any point on either the stopCh or the cancelCh chan, return, effectively "disconnecting" from all chans here.
Exit from the function where the code located. Yes, we do not wait for new messages more.
EDIT: My question is different from How to write my own Sleep function using just time.After? It has a different variant of the code that's not working for a separate reason and I needed explanation as to why.
I'm trying to solve the homework problem here: https://www.golang-book.com/books/intro/10 (Write your own Sleep function using time.After).
Here's my attempt so far based on the examples discussed in that chapter:
package main
import (
"fmt"
"time"
)
func myOwnSleep(duration int) {
for {
select {
case <-time.After(time.Second * time.Duration(duration)):
fmt.Println("slept!")
default:
fmt.Println("Waiting")
}
}
}
func main() {
go myOwnSleep(3)
var input string
fmt.Scanln(&input)
}
http://play.golang.org/p/fb3i9KY3DD
My thought process is that the infinite for will keep executing the select statement's default until the time.After function's returned channel talks. Problem with the current code being, the latter does not happen, while the default statement is called infinitely.
What am I doing wrong?
In each iteration of your for loop the select statement is executed which involves evaluating the channel operands.
In each iteration time.After() will be called and a new channel will be created!
And if duration is more than 0, this channel is not ready to receive from, so the default case will be executed. This channel will not be tested/checked again, the next iteration creates a new channel which will again not be ready to receive from, so the default case is chosen again - as always.
The solution is really simple though as can be seen in this answer:
func Sleep(sec int) {
<-time.After(time.Second* time.Duration(sec))
}
Fixing your variant:
If you want to make your variant work, you have to create one channel only (using time.After()), store the returned channel value, and always check this channel. And if the channel "kicks in" (a value is received from it), you must return from your function because more values will not be received from it and so your loop will remain endless!
func myOwnSleep(duration int) {
ch := time.After(time.Second * time.Duration(duration))
for {
select {
case <-ch:
fmt.Println("slept!")
return // MUST RETURN, else endless loop!
default:
fmt.Println("Waiting")
}
}
}
Note that though until a value is received from the channel, this function will not "rest" and just execute code relentlessly - loading one CPU core. This might even give you trouble if only 1 CPU core is available (runtime.GOMAXPROCS()), other goroutines (including the one that will (or would) send the value on the channel) might get blocked and never executed. A sleep (e.g. time.Sleep(time.Millisecond)) could release the CPU core from doing endless work (and allow other goroutines to run).