Golang: Interrupting infinite polling having time.Sleep - go

I am using the following simple polling mechanism:
func poll() {
for {
if a {
device1()
time.Sleep(time.Second * 10)
} else {
sensor1()
time.Sleep(time.Second * 10)
}
}
}
I need to poll the device1 only if "a" is true otherwise poll the sensor1. Now here "a" will be set to true by clicking the button on UI which will be a random act.
But because of time.Sleep, delay get introduced while checking the condition.
Is there any way to immediately stop time.Sleep when I get the value for "a"?
what are the possible ways to implement such interrupts while polling in golang?

You cannot interrupt a time.Sleep().
Instead if you need to listen to other "events" while waiting for something, you should use channels and the select statements. A select is similar to a switch, but with the cases all referring to communication operations.
The good thing of select is that you may list multiple cases, all refering to comm. ops (e.g. channel receives or sends), and whichever operation can proceed will be chosen (choosing one randomly if multiple can proceed). If none can proceed, it will block (wait) until one of the comm. ops can proceed (unless there's a default).
A "timeout" event may be realized using time.After(), or a series of "continuous" timeout events (ticks or heartbeats) using time.Ticker.
You should change your button handler to send a value (the button state) on a channel, so receiving from this channel can be added to the select inside poll(), and processed "immediately" (without waiting for a timeout or the next tick event).
See this example:
func poll(buttonCh <-chan bool) {
isDevice := false
read := func() {
if isDevice {
device1()
} else {
sensor1()
}
}
ticker := time.NewTicker(1 * time.Second)
defer func() { ticker.Stop() }()
for {
select {
case isDevice = <-buttonCh:
case <-ticker.C:
read()
}
}
}
func device1() { log.Println("reading device1") }
func sensor1() { log.Println("reading sensor1") }
Testing it:
buttonCh := make(chan bool)
// simulate a click after 2.5 seconds:
go func() {
time.Sleep(2500 * time.Millisecond)
buttonCh <- true
}()
go poll(buttonCh)
time.Sleep(5500 * time.Millisecond)
Output (try it on the Go Playground):
2009/11/10 23:00:01 reading sensor1
2009/11/10 23:00:02 reading sensor1
2009/11/10 23:00:03 reading device1
2009/11/10 23:00:04 reading device1
2009/11/10 23:00:05 reading device1
This solution makes it easy to add any number of event sources without changing anything. For example if you want to add a "shutdown" event and a "read-now" event to the above solution, this is how it would look like:
for {
select {
case isDevice = <-buttonCh:
case <-ticker.C:
read()
case <-readNowCh:
read()
case <-shutdownCh:
return
}
}
Where shutdownCh is a channel with any element type, and to signal shutdown, you do that by closing it:
var shutdownCh = make(chan struct{})
// Somewhere in your app:
close(shutdownCh)
Closing the channel has the advantage that you may have any number of goroutines "monitoring" it, all will receive the shutdown (this is like broadcasting). Receive from a closed channel can proceed immediately, yielding the zero-value of the channel's element type.
To issue an "immediate read" action, you would send a value on readNowCh.

Related

Concurrency in Go loop polling case branch not hit

I am implementing a very simple concurrent program in Go. There are 2 channels todo and done that are used for signaling which task is done. There are 5 routines that are executed and each one require its own time to complete. I would like to see every 100ms the status of what is happening.
However I tried but the polling branch case <-time.After(100 * time.Millisecond): seems that is never been called. It is called sometimes (not in a consisted way) if I reduce the time to something less than 100ms.
My understanding is that go func executes the method in a separate Go scheduler thread. I do not understand therefore why the case of the polling is never hit. I tried to move the specific case branch before/after the other but nothing changed.
Any suggestions?
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
func concurrent(id int, done chan int, todo chan int) {
for {
// doing a task
t := randInt(50, 100)
time.Sleep(time.Duration(t) * time.Millisecond)
done <- id
// redo again this task
t = randInt(50, 100)
time.Sleep(time.Duration(t) * time.Millisecond)
todo <- id
}
}
func randInt(min int, max int) int {
return (min + rand.Intn(max-min))
}
func seedRandom() {
rand.Seed(time.Now().UTC().UnixNano())
}
func main() {
seedRandom()
todo := make(chan int, 5)
done := make(chan int, 5)
for i := 0; i < 5; i++ {
todo <- i
}
timeout := make(chan bool)
go func() {
time.Sleep(1 * time.Second)
timeout <- true
}()
var mu sync.Mutex
var output []int
loop:
for {
select {
case <-time.After(100 * time.Millisecond):
//this branch is never hit?
fmt.Printf("\nPolling status: %v\n", output)
case <-timeout:
fmt.Printf("\nDing ding, time is up!\n")
break loop
case id := <-done:
mu.Lock()
output = append(output, id)
fmt.Printf(".")
mu.Unlock()
case id := <-todo:
go concurrent(id, done, todo)
}
}
}
Update After following the answers I created this version in Go Playgound: https://play.golang.org/p/f08t984BdPt. That works as expected
you are creating 5 goroutines (func concurrent) and in your select case using the todo channel and this channel is being used in concurrent function so you end up creating a lot of goroutines
func concurrent(id int, done chan int, todo chan int) {
for {
// doing a task
t := randInt(50, 100)
time.Sleep(time.Duration(t) * time.Millisecond)
done <- id
// redo again this task
t = randInt(50, 100)
time.Sleep(time.Duration(t) * time.Millisecond)
by doing this call you are re-crating the go-routime
todo <- id
}
}
when I ran your code I got "runtime.NumGoroutine()"
"number of goRoutines still running 347"
as you are implementing the time.After(100 * time.Millisecond) inside the for loop it gets reset every time some other case gets hit and in your case
case id := <-todo: && id := <-done: will always get hit within 100 Milliseconds that's why you didn't get the expected output (from how your code is now i would say that the number of go-routines would increase exponentially and each of em would be waiting to send value to done and few on todo channel so your loop wont get enough time(100 ms) to wait on time.After)
loop:
for {
select {
case <-time.After(100 * time.Millisecond): ->this will always get reset ( we can use time.Ticker as it will create a single object that will signal for each and every 100ms https://golang.org/pkg/time/#NewTicker
//this branch is never hit?
fmt.Printf("\nPolling status: %v\n", output)
case <-timeout:
fmt.Printf("\nDing ding, time is up!\n")
break loop
case id := <-done: -> **this will get called**
//the mutex call is actually not very usefull as this only get called once per loop and is prefectly thread safe in this code
mu.Lock()
output = append(output, id)
fmt.Printf(".")
mu.Unlock()
case id := <-todo: -> **this will get called**
go concurrent(id, done, todo)
}
}
}
https://play.golang.org/p/SmlSIUIF5jn -> I have made some modifications to make your code work as expected..
try referring this to get a better understanding of golang channels and goroutine
https://tour.golang.org/concurrency/1
time.After(100*time.Millisecond) creates a brand new channel, with a brand new timer, which starts at the moment that function is called.
So, in your loop :
for {
select {
// this statement resets the 100ms timer each time you execute the loop :
case <-time.After(100*time.Millisecond):
...
Your branch never gets hit because, with 5 goroutines sending signals within less than 100ms on one of the other cases, this time.After(100ms) never reaches completion.
You need to choose a way to keep the same timer between iterations.
Here is one way to adapt your time.After(...) call :
// store the timer in a variable *outside* the loop :
statusTimer := time.After(100*time.Millisecond)
for {
select {
case <-statusTimer:
fmt.Printf("\nPolling status: %v\n", output)
// reset the timer :
statusTimer = time.After(100*time.Millisecond)
case <-timeout:
...
Another way is, as #blackgreen suggests, to use a time.Ticker :
statusTicker := time.NewTicker(100*time.Millisecond)
for {
select {
case <-statusTicker.C:
fmt.Printf("\nPolling status: %v\n", output)
case <-timeout:
...
side notes
a. if the output slice is not shared with other goroutines, you don't need a mutex around its access :
for {
select {
case <-statusTicker.C:
fmt.Printf("\nPolling status: %v\n", output)
...
case i <-done:
// no race condition here : all happens within the same goroutine,
// the 'select' statement makes sure that 'case's are executed
// one at a time
output = append(output, id)
fmt.Printf(".")
b. For your timeout channel :
Another generic way to "signal" that some event occurred with a channel is to close the channel instead of sending a value on it :
// if you don't actually care about the value you send over this channel :
// you can make it unbuffered, and use the empty 'struct{}' type
timeout := make(chan struct{})
go func(){
// wait for some condition ...
<-time.After(1*time.Second)
close(timeout)
}()
select {
case <-statusTimer:
...
case <-timeout: // this branch will also be taken once timeout is closed
fmt.Printf("\nDing ding, time is up!\n")
break loop
case ...
The bug you will avoid is the following : suppose you want to use that timeout channel in two goroutines
if you send a value over the timeout channel, only one goroutine will get signaled - it will "eat up" the value from the channel, and the other goroutine will only have a blocking channel,
if you close the channel, both goroutines will correctly "receive" the signal
In absence of a default case, when multiple cases are ready, it executes one of them at random. It's not deterministic.
To make sure the case runs, you should run it in a separate goroutine. (In that case, you must synchronize accesses to the output variable).
Moreover you say "I would like to see every 100ms", but time.After sends on the channel only once.
To execute the case periodically, use <-time.NewTicker(100 * time.Millis).C instead.
var mu sync.Mutex
var output []int
go func() {
ticker := time.NewTicker(100 * time.Millisecond)
defer ticker.Stop()
for {
select {
case <-ticker.C:
// TODO: must synchronize access
fmt.Printf("\nPolling status: %v\n", output)
case <-timeout:
return
}
}
}()
loop:
for {
select {
// other cases
}
}

Handling timeouts and listening to channels

I have code that looks like this, where I'm listening to a channel up until a timeout interval. Let's say this goroutine 1
select {
case <-time.After(TimeoutInterval):
mu.Lock()
defer mu.Unlock()
delete(msgChMap, index)
return ""
case msg := <-msgCh:
return msg
}
Elsewhere, I have a goroutine 2 that runs something like this where it grabs the appropriate msgCh from a Map, deletes the entry in the map and then sends a message through the channel.
mu.Lock()
msgCh, ok := msgChMap[index]
delete(msgChMap, index)
mu.Unlock()
if ok {
msgCh <- "yay"
}
It seems like it is possible for me to grab the message channel msgCh from the Map, try to send a message but because TimeoutInterval has already passed, there will be nothing listening to the channel, and my code will get stuck waiting for a listener. If I put the lock after sending yay to the msgCh, it seems possible that I could deadlock as 2 will be waiting for a listener to the channel and is not releasing the lock, but 1 is no longer listening but requires the lock.
What is a general pattern to avoid getting stuck waiting for a listener? Perhaps go is smart enough to not get stuck here.
You can prevent getting stuck when waiting for a listener by using select for the sender.
By using select you can use more case for sender in this situation
mu.Lock()
msgCh, ok := msgChMap[index]
delete(msgChMap, index)
mu.Unlock()
if ok {
select {
// listener is available
case msgCh <- "yay":
fmt.Println("sent")
// if not avalable (execute immediately)
default:
fmt.Println("no available listener")
// ...just ignore or do something else
}
}
Or waiting for a short time
mu.Lock()
msgCh, ok := msgChMap[index]
delete(msgChMap, index)
mu.Unlock()
if ok {
select {
// listener is available
case msgCh <- "yay":
fmt.Println("sent")
// if not available, waiting for listener
case <-time.After(30 * time.Second):
fmt.Println("after 30 seconds, still no available listener")
// ...just ignore or do something else
}
}
The problem here is that the channel reader my stop without the writer knowing it. It should be possible to structure this solution so that this situation never happens, but ignoring that for now, for this specific problem what you need is atomic access to the channel itself, along with a flag for channel status:
type channel struct {
sync.Mutex
msgCh chan Msg
active bool
}
Writing to the channel is now done by locking it:
ch.Lock()
if ch.active {
ch.msgCh<-data
}
ch.Unlock()
And when you "inactivate" the channel, reset the flag:
case <-time.After(TimeoutInterval):
mu.Lock()
defer mu.Unlock()
ch.Lock()
defer ch.Unlock()
delete(msgChMap, index)
ch.active=false
return ""
And of course, with this now you have to keep a *channel in your map.

golang design pattern for cancelling routines inflight

I am a golang newbie who is trying to understand the correct design pattern for this problem. My current solution seems very verbose, and I'm not sure what the better approach would be.
I am trying to design a system that:
executes N goroutines
returns the result of each goroutine as soon as it is available
if a goroutine returns a particular value, it should kill other goroutines will cancel.
The goal: I want to kick off a number of goroutines, but I want to cancel the routines if one routine returns a particular result.
I'm trying to understand if my code is super "smelly" or if this is the prescribed way of doing things. I still don't have a great feeling for go, so any help would be appreciated.
Here is what I've written:
package main
import (
"context"
"fmt"
"time"
)
func main() {
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
fooCheck := make(chan bool)
barCheck := make(chan bool)
go foo(ctx, 3000, fooCheck)
go bar(ctx, 5000, barCheck)
for fooCheck != nil ||
barCheck != nil {
select {
case res, ok := <-fooCheck:
if !ok {
fooCheck = nil
continue
}
if res == false {
cancel()
}
fmt.Printf("result of foocheck: %t\n", res)
case res, ok := <-barCheck:
if !ok {
barCheck = nil
continue
}
fmt.Printf("result of barcheck: %t\n", res)
}
}
fmt.Printf("here we are at the end of the loop, ready to do some more processing...")
}
func foo(ctx context.Context, pretendWorkTime int, in chan<- bool) {
fmt.Printf("simulate doing foo work and pass ctx down to cancel down the calltree\n")
time.Sleep(time.Millisecond * time.Duration(pretendWorkTime))
select {
case <-ctx.Done():
fmt.Printf("\n\nWe cancelled this operation!\n\n")
break
default:
fmt.Printf("we have done some foo work!\n")
in <- false
}
close(in)
}
func bar(ctx context.Context, pretendWorkTime int, in chan<- bool) {
fmt.Printf("simulate doing bar work and pass ctx down to cancel down the calltree\n")
time.Sleep(time.Millisecond * time.Duration(pretendWorkTime))
select {
case <-ctx.Done():
fmt.Printf("\n\nWe cancelled the bar operation!\n\n")
break
default:
fmt.Printf("we have done some bar work!\n")
in <- true
}
close(in)
}
(play with the code here: https://play.golang.org/p/HAA-LIxWNt0)
The output works as expected, but I'm afraid I'm making some decision which will blow off my foot later.
I would use a single channel to communicate results, so it's much easier to gather the results and it "scales" automatically by its nature. If you need to identify the source of a result, simply use a wrapper which includes the source. Something like this:
type Result struct {
ID string
Result bool
}
To simulate "real" work, the workers should use a loop doing their work in an iterative manner, and in each iteration they should check the cancellation signal. Something like this:
func foo(ctx context.Context, pretendWorkMs int, resch chan<- Result) {
log.Printf("foo started...")
for i := 0; i < pretendWorkMs; i++ {
time.Sleep(time.Millisecond)
select {
case <-ctx.Done():
log.Printf("foo terminated.")
return
default:
}
}
log.Printf("foo finished")
resch <- Result{ID: "foo", Result: false}
}
In our example the bar() is the same just replace all foo word with bar.
And now executing the jobs and terminating the rest early if one does meet our expectation looks like this:
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
resch := make(chan Result, 2)
log.Println("Kicking off workers...")
go foo(ctx, 3000, resch)
go bar(ctx, 5000, resch)
for i := 0; i < cap(resch); i++ {
result := <-resch
log.Printf("Result of %s: %v", result.ID, result.Result)
if !result.Result {
cancel()
break
}
}
log.Println("Done.")
Running this app will output (try it on the Go Playground):
2009/11/10 23:00:00 Kicking off workers...
2009/11/10 23:00:00 bar started...
2009/11/10 23:00:00 foo started...
2009/11/10 23:00:03 foo finished
2009/11/10 23:00:03 Result of foo: false
2009/11/10 23:00:03 Done.
Some things to note. If we terminate early due to unexpected result, the cancel() function will be called, and we break out form the loop. It may be the rest of the workers also complete their work concurrently and send their result, which will not be a problem as we used a buffered channel, so their send will not block and they will end properly. Also, if they don't complete concurrently, they check ctx.Done() in their loop, and they terminate early, so the goroutines are cleaned up nicely.
Also note that the output of the above code does not print bar terminated. This is because the main() function terminates right after the loop, and once the main() function ends, it does not wait for other non-main goroutines to complete. For details, see No output from goroutine in Go. If the app would not terminate immediately, we would see that line printed too. If we add a time.Sleep() at the end of main():
log.Println("Done.")
time.Sleep(3 * time.Millisecond)
Output will be (try it on the Go Playground):
2009/11/10 23:00:00 Kicking off workers...
2009/11/10 23:00:00 bar started...
2009/11/10 23:00:00 foo started...
2009/11/10 23:00:03 foo finished
2009/11/10 23:00:03 Result of foo: false
2009/11/10 23:00:03 Done.
2009/11/10 23:00:03 bar terminated.
Now if you must wait for all workers to end either "normally" or "early" before moving on, you can achieve that in many ways.
One way is to use a sync.WaitGroup. For an example, see Prevent the main() function from terminating before goroutines finish in Golang. Another way would be to have each worker send a Result no matter how they end, and Result could contain the termination condition, e.g. normal or aborted. And the main() goroutine could continue the receive loop until it receives n values from resch. If this solution is chosen, you must ensure each worker sends a value (even if a panic occurs) to not block the main() in such cases (e.g. with using defer).
I'm going to share the most simplistic pattern for what you're talking about. You can extend it for more complicated scenarios.
func doStuff() {
// This can be a chan of anything.
msgCh := make(chan string)
// This is how you tell your go-routine(s) to stop, by closing this chan.
quitCh := make(chan struct{})
defer close(quitCh)
// Start all go routines.
for whileStart() {
go func() {
// Do w/e you need inside of your go-routine.
// Write back the result.
select {
case msgCh <- "my message":
// If we got here then the chan is open.
case <-quitCh:
// If we got here then the quit chan was closed.
}
}()
}
// Wait for all go routines.
for whileWait() {
// Block until a msg comes back.
msg := <-msgCh
// If you found what you want.
if msg == stopMe {
// It's safe to return because of the defer earlier.
return
}
}
}

golang channel behaviour with for loops

I'm curious about the behaviour of channels and how they work in relation to loops. Suppose I have the following code:
Consumer
tick := time.Tick(time.Duration(2) * time.Second)
for {
select {
case <-tick:
p.channel <-true
}
}
And I have a goroutine that has the following:
Processor
for {
select {
case canProcess := <-p.channel:
// synchronous process that takes longer than 2 seconds
case <-p.stop:
return
}
}
What happens when the Consumer pushes to the channel faster than the Processor can complete its synchronous process?
Do they pile up waiting for the Processor to complete or do they skip a "beat" as such?
If they pile up, is there potential for memory leaking?
I know I can put the synchronous process in a goroutine instead, but this is really to understand how channels behave. (i.e. my example has a 2-second tick, but it doesn't have to).
select will only invoke next time.Tick if previous case corresponding code was completed,
you can still have some lever by specifying size of channel i.e channel := make(chan bool,10)
See below:
func main() {
channel := make(chan bool, 10)
go func() {
tick := time.Tick(time.Duration(1) * time.Second)
for {
select {
case <-tick:
fmt.Printf("Producer: TICK %v\n", time.Now())
channel <- true
}
}
}()
for {
select {
case canProcess := <-channel:
time.Sleep(3* time.Second)
fmt.Printf("Consumer: Completed : %v\n")
fmt.Printf("%v\n", canProcess)
}
}
}

How do I iterate over a go time.Tick channel?

I'm having difficulty using time.Tick. I expect this code to print "hi" 10 times then quit after 1 second, but instead it hangs:
ticker := time.NewTicker(100 * time.Millisecond)
time.AfterFunc(time.Second, func () {
ticker.Stop()
})
for _ = range ticker.C {
go fmt.Println("hi")
}
https://play.golang.org/p/1p6-ViSvma
Looking at the source, I see that the channel isn't closed when Stop() is called. In that case, what is the idiomatic way to iterate over the ticker channel?
You're right, ticker's channel is not being closed on stop, that's stated in a documentation:
Stop turns off a ticker. After Stop, no more ticks will be sent. Stop does not close the channel, to prevent a read from the channel succeeding incorrectly.
I believe ticker is more about fire and forget and even if you want to stop it, you could even leave the routine hanging forever (depends on your application of course).
If you really need a finite ticker, you can do tricks and provide a separate channel (per ThunderCat's answer), but what I would do is providing my own implementation of ticker. This should be relatively easy and will give you flexibility with its behaviour, things like what to pass on the channel or deciding what to do with missing ticks (i.e. when reader is falling behind).
My example:
func finiteTicker(n int, d time.Duration) <-chan time.Time {
ch := make(chan time.Time, 1)
go func() {
for i := 0; i < n; i++ {
time.Sleep(d)
ch <- time.Now()
}
close(ch)
}()
return ch
}
func main() {
for range finiteTicker(10, 100*time.Millisecond) {
fmt.Println("hi")
}
}
http://play.golang.org/p/ZOwJlM8rDm
I asked on IRC as well, at got some useful insight from #Tv`.
Despite timer.Ticker looking like it should be part of a go pipeline, it does not actually play well with the pipeline idioms:
Here are the guidelines for pipeline construction:
stages close their outbound channels when all the send operations are done.
stages keep receiving values from inbound channels until those channels are closed or the senders are unblocked.
Pipelines unblock senders either by ensuring there's enough buffer for all the values that are sent or by explicitly signalling senders when the receiver may abandon the channel.
The reason for this inconsistency appears to be primarily to support the following idiom:
for {
select {
case <-ticker.C:
// do something
case <-done:
return
}
}
I don't know why this is the case, and why the pipelining idiom wasn't used:
for {
select {
case _, ok := <-ticker.C:
if ok {
// do something
} else {
return
}
}
}
(or more cleanly)
for _ = range ticker.C {
// do something
}
But this is the way go is :(

Resources