I noticed a weird behavior that is not entirely obvious when using select statement inside a loop, for example if I have:
package main
import (
"fmt"
"time"
)
func main() {
done := make(chan bool)
go func() {
fmt.Println("here we go")
for {
select {
case <-done:
fmt.Println("Bye")
return
default:
fmt.Println("this should continue to print")
}
fmt.Println("loop continues")
}
}()
time.Sleep(2 * time.Second)
done <- true
}
I would assume that the default case should only print for no longer than 2 seconds, however this is not the case, at least in my CPU it is lasting for about 10 seconds before seeing the Bye message.
However if I add time.Sleep(1 * time.Second) right after "loop continues" It is more or less in sync with the 2 second sleep before done <- true.
Given this my assumption is that print calls are getting stacked and the extended completion time is due to the call stack taking longer to get through.
Is is this a correct description of what's happening? Or am I missing something more obvious.
Edit: To put my hypothesis to test I did this and it now works as expected, it takes 2 seconds to finish the program:
go func() {
fmt.Println("here we go")
for {
select {
case <-done:
fmt.Println("Bye")
return
default:
}
}
}()
Related
I have a basic question about scheduling "cancellable" goroutines.
I want to schedule a function execution, every 3 seconds.
The function can take up to 5 seconds.
In case it takes more than 2999ms I want to stop/terminate it, to avoid overlapping w/ the next one.
I'm doing it wrong:
func main() {
fmt.Println("startProcessing")
go startProcessing()
time.Sleep(time.Second * 60)
fmt.Println("endProcessing after 60s")
}
func startProcessing() {
ticker := time.NewTicker(3 * time.Second)
for _ = range ticker.C {
ctx, _ := context.WithTimeout(context.Background(), (time.Second*3)-time.Millisecond)
fmt.Println("start doSomething")
doSomething(ctx)
}
}
func doSomething(ctx context.Context) {
executionTime := time.Duration(rand.Intn(5)+1) * time.Second
for {
select {
case <-ctx.Done():
fmt.Printf("timed out after %s\n", executionTime)
return
default:
time.Sleep(executionTime)
fmt.Printf("did something in %s\n", executionTime)
return
}
}
}
This is my output now:
startProcessing
start doSomething
did something in 2s
start doSomething
did something in 3s
start doSomething
did something in 3s
start doSomething
did something in 5s
start doSomething
did something in 2s
...
I want to read timed out after 5s instead of did something in 5s.
You just need to put the time.Sleep(executionTime) outside the select and there is no need for the for loop. I think this is somehow what you want but beware that it's not good practice. So take a look at the warning below.
func doSomething(ctx context.Context) {
executionTime := time.Duration(rand.Intn(5)+1) * time.Second
processed := make(chan int)
go func() {
time.Sleep(executionTime)
processed <- 1
}()
select {
case <-ctx.Done():
fmt.Printf("timed out after %s\n", executionTime)
case <-processed:
fmt.Printf("did something in %s\n", executionTime)
}
}
Obs: I changed the original answer a bit. We can not interrupt a goroutine in the middle of its execution. We could delegate the long-running task to another goroutine and receive the result through a dedicated channel.
Warning: I wouldn't recommend that if you expect the processing time to exceed the deadline because now you will have a leaking goroutine.
I'm posting here for the first time because I couldn't find a clean solution on the internet.
My goal is simple, I need to create a background operation (goroutine or process or whatever...) that I can kill properly (not leave in the background).
I've tried many things like using chan or context.
But I never could find the right way to avoid leaks.
Here is an example:
package main
import (
"log"
"strconv"
"runtime"
"time"
"math/rand"
)
func main() {
log.Println("goroutines: " + strconv.Itoa(runtime.NumGoroutine()))
func1()
leak := ""
if runtime.NumGoroutine() > 1 {
leak = " there is one LEAK !!"
}
log.Println("goroutines: " + strconv.Itoa(runtime.NumGoroutine()) + leak)
}
func func1() {
done := make(chan struct{})
quit := make(chan struct{})
go func() {
log.Println("goroutines: " + strconv.Itoa(runtime.NumGoroutine()))
select {
case <-quit:
log.Println("USEFUL ???")
return
default:
func2()
done<-struct{}{}
}
}()
select {
case <-time.After(4 * time.Second):
quit<-struct{}{}
log.Println("TIMEOUT")
case <-done:
log.Println("NO TIMEOUT")
}
}
func func2() {
log.Println("JOB START")
rand.Seed(time.Now().UnixNano())
val := rand.Intn(10)
log.Println("JOB DURATION: " + strconv.Itoa(val))
time.Sleep(time.Duration(val) * time.Second) // fake a long process with an unknown duration
log.Println("JOB DONE")
}
In this example, if the job is done before the 4 seconds timeout, everything is fine and the final number of goroutines will be 1, but otherwise it will be 2 like every example that I could find.
But it's just an example, maybe it's not possible with goroutines maybe it's even not possible in Go..
Your problem is here:
quit := make(chan struct{})
go func() {
for {
select {
case <-quit:
log.Println("USEFUL ???")
return
default:
func2()
quit<-struct{}{}
return
}
}
}()
This goroutine is signalling itself on an unbuffered channel. That means when it gets to the quit<-struct{}{}, that send will block forever, because it's waiting on itself to receive. It's not entirely clear how this is intended to work, though; there are a few odd things happening here:
A goroutine is signalling itself via channel, which seems wrong - it shouldn't need to communicate to itself
The channel and loop seem unnecessary; quit<-struct{}{} could be replaced with log.Println("USEFUL ???") and the whole for/select/channel business could be deleted
The function returns in each case of the select, so putting it in a loop is pointless anyway - there is no scenario where this code can ever execute a second iteration of the loop
I am confused about the following code, I write down some comment in the code which point out my confusion. And there is a result of execution at the end of the code, I also write down what result I expect.
package main
import (
"fmt"
"time"
)
func sendRPC() bool {
time.Sleep(5 * time.Second)
return true
}
func main() {
done := make(chan struct{})
ch := make(chan bool)
go func() { // goroutine A
select {
case ch <- sendRPC():
fmt.Println("RPC return")
case <-done:
fmt.Println("exit")
}
}()
select {
case <-ch:
case <-time.After(1000 * time.Millisecond):
fmt.Println("timeout")
if len(done) == 0 {
fmt.Println("1")
// here write done channel will block until sendRPC() return, why?
// I expect that the write is nonblock because goroutine A is select on done channel.
done <- struct{}{}
fmt.Println("2")
}
}
// result:
// timeout (after about 1 second)
// 1
// exit (after about 5 seconds, I expect that it is printed after about 1 second too.)
// 2
}
The specification says:
For all the cases in the statement, the channel operands of receive operations and the channel and right-hand-side expressions of send statements are evaluated exactly once, in source order, upon entering the "select" statement. The result is a set of channels to receive from or send to, and the corresponding values to send. Any side effects in that evaluation will occur irrespective of which (if any) communication operation is selected to proceed.
The set of channels for the select in goroutine A waits on the evaluation of sendRPC(). It may help to look at this equivalent goroutine:
go func() { // goroutine A
v := sendRPC() // waits for 5 seconds
select {
case ch <- v:
fmt.Println("RPC return")
case <-done:
fmt.Println("exit")
}
}()
The receive on done is delayed for 5 seconds.
given are the following 2 functions.
func main() {
index := int(0)
for {
Loop(index)
index = (index + 1) % 86400 // Max interval: 1 day
time.Sleep(1 * time.Second)
}
}
func Loop(index int) {
if index%10 == 0 {
go doSomething...
}
}
I want to execute something every 10/60/3600 seconds. So I thought an incrementing index with modulo should do this.
But what I noticed (especially on high traffic servers) that it appears to skip some of that loops.
I looked in my logs and sometimes there is something every 10 seconds but sometimes there is a gap up to 1 minute.
Does anybody know why this is happening?
I'd recommend using a time.Ticker to perform some action every N seconds. That way, you use built-in timers and only wake the CPU when something needs to be done. Even if the CPU is not heavily used, time.Sleep and a for loop is not the most reliable way to schedule tasks. For example (from the link above):
package main
import (
"fmt"
"time"
)
func main() {
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
done := make(chan bool)
go func() {
time.Sleep(10 * time.Second)
done <- true
}()
for {
select {
case <-done:
fmt.Println("Done!")
return
case t := <-ticker.C:
fmt.Println("Current time: ", t)
}
}
}
I need to read data from the Go channel for a certain period of time (say 5 seconds). The select statement with timeout doesn't work for me, as I need to read as many values as available and stop exactly after 5 seconds. So far, I've come up with a solution using an extra time channel https://play.golang.org/p/yev9CcvzRIL
package main
import "time"
import "fmt"
func main() {
// I have no control over dataChan
dataChan := make(chan string)
// this is a stub to demonstrate some data coming from dataChan
go func() {
for {
dataChan <- "some data"
time.Sleep(time.Second)
}
}()
// the following is the code I'm asking about
timeChan := time.NewTimer(time.Second * 5).C
for {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-timeChan:
fmt.Println("Time's up!")
return
}
}
}
I'm wondering is there any better or more idiomatic way for solving this problem?
That's pretty much it. But if you don't need to stop or reset the timer, simply use time.After() instead of time.NewTimer(). time.After() is "equivalent to NewTimer(d).C".
afterCh := time.After(5 * time.Second)
for {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-afterCh:
fmt.Println("Time's up!")
return
}
}
Alternatively (to your liking), you may declare the after channel in the for statement like this:
for afterCh := time.After(5 * time.Second); ; {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-afterCh:
fmt.Println("Time's up!")
return
}
}
Also I know this is just an example, but always think how a goroutine you start will properly end, as the goroutine producing data in your case will never terminate.
Also don't forget that if multiple cases may be executed without blocking, one is chosen randomly. So if dataChan is ready to receive from "non-stop", there is no guarantee that the loop will terminate immediately after the timeout. In practice this is usually not a problem (starting with that even the timeout is not a guarantee, see details at Golang Timers with 0 length), but you should not forget about it in "mission-critial" applications. For details, see related questions:
force priority of go select statement
golang: How the select worked when multiple channel involved?
It looks like context with deadline would be good fit, something like
func main() {
dataChan := make(chan string)
ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(5*time.Second))
defer cancel()
go func(ctx context.Context) {
for {
select {
case dataChan <- "some data":
time.Sleep(time.Second)
case <-ctx.Done():
fmt.Println(ctx.Err())
close(dataChan)
return
}
}
}(ctx)
for d := range dataChan {
fmt.Println("Got:", d)
}
}