I am attempting to create a poller in Go that spins up and every 24 hours executes a function.
I want to also be able to stop the polling, I'm attempting to do this by having a done channel and passing down an empty struct to stop the for loop.
In my tests, the for just loops infinitely and I can't seem to stop it, am I using the done channel incorrectly? The ticker case works as expected.
Poller struct {
HandlerFunc HandlerFunc
interval *time.Ticker
done chan struct{}
}
func (p *Poller) Start() error {
for {
select {
case <-p.interval.C:
err := p.HandlerFunc()
if err != nil {
return err
}
case <-p.done:
return nil
}
}
}
func (p *Poller) Stop() {
p.done <- struct{}{}
}
Here is the test that's exeuting the code and causing the infinite loop.
poller := poller.NewPoller(
testHandlerFunc,
time.NewTicker(1*time.Millisecond),
)
err := poller.Start()
assert.Error(t, err)
poller.Stop()
Seems like problem is in your use case, you calling poller.Start() in blocking maner, so poller.Stop() is never called. It's common, in go projects to call goroutine inside of Start/Run methods, so, in poller.Start(), i would do something like that:
func (p *Poller) Start() <-chan error {
errc := make(chan error, 1 )
go func() {
defer close(errc)
for {
select {
case <-p.interval.C:
err := p.HandlerFunc()
if err != nil {
errc <- err
return
}
case <-p.done:
return
}
}
}
return errc
}
Also, there's no need to send empty struct to done channel. Closing channel like close(p.done) is more idiomatic for go.
There is no explicit way in Go to broadcast an event to go routines for something like cancellation. Instead its idiomatic to create a channel that when closed signifies a message such as cancelling any work it has to do. Something like this is a viable pattern:
var done = make(chan struct{})
func cancelled() bool {
select {
case <-done:
return true
default:
return false
}
}
Go-routines can call cancelled to poll for a cancellation.
Then your main loop can respond to such an event but make sure you drain any channels that might cause go-routines to block.
for {
select {
case <-done:
// Drain whatever channels you need to.
for range someChannel { }
return
//.. Other cases
}
}
Related
constantly receive Json data from websocket and process them in goroutine, no idea is this writing pattern is encourage or not
ws.onmessage { //infinite receive message from websocket
go func() { //work find using this goroutine
defer processJson(message)
}()
go processJson(message) //error and program will terminated
}
func processJson(msg string) {
//code for process json
insertDatabase(processedMsg)
}
func insertDatabase(processedMsg string) {
//code insert to database
}
Below(the first goroutine) work just fine, but sometime(a week) indicates there is a data race in the code and terminate the program.
go func() {
defer processJson(message)
}()
the second goroutine, often encounter error after few minutes running, the error often is "fatal error: unexpected signal during runtime execution".
go processJson(message)
from my understanding both goroutine do the samething, why is that the first can run well and second cannot. i have try using channel, but not much difference compare to the first goroutine.
msgChan := make(chan string, 1000)
go processJson(msgChan)
for { //receive json from websocket, send to channel
msgChan <- message
}
func JsonProcessor(msg chan string) {
for { //get data from channel, process in goroutine function
msgModified := <-msg
insertDatabase(msgModified)
}
}
is there any encourage way to acheive the goal without data race, suggestions are welcome.
Appreciate and Thanks.
try to use sync.Mutex avoid data racing
mutux := sync.Mutex{}
ws.onmessage {
processJson(message)
}
func processJson(msg string) {
mutux.Lock()
// .........
mutux.Unlock()
}
if the processing function can be divided without data racing, multithread version as follows :
msgChan1 := make(chan string, 1000)
msgChan2 := make(chan string, 1000)
go func() {
for m := range msgChan1 {
// ...
}
}()
go func() {
for m := range msgChan2 {
// ...
}
}()
ws.onmessage {
msgChan1 <- message
msgChan2 <- message
}
ws.onclose {
close(msgChan1)
close(msgChan2)
}
I have the following code in Go using the semaphore library just as an example:
package main
import (
"fmt"
"context"
"time"
"golang.org/x/sync/semaphore"
)
// This protects the lockedVar variable
var lock *semaphore.Weighted
// Only one go routine should be able to access this at once
var lockedVar string
func acquireLock() {
err := lock.Acquire(context.TODO(), 1)
if err != nil {
panic(err)
}
}
func releaseLock() {
lock.Release(1)
}
func useLockedVar() {
acquireLock()
fmt.Printf("lockedVar used: %s\n", lockedVar)
releaseLock()
}
func causeDeadLock() {
acquireLock()
// calling this from a function that's already
// locked the lockedVar should cause a deadlock.
useLockedVar()
releaseLock()
}
func main() {
lock = semaphore.NewWeighted(1)
lockedVar = "this is the locked var"
// this is only on a separate goroutine so that the standard
// go "deadlock" message doesn't print out.
go causeDeadLock()
// Keep the primary goroutine active.
for true {
time.Sleep(time.Second)
}
}
Is there a way to get the acquireLock() function call to print a message after a timeout indicating that there is a potential deadlock but without unblocking the call? I would want the deadlock to persist, but a log message to be written in the event that a timeout is reached. So a TryAcquire isn't exactly what I want.
An example of what I want in psuedo code:
afterFiveSeconds := func() {
fmt.Printf("there is a potential deadlock\n")
}
lock.Acquire(context.TODO(), 1, afterFiveSeconds)
The lock.Acquire call in this example would call the afterFiveSeconds callback if the Acquire call blocked for more than 5 seconds, but it would not unblock the caller. It would continue to block.
I think I've found a solution to my problem.
func acquireLock() {
timeoutChan := make(chan bool)
go func() {
select {
case <-time.After(time.Second * time.Duration(5)):
fmt.Printf("potential deadlock while acquiring semaphore\n")
case <-timeoutChan:
break
}
}()
err := lock.Acquire(context.TODO(), 1)
close(timeoutChan)
if err != nil {
panic(err)
}
}
I'm trying to write a program in go that is similar to cron with the addition that jobs are given a max runtime and if a function exceeds this duration, the job should exit. Here is my my whole code:
package main
import (
"fmt"
"log"
"sync"
"time"
)
type Job struct {
ID string
MaxRuntime time.Duration
Frequency time.Duration
Function func()
}
func testFunc() {
log.Println("OPP11")
time.Sleep(7 * time.Second)
log.Println("OP222")
}
func New(ID, frequency, runtime string, implementation func()) Job {
r, err := time.ParseDuration(runtime)
if err != nil {
panic(err)
}
f, err := time.ParseDuration(frequency)
if err != nil {
panic(err)
}
j := Job{ID: ID, MaxRuntime: r, Frequency: f, Function: implementation}
log.Printf("Created job %#v with frequency %v and max runtime %v", ID, f, r)
return j
}
func (j Job) Run() {
for range time.Tick(j.Frequency) {
start := time.Now()
log.Printf("Job %#v executing...", j.ID)
done := make(chan int)
//quit := make(chan int)
//var wg sync.WaitGroup
//wg.Add(1)
go func() {
j.Function()
done <- 0
}()
select {
case <-done:
elapsed := time.Since(start)
log.Printf("Job %#v completed in %v \n", j.ID, elapsed)
case <-time.After(j.MaxRuntime):
log.Printf("Job %#v halted after %v", j.ID, j.MaxRuntime)
// here should exit the above goroutine
}
}
}
func main() {
// create a new job given its name, frequency, max runtime
// and the function it should run
testJob := New("my-first-job", "3s", "5s", func() {
testFunc()
})
testJob.Run()
}
What I'm trying to do is that in the second case in the select of the Run() function, it should exit the goroutine which is running the function. I tried to do this by wrapping the function in a for loop with a select statement which listens on a quit channel like this:
go func() {
for {
select {
case <-quit:
fmt.Println("quiting goroutine")
return
default:
j.Function()
done <- 0
}
}
}()
And then having quit <- 1 in the Run() function, but that doesnt seem to be doing anything. Is there a better of doing this?
As explained in the comments, the whole problem is that you want to cancel the execution of a function (j.Function) that isn't cancellable.
There's no way to "kill a goroutine". Goroutines work in a cooperative fashion. If you want to be able to "kill it", you need to ensure that the function running in that Goroutine has a mechanism for you to signal that it should stop what it's doing and return, letting the Goroutine that was running it finally terminate.
The standard way of indicating that a function is cancellable is by having it take a context.Context as its first param:
type Job struct {
// ...
Function func(context.Context)
}
Then you create the context and pass it to the j.Function. Since your cancellation logic is simply based on a timeout, there's no need to write all that select ... case <-time.After(...), as that is provided as built-in functionality with a context.Context:
func (j Job) Run() {
for range time.Tick(j.Frequency) {
go j.ExecuteOnce()
}
}
func (j Job) ExecuteOnce() {
log.Printf("Job %#v executing...", j.ID)
ctx, cancel := context.WithTimeout(context.Background(), j.MaxRuntime)
defer cancel()
j.Function(ctx)
}
Now, to finish, you have to rewrite the functions that you're going to be passing to your job scheduler so that they take context.Context and, very importantly, that they use it properly and cancel whatever they're doing when the context is cancelled.
This means that if you're writing the code for those funcs and they will somehow block, you'll be responsible for writing stuff like:
select {
case <-ctx.Done():
return ctx.Err()
case ...your blocking case...:
}
If your funcs are invoking 3rd party code, then that code needs to be aware of context and cancellation, and you'll need to pass down the ctx your funcs receive.
I'm trying to make middleware for the Gin server to handle events during a single request because I need things like IP address from context and to not pass the whole context I'd rather pass my listener which makes my method not dependent on gin.Context.
I set up a server
func main() {
router := gin.New()
router.Use(gin.Recovery())
api := router.Group("api", middleware())
{
api.GET("test/:id", endpoint)
}
router.Run("localhost:8080")
}
and made a middleware
func middleware() gin.HandlerFunc {
return func(c *gin.Context) {
listener := make(chan int)
c.Set("EventListener", listener)
go func() {
select {
case <-listener:
fmt.Println("event called")
default:
fmt.Println("default")
}
}()
c.Next()
}
}
then I got an endpoint where I get my listener and then pass it to any function I want to
func endpoint(c *gin.Context) {
listener := c.MustGet("EventListener").(chan int)
id := c.Param("id")
idInt, err := strconv.ParseInt(id, 10, 64)
if err != nil {
fmt.Println(err)
return
}
doSomething(listener, int(idInt))
}
and doSomething function to show that no context is used later on
func doSomething(listener chan int, id int) {
if id == 1 {
fmt.Println("ERROR: cause event and exit")
listener <- int(id)
return
}
if id == 2 {
fmt.Println("WARN: cause event but continue")
listener <- int(id)
}
fmt.Println("OK: everything is fine")
}
And the way it works now is:
when you call GET http://localhost:8080/api/test/1 it will trigger event and exit
when you call GET http://localhost:8080/api/test/2 it will trigger event and keep working
when you call GET http://localhost:8080/api/test/3 it will not trigger event because all is ok.
So everything is working but only for one event along request. You can not call another one, because select already passed so my question is how to fix it and allow to trigger event multiple times.
I know I can make it like
for {
select {
case <-listener:
fmt.Println("event called")
}
}
but what would be stop condition of this loop?
I know there is something like c.Done() which is chan but have no idea how to use it in my case because I can make it like
for {
select {
case <-listener:
fmt.Println("event called")
case <-c.Done():
return
}
}
but how to pass that c.Done()? This goroutine does not stop.
I found out that there is also c.Request.Context().Done() which works now, stopping goroutine and allowing to handle multiple events
go func() {
for {
select {
case <-listener:
fmt.Println("event called")
case <-c.Request.Context().Done():
return
}
}
}()
Can you read a goroutine channel into nothing? where does this channel read go to in this statement?
go func() {
<-ctx.Done()
logger.Errorf("canceled: %v", ctx.Err())
}()
Addition:
Would this code be any different than if I used the blank identifier
go func() {
_ = <-ctx.Done()
logger.Errorf("canceled: %v", ctx.Err())
}()
Yes, you can do that.
It does not need to go anywhere.
Purpose of the Done channel usually is just to signal the done event, so the value is not relevant and can be ignored.
It is the same as when you call a function and don't assign the return values to variables.
Consider this:
func getInt() int {
return 1
}
func main() {
getInt() // does not "go anywhere"
}
See this playground showing those examples:
https://play.golang.org/p/CA8P7gYpok