gocron to add and remove task dynamically with parameter - go

I am using gocron in my current project and I had encounter a few situations that not in the document.
I test this code:
gocron.Every(3).Seconds().Do(taskWithParams,2,"world")
gocron.Every(2).Seconds().Do(taskWithParams,1, "hello")
gocron.Start()
time.Sleep(10 * time.Second)
gocron.Remove(taskWithParams)//<-- remove task
...
func taskWithParams(a int, b string) {
fmt.Println(a, b)
}
When I remove task(gocron.Remove(taskWithParams)), always gocron.Every(3).Seconds().Do(taskWithParams,2,"world") is removed. even I swap them:
gocron.Every(2).Seconds().Do(taskWithParams,1, "hello")
gocron.Every(3).Seconds().Do(taskWithParams,2,"world")
Is there a way for me to specifically point out which task I want to remove, since the remove() only allow 1 argument?
The document also have a scheduler:
s := gocron.NewScheduler()
s.Every(3).Seconds().Do(task)
<- s.Start()
When is the best use case for scheduler?
If we are done with scheduler, how to remove it from memory? do scheduler.Clear() does the job? or we have to have another way to clear them from the memory?

you can handle the removal logic by deduplicating the function handlers.
package main
import (
"fmt"
)
func main() {
fn1 := func() { taskWithParams(2, "world") }
gocron.Every(3).Seconds().Do(fn1)
fn2 := func() { taskWithParams(1, "hello") }
gocron.Every(2).Seconds().Do(fn2)
gocron.Start()
time.Sleep(10 * time.Second)
gocron.Remove(fn2)
}
func taskWithParams(a int, b string) {
fmt.Println(a, b)
}
Otherwise, the scheduler.Do method returns an instance of *Job that you can pass to scheduler.RemoveByReference.
package main
import (
"fmt"
)
func main() {
job, err := gocron.Every(3).Seconds().Do(taskWithParams, 2, "ww")
if err != nil {
panic(err)
}
gocron.Every(2).Seconds().Do(taskWithParams, 1, "hh")
gocron.Start()
time.Sleep(10 * time.Second)
gocron.RemoveByReference(job)
}
func taskWithParams(a int, b string) {
fmt.Println(a, b)
}

Related

viper dynamically loading config file has data race

I would like to dynamically load config file and not restart my Go app. I wrote the below files, which runs but has data race.
config.go
package main
import (
"github.com/fsnotify/fsnotify"
"github.com/spf13/viper"
"log"
"sync"
"time"
)
var (
reloadConfig = make(chan string)
reloadConfig2 = make(chan string)
viperLock1 sync.Mutex
viperLock2 sync.Mutex
)
func setUpConfig(file string, merge bool, v *viper.Viper) {
v.AddConfigPath("./")
v.SetConfigName(file)
v.SetConfigType("yml")
if merge {
err1 := v.MergeInConfig()
checkForFatalError("fatal error occurred while reading config file!", err1)
} else {
err := v.ReadInConfig()
checkForFatalError("fatal error occurred while reading config file!", err)
}
log.Println("Initial config value: ", v.GetString("env"))
}
func loadConfigDynamically(configChannel chan string, viperLock *sync.Mutex, vipe *viper.Viper) {
viperLock.Lock()
vipe.OnConfigChange(func(e fsnotify.Event) {
viperLock.Lock()
log.Println("config file changed", e.Name)
environment := vipe.GetString("env")
configChannel <- environment
viperLock.Unlock()
})
viperLock.Unlock()
vipe.WatchConfig()
}
func loadMultipleConfigsDynamically() {
go func() {
time.Sleep(time.Millisecond * 50)
vipe2 := viper.New()
setUpConfig("config_base", false, vipe2)
loadConfigDynamically(reloadConfig2, &viperLock2, vipe2)
time.Sleep(time.Millisecond * 50)
vipe1 := viper.New()
setUpConfig("config", false, vipe1)
loadConfigDynamically(reloadConfig, &viperLock1, vipe1)
}()
}
main.go
package main
import (
log "github.com/sirupsen/logrus"
"os"
"os/signal"
"syscall"
)
var reloadConfigNow = make(chan bool)
var reloadConfigAgain = make(chan bool)
var newConfigValue string
func main() {
loadMultipleConfigsDynamically()
go printUpdatedValueOnly()
go justAnotherGoroutine()
go yetAnotherGoroutine()
shutdownAppGracefully()
}
func printUpdatedValueOnly() {
for {
select {
case updatedValue := <-reloadConfig:
newConfigValue = updatedValue
log.Println("dynamically loaded config value: ", updatedValue)
reloadConfigNow <-true
reloadConfigAgain <-true
case updatedValue1 := <-reloadConfig2:
newConfigValue = updatedValue1
log.Println("dynamically loaded config value: ", updatedValue1)
reloadConfigNow <-true
reloadConfigAgain <-true
default:
}
}
}
func justAnotherGoroutine(){
existingConfigValue := ""
for {
select {
case <-reloadConfigNow:
existingConfigValue = newConfigValue
log.Println("justAnotherGoroutine: ", existingConfigValue)
default:
}
}
}
func yetAnotherGoroutine() {
existingConfigValue := ""
for {
select {
case <-reloadConfigAgain:
existingConfigValue = newConfigValue
log.Println("yetAnotherGoroutine: ", existingConfigValue)
default:
}
}
}
func checkForFatalError(errorMsg string, err error) {
if err != nil {
log.Fatal(errorMsg, err)
}
}
func shutdownAppGracefully() {
killSignal := make(chan os.Signal, 1)
signal.Notify(killSignal, syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP, syscall.SIGQUIT)
k := <-killSignal
log.Info("OS Interrupt Signal received, application is shutting down!")
logSystemInterruptType(k)
}
func logSystemInterruptType(osInterrupt os.Signal) {
switch osInterrupt {
case syscall.SIGHUP:
log.Info("SIGHUP")
case syscall.SIGINT:
log.Info("SIGINT")
case syscall.SIGTERM:
log.Info("SIGTERM")
case syscall.SIGQUIT:
log.Info("SIGQUIT")
default:
log.Info("Unknown OS Interrupt")
}
}
config.yml
env : "LOCAL"
config_base.yml
env : "dev15"
go.mod
module reload_config
go 1.16
require (
github.com/fsnotify/fsnotify v1.4.9
github.com/spf13/viper v1.8.1
)
I learned recently that viper is not thread safe and hence I need to wrap it with mutex. I tried to do the same. In config.go file, func loadConfigDynamically, where I set OnConfigChange is the data race for read. And in the same function at the same line is previous write data race. I run the above package with
go run -race reload_config
And change the value of env in the config.yml to test if the config file is loading dynamically.This data race only occurs for the very first time config reloading dynamically. For subsequent times, it works just fine.
You lock viperLock called vipe.WatchConfig() and set vipe.OnConfigChange with a function it is also locking viperLock.
Because you already called vipe.WatchConfig() and then it started to call vipe.OnConfigChange in separate go routine. it is also try to acquire the same lock. That's why there is a race condition.
Call vipe.WatchConfig() after setting the vipe.OnConfigChange and after release the lock.
It should be corrected as below.
func loadConfigDynamically() {
go func() {
time.Sleep(time.Second)
viperLock.Lock()
vipe.OnConfigChange(func(e fsnotify.Event) {
viperLock.Lock()
log.Println("config file changed", e.Name)
environment := vipe.GetString("env")
reloadConfig <- environment
viperLock.Unlock()
})
viperLock.Unlock()
vipe.WatchConfig() //this starting call vipe.OnConfigChange
}()
}
It could be that go thinks that a variable is being modified and accessed by two goroutines at the same time and that there is no lock on the modified and accessed places.
Something like the following example:
package main
import (
"time"
)
type Foo struct {
f func(string)
}
func (f *Foo) Watch() {
go func() {
for {
time.Sleep(time.Second * 2)
if f.f != nil {
f.f("hello world")
}
}
}()
}
func (f *Foo) SetF(fun func(string)) {
f.f = fun
}
func main() {
f := Foo{}
f.Watch()
f.SetF(func(s string) {
})
time.Sleep(time.Second * 5)
}
It has a data race. If I put the same lock on both the modified and read places there will be no data race:
package main
import (
"sync"
"time"
)
var lock sync.Mutex
type Foo struct {
f func(string)
}
func (f *Foo) Watch() {
go func() {
for {
time.Sleep(time.Second * 2)
lock.Lock() // read places
if f.f != nil {
f.f("hello world")
}
lock.Unlock()
}
}()
}
func (f *Foo) SetF(fun func(string)) {
f.f = fun
}
func main() {
f := Foo{}
f.Watch()
lock.Lock() // write places
f.SetF(func(s string) {
})
lock.Unlock()
time.Sleep(time.Second * 5)
}
Or to eliminate the possibility of two goroutines reading and writing at the same time would work fine:
func main() {
f := Foo{}
f.SetF(func(s string) {
})
f.Watch()
time.Sleep(time.Second * 5)
}

How to test that a function was called in a goroutine?

I'd like to make sure that we're starting a goroutine by calling a function with the right arguments.
For example:
func MyTest(t *testing.T) {
service.EXPECT().MyMockFunc(1)
service.MyFunc()
}
func MyFunc() {
go MyMockFunc(1)
}
When I run this test, it fails because (I believe) MyMockFunc only gets called after the test has already finished running.
Is there a way to test that I started a goroutine by calling a function with the right arguments?
Note: Ideally, I'd like to keep the arguments I pass to MyMockFunc as is (not add a channel arg for instance).
Using a channel and assuming you can fire the goroutine from the test:
package main
import (
"fmt"
"testing"
"time"
)
func MyMockFunc(n int) int {
fmt.Println("MyMockFunc is called")
time.Sleep(5 * time.Second)
return n + 1
}
func TestMyMockFunc(t *testing.T) {
result := make(chan int)
go func() {
result <- MyMockFunc(1)
}()
if <-result != 2 {
t.Fatalf("Expecting 2")
}
}

Passing a function with parameters to time.AfterFunc

time.AfterFunc() accepts a duration and a function to be executed when that duration has expired. But the function cannot be a function that accepts parameters.
For example: The following function cannot be passed:
func Foo (b *Bar) {}
Although, it is possible to initialize a new function that calls the above one and then pass it:
f := func() {
Foo(somebar)
}
timer := time.AfterFunc(1*time.Second, f)
Should this really be done this way?
Why does time.AfterFunc not accept any functions that accept parameters?
Do there exist other/better ways to do this?
Create a function from the argument, return it.
package main
import (
"fmt"
"time"
)
func foo(bar string) {
fmt.Printf("in foo('%s')\n", bar)
}
func newFunc(bar string) func() {
fmt.Printf("creating func with '%s'\n", bar)
return func() {
foo(bar)
}
}
func main() {
somebar := "Here we go!"
f := newFunc(somebar)
_ = time.AfterFunc(1*time.Second, f)
time.Sleep(2 * time.Second)
}
https://play.golang.com/p/lWgeHvPLg9N
Anonymous function helps you to send functions with parameters to AfterFunc.
package main
import (
"fmt"
"time"
)
func foo(bar string) {
fmt.Printf("in foo('%s')\n", bar)
}
func main() {
somebar := "Here we go!"
time.AfterFunc(1*time.Second, func(){foo(somebar)})
time.Sleep(2 * time.Second)
}
https://play.golang.com/p/sdpiBtBWt_s

Reading from map with locks doesn't return value via channel

I tried to implement a locking version of reading/writing from a map in golang, but it doesn't return the desired result.
package main
import (
"sync"
"fmt"
)
var m = map[int]string{}
var lock = sync.RWMutex{}
func StoreUrl(id int, url string) {
for {
lock.Lock()
defer lock.Unlock()
m[id] = url
}
}
func LoadUrl(id int, ch chan string) {
for {
lock.RLock()
defer lock.RUnlock()
r := m[id]
ch <- r
}
}
func main() {
go StoreUrl(125, "www.google.com")
chb := make(chan string)
go LoadUrl(125, chb);
C := <-chb
fmt.Println("Result:", C)
}
The output is:
Result:
Meaning the value is not returned via the channel, which I don't get. Without the locking/goroutines it seems to work fine. What did I do wrong?
The code can also be found here:
https://play.golang.org/p/-WmRcMty5B
Infinite loops without sleep or some kind of IO are always bad idea.
In your code if you put a print statement at the start of StoreUrl, you will find that it never gets printed i.e the go routine was never started, the go call is setting putting the info about this new go routine in some run queue of the go scheduler but the scheduler hasn't ran yet to schedule that task. How do you run the scheduler? Do sleep/IO/channel reading/writing.
Another problem is that your infinite loop is taking lock and trying to take the lock again, which will cause it to deadlock. Defer only run after function exit and that function will never exit because of infinite loop.
Below is modified code that uses sleep to make sure every execution thread gets time to do its job.
package main
import (
"sync"
"fmt"
"time"
)
var m = map[int]string{}
var lock = sync.RWMutex{}
func StoreUrl(id int, url string) {
for {
lock.Lock()
m[id] = url
lock.Unlock()
time.Sleep(1)
}
}
func LoadUrl(id int, ch chan string) {
for {
lock.RLock()
r := m[id]
lock.RUnlock()
ch <- r
}
}
func main() {
go StoreUrl(125, "www.google.com")
time.Sleep(1)
chb := make(chan string)
go LoadUrl(125, chb);
C := <-chb
fmt.Println("Result:", C)
}
Edit: As #Jaun mentioned in the comment, you can also use runtime.Gosched() instead of sleep.
Usage of defer incorrect, defer execute at end of function, not for statement.
func StoreUrl(id int, url string) {
for {
func() {
lock.Lock()
defer lock.Unlock()
m[id] = url
}()
}
}
or
func StoreUrl(id int, url string) {
for {
lock.Lock()
m[id] = url
lock.Unlock()
}
}
We can't control the order of go routine, so add time.Sleep() to control the order.
code here:
https://play.golang.org/p/Bu8Lo46SA2

do repetitive tasks at intervals in golang using time.AfterFunc, just a sample

I just want to do repetitive background tasks in Go, using time.AfterFunc,But seems something wrong with the logic.
The out put just:
interval call
interval call
But at least 5 times to call the function if all things went normal.
package main
import (
"fmt"
"time"
"os"
"os/signal"
)
type Timer struct {
Queue chan *TimeCall
}
func NewTimer(l int) *Timer {
timer := new(Timer)
timer.Queue = make(chan *TimeCall,l)
return timer
}
type TimeCall struct {
timer *time.Timer
callback func()
}
func (this *TimeCall) CallBack() {
defer func() { recover() }()
if this.callback != nil {
this.callback()
}
}
func (this *Timer) AfterFunc(d time.Duration, callback func()) *TimeCall {
call := new(TimeCall)
call.callback = callback
call.timer = time.AfterFunc(d, func() {
this.Queue <- call
})
return call
}
type PipeService struct {
TimeCall *Timer
}
func (this *PipeService) AfterFunc(delay time.Duration, callback func()) *TimeCall {
return this.TimeCall.AfterFunc(delay, callback)
}
func (this *PipeService) IntervalCall(interval time.Duration, callback func()) {
this.TimeCall.AfterFunc(interval,func(){
if callback != nil {
callback()
}
this.AfterFunc(interval,callback)
})
}
func (this *PipeService) Run(closeSig chan bool) {
for {
select {
case <-closeSig:
return
case call := <-this.TimeCall.Queue:
call.CallBack()
}
}
}
func main() {
var closeChan chan bool
InsPipeService := &PipeService{TimeCall: NewTimer(10)}
InsPipeService.IntervalCall(2*time.Second,func(){
fmt.Println("interval call")
})
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, os.Kill)
go func(){
InsPipeService.Run(closeChan)
}()
time.Sleep(10*time.Second)
}
Run Code
time.AfterFunc() returns a *time.Timer, quoting form its doc:
The Timer type represents a single event. When the Timer expires, the current time will be sent on C, unless the Timer was created by AfterFunc.
The time.Timer returned by time.AfterFunc() does not repeat, so what you see is perfectly normal: in your PipeService.IntervalCall() you execute the callback immediately, and it gets executed after the timeout.
Also note that you pass 2 as interval for the PipeService.IntervalCall() method. This interval parameter is of type time.Duraion. So when you pass 2, that won't be 2 seconds (but actually 2 nanoseconds). You should pass a value constructed from constants from the time package like:
InsPipeService.IntervalCall(2 * time.Second, func(){
fmt.Println("interval call")
})
If you want repetition, use time.Ticker. For example the following code prints a message in every 2 seconds:
t := time.NewTicker(2 * time.Second)
for now := range t.C {
fmt.Println("tick", now)
}
Or simply if you don't need the Ticker and you don't want to shut it down:
c := time.Tick(2 * time.Second)
for now := range c {
fmt.Println("tick", now)
}
set time interval then call Start it will run user Job on each time intervals. set Enabled to false to stop it.
My Sample:
package main
import (
"fmt"
"sync"
"time"
)
type IntervalTimer struct {
Interval time.Duration
Enabled bool
Job func()
Wg sync.WaitGroup
}
func (it *IntervalTimer) isr() {
if it.Enabled {
it.Job()
time.AfterFunc(it.Interval, it.isr)
} else {
it.Wg.Done()
}
}
//trigger
func (it *IntervalTimer) Start() {
if it.Enabled {
it.Wg.Add(1)
time.AfterFunc(it.Interval, it.isr)
}
}
// user code:
var n int = 5
var it *IntervalTimer
func uerTask() {
fmt.Println(n, time.Now()) // do user job ...
n--
if n == 0 {
it.Enabled = false
}
}
func main() {
it = &IntervalTimer{Interval: 500 * time.Millisecond, Enabled: true, Job: uerTask}
it.Start()
//do some job ...
it.Wg.Wait()
fmt.Println("Bye")
}

Resources