I'm writing telegram-bot using Go (go-telegram-bot-api). It constantly parses a website and notifies about changes (every minute). Today I added simple keyboard and now can't make them work together.
The problem is that webpage parsing is in the infinite loop and program when it enters the loop ignores updates coming from the "client".
I've tried putting everything in a single loop, changing orders, etc. Maybe there is another or proper way of doing this?
Sample code:
u := tgbotapi.NewUpdate(0)
u.Timeout = 60
updates, err := bot.GetUpdatesChan(u)
for update := range updates {
if update.Message != nil {
switch update.Message.Text {
case "Show Keyboard":
Keyboard is sent
case "OptionsForParsing":
options.applied
case "StartParsing":
search bool = true
case "StopParsing":
search bool = false
}
}
if search == true{
for{
time.Sleep(1 * time.Minute)
AreThereChanges?()
if yes{
msg := tgbotapi.NewMessage(update.Message.Chat.ID, "help please")
bot.Send(msg)
}
}}
}
If I understood everything right you should start parsing loop inside of separated goroutine so updates cycle will keep working.
Try to do something like this:
// you can use cancel to stop running parsing goroutine
ctx, cancel := context.WithCancel(context.Background())
go func() {
ticker := time.NewTicker(5 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
AreThereChanges?()
if yes {
msg := tgbotapi.NewMessage(update.Message.Chat.ID, "help please")
bot.Send(msg)
}
}
}
}()
Writing from the top of my brain, so do not expect it to compile from the first attempt, but demonstrates the idea. Please, check out following links for more information regarding Ticker and contexts with cancellation: click, click.
Thanks to kimgrey I dug a bit into goroutines and channels. Infinite loop is in a separate goroutine and escaping is done through unbuffered channel (check "game changer" comments in the sample code). Now I can enter and escape infinite loop when needed. Here's how I solved my problem:
u := tgbotapi.NewUpdate(0)
u.Timeout = 60
updates, err := bot.GetUpdatesChan(u)
done := make(chan bool) //game changer
for update := range updates {
if update.Message != nil {
switch update.Message.Text {
case "Show Keyboard":
Keyboard is sent
case "OptionsForParsing":
options.applied
case "StartParsing":
search bool = true
case "StopParsing":
search bool = false
done<-true //game changer
}
go func() {
ticker := time.NewTicker(time.Minute)
defer ticker.Stop()
if search{
for {
select {
case <-done: //game changer
return
case <-ticker.C:
AnyChanges?()
}
}}
}()
}
}
Related
Consider a group of check works, each of which has independent logic, so they seem to be good to run concurrently, like:
type Work struct {
// ...
}
// This Check could be quite time-consuming
func (w *Work) Check() bool {
// return succeed or not
//...
}
func CheckAll(works []*Work) {
num := len(works)
results := make(chan bool, num)
for _, w := range works {
go func(w *Work) {
results <- w.Check()
}(w)
}
for i := 0; i < num; i++ {
if r := <-results; !r {
ReportFailed()
break;
}
}
}
func ReportFailed() {
// ...
}
When concerned about the results, if the logic is no matter which one work fails, we assert all works totally fail, the remaining values in the channel are useless. Let the remaining unfinished goroutines continue to run and send results to the channel is meaningless and waste, especially when w.Check() is quite time-consuming. The ideal effect is similar to:
for _, w := range works {
if !w.Check() {
ReportFailed()
break;
}
}
This only runs necessary check works then break, but is in sequential non-concurrent scenario.
So, is it possible to cancel these unfinished goroutines, or sending to channel?
Cancelling a (blocking) send
Your original question asked how to cancel a send operation. A send on a channel is basically "instant". A send on a channel blocks if the channel's buffer is full and there is no ready receiver.
You can "cancel" this send by using a select statement and a cancel channel which you close, e.g.:
cancel := make(chan struct{})
select {
case ch <- value:
case <- cancel:
}
Closing the cancel channel with close(cancel) on another goroutine will make the above select abandon the send on ch (if it's blocking).
But as said, the send is "instant" on a "ready" channel, and the send first evaluates the value to be sent:
results <- w.Check()
This first has to run w.Check(), and once it's done, its return value will be sent on results.
Cancelling a function call
So what you really need is to cancel the w.Check() method call. For that, the idiomatic way is to pass a context.Context value which you can cancel, and w.Check() itself must monitor and "obey" this cancellation request.
See Terminating function execution if a context is cancelled
Note that your function must support this explicitly. There is no implicit termination of function calls or goroutines, see cancel a blocking operation in Go.
So your Check() should look something like this:
// This Check could be quite time-consuming
func (w *Work) Check(ctx context.Context, workDuration time.Duration) bool {
// Do your thing and monitor the context!
select {
case <-ctx.Done():
return false
case <-time.After(workDuration): // Simulate work
return true
case <-time.After(2500 * time.Millisecond): // Simulate failure after 2.5 sec
return false
}
}
And CheckAll() may look like this:
func CheckAll(works []*Work) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
num := len(works)
results := make(chan bool, num)
wg := &sync.WaitGroup{}
for i, w := range works {
workDuration := time.Second * time.Duration(i)
wg.Add(1)
go func(w *Work) {
defer wg.Done()
result := w.Check(ctx, workDuration)
// You may check and return if context is cancelled
// so result is surely not sent, I omitted it here.
select {
case results <- result:
case <-ctx.Done():
return
}
}(w)
}
go func() {
wg.Wait()
close(results) // This allows the for range over results to terminate
}()
for result := range results {
fmt.Println("Result:", result)
if !result {
cancel()
break
}
}
}
Testing it:
CheckAll(make([]*Work, 10))
Output (try it on the Go Playground):
Result: true
Result: true
Result: true
Result: false
We get true printed 3 times (works that complete under 2.5 seconds), then the failure simulation kicks in, returns false, and terminates all other jobs.
Note that the sync.WaitGroup in the above example is not strictly needed as results has a buffer capable of holding all results, but in general it's still good practice (should you use a smaller buffer in the future).
See related: Close multiple goroutine if an error occurs in one in go
The short answer is: No.
You can not cancel or close any goroutine unless the goroutine itself reaches the return or end of its stack.
If you want to cancel something, the best approach is to pass a context.Context to them and listen to this context.Done() inside of the routine. Whenever context is canceled, you should return and the goroutine will automatically die after executing defers(if any).
package main
import "fmt"
type Work struct {
// ...
Name string
IsSuccess chan bool
}
// This Check could be quite time-consuming
func (w *Work) Check() {
// return succeed or not
//...
if len(w.Name) > 0 {
w.IsSuccess <- true
}else{
w.IsSuccess <- false
}
}
//堆排序
func main() {
works := make([]*Work,3)
works[0] = &Work{
Name: "",
IsSuccess: make(chan bool),
}
works[1] = &Work{
Name: "111",
IsSuccess: make(chan bool),
}
works[2] =&Work{
Name: "",
IsSuccess: make(chan bool),
}
for _,w := range works {
go w.Check()
}
for i,w := range works{
select {
case checkResult := <-w.IsSuccess :
fmt.Printf("index %d checkresult %t \n",i,checkResult)
}
}
}
enter image description here
I would like to pool for a token in on a timely base. The Token itself got also information about when it expires.
This should run forever until the user enters ctrl+c.
I tried the same with
span := timeLeft(*expDate)
timer := time.NewTimer(span).C
ticker := time.NewTicker(time.Second * 5).C
which also does not work (the application hangs after count down). So I decided to try it with <- time.After(...)
This is my code that does not work. You will see the count down but it never breaks on expiration.
This is is a small extract with the polling logic for simplicity sake in a main.go:
func refreshToken() (time.Time, error) {
//This should simulate a http request and returns the new target date for the next refresh
time.Sleep(2 * time.Second)
return time.Now().Add(10 * time.Second), nil
}
func timeLeft(d time.Time) time.Duration {
exactLeft := d.Sub(time.Now())
floorSeconds := math.Floor(exactLeft.Seconds())
return time.Duration(floorSeconds) * time.Second
}
func poller(expDate *time.Time) {
exp := timeLeft(*expDate)
done := make(chan bool)
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
for {
select {
// print time left on the screen
case <-time.After(3 * time.Second):
go func() {
fmt.Printf("\rNext Token refresh will be in: %v", timeLeft(*expDate))
}()
// mark as done when date is due
case <-time.After(exp):
fmt.Println("Refresh token now!")
done <- true
// exit app
case <-c:
os.Exit(0)
break
// exit function when done
case <-done:
break
}
}
}
func main() {
var expiration time.Time
expiration = time.Now().Add(10 * time.Second)
// loop and refresh token as long as the app does not exit
for {
poller(&expiration)
ex, err := refreshToken()
expiration = ex
if err != nil {
panic(err)
}
fmt.Println("next round poller")
}
}
I am also not sure if I need the done channel at all?
What is required to listen to two timers and call itself until someone hits ctrl+c?
Found a solution. While #ain was right wit the buffered done channel, it is not really required in the code now. It worked without it.
The trick did have the timeroutside of the for loop and the ticker within it. Reason is the time.Afteris a functhat return a new channel on every iteration. This seams perfectly fine for the ticker, but not for the timer.
With the following changes it worked =) ...
func poller(expDate *time.Time) {
exp := timeLeft(*expDate)
timer := time.After(exp)
fmt.Printf("Next Token refresh will be in: %v\n", exp)
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
for {
select {
// print time left on the screen
case <-time.After(3 * time.Second):
go func() {
fmt.Printf("\r ")
fmt.Printf("\r%v", timeLeft(*expDate))
}()
// mark as done when date is due
case <-timer:
fmt.Println("Refresh token now!")
return
// exit app
case <-c:
os.Exit(0)
break
}
}
}
I need to read data from the Go channel for a certain period of time (say 5 seconds). The select statement with timeout doesn't work for me, as I need to read as many values as available and stop exactly after 5 seconds. So far, I've come up with a solution using an extra time channel https://play.golang.org/p/yev9CcvzRIL
package main
import "time"
import "fmt"
func main() {
// I have no control over dataChan
dataChan := make(chan string)
// this is a stub to demonstrate some data coming from dataChan
go func() {
for {
dataChan <- "some data"
time.Sleep(time.Second)
}
}()
// the following is the code I'm asking about
timeChan := time.NewTimer(time.Second * 5).C
for {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-timeChan:
fmt.Println("Time's up!")
return
}
}
}
I'm wondering is there any better or more idiomatic way for solving this problem?
That's pretty much it. But if you don't need to stop or reset the timer, simply use time.After() instead of time.NewTimer(). time.After() is "equivalent to NewTimer(d).C".
afterCh := time.After(5 * time.Second)
for {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-afterCh:
fmt.Println("Time's up!")
return
}
}
Alternatively (to your liking), you may declare the after channel in the for statement like this:
for afterCh := time.After(5 * time.Second); ; {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-afterCh:
fmt.Println("Time's up!")
return
}
}
Also I know this is just an example, but always think how a goroutine you start will properly end, as the goroutine producing data in your case will never terminate.
Also don't forget that if multiple cases may be executed without blocking, one is chosen randomly. So if dataChan is ready to receive from "non-stop", there is no guarantee that the loop will terminate immediately after the timeout. In practice this is usually not a problem (starting with that even the timeout is not a guarantee, see details at Golang Timers with 0 length), but you should not forget about it in "mission-critial" applications. For details, see related questions:
force priority of go select statement
golang: How the select worked when multiple channel involved?
It looks like context with deadline would be good fit, something like
func main() {
dataChan := make(chan string)
ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(5*time.Second))
defer cancel()
go func(ctx context.Context) {
for {
select {
case dataChan <- "some data":
time.Sleep(time.Second)
case <-ctx.Done():
fmt.Println(ctx.Err())
close(dataChan)
return
}
}
}(ctx)
for d := range dataChan {
fmt.Println("Got:", d)
}
}
Consider this playground
messages := make(chan int, 1)
done := make(chan bool)
go func() {
for {
select {
case msg := <- messages:
fmt.Println("receiver one", msg)
case signal := <-done :
fmt.Println(signal)
return
default:
fmt.Println("no message received")
}
}
}()
go func() {
for {
select {
case msg := <- messages:
fmt.Println("receiver two", msg)
case signal := <-done :
fmt.Println(signal)
return
default:
fmt.Println("no message received")
}
}
}()
go func() {
for i := 0; i < 2; i++ {
messages<-i
}
done<-true
done<-true
done<-true
}()
<-done
I am trying to simulate communicate from one thread to two other thread, but it seems like the above code is running fine in my local but not in playground.
Is there a trick to make the program work?
You should remove the default clauses from the select statements.
Since your select statement is within a for loop and the default clause prevents it from blocking, the loop has to go through a lot of cycles before any message will be received.
Looks like Playground detects it and stops the program.
As #dev.bmax already mentioned, you use default in a wrong way.
Check this small article about using default keyword.
Also, try to add runtime.GOMAXPROCS(4) on the beginning of your program, run few times and check output to see, how it's working:
https://play.golang.org/p/TLDHNg6urB
Here's correct example for you:
https://play.golang.org/p/bc0TGdmx1Y
Note, that in this case you should use different channels for stopping additional and main goroutines.
This question already has answers here:
Timeout for WaitGroup.Wait()
(10 answers)
Closed 7 months ago.
I have come across a situation that i want to trace some goroutine to sync on a specific point, for example when all the urls are fetched. Then, we can put them all and show them in specific order.
I think this is the barrier comes in. It is in go with sync.WaitGroup. However, in real situation that we can not make sure that all the fetch operation will succeed in a short time. So, i want to introduce a timeout when wait for the fetch operations.
I am a newbie to Golang, so can someone give me some advice?
What i am looking for is like this:
wg := &sync.WaigGroup{}
select {
case <-wg.Wait():
// All done!
case <-time.After(500 * time.Millisecond):
// Hit timeout.
}
I know Wait do not support Channel.
If all you want is your neat select, you can easily convert blocking function to a channel by spawning a routine which calls a method and closes/sends on channel once done.
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
// All done!
case <-time.After(500 * time.Millisecond):
// Hit timeout.
}
Send your results to a buffered channel enough to take all results, without blocking, and read them in for-select loop in the main thread:
func work(msg string, d time.Duration, ret chan<- string) {
time.Sleep(d) // Work emulation.
select {
case ret <- msg:
default:
}
}
// ...
const N = 2
ch := make(chan string, N)
go work("printed", 100*time.Millisecond, ch)
go work("not printed", 1000*time.Millisecond, ch)
timeout := time.After(500 * time.Millisecond)
loop:
for received := 0; received < N; received++ {
select {
case msg := <-ch:
fmt.Println(msg)
case <-timeout:
fmt.Println("timeout!")
break loop
}
}
Playground: http://play.golang.org/p/PxeEEJo2dz.
See also: Go Concurrency Patterns: Timing out, moving on.
Another way to do it would be to monitor it internally, your question is limited but I'm going to assume you're starting your goroutines through a loop even if you're not you can refactor this to work for you but you could do one of these 2 examples, the first one will timeout each request to timeout individually and the second one will timeout the entire batch of requests and move on if too much time has passed
var wg sync.WaitGroup
wg.Add(1)
go func() {
success := make(chan struct{}, 1)
go func() {
// send your request and wait for a response
// pretend response was received
time.Sleep(5 * time.Second)
success <- struct{}{}
// goroutine will close gracefully after return
fmt.Println("Returned Gracefully")
}()
select {
case <-success:
break
case <-time.After(1 * time.Second):
break
}
wg.Done()
// everything should be garbage collected and no longer take up space
}()
wg.Wait()
// do whatever with what you got
fmt.Println("Done")
time.Sleep(10 * time.Second)
fmt.Println("Checking to make sure nothing throws errors after limbo goroutine is done")
Or if you just want a general easy way to timeout ALL requests you could do something like
var wg sync.WaitGroup
waiter := make(chan int)
wg.Add(1)
go func() {
success := make(chan struct{}, 1)
go func() {
// send your request and wait for a response
// pretend response was received
time.Sleep(5 * time.Second)
success <- struct{}{}
// goroutine will close gracefully after return
fmt.Println("Returned Gracefully")
}()
select {
case <-success:
break
case <-time.After(1 * time.Second):
// control the timeouts for each request individually to make sure that wg.Done gets called and will let the goroutine holding the .Wait close
break
}
wg.Done()
// everything should be garbage collected and no longer take up space
}()
completed := false
go func(completed *bool) {
// Unblock with either wait
wg.Wait()
if !*completed {
waiter <- 1
*completed = true
}
fmt.Println("Returned Two")
}(&completed)
go func(completed *bool) {
// wait however long
time.Sleep(time.Second * 5)
if !*completed {
waiter <- 1
*completed = true
}
fmt.Println("Returned One")
}(&completed)
// block until it either times out or .Wait stops blocking
<-waiter
// do whatever with what you got
fmt.Println("Done")
time.Sleep(10 * time.Second)
fmt.Println("Checking to make sure nothing throws errors after limbo goroutine is done")
This way your WaitGroup will stay in sync and you won't have any goroutines left in limbo
http://play.golang.org/p/g0J_qJ1BUT try it here you can change the variables around to see it work differently
Edit: I'm on mobile If anybody could fix the formatting that would be great thanks.
If you would like to avoid mixing concurrency logic with business logic, I wrote this library https://github.com/shomali11/parallelizer to help you with that. It encapsulates the concurrency logic so you do not have to worry about it.
So in your example:
package main
import (
"github.com/shomali11/parallelizer"
"fmt"
)
func main() {
urls := []string{ ... }
results = make([]*HttpResponse, len(urls)
options := &Options{ Timeout: time.Second }
group := parallelizer.NewGroup(options)
for index, url := range urls {
group.Add(func(index int, url string, results *[]*HttpResponse) {
return func () {
...
results[index] = &HttpResponse{url, response, err}
}
}(index, url, &results))
}
err := group.Run()
fmt.Println("Done")
fmt.Println(fmt.Sprintf("Results: %v", results))
fmt.Printf("Error: %v", err) // nil if it completed, err if timed out
}