Got a server that handles requests from clients. Each requests spins up a go routine in a for loop, how do I stop a previous go routine before a new one starts.
for {
select {
case discoveryRequest := <-discoveryRequestChannel:
for _, resourceNames := range discoveryRequest.ResourceNames {
accumulatedResourceNames = append(accumulatedResourceNames, resourceNames) //resources requested are accumulated
}
log.Infof("request resources: %v", accumulatedResourceNames)
secret, err := s.MakeSecretResponse(accumulatedResourceNames, nonce)
if err != nil {
log.Error(err) // Handling all the errors from the above layers. no need for context as they are provided in previous layers.
continue
}
if err := stream.Send(secret); err != nil {
log.Errorf("Error when sending stream to envoy %v ", err)
}
version = secret.VersionInfo
secretsRenewEndChannel <- true
close(secretsRenewEndChannel)
log.Info("kill switch")
go s.SecretsRenewer(version, accumulatedResourceNames, secretsRenewChannel, secretsRenewEndChannel)
isSecretRenewRunning = true
log.Info("set to true")
secretsRenewEndChannel = make(chan bool)
Your code isn't clear. Why do you need to select like this in a loop? What are you selecting amongst? You can just receive from the channel instead.
You may be looking for a worker pool here. Here's an example from gobyexample:
package main
import (
"fmt"
"time"
)
func worker(id int, jobs <-chan int, results chan<- int) {
for j := range jobs {
fmt.Println("worker", id, "started job", j)
time.Sleep(time.Second)
fmt.Println("worker", id, "finished job", j)
results <- j * 2
}
}
func main() {
const numJobs = 5
jobs := make(chan int, numJobs)
results := make(chan int, numJobs)
for w := 1; w <= 3; w++ {
go worker(w, jobs, results)
}
for j := 1; j <= numJobs; j++ {
jobs <- j
}
close(jobs)
for a := 1; a <= numJobs; a++ {
<-results
}
}
To adjust it to your problem, the worker would handle requests and the main goroutine would feed requests into a channel. You can easily control how many goroutines run concurrently with this approach.
Another pattern that may be useful in your case is rate limiting
Related
I made some edits from the gobyexample:
import (
"fmt"
"math/rand"
"time"
)
type DemoResult struct {
Name string
Rate int
}
func random(min, max int) int {
rand.Seed(time.Now().UTC().UnixNano())
return rand.Intn(max-min) + min
}
func worker(id int, jobs <-chan int, results chan<- DemoResult) {
for j := range jobs {
fmt.Println("worker", id, "started job", j)
time.Sleep(time.Second)
fmt.Println("worker", id, "finished job", j)
myrand := random(1, 4)
if myrand == 2 {
results <- DemoResult{Name: "succ", Rate: j}
}
// else {
// results <- DemoResult{Name: "failed", Rate: 999}
// }
}
}
func main() {
const numJobs = 5
jobs := make(chan int, numJobs)
results := make(chan DemoResult)
for w := 1; w <= 3; w++ {
go worker(w, jobs, results)
}
for j := 1; j <= numJobs; j++ {
jobs <- j
}
close(jobs)
for a := 1; a <= numJobs; a++ {
out := <-results
if out.Name == "succ" {
fmt.Printf("%v\n", out)
}
}
}
I commented the following code intentional to make it stuck forever:
// else {
// results <- DemoResult{Name: "failed", Rate: 999}
// }
It seems like we should make the result's length the same as jobs'. I was wondering if we could make it have different length?
Use a wait group to detect when the workers are done. Close the results channel when the workers are done. Receive results until the channel is closed.
func worker(wg *sync.WaitGroup, id int,
jobs <-chan int,
results chan<- DemoResult) {
// Decrement wait group counter on return from
// function.
defer wg.Done()
⋮
}
func main() {
⋮
// Declare wait group and increment counter for
// each worker.
var wg sync.WaitGroup
for w := 1; w <= 3; w++ {
wg.Add(1)
go worker(&wg, w, jobs, results)
}
⋮
// Wait for workers to decrement wait group
// counter to zero and close channel.
// Execute in goroutine so we can continue on
// to receiving values from results in main.
go func() {
wg.Wait()
close(results)
}()
⋮
// Loop until results is closed.
for out := range results {
⋮
}
}
https://go.dev/play/p/FOQwybMl7tM
I was wondering if we could make it have different length?
Absolutely but you need some way of determining when you have reached the end of the results. This is the reason your example fails - currently the function assumes there will be numJobs (one result per job) results and waits for that many.
An alternative would be to use the channels closure to indicate this i.e. (playground)
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
type DemoResult struct {
Name string
Rate int
}
func random(min, max int) int {
rand.Seed(time.Now().UTC().UnixNano())
return rand.Intn(max-min) + min
}
func worker(id int, jobs <-chan int, results chan<- DemoResult) {
for j := range jobs {
fmt.Println("worker", id, "started job", j)
time.Sleep(time.Second)
fmt.Println("worker", id, "finished job", j)
myrand := random(1, 4)
if myrand == 2 {
results <- DemoResult{Name: "succ", Rate: j}
} // else {
// results <- DemoResult{Name: "failed", Rate: 999}
//}
}
}
func main() {
const numWorkers = 3
const numJobs = 5
jobs := make(chan int, numJobs)
results := make(chan DemoResult)
var wg sync.WaitGroup
wg.Add(numWorkers)
for w := 1; w <= numWorkers; w++ {
go func() {
worker(w, jobs, results)
wg.Done()
}()
}
go func() {
wg.Wait() // Wait for go routines to complete then close results channel
close(results)
}()
for j := 1; j <= numJobs; j++ {
jobs <- j
}
close(jobs)
for out := range results {
if out.Name == "succ" {
fmt.Printf("%v\n", out)
}
}
}
I have a worker pool listening on a jobs channel, and responding on a results channel.
The jobs producer must run on a fixed ticker interval. Results must be flushed before reading just enough new jobs to fill up the buffer. It's critical to flush results, and read new jobs, in batches.
See example code below, run it on the playground here.
Is it possible to rewrite this without an atomic counter for keeping track of inflight jobs?
// Worker pool with buffered jobs and fixed polling interval
package main
import (
"fmt"
"math/rand"
"os"
"os/signal"
"strings"
"sync"
"sync/atomic"
"syscall"
"time"
)
func main() {
rand.Seed(time.Now().UnixNano())
// buf is the size of the jobs buffer
buf := 5
// workers is the number of workers to start
workers := 3
// jobs chan for workers
jobs := make(chan int, buf)
// results chan for workers
results := make(chan int, buf*2)
// jobID is incremented for each job sent on the jobs chan
var jobID int
// inflight is a count of the items in the jobs chan buffer
var inflight uint64
// pollInterval for jobs producer
pollInterval := 500 * time.Millisecond
// pollDone chan to stop polling
pollDone := make(chan bool)
// jobMultiplier on pollInterval for random job processing times
jobMultiplier := 5
// done chan to exit program
done := make(chan bool)
// Start workers
wg := sync.WaitGroup{}
for n := 0; n < workers; n++ {
wg.Add(1)
go (func(n int) {
defer wg.Done()
for {
// Receive from channel or block
jobID, more := <-jobs
if more {
// To subtract a signed positive constant value...
// https://golang.org/pkg/sync/atomic/#AddUint64
c := atomic.AddUint64(&inflight, ^uint64(0))
fmt.Println(
fmt.Sprintf("worker %v processing %v - %v jobs left",
n, jobID, c))
// Processing the job...
m := rand.Intn(jobMultiplier)
time.Sleep(time.Duration(m) * pollInterval)
results <- jobID
} else {
fmt.Println(fmt.Sprintf("worker %v exited", n))
return
}
}
})(n)
}
// Signal to exit
sig := make(chan os.Signal, 1)
signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM)
fmt.Println("ctrl+c to exit")
go (func() {
ticker := time.NewTicker(pollInterval)
r := make([]string, 0)
flushResults := func() {
fmt.Println(
fmt.Sprintf("===> results: %v", strings.Join(r, ",")))
r = make([]string, 0)
}
for {
select {
case <-ticker.C:
flushResults()
// Fetch jobs
c := atomic.LoadUint64(&inflight)
d := uint64(buf) - c
for i := 0; i < int(d); i++ {
jobID++
jobs <- jobID
atomic.AddUint64(&inflight, 1)
}
fmt.Println(fmt.Sprintf("===> send %v jobs", d))
case jobID := <-results:
r = append(r, fmt.Sprintf("%v", jobID))
case <-pollDone:
// Stop polling for new jobs
ticker.Stop()
// Close jobs channel to stop workers
close(jobs)
// Wait for workers to exit
wg.Wait()
close(results)
// Flush remaining results
for {
jobID, more := <-results
if more {
r = append(r, fmt.Sprintf("%v", jobID))
} else {
break
}
}
flushResults()
// Done!
done <- true
return
}
}
})()
// Wait for exit signal
<-sig
fmt.Println("---------| EXIT |---------")
pollDone <- true
<-done
fmt.Println("...done")
}
Here is a channel-based version of your code, functionally equivalent to the intent of the example above. The key points is that we're not using any atomic values to vary the logic of the code, because that offers no synchronization between the goroutines. All interactions between the goroutines are synchronized using channels, sync.WaitGroup, or context.Context. There are probably better ways to solve the problem at hand, but this demonstrates that there are no atomics necessary to coordinate the queue and workers.
The only value that is still left uncoordinated between goroutines here is the use of len(jobs) in the log output. Whether it makes sense to use it or not is up to you, as its value is meaningless in a concurrent world, but it's safe because it's synchronized for concurrent use and there is no logic based on the value.
buf := 5
workers := 3
jobs := make(chan int, buf)
// results buffer must always be larger than workers + buf to prevent deadlock
results := make(chan int, buf*2)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Start workers
var wg sync.WaitGroup
for n := 0; n < workers; n++ {
wg.Add(1)
go func(n int) {
defer wg.Done()
for jobID := range jobs {
fmt.Printf("worker %v processing %v - %v jobs left\n", n, jobID, len(jobs))
time.Sleep(time.Duration(rand.Intn(5)) * pollInterval)
results <- jobID
}
fmt.Printf("worker %v exited", n)
}(n)
}
var done sync.WaitGroup
done.Add(1)
go func() {
defer done.Done()
ticker := time.NewTicker(pollInterval)
r := make([]string, 0)
flushResults := func() {
fmt.Printf("===> results: %v\n", strings.Join(r, ","))
r = r[:0]
}
for {
select {
case <-ticker.C:
flushResults()
// send max buf jobs, or fill the queue
for i := 0; i < buf; i++ {
jobID++
select {
case jobs <- jobID:
continue
}
break
}
fmt.Printf("===> send %v jobs\n", i)
case jobID := <-results:
r = append(r, fmt.Sprintf("%v", jobID))
case <-ctx.Done():
// Close jobs channel to stop workers
close(jobs)
// Wait for workers to exit
wg.Wait()
// we can close results for easy iteration because we know
// there are no more workers.
close(results)
// Flush remaining results
for jobID := range results {
r = append(r, fmt.Sprintf("%v", jobID))
}
flushResults()
return
}
}
}()
// _Closing_ a channel indicates that no more values
// will be sent on it. This can be useful to communicate
// completion to the channel's receivers.
package main
import "fmt"
// In this example we'll use a `jobs` channel to
// communicate work to be done from the `main()` goroutine
// to a worker goroutine. When we have no more jobs for
// the worker we'll `close` the `jobs` channel.
func main() {
jobs := make(chan int, 5)
done := make(chan bool)
// Here's the worker goroutine. It repeatedly receives
// from `jobs` with `j, more := <-jobs`. In this
// special 2-value form of receive, the `more` value
// will be `false` if `jobs` has been `close`d and all
// values in the channel have already been received.
// We use this to notify on `done` when we've worked
// all our jobs.
for i := 1; i <= 3; i++ {
go func() {
for {
j, more := <-jobs
if more {
fmt.Println("received job", j)
} else {
fmt.Println("received all jobs")
done <- true
return
}
}
}()
}
// This sends 3 jobs to the worker over the `jobs`
// channel, then closes it.
j := 0
for {
j++
jobs <- j
fmt.Println("sent job", j)
}
close(jobs)
fmt.Println("sent all jobs")
// We await the worker using the
// [synchronization](channel-synchronization) approach
// we saw earlier.
<-done
}
https://play.golang.org/p/x28R_g8ftS
What I'm trying to do is get all the responses from a paginated url endpoint. jobs is a channel storing the page number. I have a function in if more{} checking for empty reponse and I have
done <- true
return
I thought this would close the go routine.
But, the page generator for{j++; jobs <- j} is causing it to get stuck in a loop. Any idea how this can be resolved?
By definition a for loop without conditions is an infinite loop. Unless you put some logic to break this infinite loop, you'll never get out of it.
In your playground your comment implies that you want to send 3 jobs. You should change your for loop accordingly:
for j := 0; j < 3; j++ {
jobs <- j
fmt.Println("sent job", j)
}
This is a simplified version of a worker.. Its not very useful for production level traffic, but should serve as a simple example, there are tons of them :-)
package main
import (
"log"
"sync"
)
type worker struct {
jobs chan int
wg *sync.WaitGroup
}
func main() {
w := worker{
jobs: make(chan int, 5), // I only want to work on 5 jobs at any given time
wg: new(sync.WaitGroup),
}
for i := 0; i < 3; i++ {
w.wg.Add(1)
go func(i int) {
defer w.wg.Done()
w.jobs <- i
}(i)
}
// wait in the background so that i can move to line 34 and start consuming my job queue
go func() {
w.wg.Wait()
close(w.jobs)
}()
for job := range w.jobs {
log.Println("Got job, I should do something with it", job)
}
}
This was I was looking for. I have a number generator in an infinite while loop. And the program exits on some condition, in this example, it is on the j value, but it can also be something else.
https://play.golang.org/p/Ud4etTjrmx
package main
import "fmt"
func jobs(job chan int) {
i := 1
for {
job <- i
i++
}
}
func main() {
jobsChan := make(chan int, 5)
done := false
j := 0
go jobs(jobsChan)
for !done {
j = <-jobsChan
if j < 20 {
fmt.Printf("job %d\n", j)
} else {
done = true
}
}
}
I wonder what would be idiomatic way to do as following.
I have N slow API queries, and one database connection, I want to have a buffered channel, where responses will come, and one database transaction which I will use to write data.
I could only come up with semaphore thing as following makeup example:
func myFunc(){
//10 concurrent API calls
sem := make(chan bool, 10)
//A concurrent safe map as buffer
var myMap MyConcurrentMap
for i:=0;i<N;i++{
sem<-true
go func(i int){
defer func(){<-sem}()
resp:=slowAPICall(fmt.Sprintf("http://slow-api.me?%d",i))
myMap.Put(resp)
}(i)
}
for j=0;j<cap(sem);j++{
sem<-true
}
tx,_ := db.Begin()
for data:=range myMap{
tx.Exec("Insert data into database")
}
tx.Commit()
}
I am nearly sure there is simpler, cleaner and more proper solution, but it is seems complicated to grasp for me.
EDIT:
Well, I come with following solution, this way I do not need the buffer map, so once data comes to resp channel the data is printed or can be used to insert into a database, it works, I am still not sure if everything OK, at last there are no race.
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
//Gloab waitGroup
var wg sync.WaitGroup
func init() {
//just for fun sake, make rand seeded
rand.Seed(time.Now().UnixNano())
}
//Emulate a slow API call
func verySlowAPI(id int) int {
n := rand.Intn(5)
time.Sleep(time.Duration(n) * time.Second)
return n
}
func main() {
//Amount of tasks
N := 100
//Concurrency level
concur := 10
//Channel for tasks
tasks := make(chan int, N)
//Channel for responses
resp := make(chan int, 10)
//10 concurrent groutinezs
wg.Add(concur)
for i := 1; i <= concur; i++ {
go worker(tasks, resp)
}
//Add tasks
for i := 0; i < N; i++ {
tasks <- i
}
//Collect data from goroutiens
for i := 0; i < N; i++ {
fmt.Printf("%d\n", <-resp)
}
//close the tasks channel
close(tasks)
//wait till finish
wg.Wait()
}
func worker(task chan int, resp chan<- int) {
defer wg.Done()
for {
task, ok := <-task
if !ok {
return
}
n := verySlowAPI(task)
resp <- n
}
}
There's no need to use channels for a semaphore, sync.WaitGroup was made for waiting for a set of routines to complete.
If you're using the channel to limit throughput, you're better off with a worker pool, and using the channel to pass jobs to the workers:
type job struct {
i int
}
func myFunc(N int) {
// Adjust as needed for total number of tasks
work := make(chan job, 10)
// res being whatever type slowAPICall returns
results := make(chan res, 10)
resBuff := make([]res, 0, N)
wg := new(sync.WaitGroup)
// 10 concurrent API calls
for i = 0; i < 10; i++ {
wg.Add(1)
go func() {
for j := range work {
resp := slowAPICall(fmt.Sprintf("http://slow-api.me?%d", j.i))
results <- resp
}
wg.Done()
}()
}
go func() {
for r := range results {
resBuff = append(resBuff, r)
}
}
for i = 0; i < N; i++ {
work <- job{i}
}
close(work)
wg.Wait()
close(results)
}
Maybe this will work for you. Now you can get rid of your concurrent map. Here is a code snippet:
func myFunc() {
//10 concurrent API calls
sem := make(chan bool, 10)
respCh := make(chan YOUR_RESP_TYPE, 10)
var responses []YOUR_RESP_TYPE
for i := 0; i < N; i++ {
sem <- true
go func(i int) {
defer func() {
<-sem
}()
resp := slowAPICall(fmt.Sprintf("http://slow-api.me?%d",i))
respCh <- resp
}(i)
}
respCollected := make(chan struct{})
go func() {
for i := 0; i < N; i++ {
responses = append(responses, <-respCh)
}
close(respCollected)
}()
<-respCollected
tx,_ := db.Begin()
for _, data := range responses {
tx.Exec("Insert data into database")
}
tx.Commit()
}
Than we need to use one more goroutine that will collect all responses in some slice or map from a response channel.
I am playing around with channels by making a workerpool of a 1000 workers. Currently I am getting the following error:
fatal error: all goroutines are asleep - deadlock!
Here is my code:
package main
import "fmt"
import "time"
func worker(id int, jobs <-chan int, results chan<- int) {
for j := range jobs {
fmt.Println("worker", id, "started job", j)
time.Sleep(time.Second)
fmt.Println("worker", id, "finished job", j)
results <- j * 2
}
}
func main() {
jobs := make(chan int, 100)
results := make(chan int, 100)
for w := 1; w <= 1000; w++ {
go worker(w, jobs, results)
}
for j := 1; j < 1000000; j++ {
jobs <- j
}
close(jobs)
fmt.Println("==========CLOSED==============")
for i:=0;i<len(results);i++ {
<-results
}
}
Why is this happening? I am still new to go and I am hoping to make sense of this.
While Thomas' answer is basically correct, I post my version which is IMO better Go and also works with unbuffered channels:
func main() {
jobs := make(chan int)
results := make(chan int)
var wg sync.WaitGroup
// you could init the WaitGroup's count here with one call but this is error
// prone - if you change the loop's size you could forget to change the
// WG's count. So call wg.Add in loop
//wg.Add(1000)
for w := 1; w <= 1000; w++ {
wg.Add(1)
go func() {
defer wg.Done()
worker(w, jobs, results)
}()
}
go func() {
for j := 1; j < 2000; j++ {
jobs <- j
}
close(jobs)
fmt.Println("==========CLOSED==============")
}()
// in this gorutine we wait until all "producer" routines are done
// then close the results channel so that the consumer loop stops
go func() {
wg.Wait()
close(results)
}()
for i := range results {
fmt.Print(i, " ")
}
fmt.Println("==========DONE==============")
}
The problem is that your channels are filling up. The main() routine tries to put all jobs into the jobs channel before reading any results. But the results channel only has space for 100 results before any write to the channel will block, so all the workers will eventually block waiting for space in this channel – space that will never come, because main() has not started reading from results yet.
To quickly fix this, you can either make jobs big enough to hold all jobs, so the main() function can continue to the reading phase; or you can make results big enough to hold all results, so the workers can output their results without blocking.
A nicer approach is to make another goroutine to fill up the jobs queue, so main() can go straight to reading results:
func main() {
jobs := make(chan int, 100)
results := make(chan int, 100)
for w := 1; w <= 1000; w++ {
go worker(w, jobs, results)
}
go func() {
for j := 1; j < 1000000; j++ {
jobs <- j
}
close(jobs)
fmt.Println("==========CLOSED==============")
}
for i := 1; i < 1000000; i++ {
<-results
}
}
Note that I had to change the final for loop to a fixed number of iterations, otherwise it might terminate before all the results have been read.
The following code:
for j := 1; j < 1000000; j++ {
jobs <- j
}
should run in a separate goroutine, since all the workers will block waiting for the main gorourine to receive on the results channel, while the main goroutine is stuck in the loop.
package main
import (
"fmt"
"sync"
"time"
)
func worker(id int, jobs <-chan int, results chan<- int, wg *sync.WaitGroup) {
defer wg.Done()
for j := range jobs {
fmt.Println("worker", id, "started job", j)
time.Sleep(time.Millisecond * time.Duration(10))
fmt.Println("worker", id, "finished job", j)
results <- j * 2
}
}
func main() {
jobs := make(chan int, 100)
results := make(chan int, 100)
wg := new(sync.WaitGroup)
wg.Add(1000)
for w := 1; w <= 1000; w++ {
go worker(w, jobs, results, wg)
}
go func() {
wg.Wait()
close(results)
}()
go func() {
for j := 1; j < 1000000; j++ {
jobs <- j
}
close(jobs)
}()
sum := 0
for v := range results {
sum += v
}
fmt.Println("==========CLOSED==============")
fmt.Println("sum", sum)
}