Selecting inside range over go channel - go

I'm following this post to parallelise my app. I need to tailor this code:
func sq(done <-chan struct{}, in <-chan int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for n := range in {
select {
case out <- n * n:
case <-done:
return
}
}
}()
return out
}
I don't fully understand the line case out <- n * n:. I can see it's saying that if there's a value for n, then square it and send it down the channel, but I don't understand why. Does select just take the first true case? Could it be rewritten:
for n := range in {
select {
case n:
out <- n * n
case <-done:
return
}
}
Anyway, I need to replace the line case out <- n * n: with a function call. I've changed it to the following:
out := make(chan structs.Ticket)
go func() {
defer close(out)
for url := range inputChannel {
select {
case url:
data, err := GetData(url)
fmt.Println("Got error: ", err)
out <- data
case <-done:
return
}
}
}()
return out
It looks like this will compile (I can't compile it yet), but because it's not simple to debug parallel code I wanted to check that using case url was the right way to select on a channel in a range. Is this right?
Update
OK I've removed the remaining issues with my code, and now when I try to compile I get the error messages:
url evaluated but not used
select case must be receive, send or assign recv

Being in a range or not doesn't have any impact on what select is doing here.
No, select doesn't take the first true expression... it doesn't take expressions at all. The only things that can appear as the cases of an expression are channel sends, channel receives, and assignments with channel receives on their right side.
select {
case out <- n * n:
case <-done:
return
}
says "if sending on out is possible (i.e. it has remaining capacity or an active reader), then send the value n * n to it and continue. If receiving from done is possible, return from the function. If both are possible, choose one at random and do it. If neither is possible, wait until one of them becomes possible." (See Select Statements in the spec).
If the value you want to send needs to be computed (and it's too complex to put on the right hand side of the channel send), simply do it before the select. The spec makes it clear that all of the expressions in send statements in a select are computed ahead of time anyway, so nothing is lost.

I don't fully understand the line case out <- n * n:. I can see it's saying that if there's a value for n, then square it and send it down the channel, but I don't understand why.
That's not correct. case out <- n * n checks to see if out is ready to read, and sends n * n to out if it is. Unless done is also ready.
select is used when you have multiple channels to talk to. Whichever channel is ready, it will do that case. If multiple channels are ready, it will select one at random.
select {
case out <- n * n:
case <-done:
return
}
}
This will select over out and done. If either is ready to proceed, ie. out is ready to read or there's something to read from done, it will pick one of those cases. The order is random, so it is possible to send more down out even if there's something to be read from done.
This pattern is used to shut down infinite goroutines. If you stop reading from its output channel, it won't do any more work, but it will hang around in memory. So by passing a value to done you can tell the goroutine to shut down.
UPDATE: In your original case, where the goroutine is looping over an input channel and sending output, done is an unnecessary complication. Once the input channel is closed, the function will return.
func sq(in <-chan int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for n := range in {
out <- n * n
}
}()
return out
}
func main() {
in := make(chan int)
out := sq(in)
for _,i := range []int{1,2,3,4} {
in <- i
fmt.Println(<-out)
}
// The `range` inside the goroutine from sq() will exit,
// and the goroutine will return.
close(in)
}
If it just spat out an ever increasing set of squares, then done would be necessary inside an infinite loop.
func sq(done chan bool) <-chan int {
out := make(chan int)
go func() {
defer close(out)
n := 0
for {
select {
case <-done:
return
case out<-n*n:
n++
}
}
}()
return out
}
func main() {
done := make(chan bool)
out := sq(done)
for range []int{1,2,3,4} {
fmt.Println(<-out)
}
// The switch in the goroutine will be able to read
// from done (out's buffer being already full) and return.
done <- true
}

Related

Is it possible to cancel unfinished goroutines?

Consider a group of check works, each of which has independent logic, so they seem to be good to run concurrently, like:
type Work struct {
// ...
}
// This Check could be quite time-consuming
func (w *Work) Check() bool {
// return succeed or not
//...
}
func CheckAll(works []*Work) {
num := len(works)
results := make(chan bool, num)
for _, w := range works {
go func(w *Work) {
results <- w.Check()
}(w)
}
for i := 0; i < num; i++ {
if r := <-results; !r {
ReportFailed()
break;
}
}
}
func ReportFailed() {
// ...
}
When concerned about the results, if the logic is no matter which one work fails, we assert all works totally fail, the remaining values in the channel are useless. Let the remaining unfinished goroutines continue to run and send results to the channel is meaningless and waste, especially when w.Check() is quite time-consuming. The ideal effect is similar to:
for _, w := range works {
if !w.Check() {
ReportFailed()
break;
}
}
This only runs necessary check works then break, but is in sequential non-concurrent scenario.
So, is it possible to cancel these unfinished goroutines, or sending to channel?
Cancelling a (blocking) send
Your original question asked how to cancel a send operation. A send on a channel is basically "instant". A send on a channel blocks if the channel's buffer is full and there is no ready receiver.
You can "cancel" this send by using a select statement and a cancel channel which you close, e.g.:
cancel := make(chan struct{})
select {
case ch <- value:
case <- cancel:
}
Closing the cancel channel with close(cancel) on another goroutine will make the above select abandon the send on ch (if it's blocking).
But as said, the send is "instant" on a "ready" channel, and the send first evaluates the value to be sent:
results <- w.Check()
This first has to run w.Check(), and once it's done, its return value will be sent on results.
Cancelling a function call
So what you really need is to cancel the w.Check() method call. For that, the idiomatic way is to pass a context.Context value which you can cancel, and w.Check() itself must monitor and "obey" this cancellation request.
See Terminating function execution if a context is cancelled
Note that your function must support this explicitly. There is no implicit termination of function calls or goroutines, see cancel a blocking operation in Go.
So your Check() should look something like this:
// This Check could be quite time-consuming
func (w *Work) Check(ctx context.Context, workDuration time.Duration) bool {
// Do your thing and monitor the context!
select {
case <-ctx.Done():
return false
case <-time.After(workDuration): // Simulate work
return true
case <-time.After(2500 * time.Millisecond): // Simulate failure after 2.5 sec
return false
}
}
And CheckAll() may look like this:
func CheckAll(works []*Work) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
num := len(works)
results := make(chan bool, num)
wg := &sync.WaitGroup{}
for i, w := range works {
workDuration := time.Second * time.Duration(i)
wg.Add(1)
go func(w *Work) {
defer wg.Done()
result := w.Check(ctx, workDuration)
// You may check and return if context is cancelled
// so result is surely not sent, I omitted it here.
select {
case results <- result:
case <-ctx.Done():
return
}
}(w)
}
go func() {
wg.Wait()
close(results) // This allows the for range over results to terminate
}()
for result := range results {
fmt.Println("Result:", result)
if !result {
cancel()
break
}
}
}
Testing it:
CheckAll(make([]*Work, 10))
Output (try it on the Go Playground):
Result: true
Result: true
Result: true
Result: false
We get true printed 3 times (works that complete under 2.5 seconds), then the failure simulation kicks in, returns false, and terminates all other jobs.
Note that the sync.WaitGroup in the above example is not strictly needed as results has a buffer capable of holding all results, but in general it's still good practice (should you use a smaller buffer in the future).
See related: Close multiple goroutine if an error occurs in one in go
The short answer is: No.
You can not cancel or close any goroutine unless the goroutine itself reaches the return or end of its stack.
If you want to cancel something, the best approach is to pass a context.Context to them and listen to this context.Done() inside of the routine. Whenever context is canceled, you should return and the goroutine will automatically die after executing defers(if any).
package main
import "fmt"
type Work struct {
// ...
Name string
IsSuccess chan bool
}
// This Check could be quite time-consuming
func (w *Work) Check() {
// return succeed or not
//...
if len(w.Name) > 0 {
w.IsSuccess <- true
}else{
w.IsSuccess <- false
}
}
//堆排序
func main() {
works := make([]*Work,3)
works[0] = &Work{
Name: "",
IsSuccess: make(chan bool),
}
works[1] = &Work{
Name: "111",
IsSuccess: make(chan bool),
}
works[2] =&Work{
Name: "",
IsSuccess: make(chan bool),
}
for _,w := range works {
go w.Check()
}
for i,w := range works{
select {
case checkResult := <-w.IsSuccess :
fmt.Printf("index %d checkresult %t \n",i,checkResult)
}
}
}
enter image description here

How to exit from main thread

func GoCountColumns(in chan []string, r chan Result, quit chan int) {
for {
select {
case data := <-in:
r <- countColumns(data) // some calculation function
case <-quit:
return // stop goroutine
}
}
}
func main() {
fmt.Println("Welcome to the csv Calculator")
file_path := os.Args[1]
fd, _ := os.Open(file_path)
reader := csv.NewReader(bufio.NewReader(fd))
var totalColumnsCount int64 = 0
var totallettersCount int64 = 0
linesCount := 0
numWorkers := 10000
rc := make(chan Result, numWorkers)
in := make(chan []string, numWorkers)
quit := make(chan int)
t1 := time.Now()
for i := 0; i < numWorkers; i++ {
go GoCountColumns(in, rc, quit)
}
//start worksers
go func() {
for {
record, err := reader.Read()
if err == io.EOF {
break
}
if err != nil {
log.Fatal(err)
}
if linesCount%1000000 == 0 {
fmt.Println("Adding to the channel")
}
in <- record
//data := countColumns(record)
linesCount++
//totalColumnsCount = totalColumnsCount + data.ColumnCount
//totallettersCount = totallettersCount + data.LettersCount
}
close(in)
}()
for i := 0; i < numWorkers; i++ {
quit <- 1 // quit goroutines from main
}
close(rc)
for i := 0; i < linesCount; i++ {
data := <-rc
totalColumnsCount = totalColumnsCount + data.ColumnCount
totallettersCount = totallettersCount + data.LettersCount
}
fmt.Printf("I counted %d lines\n", linesCount)
fmt.Printf("I counted %d columns\n", totalColumnsCount)
fmt.Printf("I counted %d letters\n", totallettersCount)
elapsed := time.Now().Sub(t1)
fmt.Printf("It took %f seconds\n", elapsed.Seconds())
}
My Hello World is a program that reads a csv file and passes it to a channel. Then the goroutines should consume from this channel.
My Problem is I have no idea how to detect from the main thread that all data was processed and I can exit my program.
on top of other answers.
Take (great) care that closing a channel should happen on the write call site, not the read call site. In GoCountColumns the r channel being written, the responsibility to close the channel are onto GoCountColumns function. Technical reasons are, it is the only actor knowing for sure that the channel will not being written anymore and thus is safe for close.
func GoCountColumns(in chan []string, r chan Result, quit chan int) {
defer close(r) // this line.
for {
select {
case data := <-in:
r <- countColumns(data) // some calculation function
case <-quit:
return // stop goroutine
}
}
}
The function parameters naming convention, if i might say, is to have the destination as first parameter, the source as second, and others parameters along. The GoCountColumns is preferably written:
func GoCountColumns(dst chan Result, src chan []string, quit chan int) {
defer close(dst)
for {
select {
case data := <-src:
dst <- countColumns(data) // some calculation function
case <-quit:
return // stop goroutine
}
}
}
You are calling quit right after the process started. Its illogical. This quit command is a force exit sequence, it should be called once an exit signal is detected, to force exit the current processing in best state possible, possibly all broken. In other words, you should be relying on the signal.Notify package to capture exit events, and notify your workers to quit. see https://golang.org/pkg/os/signal/#example_Notify
To write better parallel code, list at first the routines you need to manage the program lifetime, identify those you need to block onto to ensure the program has finished before exiting.
In your code, exists read, map. To ensure complete processing, the program main function must ensure that it captures a signal when map exits before exiting itself. Notice that the read function does not matter.
Then, you will also need the code required to capture an exit event from user input.
Overall, it appears we need to block onto two events to manage lifetime. Schematically,
func main(){
go read()
go map(mapDone)
go signal()
select {
case <-mapDone:
case <-sig:
}
}
This simple code is good to process or die. Indeed, when the user event is caught, the program exits immediately, without giving a chance to others routines to do something required upon stop.
To improve those behaviors, you need first a way to signal the program wants to leave to other routines, second, a way to wait for those routines to finish their stop sequence before leaving.
To signal exit event, or cancellation, you can make use of a context.Context, pass it around to the workers, make them listen to it.
Again, schematically,
func main(){
ctx,cancel := context.WithCancel(context.WithBackground())
go read(ctx)
go map(ctx,mapDone)
go signal()
select {
case <-mapDone:
case <-sig:
cancel()
}
}
(more onto read and map later)
To wait for completion, many things are possible, for as long as they are thread safe. Usually, a sync.WaitGroup is being used. Or, in cases like yours where there is only one routine to wait for, we can re use the current mapDone channel.
func main(){
ctx,cancel := context.WithCancel(context.WithBackground())
go read(ctx)
go map(ctx,mapDone)
go signal()
select {
case <-mapDone:
case <-sig:
cancel()
<-mapDone
}
}
That is simple and straight forward. But it is not totally correct. The last mapDone chan might block forever and make the program unstoppable. So you might implement a second signal handler, or a timeout.
Schematically, the timeout solution is
func main(){
ctx,cancel := context.WithCancel(context.WithBackground())
go read(ctx)
go map(ctx,mapDone)
go signal()
select {
case <-mapDone:
case <-sig:
cancel()
select {
case <-mapDone:
case <-time.After(time.Second):
}
}
}
You might also accumulate a signal handling and a timeout in the last select.
Finally, there are few things to tell about read and map context listening.
Starting with map, the implementation requires to read for context.Done channel regularly to detect cancellation.
It is the easy part, it requires to only update the select statement.
func GoCountColumns(ctx context.Context, dst chan Result, src chan []string) {
defer close(dst)
for {
select {
case <-ctx.Done():
<-time.After(time.Minute) // do something more useful.
return // quit. Notice the defer will be called.
case data := <-src:
dst <- countColumns(data) // some calculation function
}
}
}
Now the read part is bit more tricky as it is an IO it does not provide a selectable programming interface and listening to the context channel cancellation might seem contradictory. It is. As IOs are blocking, impossible to listen the context. And while reading from the context channel, impossible to read the IO. In your case, the solution requires to understand that your read loop is not relevant to your program lifetime (recall we only listen onto mapDone?), and that we can just ignore the context.
In other cases, if for example you wanted to restart at last byte read (so at every read, we increment an n, counting bytes, and we want to save that value upon stop). Then, a new routine is required to be started, and thus, multiple routines are to wait for completion. In such cases a sync.WaitGroup will be more appropriate.
Schematically,
func main(){
var wg sync.WaitGroup
processDone:=make(chan struct{})
ctx,cancel := context.WithCancel(context.WithBackground())
go read(ctx)
wg.Add(1)
go saveN(ctx,&wg)
wg.Add(1)
go map(ctx,&wg)
go signal()
go func(){
wg.Wait()
close(processDone)
}()
select {
case <-processDone:
case <-sig:
cancel()
select {
case <-processDone:
case <-time.After(time.Second):
}
}
}
In this last code, the waitgroup is being passed around. Routines are responsible to call for wg.Done(), when all routines are done, the processDone channel is closed, to signal the select.
func GoCountColumns(ctx context.Context, dst chan Result, src chan []string, wg *sync.WaitGroup) {
defer wg.Done()
defer close(dst)
for {
select {
case <-ctx.Done():
<-time.After(time.Minute) // do something more useful.
return // quit. Notice the defer will be called.
case data := <-src:
dst <- countColumns(data) // some calculation function
}
}
}
It is undecided which patterns is preferred, but you might also see waitgroup being managed at call sites only.
func main(){
var wg sync.WaitGroup
processDone:=make(chan struct{})
ctx,cancel := context.WithCancel(context.WithBackground())
go read(ctx)
wg.Add(1)
go func(){
defer wg.Done()
saveN(ctx)
}()
wg.Add(1)
go func(){
defer wg.Done()
map(ctx)
}()
go signal()
go func(){
wg.Wait()
close(processDone)
}()
select {
case <-processDone:
case <-sig:
cancel()
select {
case <-processDone:
case <-time.After(time.Second):
}
}
}
Beyond all of that and OP questions, you must always evaluate upfront the pertinence of parallel processing for a given task. There is no unique recipe, practice and measure your code performances. see pprof.
There is way too much going on in this code. You should restructure your code into short functions that serve specific purposes to make it possible for someone to help you out easily (and help yourself as well).
You should read the following Go article, which goes into concurrency patterns:
https://blog.golang.org/pipelines
There are multiple ways to make one go-routine wait on some other work to finish. The most common ways are with wait groups (example I have provided) or channels.
func processSomething(...) {
...
}
func main() {
workers := &sync.WaitGroup{}
for i := 0; i < numWorkers; i++ {
workers.Add(1) // you want to call this from the calling go-routine and before spawning the worker go-routine
go func() {
defer workers.Done() // you want to call this from the worker go-routine when the work is done (NOTE the defer, which ensures it is called no matter what)
processSomething(....) // your async processing
}()
}
// this will block until all workers have finished their work
workers.Wait()
}
You can use a channel to block main until completion of a goroutine.
package main
import (
"log"
"time"
)
func main() {
c := make(chan struct{})
go func() {
time.Sleep(3 * time.Second)
log.Println("bye")
close(c)
}()
// This blocks until the channel is closed by the routine
<-c
}
No need to write anything into the channel. Reading is blocked until data is read or, which we use here, the channel is closed.

Idiomatic way for reading from the channel for a certain time

I need to read data from the Go channel for a certain period of time (say 5 seconds). The select statement with timeout doesn't work for me, as I need to read as many values as available and stop exactly after 5 seconds. So far, I've come up with a solution using an extra time channel https://play.golang.org/p/yev9CcvzRIL
package main
import "time"
import "fmt"
func main() {
// I have no control over dataChan
dataChan := make(chan string)
// this is a stub to demonstrate some data coming from dataChan
go func() {
for {
dataChan <- "some data"
time.Sleep(time.Second)
}
}()
// the following is the code I'm asking about
timeChan := time.NewTimer(time.Second * 5).C
for {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-timeChan:
fmt.Println("Time's up!")
return
}
}
}
I'm wondering is there any better or more idiomatic way for solving this problem?
That's pretty much it. But if you don't need to stop or reset the timer, simply use time.After() instead of time.NewTimer(). time.After() is "equivalent to NewTimer(d).C".
afterCh := time.After(5 * time.Second)
for {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-afterCh:
fmt.Println("Time's up!")
return
}
}
Alternatively (to your liking), you may declare the after channel in the for statement like this:
for afterCh := time.After(5 * time.Second); ; {
select {
case d := <-dataChan:
fmt.Println("Got:", d)
case <-afterCh:
fmt.Println("Time's up!")
return
}
}
Also I know this is just an example, but always think how a goroutine you start will properly end, as the goroutine producing data in your case will never terminate.
Also don't forget that if multiple cases may be executed without blocking, one is chosen randomly. So if dataChan is ready to receive from "non-stop", there is no guarantee that the loop will terminate immediately after the timeout. In practice this is usually not a problem (starting with that even the timeout is not a guarantee, see details at Golang Timers with 0 length), but you should not forget about it in "mission-critial" applications. For details, see related questions:
force priority of go select statement
golang: How the select worked when multiple channel involved?
It looks like context with deadline would be good fit, something like
func main() {
dataChan := make(chan string)
ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(5*time.Second))
defer cancel()
go func(ctx context.Context) {
for {
select {
case dataChan <- "some data":
time.Sleep(time.Second)
case <-ctx.Done():
fmt.Println(ctx.Err())
close(dataChan)
return
}
}
}(ctx)
for d := range dataChan {
fmt.Println("Got:", d)
}
}

Go using timeouts with channels

I am using goroutines/channels to check if list of urls are reachable. Here is my code. This seems to always return true. Why is the timeout case not getting executed? The goal is to return false even if one of the urls is not reachable
import "fmt"
import "time"
func check(u string) bool {
time.Sleep(4 * time.Second)
return true
}
func IsReachable(urls []string) bool {
ch := make(chan bool, 1)
for _, url := range urls {
go func(u string) {
select {
case ch <- check(u):
case <-time.After(time.Second):
ch<-false
}
}(url)
}
return <-ch
}
func main() {
fmt.Println(IsReachable([]string{"url1"}))
}
check(u) will sleep in the current goroutine, i.e. the one that's running func. The select statement is only run properly once it returns, and by that time, both branches are runnable and the runtime can pick whichever one it pleases.
You can solve it by running check inside yet another goroutine:
package main
import "fmt"
import "time"
func check(u string, checked chan<- bool) {
time.Sleep(4 * time.Second)
checked <- true
}
func IsReachable(urls []string) bool {
ch := make(chan bool, 1)
for _, url := range urls {
go func(u string) {
checked := make(chan bool)
go check(u, checked)
select {
case ret := <-checked:
ch <- ret
case <-time.After(1 * time.Second):
ch <- false
}
}(url)
}
return <-ch
}
func main() {
fmt.Println(IsReachable([]string{"url1"}))
}
It seems you want to check reachability of a set of URLs, and return true if one of them is available. If the timeout is long compared to the time it takes to spin up a goroutine, you could simplify this by having just one timeout for all URLs together. But we need to make sure that the channel is large enough to hold the answers from all checks, or the ones that don't "win" will block forever:
package main
import "fmt"
import "time"
func check(u string, ch chan<- bool) {
time.Sleep(4 * time.Second)
ch <- true
}
func IsReachable(urls []string) bool {
ch := make(chan bool, len(urls))
for _, url := range urls {
go check(url, ch)
}
time.AfterFunc(time.Second, func() { ch <- false })
return <-ch
}
func main() {
fmt.Println(IsReachable([]string{"url1", "url2"}))
}
The reason this always returns true is you are calling check(u) within your select statement. You need to call it within a go routine and then use a select to either wait for the result or timeout.
In case you want to check the reachability of multiple URLs in parallel you need to restructure your code.
First create a function which checks the reachability of one URL:
func IsReachable(url string) bool {
ch := make(chan bool, 1)
go func() { ch <- check(url) }()
select {
case reachable := <-ch:
return reachable
case <-time.After(time.Second):
// call timed out
return false
}
}
Then call this function from a loop:
urls := []string{"url1", "url2", "url3"}
for _, url := range urls {
go func() { fmt.Println(IsReachable(url)) }()
}
Play
change the line
ch := make(chan bool, 1)
to
ch := make(chan bool)
You did open a asynchronous (= non blocking) channel, but you need a blocking channel to get it work.
The result of true being returned here is deterministic in this scenario, it's not a random one picked up by the runtime, because there's only true value available (however long it may take for it to become available!) being sent into the channel, the false result would never be available for the channel since the time.After() call statement would never get the chance to be executed in the first place!
In this select, the first executable line it sees is check(u) call, not the channel sending call in the first case branch, or any other call at all! And it's only after this first check(u) execution has returned here, would select branch cases get checked and called upon, by which point, the value of true is already to be pushed into the first branch case channel, so no channel blocking here to the select statement, the select can fulfil its purpose promptly here without needing to check its remaining branch cases!
so looks like it's the use of select here that wouldn't seem quite correct in this scenario.
the select branch cases are supposed to listen to channel sending and receiving values directly, or optionally with a default to escape the blocking when necessary.
so the fix is as some people pointed out here already, putting the long running task or process into a separate goroutine, and have it send result into channel,
and then in the main goroutine (or whichever other routine that needs that value off the channel), use the select branch cases to either listen on that specific channel for a value, or on the channel provided by the time.After(time.Second) call.
Basically, this line: case ch <- check(u) is correct in the sense of sending a value into a channel, but it's just not for its intended use (i.e. blocking this branch case), because the case channel<- is not being blocked there at all (the time check(u) spends on is all happening before the channel gets involved), since in a separate goroutine, aka, the main one: return <-ch, it's already ready to read that value whenever it gets pushed through. That is why time.After() call statement in the second case branch would never even get a chance to be evaluated, in the first instance!
see this example for a simple solution, ie. the correct use of a select in conjunction of separate goroutines: https://gobyexample.com/timeouts
In case it's useful, here's a generalised version of #Thomas 's answer, much simplified by #mh-cbon
func WithTimeout(delegate func() interface{}, timeout time.Duration) (ret interface{}, ok bool) {
ch := make(chan interface{}, 1) // buffered
go func() { ch <- delegate() }()
select {
case ret = <-ch:
return ret, true
case <-time.After(timeout):
}
return nil, false
}
Then you can call to 'timeout' any function
if value,ok := WithTimeout(myFunc, time.Second); ok {
// returned
} else {
// didn't return
}
Call like this to wait for a channel
if value,ok := WithTimeout(func()interface{}{return <- inbox}, time.Second); ok {
// returned
} else {
// didn't return
}
Like this to try sending
_,ok = WithTimeout(func()interface{}{outbox <- myValue; return nil}, time.Second)
if !ok{...

Do go channels preserve order when blocked?

I have a slice of channels that all receive the same message:
func broadcast(c <-chan string, chans []chan<- string) {
for msg := range c {
for _, ch := range chans {
ch <- msg
}
}
}
However, since each of the channels in chans are potentially being read at a different rate, I don't want to block the other channels when I get a slow consumer. I've solved this with goroutines:
func broadcast(c <-chan string, chans []chan<- string) {
for msg := range c {
for _, ch := range chans {
go func() { ch <- msg }()
}
}
}
However, the order of the messages that get passed to each channel is important. I looked to the spec to see if channels preserve order when blocked, and all I found was this:
If the capacity is greater than zero, the channel is asynchronous: communication operations succeed without blocking if the buffer is not full (sends) or not empty (receives), and elements are received in the order they are sent.
To me, if a write is blocked, then it is not "sent", but waiting to be sent. With that assumption, the above says nothing about order of sending when multiple goroutines are blocked on writing.
Are there any guarantees about the order of sends after a channel becomes unblocked?
No, there are no guarantees.
Even when the channel is not full, if two goroutines are started at about the same time to send to it, I don't think there is any guarantee that the goroutine that was started first would actually execute first. So you can't count on the messages arriving in order.
You can drop the message if the channel is full (and then set a flag to pause the client and send them a message that they're dropping messages or whatever).
Something along the lines of (untested):
type Client struct {
Name string
ch chan<-string
}
func broadcast(c <-chan string, chans []*Client) {
for msg := range c {
for _, ch := range chans {
select {
case ch.ch <- msg:
// all okay
default:
log.Printf("Channel was full sending '%s' to client %s", msg, ch.Name)
}
}
}
}
In this code, no guarantees.
The main problem with the given sample code lies not in the channel behavior, but rather in the numerous created goroutines. All the goroutines are "fired" inside the same imbricated loop without further synchronization, so even before they start to send messages, we simply don't know which ones will execute first.
However this rises a legitimate question in general : if we somehow garantee the order of several blocking send instructions, are we guaranteed to receive them in the same order?
The "happens-before" property of the sendings is difficult to create. I fear it is impossible because :
Anything can happen before the sending instruction : for example, other goroutines performing their own sendings or not
A goroutine being blocked in a sending cannot simultaneously manage other sorts of synchronization
For example, if I have 10 goroutines numbered 1 to 10, I have no way of letting them send their own number to the channel, concurrently, in the right order. All I can do is use various kinds of sequential tricks like doing the sorting in 1 single goroutine.
This is an addition to the already posted answers.
As practically everyone stated, that the problem is the order of execution of the goroutines,
you can easily coordinate goroutine execution using channels by passing around the number of the
goroutine you want to run:
func coordinated(coord chan int, num, max int, work func()) {
for {
n := <-coord
if n == num {
work()
coord <- (n+1) % max
} else {
coord <- n
}
}
}
coord := make(chan int)
go coordinated(coord, 0, 3, func() { println("0"); time.Sleep(1 * time.Second) })
go coordinated(coord, 1, 3, func() { println("1"); time.Sleep(1 * time.Second) })
go coordinated(coord, 2, 3, func() { println("2"); time.Sleep(1 * time.Second) })
coord <- 0
or by using a central goroutine which executes the workers in a ordered manner:
func executor(funs chan func()) {
for {
worker := <-funs
worker()
funs <- worker
}
}
funs := make(chan func(), 3)
funs <- func() { println("0"); time.Sleep(1 * time.Second) }
funs <- func() { println("1"); time.Sleep(1 * time.Second) }
funs <- func() { println("2"); time.Sleep(1 * time.Second) }
go executor(funs)
These methods will, of course, remove all parallelism due to synchronization. However,
the concurrent aspect of your program remains.

Resources