I'm writing some code where I'm passing data from one channel to another one. Following some intuition and this answer I expected the following code to work (other is a sufficiently big buffered channel and out is the source channel):
for {
select {
case other <- (<-out):
log.Warn("C")
}
}
And it does! But the other cases don't trigger at all, e.g. there are no Ds in the logs for the code below:
for {
select {
case other <- (<-out):
log.Warn("C")
default:
log.Warn("D")
}
}
Using the more traditional solution, there are Ds all over the logs:
for {
select {
case msg := <-out:
other <- msg
log.Warn("C")
default:
log.Warn("D")
}
}
Obviously, I'm going with the usual solution, but I still don't know why the unusual one does not work as expected.
I suspect the answer lies somewhere in The Go Memory Model but I can't quite figure out what is exactly happening in this case.
I have put together some playgrounds where you can check out this behavior:
Unusal (there are no Ds at all)
Usual (there are a lot of Ds, you might have to try it locally to see anything other than Ds)
Thanks in advance to anyone who can shed some light on this!
When you have this:
ch := make(chan int, 10)
// ...
select {
case ch <- <-out:
fmt.Println("C")
default:
fmt.Println("D")
}
The communication op of the first case is ch <- something, where something is <-out. But the something is evaluated first, and only then is checked which communication op of the cases can proceed.
So <-out will block as long as it needs, and then ch <- something is checked if it can proceed. Since you used a big enough buffer, it can always proceed in your example, so default is never chosen.
Spec: Select statements:
Execution of a "select" statement proceeds in several steps:
For all the cases in the statement, the channel operands of receive operations and the channel and right-hand-side expressions of send statements are evaluated exactly once, in source order, upon entering the "select" statement. The result is a set of channels to receive from or send to, and the corresponding values to send. Any side effects in that evaluation will occur irrespective of which (if any) communication operation is selected to proceed. Expressions on the left-hand side of a RecvStmt with a short variable declaration or assignment are not yet evaluated.
If one or more of the communications can proceed, a single one that can proceed is chosen via a uniform pseudo-random selection. Otherwise, if there is a default case, that case is chosen. If there is no default case, the "select" statement blocks until at least one of the communications can proceed.
Unless the selected case is the default case, the respective communication operation is executed.
If the selected case is a RecvStmt with a short variable declaration or an assignment, the left-hand side expressions are evaluated and the received value (or values) are assigned.
The statement list of the selected case is executed.
If you lower the buffer of ch, you will see occasional Ds printed in the output (try it on the Go Playground).
ch := make(chan int, 2)
Related
I don't understand how the Done() channel in context.Context can work as intended. The module documentation (and source code using it) relies on this pattern:
select {
case <-ctx.Done():
return ctx.Err()
case results <- result:
}
The channel return by Done() is closed if the Context is canceled or timed out, and the Err() variable holds the reason.
I have two questions regarding this approach:
What's the behavior of select when a channel is closed? When and why is the case entered? Does the fact that there's no assignment have relevance?
According to the language reference:
If one or more of the communications can proceed, a single one that can proceed is chosen via a uniform pseudo-random selection.
If the choice is random, how does that pattern guarantee that I won't be sending results down a pipeline when the Context is canceled? I would understand if the cases were evaluated in declaration order (and closed channel cases were selected).
If I'm completely off the track here, please explain this to me from a better angle.
This case:
case <-ctx.Done():
Has the communication op:
<-ctx.Done()
It's a receive from a channel. Spec: Receive operator:
A receive operation on a closed channel can always proceed immediately, yielding the element type's zero value after any previously sent values have been received.
So when the channel returned by ctx.Done() is closed, a receive from it can proceed immediately. Thus control flow can enter into this case.
If the other case (results <- result) can also proceed when the context is cancelled, one is chosen (pseudo-)randomly, there is no guarantee which one it will be.
If you don't want to send a value on results if the context is already cancelled, check the ctx.Done() channel before the select with another non-blocking select:
select {
case <-ctx.Done():
return ctx.Err()
default:
}
select {
case <-ctx.Done():
return ctx.Err()
case results <- result:
}
Note that you must add a default branch to the first select, else it would block until the context is cancelled. If there is a default branch and the context is not yet cancelled, default branch is chosen and so the control flow can go to the second select.
See related questions:
Force priority of go select statement
How does select work when multiple channels are involved?
incoming := make(chan int)
done := make(chan struct{})
...
close(done)
...
select {
case incoming <- order:
// logic
case <-done:
return
}
Let's say you close the done channel before the select statement:
1.The case <-done will be selected.
2.Is this because when close(done) executes, the done channel will be in a state, in which the select statement will consider the "case <-done" a match over the writing case "incoming <- order"?
The select statement blocks until one or more of the communication operations in its cases can proceed. As soon as one or more communication operations can proceed select will randomly choose one of them.
A receive operation on a closed channel can proceed immediately.
Select Statement:
Execution of a "select" statement proceeds in several steps:
For all the cases in the statement, the channel operands of
receive operations and the channel and right-hand-side expressions
of send statements are evaluated exactly once, in source order,
upon entering the "select" statement. The result is a set of
channels to receive from or send to, and the corresponding values
to send. Any side effects in that evaluation will occur
irrespective of which (if any) communication operation is selected
to proceed. Expressions on the left-hand side of a RecvStmt with a > short variable declaration or assignment are not yet evaluated.
If one or more of the communications can proceed, a single one
that can proceed is chosen via a uniform pseudo-random selection.
Otherwise, if there is a default case, that case is chosen. If
there is no default case, the "select" statement blocks until at
least one of the communications can proceed.
Unless the selected case is the default case, the respective
communication operation is executed.
If the selected case is a RecvStmt with a short variable
declaration or an assignment, the left-hand side expressions are
evaluated and the received value (or values) are assigned.
The statement list of the selected case is executed.
Receive operator:
A receive operation on a closed channel can always proceed
immediately, yielding the element type's zero value after any
previously sent values have been received.
I have an application with multiple goroutines that are running for loops, and need a way to signal these for loops to break, and to test whether the timeout case occurred. I was looking into using a shared channel with select statements to accomplish this as follows:
// elsewhere in the code, this channel is created, and passed below
done := make(chan struct{})
time.AfterFunc(timeout, func() { close(done) })
...
go func() {
Loop:
for {
select {
case <-done:
break Loop
default:
foo()
time.Sleep(1 * time.Second)
}
}
select {
case <-done:
panic("timed out!")
default:
// ok
}
}()
Is this a valid way to accomplish this? What I'm most concerned about is that the branch of a select that is chosen could be non-deterministic, so that default may be chosen even if one of the cases is ready. Is this possible? Is there any documentation that states that a matching case is guaranteed to have preference over a default. The concern is that the for loop above could loop several times after done is closed and/or report success even though a timeout occurred.
The Go Programming Language Specification
Select statements
Execution of a "select" statement proceeds in several steps:
For all the cases in the statement, the channel operands of receive operations and the channel and right-hand-side expressions of send
statements are evaluated exactly once, in source order, upon entering
the "select" statement. The result is a set of channels to receive
from or send to, and the corresponding values to send. Any side
effects in that evaluation will occur irrespective of which (if any)
communication operation is selected to proceed. Expressions on the
left-hand side of a RecvStmt with a short variable declaration or
assignment are not yet evaluated.
If one or more of the communications can proceed, a single one that can proceed is chosen via a uniform pseudo-random selection.
Otherwise, if there is a default case, that case is chosen. If there
is no default case, the "select" statement blocks until at least one
of the communications can proceed.
Unless the selected case is the default case, the respective communication operation is executed.
If the selected case is a RecvStmt with a short variable declaration or an assignment, the left-hand side expressions are
evaluated and the received value (or values) are assigned.
The statement list of the selected case is executed.
"What I'm most concerned about is that the branch of a select that is
chosen could be non-deterministic, so that default may be chosen even
if one of the cases is ready. Is this possible?"
No. See step 2 of the select specification.
Could not fully understand it from the docs or google:
What are the differences between the two and in
Which case would you use each one?
for{
select{
case s := <-something:
fmt.Println(s)
case done := <-true:
return
}
}
and
select{
case s := <-something:
fmt.Println(s)
case done := <-true:
return
}
Thanks
Code with loop will keep printing data from channel something until it receives anything on channel done.
Select-only code will either print data from channel something or will quit when it receives anything on channel done. Only one case will be executed. Keep in mind there is no fallthrough in Go select and switch statements by default.
select statement executes through its cases (sending/receiving a the channel) once. If none of its cases are ready to be executed, it blocks until at least one of the case is ready to execute. If more than one cases are ready at the same time, one of the ready case is selected to be executed at random.
So in second case, if there is some data on the something channel, it would be read and put into s. But there is also a chance of true being sent on done while the case s := <-something: will never be executed.
In the first case, you probably want something like this (also in second case):
for{
select{
case s := <-something:
fmt.Println(s)
case <-done: // note the difference
return
}
}
What this now does is that it waits for data on something and also keeps an eye on done. If there is data on something channel (and no data on done), it will be read and put into s (case branch case s := <-something: will be executed with s having the value read from something). This will account for one full execution of select statement and the control will go back to for loop and it will start over again.
If there is no data on something channel, select blocks and waits for data on either something or done. If data arrives on something, similar execution as above happens, otherwise if it arrives in done, the function returns (breaks out of loop). This way some other process can write to done and signal the function containing above for loop to stop processing something and return.
If your program sends data to the 'something' channel a bunch of times you are going to want to repeat the select clause until you receive a done signal.
For example imagine you are running the following routine
.... (some calculations)
something <- x
.... (some calculations)
something <- y
true <- z
If your routine doesn't have the for loop it will only receive the value 'x' and won't receive y or z.
How do I check whether a channel has a value for me to read?
I don't want to block when reading a channel. I want to see whether it has a value. If it does have one, I'll read it. If it doesn't have one (yet), I'll do something else and check back again later.
The only non-blocking operation I know of to read from a channel is inside a select block having a default case :
select {
case x, ok := <-ch:
if ok {
fmt.Printf("Value %d was read.\n", x)
} else {
fmt.Println("Channel closed!")
}
default:
fmt.Println("No value ready, moving on.")
}
Please try the non-blocking here
Note about previous answers: the receive operator itself is now a blocking operation, as of Go 1.0.3 . The spec has been modified. Please try the blocking here (deadlock)
If you're doing this often then it's probably not a great design and you might be better off spawning another goroutine to do whatever work you're planning to do when there isn't anything to read from the channel. The synchronous/blocking nature of Go's channels make code easier to read and reason about while the scheduler and cheap goroutines means that async calls are unnecessary since waiting goroutines take up very little resources.
Unfortunately, the previous answers are incorrect. The spec clearly says that you CAN use channels this way using len() function, but only if you specified the channel capacity - the buffer length for a channel while making it. If you omitted a channel capacity while making it - the channel operations are always blocking.
You don't, at least not for synchronous (unbuffered) channels. There is no way to tell if a value is waiting without asking to take the value from the channel.
For buffered channels, you technically can use the len function to do what you describe, but you really, really shouldn't. Your technique is invalid.
The reason is that it represents a race condition. Given channel ch, your goroutine might see that len(ch) > 0 and conclude that there is a value waiting. It cannot conclude however, that it can read from the channel without blocking--another goroutine might empty the channel between the time you check len and the time your receive operation runs.
For the purpose you described, use select with a default case as Ripounet showed.
WARNING: This is no longer accurate, see the answer below.
From the docs:
If a receive expression is used in an
assignment or initialization of the
form
x, ok = <-ch
x, ok := <-ch
var x, ok = <-ch
the receive operation becomes
non-blocking. If the operation can
proceed, the boolean variable ok will
be set to true and the value stored in
x; otherwise ok is set to false and x
is set to the zero value for its type
In most cases relying on such information is a really bad design choice. Not even saying about how it's dirty in it's implementation.
So, do not implement the following steps to detect if channel is ready for read at runtime:
define hchan waitq sudog structs as defined here - https://golang.org/src/runtime/chan.go
use "unsafe" package to cast channel to pointer to hchan struct
read sendq field of this struct to get listeners
read first sudog and read msg field from there.
cast msg to the appropriate type for the channels using "reflect" and "unsafe"