Could not fully understand it from the docs or google:
What are the differences between the two and in
Which case would you use each one?
for{
select{
case s := <-something:
fmt.Println(s)
case done := <-true:
return
}
}
and
select{
case s := <-something:
fmt.Println(s)
case done := <-true:
return
}
Thanks
Code with loop will keep printing data from channel something until it receives anything on channel done.
Select-only code will either print data from channel something or will quit when it receives anything on channel done. Only one case will be executed. Keep in mind there is no fallthrough in Go select and switch statements by default.
select statement executes through its cases (sending/receiving a the channel) once. If none of its cases are ready to be executed, it blocks until at least one of the case is ready to execute. If more than one cases are ready at the same time, one of the ready case is selected to be executed at random.
So in second case, if there is some data on the something channel, it would be read and put into s. But there is also a chance of true being sent on done while the case s := <-something: will never be executed.
In the first case, you probably want something like this (also in second case):
for{
select{
case s := <-something:
fmt.Println(s)
case <-done: // note the difference
return
}
}
What this now does is that it waits for data on something and also keeps an eye on done. If there is data on something channel (and no data on done), it will be read and put into s (case branch case s := <-something: will be executed with s having the value read from something). This will account for one full execution of select statement and the control will go back to for loop and it will start over again.
If there is no data on something channel, select blocks and waits for data on either something or done. If data arrives on something, similar execution as above happens, otherwise if it arrives in done, the function returns (breaks out of loop). This way some other process can write to done and signal the function containing above for loop to stop processing something and return.
If your program sends data to the 'something' channel a bunch of times you are going to want to repeat the select clause until you receive a done signal.
For example imagine you are running the following routine
.... (some calculations)
something <- x
.... (some calculations)
something <- y
true <- z
If your routine doesn't have the for loop it will only receive the value 'x' and won't receive y or z.
Related
I'm writing some code where I'm passing data from one channel to another one. Following some intuition and this answer I expected the following code to work (other is a sufficiently big buffered channel and out is the source channel):
for {
select {
case other <- (<-out):
log.Warn("C")
}
}
And it does! But the other cases don't trigger at all, e.g. there are no Ds in the logs for the code below:
for {
select {
case other <- (<-out):
log.Warn("C")
default:
log.Warn("D")
}
}
Using the more traditional solution, there are Ds all over the logs:
for {
select {
case msg := <-out:
other <- msg
log.Warn("C")
default:
log.Warn("D")
}
}
Obviously, I'm going with the usual solution, but I still don't know why the unusual one does not work as expected.
I suspect the answer lies somewhere in The Go Memory Model but I can't quite figure out what is exactly happening in this case.
I have put together some playgrounds where you can check out this behavior:
Unusal (there are no Ds at all)
Usual (there are a lot of Ds, you might have to try it locally to see anything other than Ds)
Thanks in advance to anyone who can shed some light on this!
When you have this:
ch := make(chan int, 10)
// ...
select {
case ch <- <-out:
fmt.Println("C")
default:
fmt.Println("D")
}
The communication op of the first case is ch <- something, where something is <-out. But the something is evaluated first, and only then is checked which communication op of the cases can proceed.
So <-out will block as long as it needs, and then ch <- something is checked if it can proceed. Since you used a big enough buffer, it can always proceed in your example, so default is never chosen.
Spec: Select statements:
Execution of a "select" statement proceeds in several steps:
For all the cases in the statement, the channel operands of receive operations and the channel and right-hand-side expressions of send statements are evaluated exactly once, in source order, upon entering the "select" statement. The result is a set of channels to receive from or send to, and the corresponding values to send. Any side effects in that evaluation will occur irrespective of which (if any) communication operation is selected to proceed. Expressions on the left-hand side of a RecvStmt with a short variable declaration or assignment are not yet evaluated.
If one or more of the communications can proceed, a single one that can proceed is chosen via a uniform pseudo-random selection. Otherwise, if there is a default case, that case is chosen. If there is no default case, the "select" statement blocks until at least one of the communications can proceed.
Unless the selected case is the default case, the respective communication operation is executed.
If the selected case is a RecvStmt with a short variable declaration or an assignment, the left-hand side expressions are evaluated and the received value (or values) are assigned.
The statement list of the selected case is executed.
If you lower the buffer of ch, you will see occasional Ds printed in the output (try it on the Go Playground).
ch := make(chan int, 2)
I'm reading a book about concurrency in Go (I'm learning it now) and I found this code:
c := sync.NewCond(&sync.Mutex{})
queue := make([]interface{}, 0, 10)
removeFromQueue := func(delay time.Duration) {
time.Sleep(delay)
c.L.Lock()
queue = queue[1:]
fmt.Println("Removed from queue")
c.L.Unlock() c.Signal()
}
for i := 0; i < 10; i++ {
c.L.Lock()
// Why this loop?
for len(queue) == 2 {
c.Wait()
}
fmt.Println("Adding to queue")
queue = append(queue, struct{}{})
go removeFromQueue(1*time.Second)
c.L.Unlock()
}
The problem is that I don't understand why the author introduces the for loop marked by the comment. As far as I can see, the program would be correct without it, but the author says that the loop is there because Cond will signal that something has happened only, but that doesn't mean that the state has truly changed.
In what case could that be possible?
Without the actual book at hand, and instead just some code snippets that seem out of context, it is hard to say what the author had in mind in particular. But we can guess. There is a general point about condition variables in most languages, including Go: waiting for some condition to be satisfied does require a loop in general. In some specific cases, the loop is not required.
The Go documentation is, I think, clearer about this. In particular, the text description for sync's func (c *Cond) Wait() says:
Wait atomically unlocks c.L and suspends execution of the calling goroutine. After later resuming execution, Wait locks c.L before returning. Unlike in other systems, Wait cannot return unless awoken by Broadcast or Signal.
Because c.L is not locked when Wait first resumes, the caller typically cannot assume that the condition is true when Wait returns. Instead, the caller should Wait in a loop:
c.L.Lock()
for !condition() {
c.Wait()
}
... make use of condition ...
c.L.Unlock()
I added bold emphasis to the phrase that explains the reason for the loop.
Whether you can omit the loop depends on more than one thing:
Under what condition(s) does another goroutine invoke Signal and/or Broadcast?
How many goroutines are running, and what might they be doing in parallel?
As the Go documentation says, there's one case we don't have to worry about in Go, that we might in some other systems. In some systems, the equivalent of Wait is sometimes resumed (via the equivalent of Signal) when Signal (or its equivalent) has not actually been invoked on the condition variable.
The queue example you've quoted is particularly odd because there is only one goroutine—the one running the for loop that counts to ten—that can add entries to the queue. The remaining goroutines only remove entries. So if the queue length is 2, and we pause and wait for a signal that the queue length has changed, the queue length can only have changed to either one or zero: no other goroutine can add to it and only the two goroutines we have created at this point can remove from it. This means that given this particular example, we have one of those cases where the loop is not required after all.
(It's also odd in that queue is given an initial capacity of 10, which is as many items as we'll put in, and then we start waiting when its length is exactly 2, so that we should not reach that capacity anyway. If we were to spin off additional goroutines that might add to the queue, the loop that waits while len(queue) == 2 could indeed be signaled by a removal that drops the count from 2 to 1 but not get a chance to resume until insertion occurs, pushing the count back up to 2. However, depending on the situation, that loop might not be resumed until two other goroutines have each added an entry, pushing the count to 3, for instance. So why repeat the loop when the length is exactly two? If the idea is to preserve queue slots, we should loop while the count is greater than or equal to 2.)
(Besides all this, the initial capacity is not relevant as the queue will be dynamically resized to a large slice if necessary.)
Let's say I have a bunch of goroutines running that do a HTTP request and will return at a random time. I start the goroutines (with waitgroups and one cancel among them) and I want them to keep executing the request until it returns an OK (may time out). When the first one returns, it calls the cancel and every other goroutine should just stop.
I'm not sure how I can set up the goroutine function case that will execute the code if context is not done?
As an example, I know how to use time.After() but how do I make it such that it executes the code immediately instead of after a duration?
for {
select {
case <-ctx.Done():
wg.Done()
return
case <-time.After(time.Second):
// code goes here
}
}
What can I replace the time case with?
If I understand you correctly, you have multiple goroutines racing to compute the result to the same request. I believe you're looking for something like this:
for {
select {
case <-ctx.Done():
wg.Done()
return
default:
}
// Do stuff
}
The select will immediately drop to default if context is not canceled. However, context may still cancel while you're running your code, so you may end up with multiple goroutines trying to write the result, so you have to synchronize that as well. You can write your result to a channel, and when received, you can cancel the context. However you may still have goroutines waiting to write to that channel, so you have to keep reading from the channel until all goroutines terminate.
The output of the code given bellow and is somewhat confusing, Please help me understand the behaviour of the channels and goroutines and how
does the execution actually takes place.
I have tried to understand the flow of the program but the statement after the "call of goroutine" gets executed, even though the goroutine is called,
later on the statements in goroutines are executed,
on second "call of goroutine" the behaviour is different and the sequence of printing/flow of program changes.
Following is the code:
package main
import "fmt"
func main() {
fmt.Println("1")
done := make(chan string)
go test(done)
fmt.Println("7")
fmt.Println(<-done)
fmt.Println("8")
fmt.Println(<-done)
fmt.Println("9")
fmt.Println(<-done)
}
func test(done chan string) {
fmt.Println("2")
done <- "3"
done <- "10"
fmt.Println("4")
done <- "5"
fmt.Println("6")
}
The result of the above code:
1
7
2
3
8
10
9
4
6
5
Please help me understand why and how this result comes out.
Concept 1: Channels
Visualize a channel as a tube where data goes in one end and out the other. The first data in is the first data that comes out the other side. There are buffered channels and non-buffered channels but for your example you only need to understand the default channel, which is unbuffered. Unbuffered channels only allow one value in the channel at a time.
Writing to an Unbuffered Channel
Code that looks like this writes data into one end of the channel.
ch <- value
Now, this code actually waits to be done executing until something reads the value out of the channel. An unbuffered channel only allows for one value at a time to be within it, and doesn't continue executing until it is read. We'll see later how this affects the ordering of how your code is executed.
Reading from an Unbuffered Channel
To read from an unbuffered channel (visualize taking a value out of the channel), the code to do this looks like
[value :=] <-ch
when you read code documentation [things in] square brackets indicate that what's within them is optional. Above, without the [value :=] you'll just take a value out of the channel and don't use it for anything.
Now when there's a value in the channel, this code has two side effects. One, it reads the value out of a channel in whatever routine we are in now, and proceeds with the value. The other effect it has is to allow the goroutine which put the value into the channel to continue. This is the critical bit that's necessary to understand your example program.
In the event there is NO value in the channel yet, it will wait for a value to be written into the channel before continuing. In other words, the thread blocks until the channel has a value to read.
Concept 2: Goroutines
A goroutine allows your code to continue executing two pieces of code concurrently. This can be used to allow your code to execute faster, or attend to multiple problems at the same time (think of a server where multiple users are loading pages from it at the same time).
Your question arises when you try to figure out the ordering that code is executed when you have multiple routines executing concurrently. This is a good question and others have correctly stated that it depends. When you spawn two goroutines, the ordering of which lines of code are executed is arbitrary.
The code below with a goroutine may print executing a() or end main() first. This is due to the fact that spawning a gorouting means there are two concurrent streams (threads) of execution happening at the same time. In this case, one thread stays in main() and the other starts executing the first line in a(). How the runtime decides to choose which to run first is arbitrary.
func main() {
fmt.Println("start main()")
go a()
fmt.Println("end main()")
}
func a() {
fmt.Println("executing a()")
}
Goroutines + Channels
Now let's use a channel to control the ordering of what get's executed, when.
The only difference now is we create a channel, pass it into the goroutine, and wait for it's value to be written before continuing in main. From earlier, we discussed how the routine reading the value from a channel needs to wait until there's a value in the channel before continuing. Since executing a() is always printed before the channel is written to, we will always wait to read the value put into the channel until executing a() has printed. Since we read from the channel (which happens after the channel is written) before printing end main(), executing a() will always print before end main(). I made this playground so you can run it for yourself.
func main() {
fmt.Println("start main()")
ch := make(chan int)
go a(ch)
<-ch
fmt.Println("end main()")
}
func a(ch chan int) {
fmt.Println("executing a()")
ch <- 0
}
Your Example
I think at this point you could figure out what happens when, and what might happen in a different order. My own first attempt was wrong when I went through it in my head (see edit history). You have to be careful! I'll not give the right answer, upon editing, since I realized this may be a homework assignment.
EDIT: more semantics about <-done
On my first go through, I forgot to mention that fmt.Println(<-done) is conceptually the same as the following.
value := <-done
fmt.Println(value)
This is important because it helps you see that when the main() thread reads from the done channel, it doesn't print it at the same time. These are two separate steps to the runtime.
Here is a quote from 50 Shades Of Go: Traps, Gotchas and Common mistakes:
You can also use a special cancellation channel to interrupt the
workers.
func First(query string, replicas ...Search) Result {
c := make(chan Result)
done := make(chan struct{})
defer close(done)
searchReplica := func(i int) {
select {
case c <- replicas[i](query):
case <- done:
}
}
for i := range replicas {
go searchReplica(i)
}
return <-c
}
As far as understand, it means that we use channel done to interrupt the workers ahead of time without waiting for full execution (in our case execution of replicas[i](query). Therefore, we can receive a result from the fastest worker ("First Wins Pattern") and then cancel the work in all other workers and save the resources.
On the other hand, according to the specification:
For all the cases in the statement, the channel operands of receive
operations and the channel and right-hand-side expressions of send
statements are evaluated exactly once, in source order, upon entering
the "select" statement.
As far as I understand, it means we cannot interrupt the workers, as in any case, all workers will evaluate function replicas[i]query and only then select case <- done and finish their execution.
Could you please point out at the mistake in my reasoning?
Your reasoning is correct, the wording on the site is not completely clear. What this "construct" achieves is that the goroutines will not be left hanging forever, but once the searches finish, the goroutines will end properly. Nothing more is happening there.
In general, you can't interrupt any goroutine from the outside, the goroutine itself has to support some kind of termination (e.g. shutdown channel, context.Context etc.). See cancel a blocking operation in Go.
So yes, in the example you posted, all searches will be launched, concurrently, result of the fastest one will be returned as it arrives, the rest of the goroutines will continue to run as long as their search is finished.
What happens to the rest? The rest will be discarded (case <- done will be chosen, as an unbuffered channel cannot hold any elements, and there will be no one else receiving more from the channel).
You can verify this in this Go Playground example.