I am unable to figure out why the method requires you to specifically provide a buffered channel.
From the documentation,
func (*Client) Go
func (client *Client) Go(serviceMethod string, args interface{}, reply interface{}, done chan *Call) *Call
Go invokes the function asynchronously. It returns the Call structure
representing the invocation. The done channel will signal when the
call is complete by returning the same Call object. If done is nil, Go
will allocate a new channel. If non-nil, done must be buffered or Go
will deliberately crash.
LeGEC alluded to this in their comment.
Digging in further you will find this bit in client.go
func (call *Call) done() {
select {
case call.Done <- call:
// ok
default:
// We don't want to block here. It is the caller's responsibility to make
// sure the channel has enough buffer space. See comment in Go().
if debugLog {
log.Println("rpc: discarding Call reply due to insufficient Done chan capacity")
}
}
}
From what you can see here is that the library expects the call to be completely asynchronous. This means the done channel must have enough capacity to completely decouple the two processes (i.e. no blocking at all).
Further when the select statement is used as seen, it is the idiomatic way to do a non-blocking channel operation.
Related
This is the second of two questions (this is the first one) to help make sense of the Go generics proposal examples.
In particular I am having trouble-so far-understanding two bits of code from the examples section of the proposal entitled "Channels":
The second issue I have is in the following definition of the the Ranger function.
Namely, I don't understand the need to call runtime.SetFinalizer(r,r.finalize) where in fact what the finalize) method of the *Receiver[T] type is supposed to do is simply to signal that the receiver is done receiving values (close(r.done)).
The way I see it, by providing a finalizer for a *Receiver[T] the code is delegating the obligation to close the receiver to the runtime.
The way I understand this piece of code, is that the *Receiver[T] signals to the *Sender[T] that it won't be receiving any more values when the GC decides that the former is unreachable ie no more references are available to it.
If my interpretation is correct, why wait that long for the receiver to signal it's done? Is't it possible, to explicitly handle the close operation in the code somehow?
Thanks.
Code:
// Ranger provides a convenient way to exit a goroutine sending values
// when the receiver stops reading them.
//
// Ranger returns a Sender and a Receiver. The Receiver provides a
// Next method to retrieve values. The Sender provides a Send method
// to send values and a Close method to stop sending values. The Next
// method indicates when the Sender has been closed, and the Send
// method indicates when the Receiver has been freed.
func Ranger[T any]() (*Sender[T], *Receiver[T]) {
c := make(chan T)
d := make(chan bool)
s := &Sender[T]{values: c, done: d}
r := &Receiver[T]{values: c, done: d}
// The finalizer on the receiver will tell the sender
// if the receiver stops listening.
runtime.SetFinalizer(r, r.finalize)
return s, r
}
// A Sender is used to send values to a Receiver.
type Sender[T any] struct {
values chan<- T
done <-chan bool
}
// Send sends a value to the receiver. It reports whether any more
// values may be sent; if it returns false the value was not sent.
func (s *Sender[T]) Send(v T) bool {
select {
case s.values <- v:
return true
case <-s.done:
// The receiver has stopped listening.
return false
}
}
// Close tells the receiver that no more values will arrive.
// After Close is called, the Sender may no longer be used.
func (s *Sender[T]) Close() {
close(s.values)
}
// A Receiver receives values from a Sender.
type Receiver[T any] struct {
values <-chan T
done chan<- bool
}
// Next returns the next value from the channel. The bool result
// reports whether the value is valid. If the value is not valid, the
// Sender has been closed and no more values will be received.
func (r *Receiver[T]) Next() (T, bool) {
v, ok := <-r.values
return v, ok
}
// finalize is a finalizer for the receiver.
// It tells the sender that the receiver has stopped listening.
func (r *Receiver[T]) finalize() {
close(r.done)
}
TLDR: Your understanding is correct, the done channel may simply be closed by the receiver "manually" to signal the lost of interest (to stop the communication and relieve the sender from its duty).
Channels are used for goroutines to communicate in a concurrency safe manner. The idiomatic use is that the sender party keeps sending values, and once there are no more values to send, it is signaled by the sender closing the channel.
The receiver party keeps receiving from the channel until it is closed, which signals there won't be (can't be) any more values coming on the channel. This is usually / easiest done using a for range over the channel.
So usually the receiver has to keep receiving until the channel is closed, else the sender party would get blocked forever. Often this is OK / sufficient.
The demonstrated Ranger() construct is for the non-general case when there's need / possibility for the receiver to stop the communication.
A single channel does not provide a mean for the receiver party to signal the sender that the receiver has lost interest, and no more values are needed. This requires an additional channel which the receiver has to close (and the sender has to monitor of course). As long as there's a single receiver, this is also OK. But if there are multiple receivers, closing the done channel gets a little more complicated: it's not OK for all the receivers to close the done channel: closing an already closed channel panics. So the receivers also have to be coordinated, so only a single receiver, or rather the coordinator party itself closes the done channel, once only; and this has to happen after all receivers "abandoned" the channel.
Ranger() helps with this, and in a simple way by delegating closing the done channel using a finalizer. This is acceptable because usually it wouldn't even be the receiver(s) task to stop the communication, but in the rare case if this still arises, it will be dealt with (in an easy way, without the need of an additional, coordinator goroutine).
I was working on some Go code and wanted to close a channel at the end of it. However, I wanted to test if it'd actually close the channel if I use an anonymous function, passing the channel as an argument and use the function with the defer keyword.
channel := make(chan string, 2)
defer func (channel chan string) {
close(channel)
value, ok := <-channel
fmt.Println("Value:", value, "Open channel?", ok)
}(channel)
To my understanding everything in Go is passed by value and not by reference, and make's documentation says that it returns a channel when used to spawn a channel, not a pointer to a channel. Does this imply that if I pass the channel as argument to a function and close the channel within that function, the channel outside the function would remain open? If so, I'm guessing I'd need to pass the channel as a pointer to the function to achieve that behavior?
I'm new to Go, so I'd appreciate if you can explain me the "why" rather than "Oh, just swap this chunk for this other chunk of code" just so I can better understand how it's actually working:)
Thanks!
Channels behaves like slice, map, strings or functions, as in they actually store a wrapped pointer to the underlying data. They behave like opaque pointers.
So passing a channel is copying it by value, but just like a slice, it's "value" is literally just a reference to the actual low-level channel data structure. So there is no need to explicitly pass a channel pointer, since that would be like passing a pointer of a pointer.
To answer your questions: Yes it can and yes it's safe to pass channels by value because they are "pointers" in and of themselves.
You can also verify it with a slightly modified version of your code:
package main
import "fmt"
func main() {
channel := make(chan string, 2)
// channel is open here.
// close the channel by passing it into a function
// by value and closing it within the function.
func(channel chan string) {
close(channel)
}(channel)
// is the channel closed?
value, ok := <-channel
fmt.Println("Value:", value, "Open channel?", ok)
}
Output (go version go1.16 darwin/amd64):
Value: Open channel? false
Does this imply that if I pass the channel as argument to a function and close the channel within that function, the channel outside the function would remain open?
No. Copying a chan variable doesn't produce a copy of the channel itself. Variables of type chan are handles to the underlying channel created by make. Closing any copy of the chan variable closes the channel.
func (s *server) send(m *message) error {
go func() {
s.outgoingMessageChan <- message
}()
return nil
}
func main(s *server) {
for {
select {
case <-someChannel:
// do something
case msg := <-s.outGoingMessageChan:
// take message sent from "send" and do something
}
}
}
I am pulling out of this s.outgoingMessageChan in another function, before using an anonymous go function, a call to this function would usually block - meaning whenever send is called, s.outgoingMessageChan <- message would block until something is pulling out of it. However after wrapping it like this it doesn't seem to block anymore. I understand that it kind of sends this operation to the background and proceeds as usual, but I'm not able to wrap my head around how this doesn't affect the current function call.
Each time send is called a new goroutine is created, and returns immediately. (BTW there is no reason to return an error if there can never be an error.) The goroutine (which has it's own "thread" of execution) will block if nothing is ready to read from the chan (assuming it's unbuffered). Once the message is read off the chan the goroutine will continue but since it does nothing else it will simply end.
I should point out that there is no such thing as an anonymous goroutine. Goroutines have no identifier at all (except for a number that you should only use for debugging purposes). You have an anonymous function which you put the go keyword in front causing it to run in a separate goroutine.
For a send function that blocks as you seem to want then just use:
func (s *server) send(m *message) {
s.outgoingMessageChan <- message
}
However, I can't see any point in this function (though it would be inlined and just as efficient as not using a function).
I suspect you may be calling send many times before anything is read from the chan. In this case many new goroutines will be created (each time you call send) which will all block. Each time the chan is read from one will unblock delivering its value and that goroutine will terminate. Doing this you are simply creating an inefficient buffering mechanism. Moreover, if send is called for a prolonged period at a faster rate than the values can be read from the chan then you will eventually run out of memory. Better would be to use a buffered chan (and no goroutines) that once it (the chan) became full exerted "back-pressure" on whatever was producing the messages.
Another point is that the function name main is used to identify the entry point to a program. Please use another name for your 2nd function above. It also seems like it should be a method (using s *server receiver) than a function.
I have two (but later I'll be three) go routines that are handling incoming messages from a remote server (from a ampq channel). But because they are handling on the same data/state, I want to block all other go routines, except the one running.
I come up with a solution to use chan bool where each go routine blocks and then release it, the code is like:
package main
func a(deliveries <-chan amqp, handleDone chan bool) {
for d := range deliveries {
<-handleDone // Data comes always, wait for other channels
handleDone <- false // Block other channels
// Do stuff with data...
handleDone <- true // I'm done, other channels are free to do anything
}
}
func b(deliveries <-chan amqp, handleDone chan bool) {
for d := range deliveries {
<-handleDone
handleDone <- false
// Do stuff with data...
handleDone <- true
}
}
func main() {
handleDone := make(chan bool, 1)
go a(arg1, handleDone)
go b(arg2, handleDone)
// go c(arg3, handleDone) , later
handleDone <- true // kickstart
}
But for the first time each of the function will get handleDone <- true, which they will be executed. And later if I add another third function, things will get more complicated. How can block all other go routines except the running? Any other better solutions?
You want to look at the sync package.
http://golang.org/pkg/sync/
You would do this with a mutex.
If you have an incoming stream of messages and you have three goroutines listening on that stream and processing and you want to ensure that only one goroutine is running at a time, the solution is quite simple: kill off two of the goroutines.
You're spinning up concurrency and adding complexity and then trying to prevent them from running concurrently. The end result is the same as a single stream reader, but with lots of things that can go wrong.
I'm puzzled why you want this - why can't each message on deliveries be handled independently? and why are there two different functions handling those message? If each is responsible for a particular type of message, it seems like you want one deliveries receiver that dispatches to appropriate logic for the type.
But to answer your question, I don't think it's true that each function will get a true from handleDone on start. One (let's say it's a) is receiving the true sent from main; the other (b then) is getting the false sent from the first. Because you're discarding the value received, you can't tell this. And then both are running, and you're using a buffered channel (you probably want make(chan bool) instead for an unbuffered one), so confusion ensues, particularly when you add that third goroutine.
The handleDone <- false doesn't actually accomplish anything. Just treat any value on handleDone as the baton in a relay race. Once a goroutine receives this value, it can do its thing; when it's done, it should send it to the channel to hand it to the next goroutine.
So, right now, I just pass a pointer to a Queue object (implementation doesn't really matter) and call queue.add(result) at the end of goroutines that should add things to the queue.
I need that same sort of functionality—and of course doing a loop checking completion with the comma ok syntax is unacceptable in terms of performance versus the simple queue add function call.
Is there a way to do this better, or not?
There are actually two parts to your question: how does one queue data in Go, and how does one use a channel without blocking.
For the first part, it sounds like what you need to do is instead of using the channel to add things to the queue, use the channel as a queue. For example:
var (
ch = make(chan int) // You can add an int parameter to this make call to create a buffered channel
// Do not buffer these channels!
gFinished = make(chan bool)
processFinished = make(chan bool)
)
func f() {
go g()
for {
// send values over ch here...
}
<-gFinished
close(ch)
}
func g() {
// create more expensive objects...
gFinished <- true
}
func processObjects() {
for val := range ch {
// Process each val here
}
processFinished <- true
}
func main() {
go processObjects()
f()
<-processFinished
}
As for how you can make this more asynchronous, you can (as cthom06 pointed out) pass a second integer to the make call in the second line which will make send operations asynchronous until the channel's buffer is full.
EDIT: However (as cthom06 also pointed out), because you have two goroutines writing to the channel, one of them has to be responsible for closing the channel. Also, my previous revision would exit before processObjects could complete. The way I chose to synchronize the goroutines is by creating a couple more channels that pass around dummy values to ensure that the cleanup gets finished properly. Those channels are specifically unbuffered so that the sends happen in lock-step.