I'm writing some package, where channel is only used to interrupt some process, so it is never read any value, just detects that it need to stop, like this:
func myfunc(stop_chan chan /*bool*/) {
for {
//do something time consuming
// ....
// check on channel
select{
case <-stop_chan:
//cleanup
return
default:
}
// continue working
}
}
later I wish this function to accept any type of channel. Is that possible?
I wish this function to accept any type of channel. Is that possible?
No, this is not possible. There are no untyped channels in Go. You could accept a channel of any type, using interface{}, then using reflection to interact with that channel. But this is cumbersome and slow, so not recommended.
A better approach, probably, is to settle on a single type of channel (chan struct{} is the natural choice for your use-case), or use a context variable instead, which can handle cancellation in a standardized way.
Related
This is the second of two questions (this is the first one) to help make sense of the Go generics proposal examples.
In particular I am having trouble-so far-understanding two bits of code from the examples section of the proposal entitled "Channels":
The second issue I have is in the following definition of the the Ranger function.
Namely, I don't understand the need to call runtime.SetFinalizer(r,r.finalize) where in fact what the finalize) method of the *Receiver[T] type is supposed to do is simply to signal that the receiver is done receiving values (close(r.done)).
The way I see it, by providing a finalizer for a *Receiver[T] the code is delegating the obligation to close the receiver to the runtime.
The way I understand this piece of code, is that the *Receiver[T] signals to the *Sender[T] that it won't be receiving any more values when the GC decides that the former is unreachable ie no more references are available to it.
If my interpretation is correct, why wait that long for the receiver to signal it's done? Is't it possible, to explicitly handle the close operation in the code somehow?
Thanks.
Code:
// Ranger provides a convenient way to exit a goroutine sending values
// when the receiver stops reading them.
//
// Ranger returns a Sender and a Receiver. The Receiver provides a
// Next method to retrieve values. The Sender provides a Send method
// to send values and a Close method to stop sending values. The Next
// method indicates when the Sender has been closed, and the Send
// method indicates when the Receiver has been freed.
func Ranger[T any]() (*Sender[T], *Receiver[T]) {
c := make(chan T)
d := make(chan bool)
s := &Sender[T]{values: c, done: d}
r := &Receiver[T]{values: c, done: d}
// The finalizer on the receiver will tell the sender
// if the receiver stops listening.
runtime.SetFinalizer(r, r.finalize)
return s, r
}
// A Sender is used to send values to a Receiver.
type Sender[T any] struct {
values chan<- T
done <-chan bool
}
// Send sends a value to the receiver. It reports whether any more
// values may be sent; if it returns false the value was not sent.
func (s *Sender[T]) Send(v T) bool {
select {
case s.values <- v:
return true
case <-s.done:
// The receiver has stopped listening.
return false
}
}
// Close tells the receiver that no more values will arrive.
// After Close is called, the Sender may no longer be used.
func (s *Sender[T]) Close() {
close(s.values)
}
// A Receiver receives values from a Sender.
type Receiver[T any] struct {
values <-chan T
done chan<- bool
}
// Next returns the next value from the channel. The bool result
// reports whether the value is valid. If the value is not valid, the
// Sender has been closed and no more values will be received.
func (r *Receiver[T]) Next() (T, bool) {
v, ok := <-r.values
return v, ok
}
// finalize is a finalizer for the receiver.
// It tells the sender that the receiver has stopped listening.
func (r *Receiver[T]) finalize() {
close(r.done)
}
TLDR: Your understanding is correct, the done channel may simply be closed by the receiver "manually" to signal the lost of interest (to stop the communication and relieve the sender from its duty).
Channels are used for goroutines to communicate in a concurrency safe manner. The idiomatic use is that the sender party keeps sending values, and once there are no more values to send, it is signaled by the sender closing the channel.
The receiver party keeps receiving from the channel until it is closed, which signals there won't be (can't be) any more values coming on the channel. This is usually / easiest done using a for range over the channel.
So usually the receiver has to keep receiving until the channel is closed, else the sender party would get blocked forever. Often this is OK / sufficient.
The demonstrated Ranger() construct is for the non-general case when there's need / possibility for the receiver to stop the communication.
A single channel does not provide a mean for the receiver party to signal the sender that the receiver has lost interest, and no more values are needed. This requires an additional channel which the receiver has to close (and the sender has to monitor of course). As long as there's a single receiver, this is also OK. But if there are multiple receivers, closing the done channel gets a little more complicated: it's not OK for all the receivers to close the done channel: closing an already closed channel panics. So the receivers also have to be coordinated, so only a single receiver, or rather the coordinator party itself closes the done channel, once only; and this has to happen after all receivers "abandoned" the channel.
Ranger() helps with this, and in a simple way by delegating closing the done channel using a finalizer. This is acceptable because usually it wouldn't even be the receiver(s) task to stop the communication, but in the rare case if this still arises, it will be dealt with (in an easy way, without the need of an additional, coordinator goroutine).
I was working on some Go code and wanted to close a channel at the end of it. However, I wanted to test if it'd actually close the channel if I use an anonymous function, passing the channel as an argument and use the function with the defer keyword.
channel := make(chan string, 2)
defer func (channel chan string) {
close(channel)
value, ok := <-channel
fmt.Println("Value:", value, "Open channel?", ok)
}(channel)
To my understanding everything in Go is passed by value and not by reference, and make's documentation says that it returns a channel when used to spawn a channel, not a pointer to a channel. Does this imply that if I pass the channel as argument to a function and close the channel within that function, the channel outside the function would remain open? If so, I'm guessing I'd need to pass the channel as a pointer to the function to achieve that behavior?
I'm new to Go, so I'd appreciate if you can explain me the "why" rather than "Oh, just swap this chunk for this other chunk of code" just so I can better understand how it's actually working:)
Thanks!
Channels behaves like slice, map, strings or functions, as in they actually store a wrapped pointer to the underlying data. They behave like opaque pointers.
So passing a channel is copying it by value, but just like a slice, it's "value" is literally just a reference to the actual low-level channel data structure. So there is no need to explicitly pass a channel pointer, since that would be like passing a pointer of a pointer.
To answer your questions: Yes it can and yes it's safe to pass channels by value because they are "pointers" in and of themselves.
You can also verify it with a slightly modified version of your code:
package main
import "fmt"
func main() {
channel := make(chan string, 2)
// channel is open here.
// close the channel by passing it into a function
// by value and closing it within the function.
func(channel chan string) {
close(channel)
}(channel)
// is the channel closed?
value, ok := <-channel
fmt.Println("Value:", value, "Open channel?", ok)
}
Output (go version go1.16 darwin/amd64):
Value: Open channel? false
Does this imply that if I pass the channel as argument to a function and close the channel within that function, the channel outside the function would remain open?
No. Copying a chan variable doesn't produce a copy of the channel itself. Variables of type chan are handles to the underlying channel created by make. Closing any copy of the chan variable closes the channel.
func (s *server) send(m *message) error {
go func() {
s.outgoingMessageChan <- message
}()
return nil
}
func main(s *server) {
for {
select {
case <-someChannel:
// do something
case msg := <-s.outGoingMessageChan:
// take message sent from "send" and do something
}
}
}
I am pulling out of this s.outgoingMessageChan in another function, before using an anonymous go function, a call to this function would usually block - meaning whenever send is called, s.outgoingMessageChan <- message would block until something is pulling out of it. However after wrapping it like this it doesn't seem to block anymore. I understand that it kind of sends this operation to the background and proceeds as usual, but I'm not able to wrap my head around how this doesn't affect the current function call.
Each time send is called a new goroutine is created, and returns immediately. (BTW there is no reason to return an error if there can never be an error.) The goroutine (which has it's own "thread" of execution) will block if nothing is ready to read from the chan (assuming it's unbuffered). Once the message is read off the chan the goroutine will continue but since it does nothing else it will simply end.
I should point out that there is no such thing as an anonymous goroutine. Goroutines have no identifier at all (except for a number that you should only use for debugging purposes). You have an anonymous function which you put the go keyword in front causing it to run in a separate goroutine.
For a send function that blocks as you seem to want then just use:
func (s *server) send(m *message) {
s.outgoingMessageChan <- message
}
However, I can't see any point in this function (though it would be inlined and just as efficient as not using a function).
I suspect you may be calling send many times before anything is read from the chan. In this case many new goroutines will be created (each time you call send) which will all block. Each time the chan is read from one will unblock delivering its value and that goroutine will terminate. Doing this you are simply creating an inefficient buffering mechanism. Moreover, if send is called for a prolonged period at a faster rate than the values can be read from the chan then you will eventually run out of memory. Better would be to use a buffered chan (and no goroutines) that once it (the chan) became full exerted "back-pressure" on whatever was producing the messages.
Another point is that the function name main is used to identify the entry point to a program. Please use another name for your 2nd function above. It also seems like it should be a method (using s *server receiver) than a function.
I am unable to figure out why the method requires you to specifically provide a buffered channel.
From the documentation,
func (*Client) Go
func (client *Client) Go(serviceMethod string, args interface{}, reply interface{}, done chan *Call) *Call
Go invokes the function asynchronously. It returns the Call structure
representing the invocation. The done channel will signal when the
call is complete by returning the same Call object. If done is nil, Go
will allocate a new channel. If non-nil, done must be buffered or Go
will deliberately crash.
LeGEC alluded to this in their comment.
Digging in further you will find this bit in client.go
func (call *Call) done() {
select {
case call.Done <- call:
// ok
default:
// We don't want to block here. It is the caller's responsibility to make
// sure the channel has enough buffer space. See comment in Go().
if debugLog {
log.Println("rpc: discarding Call reply due to insufficient Done chan capacity")
}
}
}
From what you can see here is that the library expects the call to be completely asynchronous. This means the done channel must have enough capacity to completely decouple the two processes (i.e. no blocking at all).
Further when the select statement is used as seen, it is the idiomatic way to do a non-blocking channel operation.
I have a struct called Hub with a Run() method which is executed in its own goroutine. This method sequentially handles incoming messages. Messages arrive concurrently from multiple producers (separate goroutines). Of course I use a channel to accomplish this task. But now I want to hide the Hub behind an interface to be able to choose from its implementations. So, using a channel as a simple Hub's field isn't appropriate.
package main
import "fmt"
import "time"
type Hub struct {
msgs chan string
}
func (h *Hub) Run() {
for {
msg, hasMore := <- h.msgs
if !hasMore {
return
}
fmt.Println("hub: msg received", msg)
}
}
func (h *Hub) SendMsg(msg string) {
h.msgs <- msg
}
func send(h *Hub, prefix string) {
for i := 0; i < 5; i++ {
fmt.Println("main: sending msg")
h.SendMsg(fmt.Sprintf("%s %d", prefix, i))
}
}
func main() {
h := &Hub{make(chan string)}
go h.Run()
for i := 0; i < 10; i++ {
go send(h, fmt.Sprintf("msg sender #%d", i))
}
time.Sleep(time.Second)
}
So I've introduced Hub.SendMsg(msg string) function that just calls h.msgs <- msg and which I can add to the HubInterface. And as a Go-newbie I wonder, is it safe from the concurrency perspective? And if so - is it a common approach in Go?
Playground here.
Channel send semantics do not change when you move the send into a method. Andrew's answer points out that the channel needs to be created with make to send successfully, but that was always true, whether or not the send is inside a method.
If you are concerned about making sure callers can't accidentally wind up with invalid Hub instances with a nil channel, one approach is to make the struct type private (hub) and have a NewHub() function that returns a fully initialized hub wrapped in your interface type. Since the struct is private, code in other packages can't try to initialize it with an incomplete struct literal (or any struct literal).
That said, it's often possible to create invalid or nonsense values in Go and that's accepted: net.IP("HELLO THERE BOB") is valid syntax, or net.IP{}. So if you think it's better to expose your Hub type go ahead.
Easy answer
Yes
Better answer
No
Channels are great for emitting data from unknown go-routines. They do so safely, however I would recommend being careful with a few parts. In the listed example the channel is created with the construction of the struct by the consumer (and not not by a consumer).
Say the consumer creates the Hub like the following: &Hub{}. Perfectly valid... Apart from the fact that all the invokes of SendMsg() will block for forever. Luckily you placed those in their own go-routines. So you're still fine right? Wrong. You are now leaking go-routines. Seems fine... until you run this for a period of time. Go encourages you to have valid zero values. In this case &Hub{} is not valid.
Ensuring SendMsg() won't block could be achieved via a select{} however you then have to decide what to do when you encounter the default case (e.g. throw data away). The channel could block for more reasons than bad setup too. Say later you do more than simply print the data after reading from the channel. What if the read gets very slow, or blocks on IO. You then will start pushing back on the producers.
Ultimately, channels allow you to not think much about concurrency... However if this is something of high-throughput, then you have quite a bit to consider. If it is production code, then you need to understand that your API here involves SendMsg() blocking.