func (s *server) send(m *message) error {
go func() {
s.outgoingMessageChan <- message
}()
return nil
}
func main(s *server) {
for {
select {
case <-someChannel:
// do something
case msg := <-s.outGoingMessageChan:
// take message sent from "send" and do something
}
}
}
I am pulling out of this s.outgoingMessageChan in another function, before using an anonymous go function, a call to this function would usually block - meaning whenever send is called, s.outgoingMessageChan <- message would block until something is pulling out of it. However after wrapping it like this it doesn't seem to block anymore. I understand that it kind of sends this operation to the background and proceeds as usual, but I'm not able to wrap my head around how this doesn't affect the current function call.
Each time send is called a new goroutine is created, and returns immediately. (BTW there is no reason to return an error if there can never be an error.) The goroutine (which has it's own "thread" of execution) will block if nothing is ready to read from the chan (assuming it's unbuffered). Once the message is read off the chan the goroutine will continue but since it does nothing else it will simply end.
I should point out that there is no such thing as an anonymous goroutine. Goroutines have no identifier at all (except for a number that you should only use for debugging purposes). You have an anonymous function which you put the go keyword in front causing it to run in a separate goroutine.
For a send function that blocks as you seem to want then just use:
func (s *server) send(m *message) {
s.outgoingMessageChan <- message
}
However, I can't see any point in this function (though it would be inlined and just as efficient as not using a function).
I suspect you may be calling send many times before anything is read from the chan. In this case many new goroutines will be created (each time you call send) which will all block. Each time the chan is read from one will unblock delivering its value and that goroutine will terminate. Doing this you are simply creating an inefficient buffering mechanism. Moreover, if send is called for a prolonged period at a faster rate than the values can be read from the chan then you will eventually run out of memory. Better would be to use a buffered chan (and no goroutines) that once it (the chan) became full exerted "back-pressure" on whatever was producing the messages.
Another point is that the function name main is used to identify the entry point to a program. Please use another name for your 2nd function above. It also seems like it should be a method (using s *server receiver) than a function.
Related
I am unable to figure out why the method requires you to specifically provide a buffered channel.
From the documentation,
func (*Client) Go
func (client *Client) Go(serviceMethod string, args interface{}, reply interface{}, done chan *Call) *Call
Go invokes the function asynchronously. It returns the Call structure
representing the invocation. The done channel will signal when the
call is complete by returning the same Call object. If done is nil, Go
will allocate a new channel. If non-nil, done must be buffered or Go
will deliberately crash.
LeGEC alluded to this in their comment.
Digging in further you will find this bit in client.go
func (call *Call) done() {
select {
case call.Done <- call:
// ok
default:
// We don't want to block here. It is the caller's responsibility to make
// sure the channel has enough buffer space. See comment in Go().
if debugLog {
log.Println("rpc: discarding Call reply due to insufficient Done chan capacity")
}
}
}
From what you can see here is that the library expects the call to be completely asynchronous. This means the done channel must have enough capacity to completely decouple the two processes (i.e. no blocking at all).
Further when the select statement is used as seen, it is the idiomatic way to do a non-blocking channel operation.
I have a struct called Hub with a Run() method which is executed in its own goroutine. This method sequentially handles incoming messages. Messages arrive concurrently from multiple producers (separate goroutines). Of course I use a channel to accomplish this task. But now I want to hide the Hub behind an interface to be able to choose from its implementations. So, using a channel as a simple Hub's field isn't appropriate.
package main
import "fmt"
import "time"
type Hub struct {
msgs chan string
}
func (h *Hub) Run() {
for {
msg, hasMore := <- h.msgs
if !hasMore {
return
}
fmt.Println("hub: msg received", msg)
}
}
func (h *Hub) SendMsg(msg string) {
h.msgs <- msg
}
func send(h *Hub, prefix string) {
for i := 0; i < 5; i++ {
fmt.Println("main: sending msg")
h.SendMsg(fmt.Sprintf("%s %d", prefix, i))
}
}
func main() {
h := &Hub{make(chan string)}
go h.Run()
for i := 0; i < 10; i++ {
go send(h, fmt.Sprintf("msg sender #%d", i))
}
time.Sleep(time.Second)
}
So I've introduced Hub.SendMsg(msg string) function that just calls h.msgs <- msg and which I can add to the HubInterface. And as a Go-newbie I wonder, is it safe from the concurrency perspective? And if so - is it a common approach in Go?
Playground here.
Channel send semantics do not change when you move the send into a method. Andrew's answer points out that the channel needs to be created with make to send successfully, but that was always true, whether or not the send is inside a method.
If you are concerned about making sure callers can't accidentally wind up with invalid Hub instances with a nil channel, one approach is to make the struct type private (hub) and have a NewHub() function that returns a fully initialized hub wrapped in your interface type. Since the struct is private, code in other packages can't try to initialize it with an incomplete struct literal (or any struct literal).
That said, it's often possible to create invalid or nonsense values in Go and that's accepted: net.IP("HELLO THERE BOB") is valid syntax, or net.IP{}. So if you think it's better to expose your Hub type go ahead.
Easy answer
Yes
Better answer
No
Channels are great for emitting data from unknown go-routines. They do so safely, however I would recommend being careful with a few parts. In the listed example the channel is created with the construction of the struct by the consumer (and not not by a consumer).
Say the consumer creates the Hub like the following: &Hub{}. Perfectly valid... Apart from the fact that all the invokes of SendMsg() will block for forever. Luckily you placed those in their own go-routines. So you're still fine right? Wrong. You are now leaking go-routines. Seems fine... until you run this for a period of time. Go encourages you to have valid zero values. In this case &Hub{} is not valid.
Ensuring SendMsg() won't block could be achieved via a select{} however you then have to decide what to do when you encounter the default case (e.g. throw data away). The channel could block for more reasons than bad setup too. Say later you do more than simply print the data after reading from the channel. What if the read gets very slow, or blocks on IO. You then will start pushing back on the producers.
Ultimately, channels allow you to not think much about concurrency... However if this is something of high-throughput, then you have quite a bit to consider. If it is production code, then you need to understand that your API here involves SendMsg() blocking.
EDIT: My question is different from How to write my own Sleep function using just time.After? It has a different variant of the code that's not working for a separate reason and I needed explanation as to why.
I'm trying to solve the homework problem here: https://www.golang-book.com/books/intro/10 (Write your own Sleep function using time.After).
Here's my attempt so far based on the examples discussed in that chapter:
package main
import (
"fmt"
"time"
)
func myOwnSleep(duration int) {
for {
select {
case <-time.After(time.Second * time.Duration(duration)):
fmt.Println("slept!")
default:
fmt.Println("Waiting")
}
}
}
func main() {
go myOwnSleep(3)
var input string
fmt.Scanln(&input)
}
http://play.golang.org/p/fb3i9KY3DD
My thought process is that the infinite for will keep executing the select statement's default until the time.After function's returned channel talks. Problem with the current code being, the latter does not happen, while the default statement is called infinitely.
What am I doing wrong?
In each iteration of your for loop the select statement is executed which involves evaluating the channel operands.
In each iteration time.After() will be called and a new channel will be created!
And if duration is more than 0, this channel is not ready to receive from, so the default case will be executed. This channel will not be tested/checked again, the next iteration creates a new channel which will again not be ready to receive from, so the default case is chosen again - as always.
The solution is really simple though as can be seen in this answer:
func Sleep(sec int) {
<-time.After(time.Second* time.Duration(sec))
}
Fixing your variant:
If you want to make your variant work, you have to create one channel only (using time.After()), store the returned channel value, and always check this channel. And if the channel "kicks in" (a value is received from it), you must return from your function because more values will not be received from it and so your loop will remain endless!
func myOwnSleep(duration int) {
ch := time.After(time.Second * time.Duration(duration))
for {
select {
case <-ch:
fmt.Println("slept!")
return // MUST RETURN, else endless loop!
default:
fmt.Println("Waiting")
}
}
}
Note that though until a value is received from the channel, this function will not "rest" and just execute code relentlessly - loading one CPU core. This might even give you trouble if only 1 CPU core is available (runtime.GOMAXPROCS()), other goroutines (including the one that will (or would) send the value on the channel) might get blocked and never executed. A sleep (e.g. time.Sleep(time.Millisecond)) could release the CPU core from doing endless work (and allow other goroutines to run).
I have two (but later I'll be three) go routines that are handling incoming messages from a remote server (from a ampq channel). But because they are handling on the same data/state, I want to block all other go routines, except the one running.
I come up with a solution to use chan bool where each go routine blocks and then release it, the code is like:
package main
func a(deliveries <-chan amqp, handleDone chan bool) {
for d := range deliveries {
<-handleDone // Data comes always, wait for other channels
handleDone <- false // Block other channels
// Do stuff with data...
handleDone <- true // I'm done, other channels are free to do anything
}
}
func b(deliveries <-chan amqp, handleDone chan bool) {
for d := range deliveries {
<-handleDone
handleDone <- false
// Do stuff with data...
handleDone <- true
}
}
func main() {
handleDone := make(chan bool, 1)
go a(arg1, handleDone)
go b(arg2, handleDone)
// go c(arg3, handleDone) , later
handleDone <- true // kickstart
}
But for the first time each of the function will get handleDone <- true, which they will be executed. And later if I add another third function, things will get more complicated. How can block all other go routines except the running? Any other better solutions?
You want to look at the sync package.
http://golang.org/pkg/sync/
You would do this with a mutex.
If you have an incoming stream of messages and you have three goroutines listening on that stream and processing and you want to ensure that only one goroutine is running at a time, the solution is quite simple: kill off two of the goroutines.
You're spinning up concurrency and adding complexity and then trying to prevent them from running concurrently. The end result is the same as a single stream reader, but with lots of things that can go wrong.
I'm puzzled why you want this - why can't each message on deliveries be handled independently? and why are there two different functions handling those message? If each is responsible for a particular type of message, it seems like you want one deliveries receiver that dispatches to appropriate logic for the type.
But to answer your question, I don't think it's true that each function will get a true from handleDone on start. One (let's say it's a) is receiving the true sent from main; the other (b then) is getting the false sent from the first. Because you're discarding the value received, you can't tell this. And then both are running, and you're using a buffered channel (you probably want make(chan bool) instead for an unbuffered one), so confusion ensues, particularly when you add that third goroutine.
The handleDone <- false doesn't actually accomplish anything. Just treat any value on handleDone as the baton in a relay race. Once a goroutine receives this value, it can do its thing; when it's done, it should send it to the channel to hand it to the next goroutine.
So, right now, I just pass a pointer to a Queue object (implementation doesn't really matter) and call queue.add(result) at the end of goroutines that should add things to the queue.
I need that same sort of functionality—and of course doing a loop checking completion with the comma ok syntax is unacceptable in terms of performance versus the simple queue add function call.
Is there a way to do this better, or not?
There are actually two parts to your question: how does one queue data in Go, and how does one use a channel without blocking.
For the first part, it sounds like what you need to do is instead of using the channel to add things to the queue, use the channel as a queue. For example:
var (
ch = make(chan int) // You can add an int parameter to this make call to create a buffered channel
// Do not buffer these channels!
gFinished = make(chan bool)
processFinished = make(chan bool)
)
func f() {
go g()
for {
// send values over ch here...
}
<-gFinished
close(ch)
}
func g() {
// create more expensive objects...
gFinished <- true
}
func processObjects() {
for val := range ch {
// Process each val here
}
processFinished <- true
}
func main() {
go processObjects()
f()
<-processFinished
}
As for how you can make this more asynchronous, you can (as cthom06 pointed out) pass a second integer to the make call in the second line which will make send operations asynchronous until the channel's buffer is full.
EDIT: However (as cthom06 also pointed out), because you have two goroutines writing to the channel, one of them has to be responsible for closing the channel. Also, my previous revision would exit before processObjects could complete. The way I chose to synchronize the goroutines is by creating a couple more channels that pass around dummy values to ensure that the cleanup gets finished properly. Those channels are specifically unbuffered so that the sends happen in lock-step.