i'm trying to implement a pipeline using buffered channel with size one, it will accept incoming message, print it out and wait some time then pass it forward. The main problem is that reading gobyexample tutorial I decided to implement it using select statement, but I cannot figure out how to make channel block until it will pass the message forward.
func runVertex(vertex Vertex) {
newVertex := findVertex(vertex)
var packet handledPacket
for {
select {
case packet.packet = <-someChannel:
fmt.Println("msg received")
time.Sleep(time.Second)
case someOtherChannel <- packet.packet:
time.Sleep(time.Second)
default:
time.Sleep(10 * time.Millisecond)
}
}
}
You don't need a select to do the thing you described:
for {
packet.packet =<- someChannel:
fmt.Println("msg received");
time.Sleep(time.Second);
someOtherChannel<-packet.packet:
}
The operation of receiving something from a channel is blocking, you don't need to do something special. And selects are intended to let you either handle multiple channels or somehow react if there is nothing new in the channel(s) you are watching. Here is an example:
package main
import (
"fmt"
"time"
)
func messagesPrinter(channel1 chan string, channel2 chan string) {
var message string
for {
select {
case message = <-channel1:
fmt.Println("A message from channel 1:", message)
time.Sleep(time.Second)
case message = <-channel2:
fmt.Println("A message from channel 2:", message)
time.Sleep(time.Second)
default: // Optional part
fmt.Println("Now message yet...")
time.Sleep(500 * time.Millisecond)
}
}
}
func main() {
c1 := make(chan string)
c2 := make(chan string)
go messagesPrinter(c1, c2)
c1 <- "Hello"
c1 <- "How are you?"
time.Sleep(5 * time.Second)
c2 <- "I'm fine!"
}
Related
I am trying to pause and resume groutine. I understand I can sleep the run, but I am looking for is like a button "pause/resume" rather than a timer.
Here is my attempt. I am using the blocking feature of channel to pause, and select to switch what to execute based on channel value. However, the output is always Running in my case.
func main() {
ctx := wctx{}
go func(ctx wctx) {
for {
time.Sleep(1 * time.Second)
select {
case <-ctx.pause:
fmt.Print("Paused")
<-ctx.pause
case <-ctx.resume:
fmt.Print("Resumed")
default:
fmt.Print("Running \n")
}
}
}(ctx)
ctx.pause <- struct{}{}
ctx.resume <- struct{}{}
}
type wctx struct {
pause chan struct{}
resume chan struct{}
}
A select with multiple ready cases chooses one pseudo-randomly. So if the goroutine is "slow" to check those channels, you might send a value on both pause and resume (assuming they are buffered) so receiving from both channels could be ready, and resume could be chosen first, and in a later iteration the pause when the goroutine should not be paused anymore.
For this you should use a "state" variable synchronized by a mutex. Something like this:
const (
StateRunning = iota
StatePaused
)
type wctx struct {
mu sync.Mutex
state int
}
func (w *wctx) SetState(state int) {
w.mu.Lock()
defer w.mu.Unlock()
w.state = state
}
func (w *wctx) State() int {
w.mu.Lock()
defer w.mu.Unlock()
return w.state
}
Testing it:
ctx := &wctx{}
go func(ctx *wctx) {
for {
time.Sleep(1 * time.Millisecond)
switch state := ctx.State(); state {
case StatePaused:
fmt.Println("Paused")
default:
fmt.Println("Running")
}
}
}(ctx)
time.Sleep(3 * time.Millisecond)
ctx.SetState(StatePaused)
time.Sleep(3 * time.Millisecond)
ctx.SetState(StateRunning)
time.Sleep(2 * time.Millisecond)
Output (try it on the Go Playground):
Running
Running
Running
Paused
Paused
Paused
Running
Running
You need to initialize your channels, remember that reads from nil channels always blocks.
A select with a default case never blocks.
Here is a modified version of your program, that fixes the above mentioned issues:
package main
import (
"fmt"
"time"
)
func main() {
ctx := wctx{
pause: make(chan struct{}),
resume: make(chan struct{}),
}
go func(ctx wctx) {
for {
select {
case <-ctx.pause:
fmt.Println("Paused")
case <-ctx.resume:
fmt.Println("Resumed")
}
fmt.Println("Running")
time.Sleep(time.Second)
}
}(ctx)
ctx.pause <- struct{}{}
ctx.resume <- struct{}{}
}
type wctx struct {
pause chan struct{}
resume chan struct{}
}
How we can set something like listener on go channels that when someone has read something from the channel, that notify us?
Imagine we have a sequence number for channel entries and we wanna decrement it when someone had read a value from our channel somewhere out of our package.
Unbuffered channels hand off data synchronously, so you already know when the data is read. Buffered channels work similarly when the buffer is full, but otherwise they don't block the same, so this approach wouldn't tell you quite the same thing. Depending on what your needs really are, consider also using tools like sync.WaitGroup.
ch = make(chan Data)
⋮
for {
⋮
// make data available
ch <- data
// now you know it was read
sequenceNumber--
⋮
}
You could create a channel relay mechanism, to capture read events in realtime.
So for example:
func relayer(in <-chan MyStruct) <-chan MyStruct {
out := make(chan MyStruct) // non-buffered chan (see below)
go func() {
defer close(out)
readCountLimit := 10
for item := range in {
out <- item
// ^^^^ so this will block until some worker has read from 'out'
readCountLimit--
}
}()
return out
}
Usage:
type MyStruct struct {
// put your data fields here
}
ch := make(chan MyStruct) // <- original channel - used by producer to write to
rch := relayer(ch) // <- relay channel - used to read from
// consumers
go worker("worker 1", rch)
go worker("worker 2", rch)
// producer
for { ch <- MyStruct{} }
You can do it in manual mode. implement some sort of ACK marker to the message.
Something like this:
type Msg struct {
Data int
ack bool
}
func (m *Msg) Ack() {
m.ack = true
}
func (m *Msg) Acked() bool {
return m.ack
}
func main() {
ch := make(chan *Msg)
msg := &Msg{Data: 1}
go func() {
for {
if msg.Acked() {
// do smth
}
time.Sleep(10 * time.Second)
}
}()
ch <- msg
for msg := range ch {
msg.Ack()
}
}
Code not tested.
You can also add some additional information to Ack() method, say meta information about package and func, from where Ack() was called, this answer may be related: https://stackoverflow.com/a/35213181/3782382
Hello i learn about go routine and channel.
I do some experiment with channel, i send a data over channel and try to catch it in 2 functions. But my second function not run
Here is my code :
package main
import (
"fmt"
"os"
"time"
)
func timeout(duration int, ch chan<- bool) {
time.AfterFunc(time.Duration(duration)*time.Second, func() {
ch <- true
})
}
func watcher(duration int, ch <-chan bool) {
<-ch
fmt.Println("\nTimeout! no Answer after", duration, "seconds")
os.Exit(0)
}
func watcher2(duration int, ch <-chan bool) {
<-ch
fmt.Println("This is watcher 2 as a second receiver")
}
func main() {
var data = make(chan bool)
var duration = 5
go timeout(duration, data)
go watcher(duration, data)
go watcher2(duration, data)
var input string
fmt.Print("What is 725/25 ? ")
fmt.Scan(&input)
if input == "29" {
fmt.Println("Correct")
} else {
fmt.Println("Wrong!")
}
}
Can you tell me some explanation about it?
Thank you
As #Andy Schweig mentioned, you can pull from Go channel only once. If you still want to receive message twice, you can use Observer design pattern:
import "fmt"
type Observer interface {
Notify(message string)
}
type Watcher struct {
name string
}
func (w Watcher) Notify(message string) {
fmt.Printf("Watcher %s got message %s\n", w.name, message)
}
var watchers = [...]Watcher {{name: "Watcher 1"}, {name: "Watcher 2"}}
var c = make(chan string)
func notifier() {
var message string
for {
// Messaged pulled only once
message = <- c
// But all watchers still receive it
for _, w := range watchers {
w.Notify(message)
}
}
}
func main() {
go notifier()
c <- "hello"
c <- "how are you?"
}
The channel you declared can only deal with one receiver. By default channels are unbuffered, meaning that they will only accept sends if there is a corresponding receiver to receive the sent value. Whereas a buffered channel accept a limited number of values without a corresponding receiver for those values. If you are looking to inject multiple input and its subsequent receive then you need declare your channel as buffered channel.
ch := make(chan bool, n) //n being the number of items to buffer
I would like to broadcast data received from a channel to a list of channel. The list of channel is dynamic an can be modified during the run phase.
As a new developper in Go, I wrote this code. I found it quite heavy for what I want. Is-there a better way to do this?
package utils
import "sync"
// StringChannelBroadcaster broadcasts string data from a channel to multiple channels
type StringChannelBroadcaster struct {
Source chan string
Subscribers map[string]*StringChannelSubscriber
stopChannel chan bool
mutex sync.Mutex
capacity uint64
}
// NewStringChannelBroadcaster creates a StringChannelBroadcaster
func NewStringChannelBroadcaster(capacity uint64) (b *StringChannelBroadcaster) {
return &StringChannelBroadcaster{
Source: make(chan string, capacity),
Subscribers: make(map[string]*StringChannelSubscriber),
capacity: capacity,
}
}
// Dispatch starts dispatching message
func (b *StringChannelBroadcaster) Dispatch() {
b.stopChannel = make(chan bool)
for {
select {
case val, ok := <-b.Source:
if ok {
b.mutex.Lock()
for _, value := range b.Subscribers {
value.Channel <- val
}
b.mutex.Unlock()
}
case <-b.stopChannel:
return
}
}
}
// Stop stops the Broadcaster
func (b *StringChannelBroadcaster) Stop() {
close(b.stopChannel)
}
// StringChannelSubscriber defines a subscriber to a StringChannelBroadcaster
type StringChannelSubscriber struct {
Key string
Channel chan string
}
// NewSubscriber returns a new subsriber to the StringChannelBroadcaster
func (b *StringChannelBroadcaster) NewSubscriber() *StringChannelSubscriber {
key := RandString(20)
newSubscriber := StringChannelSubscriber{
Key: key,
Channel: make(chan string, b.capacity),
}
b.mutex.Lock()
b.Subscribers[key] = &newSubscriber
b.mutex.Unlock()
return &newSubscriber
}
// RemoveSubscriber removes a subscrber from the StringChannelBroadcaster
func (b *StringChannelBroadcaster) RemoveSubscriber(subscriber *StringChannelSubscriber) {
b.mutex.Lock()
delete(b.Subscribers, subscriber.Key)
b.mutex.Unlock()
}
Thank you,
Julien
I think you can simplify it a bit: get rid of stopChannel and the Stop method. You can just close Source instead of calling Stop, and detect that in Dispatch (ok will be false) to quit (you can just range over the source channel actually).
You can get rid of Dispatch, and just start a goroutine in NewStringChannelBroadcaster with the for cycle, so external code doesn't have to start the dispatch cycle separately.
You can use a channel type as the map key, so your map can become map[chan string]struct{} (empty struct because you don't need the map value). So your NewSubscriber can take a channel type parameter (or create a new channel and return it), and insert that into the map, you don't need the random string or the StringChannelSubscriber type.
I also made some improvements, like closing the subscriber channels:
package main
import "sync"
import (
"fmt"
"time"
)
// StringChannelBroadcaster broadcasts string data from a channel to multiple channels
type StringChannelBroadcaster struct {
Source chan string
Subscribers map[chan string]struct{}
mutex sync.Mutex
capacity uint64
}
// NewStringChannelBroadcaster creates a StringChannelBroadcaster
func NewStringChannelBroadcaster(capacity uint64) *StringChannelBroadcaster {
b := &StringChannelBroadcaster{
Source: make(chan string, capacity),
Subscribers: make(map[chan string]struct{}),
capacity: capacity,
}
go b.dispatch()
return b
}
// Dispatch starts dispatching message
func (b *StringChannelBroadcaster) dispatch() {
// for iterates until the channel is closed
for val := range b.Source {
b.mutex.Lock()
for ch := range b.Subscribers {
ch <- val
}
b.mutex.Unlock()
}
b.mutex.Lock()
for ch := range b.Subscribers {
close(ch)
// you shouldn't be calling RemoveSubscriber after closing b.Source
// but it's better to be safe than sorry
delete(b.Subscribers, ch)
}
b.Subscribers = nil
b.mutex.Unlock()
}
func (b *StringChannelBroadcaster) NewSubscriber() chan string {
ch := make(chan string, b.capacity)
b.mutex.Lock()
if b.Subscribers == nil {
panic(fmt.Errorf("NewSubscriber called on closed broadcaster"))
}
b.Subscribers[ch] = struct{}{}
b.mutex.Unlock()
return ch
}
// RemoveSubscriber removes a subscrber from the StringChannelBroadcaster
func (b *StringChannelBroadcaster) RemoveSubscriber(ch chan string) {
b.mutex.Lock()
if _, ok := b.Subscribers[ch]; ok {
close(ch) // this line does have to be inside the if to prevent close of closed channel, in case RemoveSubscriber is called twice on the same channel
delete(b.Subscribers, ch) // this line doesn't need to be inside the if
}
b.mutex.Unlock()
}
func main() {
b := NewStringChannelBroadcaster(0)
var toberemoved chan string
for i := 0; i < 3; i++ {
i := i
ch := b.NewSubscriber()
if i == 1 {
toberemoved = ch
}
go func() {
for v := range ch {
fmt.Printf("receive %v: %v\n", i, v)
}
fmt.Printf("Exit %v\n", i)
}()
}
b.Source <- "Test 1"
b.Source <- "Test 2"
// This is a race condition: the second reader may or may not receive the first two messages.
b.RemoveSubscriber(toberemoved)
b.Source <- "Test 3"
// let the reader goroutines receive the last message
time.Sleep(2 * time.Second)
close(b.Source)
// let the reader goroutines write close message
time.Sleep(1 * time.Second)
}
https://play.golang.org/p/X-NcikvbDM
Edit: I've added your edit to fix the panic when calling RemoveSubscriber after closing Source, but you shouldn't be doing that, you should let the struct and everything in it be garbage collected after the channel is closed.
I've also added a panic to NewSubscriber if it's called after closing Source. Previously you could do that and it'd leak the created channel and presumably the goroutine that will block forever on that channel.
If you can call NewSubscriber (or RemoveSubscriber) on an already closed broadcaster, that probably means there's an error in your code somewhere, since you're holding on to a broadcaster that you shouldn't be.
I'm pretty new to Go so sorry if the topic is wrong but I hope you understand my question. I want to process events to different go routines via a channel. Here is some sample code
type Event struct {
Host string
Command string
Output string
}
var (
incoming = make(chan Event)
)
func processEmail(ticker* time.Ticker) {
for {
select {
case t := <-ticker.C:
fmt.Println("Email Tick at", t)
case e := <-incoming:
fmt.Println("EMAIL GOT AN EVENT!")
fmt.Println(e)
}
}
}
func processPagerDuty(ticker* time.Ticker) {
for {
select {
case t := <-ticker.C:
fmt.Println("Pagerduty Tick at", t)
case e := <-incoming:
fmt.Println("PAGERDUTY GOT AN EVENT!")
fmt.Println(e)
}
}
}
func main() {
err := gcfg.ReadFileInto(&cfg, "dispatch-api.cfg")
if err != nil {
fmt.Printf("Error loading the config")
}
ticker := time.NewTicker(time.Second * 10)
go processEmail(ticker)
ticker := time.NewTicker(time.Second * 1)
go processPagerDuty(ticker)
}
func eventAdd(r render.Render, params martini.Params, req *http.Request) {
// create an event now
e := Event{Host: "web01-east.domain.com", Command: "foo", Output: "bar"}
incoming <- e
}
So the ticker events work just create. When I issue an API call to create an event I just get output from the processEmail function. Its whatever go routine is called first will get the event over the channel.
Is there a way for both functions to get that event?
You can use fan in and fan out (from Rob Pike's speech):
package main
func main() {
// feeders - feeder1, feeder2 and feeder3 are used to fan in
// data into one channel
go func() {
for {
select {
case v1 := <-feeder1:
mainChannel <- v1
case v2 := <-feeder2:
mainChannel <- v2
case v3 := <-feeder3:
mainChannel <- v3
}
}
}()
// dispatchers - not actually fan out rather dispatching data
go func() {
for {
v := <-mainChannel
// use this to prevent leaking goroutines
// (i.e. when one consumer got stuck)
done := make(chan bool)
go func() {
consumer1 <- v
done <- true
}()
go func() {
consumer2 <- v
done <- true
}()
go func() {
consumer3 <- v
done <- true
}()
<-done
<-done
<-done
}
}()
// or fan out (when processing the data by just one consumer is enough)
go func() {
for {
v := <-mainChannel
select {
case consumer1 <- v:
case consumer2 <- v:
case consumer3 <- v:
}
}
}()
// consumers(your logic)
go func() { <-consumer1 /* using the value */ }()
go func() { <-consumer2 /* using the value */ }()
go func() { <-consumer3 /* using the value */ }()
}
type payload int
var (
feeder1 = make(chan payload)
feeder2 = make(chan payload)
feeder3 = make(chan payload)
mainChannel = make(chan payload)
consumer1 = make(chan payload)
consumer2 = make(chan payload)
consumer3 = make(chan payload)
)
Channels are a point to point communication method, not a broadcast communication method, so no, you can't get both functions to get the event without doing something special.
You could have separate channels for both goroutines and send the message into each. This is probably the simplest solution.
Or alternatively you could get one goroutine to signal the next one.
Go has two mechanisms for doing broadcast signalling as far as I know. One is closing a channel. This only works a single time though.
The other is to use a sync.Cond lock. These are moderately tricky to use, but will allow you to have multiple goroutines woken up by a single event.
If I was you, I'd go for the first option, send the event to two different channels. That seems to map the problem quite well.