I'm just playing around with Go, taking it for a test drive so to speak. I'm having a problem where a go routine that is mean to receive 3 integers only seems to receive one.
type simpleFunction func() int
func run(fChan chan simpleFunction, result chan int) {
for{
select {
case fn := <-fChan:
fmt.Printf("sending: %d down result chan\n", fn())
result <- fn()
case <-time.After(time.Second * 2):
close(fChan)
}
}
}
func recieve(result chan int){
for {
select {
case x := <-result:
fmt.Printf("recieved: %d from result chan\n", x)
case <-time.After(time.Second * 2):
close(result)
}
}
}
So, as you can see the run routine receives functions, evaluates them, and then sends the result down the result channel.
Here's my main/test:
func main() {
fns := []simpleFunction{
func() int {return 1},
func() int {return 2},
func() int {return 3},
}
fChan := make(chan simpleFunction)
result := make(chan int)
go run(fChan, result)
go recieve(result)
for _, fn := range fns {
fmt.Printf("sending a function that returns: %d down function chan\n", fn())
fChan <- fn
}
}
And here's my output:
sending a function that returns: 1 down function chan
sending: 1 down result chan
recieved: 1 from result chan
sending a function that returns: 2 down function chan
sending a function that returns: 3 down function chan
sending: 2 down result chan
sending: 3 down result chan
So, as you can see, everything seems to go swimmingly for the first function, but it's not so hot afterwards. Any tips or suggestions?
There are a couple of issues with this code:
The program terminates when main returns. It does not wait for the run and receive goroutines to complete.
There's a race on closing the channels. There's no guarantee that the sender will top sending before the timeout.
If main does not exit, then the for { select { } } loops will spin forever printing zero values. Receive on a closed channel returns the zero value.
Related
What I am trying to do is to essentially create a buffered channel effect using goroutines.
What I have is a function called Elem that looks as follows:
func Elem(in, out chan int) {
for {
buf := <-in
out <- buf
}
}
Originally the purpose of this function was to avoid blocking of a goroutine when it attempts to send an int value to another goroutine via a channel, and that goroutine is not receiving the value. So instead Elem acts as an intermediary which collects the value from the channel and blocks on behalf of the sending goroutine (assuming that no one is trying to receive the value).
Coming back to the point of the question, is there a way to chain an arbitrary number of these Elem goroutines together. So essentially what I am looking for is a function that takes 2 channels and an integer. The 2 channels represent the input channel for the 1st Elem goroutine and the output channel for the last Elem goroutine in the pipeline.
func ChainElems(in, out chan int, numOfElemsToChain int) {
for i:=0; i<numOfElemsToChain; i++ { //this doesn't have to be a loop I just thought that this is how you would go about it
//chaining logic
}
}
What I did is something like this:
package main
func Elem(in, out chan int) {
for {
buf := <-in
out <- buf
}
}
func main() {
in := make(chan int)
out1 := make(chan int)
out2 := make(chan int)
out3 := make(chan int)
go Elem(in, out1)
go Elem(out1, out2)
go Elem(out2, out3)
in <- 3
in <- 6
in <- 7
fmt.Println(<-out3)
fmt.Println(<-out3)
fmt.Println(<-out3)
}
However this is sort of "hardcoded" (only for 3 goroutines). I want to make this process a function for any number of Elems that I want to chain. Is this possible?
go version go1.11.4 darwin/amd64
A new channel and goroutine were created, and the content of the old channel was transferred to the new channel through goroutine. It should not block, but after testing, it was found to be blocked.
thanks.
type waiter struct {
ch1 chan struct{}
ch2 <-chan time.Time
limit int
count int
}
func (w *waiter) recv1Block() chan struct{} {
ch := make(chan struct{})
go func() {
for m := range w.ch1 {
ch <- m
}
}()
return ch
}
func (w *waiter) runBlock(wg *sync.WaitGroup) {
defer wg.Done()
i := 0
for i < w.limit {
select {
case <-w.recv1Block(): **// why block here?**
i++
case <-w.recv2():
}
}
w.count = i
}
why recv1Block will be block.
Every time you call recv1Block(), it creates a new channel and launches a background goroutine that tries to read all of the data from it. Since you're calling it in a loop, you will have many things all trying to consume the data from the channel; since the channel never closes, all of the goroutines will run forever.
As an exercise, you might try changing your code to pass around a chan int instead of a chan struct{}, and write a series of sequential numbers, and print them out as they're ultimately received. A sequence like this is valid:
run on goroutine #1 calls recv1Block().
recv1Block() on GR#1 spawns GR#2, and returns channel#2.
run on GR#1 blocks receiving on channel#2.
recv1Block() on GR#2 reads 0 from w.c1.
recv1Block() on GR#2 writes 0 to channel#2 (where run on GR#1 is ready to read).
recv1Block() on GR#2 reads 1 from w.c1.
recv1Block() on GR#2 wants to write 0 to channel#2 but blocks.
run on GR#1 wakes up, and receives the 0.
run on GR#1 calls recv1Block().
recv1Block() on GR#1 spawns GR#3, and returns channel #3.
recv1Block() on GR#3 reads 2 from w.c1.
...
Notice that the value 1 in this sequence will never be written anywhere, and in fact there is nothing left that could read it.
The easy solution here is to not call the channel-creating function in a loop:
func (w *waiter) runBlock(wg *sync.WaitGroup) {
defer wg.Done()
ch1 := w.recv1Block()
ch2 := w.recv2()
for {
select {
case _, ok := <-ch1:
if !ok {
return
}
w.count++
case <-ch2:
}
}
It's also standard practice to close channels when you're done with them. This will terminate a for ... range ch loop, and it will appear as readable to a select statement. In your top-level generator function:
for i := 0; i < w.limit; i++ {
w.ch1 <- struct{}{}
}
close(w.ch1)
And in your "copy the channel" function:
func (w *waiter) recv1Block() chan struct{} {
ch := make(chan struct{})
go func() {
for m := range w.ch1 {
ch <- m
}
close(ch)
}()
return ch
}
This also means that you don't need to run the main loop by "dead reckoning", expecting it to produce exactly 100 items then stop; you can stop whenever its channel exits. The consumer loop I show above does this.
I have a goroutine which can generate an infinite number of values (each more suitable than the last), but it takes progressively longer to find each values. I'm trying to find a way to add a time limit, say 10 seconds, after which my function does something with the best value received so far.
This is my current "solution", using a channel and timer:
// the goroutine which runs infinitely
// (or at least a very long time for high values of depth)
func runSearch(depth int, ch chan int) {
for i := 1; i <= depth; i++ {
fmt.Printf("Searching to depth %v\n", i)
ch <- search(i)
}
}
// consumes progressively better values until the channel is closed
func awaitBestResult(ch chan int) {
var result int
for result := range ch {
best = result
}
// do something with best result here
}
// run both consumer and producer
func main() {
timer := time.NewTimer(time.Millisecond * 2000)
ch := make(chan int)
go runSearch(1000, ch)
go awaitBestResult(ch)
<-timer.C
close(ch)
}
This mostly works - the best result is processed after the timer ends and the channel is closed. However, I then get a panic (panic: send on closed channel) from the runSearch goroutine, since the channel has been closed by the main function.
How can I stop the first goroutine running after the timer has completed? Any help is very appreciated.
You need to ensure that the goroutine knows when it is done processing, so that it doesn't attempt to write to a closed channel, and panic.
This sounds like a perfect case for the context package:
func runSearch(ctx context.Context, depth int, ch chan int) {
for i := 1; i <= depth; i++ {
select {
case <- ctx.Done()
// Context cancelled, return
return
default:
}
fmt.Printf("Searching to depth %v\n", i)
ch <- search(i)
}
}
Then in main():
// run both consumer and producer
func main() {
ctx := context.WithTimeout(context.Background, 2 * time.Second)
ch := make(chan int)
go runSearch(ctx, 1000, ch)
go awaitBestResult(ch)
close(ch)
}
You are getting a panic because your sending goroutine runSearch apparently outlives the timer and it is trying to send a value on the channel which is already closed by your main goroutine. You need to devise a way to signal the sending go routine not to send any values once your timer is lapsed and before you close the channel in main. On the other hand if your search gets over sooner you also need to communicate to main to move on. You can use one channel and synchronize so that there are no race conditions. And finally you need to know when your consumer has processed all the data before you can exit main.
Here's something which may help.
package main
import (
"fmt"
"sync"
"time"
)
var mu sync.Mutex //To protect the stopped variable which will decide if a value is to be sent on the signalling channel
var stopped bool
func search(i int) int {
time.Sleep(1 * time.Millisecond)
return (i + 1)
}
// (or at least a very long time for high values of depth)
func runSearch(depth int, ch chan int, stopSearch chan bool) {
for i := 1; i <= depth; i++ {
fmt.Printf("Searching to depth %v\n", i)
n := search(i)
select {
case <-stopSearch:
fmt.Println("Timer over! Searched till ", i)
return
default:
}
ch <- n
fmt.Printf("Sent depth %v result for processing\n", i)
}
mu.Lock() //To avoid race condition with timer also being
//completed at the same time as execution of this code
if stopped == false {
stopped = true
stopSearch <- true
fmt.Println("Search completed")
}
mu.Unlock()
}
// consumes progressively better values until the channel is closed
func awaitBestResult(ch chan int, doneProcessing chan bool) {
var best int
for result := range ch {
best = result
}
fmt.Println("Best result ", best)
// do something with best result here
//and communicate to main when you are done processing the result
doneProcessing <- true
}
func main() {
doneProcessing := make(chan bool)
stopSearch := make(chan bool)
// timer := time.NewTimer(time.Millisecond * 2000)
timer := time.NewTimer(time.Millisecond * 12)
ch := make(chan int)
go runSearch(1000, ch, stopSearch)
go awaitBestResult(ch, doneProcessing)
select {
case <-timer.C:
//If at the same time runsearch is also completed and trying to send a value !
//So we hold a lock before sending value on the channel
mu.Lock()
if stopped == false {
stopped = true
stopSearch <- true
fmt.Println("Timer expired")
}
mu.Unlock()
case <-stopSearch:
fmt.Println("runsearch goroutine completed")
}
close(ch)
//Wait for your consumer to complete processing
<-doneProcessing
//Safe to exit now
}
On playground. Change the value of timer to observe both the scenarios.
I have a channel that will receive bursts of writes to it. I want to wait until a burst of sends on the channel have finished before triggering an action.
I have looked at this gist, however, it will send on the output every intervalif there is data in the buffer:
func debounceChannel(interval time.Duration, output chan int) chan int {
input := make(chan int)
go func() {
var buffer int
var ok bool
// We do not start waiting for interval until called at least once
buffer, ok = <-input
// If channel closed exit, we could also close output
if !ok {
return
}
// We start waiting for an interval
for {
select {
case buffer, ok = <-input:
// If channel closed exit, we could also close output
if !ok {
return
}
case <-time.After(interval):
// Interval has passed and we have data, so send it
output <- buffer
// Wait for data again before starting waiting for an interval
buffer, ok = <-input
if !ok {
return
}
// If channel is not closed we have more data and start waiting for interval
}
}
}()
return input
}
In my case, I want to wait until there is no longer any data being sent on the input channel for this burst before triggering or sending on the output.
How do I achieve this?
Sounds like you need synchronization between goroutines, perhaps along this line.
func main() {
// Create a channel for our input
input := make(chan int, 1)
// Create another for synchronization between main and forked goroutines
done := make(chan bool)
go func() {
// block-wait for received value
<-input
// do some more things here
// when done, send signal to the main goroutine
done <- true
}()
// Do something while wait for the forked goroutine
// this block until `<-done`
<-done
close(mychan)
}
This post explains quite clear about synchronization using channels and sync group.
This is what I ended up implementing as my debouncer:
func Debounce(lull time.Duration, in chan struct{}, out chan struct{}) {
go func() {
var last int64 = 0
for {
select {
case <-in:
last = time.Now().Unix()
case <-time.Tick(lull):
if last != 0 && time.Now().Unix() >= last+int64(lull.Seconds()) {
last = 0
out <- struct{}{}
}
}
}
}()
}
It takes a lull time which is the duration where if we do not receive on the input, then we assume there is a break in the bursts of data. There are 2 channels, 1 input and 1 output. The bursts of data arrives on the input, and for each burst, we write to the output channel at the end of the burst.
The implementation is extremely simplistic. I just store the current unix timestamp every time I receive from the input channel. Then, I have a ticker ticking with a duration of the lull time. All this does is check to see if we've exceeded the wait time for the last burst. If so, it resets last to 0 an emits an event on the output channel.
Here's some code using the debounce function with a lull time of 2 seconds which sends random bursts on the input channel:
func main() {
out := make(chan struct{})
in := make(chan struct{})
Debounce(2*time.Second, in, out)
// Generating bursts of input data
go func(in chan struct{}) {
for {
select {
case <-time.Tick(1 * time.Second):
in <- struct{}{}
fmt.Println("Sending!")
shouldSleep := rand.Intn(2)
if shouldSleep == 1 {
time.Sleep(5 * time.Second)
}
}
}
}(in)
// Listening for output events
go func(out chan struct{}) {
for _ = range out {
fmt.Println("Got an event!")
}
}(out)
// Do not let main terminate.
done := make(chan struct{})
<-done
}
What I've used as a debouncer:
package pkg
import (
"context"
"time"
)
// Debounce takes a channel, and will notify the output channel with the last received message after a lull duration.
// Upon cancel, it will check one last time for a message.
// Cancelling the ctx will cause the goroutine to exit.
func Debounce[T any](ctx context.Context, lull time.Duration, input <-chan T, output chan<- T) {
go func() {
var (
buffer *T
minTimer = time.NewTimer(min)
flush = func() {
if buffer != nil {
// don't block if we couldn't send
select {
case output <- *buffer:
default:
}
buffer = nil
}
}
)
defer minTimer.Stop()
hits := 0
for {
select {
case <-ctx.Done():
// try and get last message
select {
case tmpBuf, ok := <-input:
if !ok {
break
}
buffer = &tmpBuf
default:
}
flush()
return
case tmpBuf, ok := <-input:
if !ok {
flush()
return
}
hits++
buffer = &tmpBuf
case <-minTimer.C:
flush()
minTimer.Reset(min)
}
}
}()
}
I try to write a simple code in GO, in which two go routine (Send and Receive) send each other integers. I give the code below. Can anybody help me why the output of this program is [no output]. Is there any silly mistake (sorry, I am new in GO) ?
package main
func Send (in1 <-chan int, out1 chan<- int) {
i := 2
out1 <- i
print(i, "\n")
}
func Receive (in <-chan int, out chan<- int) {
i := <-in
print(i, "\n")
out <- i
}
func main() {
for i := 0; i < 10; i++ {
ch1 := make(chan int)
ch := make(chan int)
go Send (ch1 , ch)
go Receive (ch , ch1)
ch = ch1
ch1 = ch
}
}
How about this:
package main
func Send (ch chan<- int) {
for i := 0; i < 10; i++ {
print(i, " sending\n")
ch <- i
}
}
func Receive (ch <-chan int) {
for i := 0; i < 10; i++ {
print(<-ch, " received\n")
}
}
func main() {
ch := make(chan int)
go Receive(ch)
Send(ch)
}
The output of this when I run it on golang.org is:
0 sending
0 received
1 sending
2 sending
1 received
2 received
3 sending
4 sending
3 received
4 received
5 sending
6 sending
5 received
6 received
7 sending
8 sending
7 received
8 received
9 sending
I'm not sure why the 9 was never received. There should be some way to sleep the main thread until the Receive goroutine has completed. Also, it's inelegant that the receiver and the sender both know that they are going to send 10 numbers. One of the goroutines should just shut off when the other has finished its work. I'm not sure how to do that.
EDIT1:
Here is an implementation with two channels, and the go routines send ints back and forth between eachother. One is designated as the responder, who only sends and int after it receives an int. The responder simply adds two to the int he receives, and then sends it back.
package main
func Commander(commands chan int, responses chan int) {
for i := 0; i < 10; i++ {
print(i, " command\n")
commands <- i
print(<-responses, " response\n");
}
close(commands)
}
func Responder(commands chan int, responses chan int) {
for {
x, open := <-commands
if !open {
return;
}
responses <- x + 2
}
}
func main() {
commands := make(chan int)
responses := make(chan int)
go Commander(commands, responses)
Responder(commands, responses)
}
The output when I run it on golang.org is:
0 command
2 response
1 command
3 response
2 command
4 response
3 command
5 response
4 command
6 response
5 command
7 response
6 command
8 response
7 command
9 response
8 command
10 response
9 command
11 response
David Grayson's fix works, but the explanation of why 9 is not always received wouldn't fit into a comment!
When the sending goroutine sends the value on the channel, it blocks until the receiving goroutine receives it. At that point, it's unblocked. The go scheduler might return from main almost immediately before it gives the receiving goroutine a chance to print anything. If you set GOMAXPROCS to something greater than 1, this happens less frequently because the receiving goroutine won't block while the main goroutine is active.
If you want to stop main from returning until all goroutines are finished, you can return a channel from Receive(), like this:
package main
import "fmt"
func Send(ch chan<- int) {
for i := 0; i < 10; i++ {
fmt.Println(i, " sending")
ch <- i
}
close(ch)
}
func Receive(ch <-chan int) (<-chan bool) {
done := make(chan bool)
go func() {
for {
i, ok := <-ch
if !ok {
break
}
fmt.Println(i, " received")
}
done <- true
}()
return done
}
func main() {
ch := make(chan int)
d := Receive(ch)
Send(ch)
_ = <-d
}
As for why your output was empty in your original example: you make two goroutines, and thus main is totally unblocked, so it just returns and the program exits! I've definitely made this mistake too, but the runtime only joins with the main goroutine, not with ALL goroutines, like you may expect.