I'm trying to build a function that I pass a channel to and when run in a go routine it will constantly post updates (in this instance, values of sin) to a channel. When data is sent the the channel, I then want to send it over a web socket.
func sineWave(value chan float64) {
var div float64
sinMult := 6.2839
i := 0
log.Println("started")
for {
div = (float64(i+1) / sinMult)
log.Println(math.Sin(div))
time.Sleep(100 * time.Millisecond)
value <- math.Sin(div)
// log.Println()
i++
if i == 45 {
i = 0
}
}
// log.Println(math.Sin(div * math.Pi))
}
It seems to get stuck at value <- main.Sin(div) stopping the rest of main() from running. How do i get sineWave to run indefinitely in the background and to print its output in another function as they arrive?
There is several mistakes in this code,
the value chan is never drained, so any write will block
the value chan is never closed, so any drain will be infinite
A channel must always be drained, a channel must be closed at some point.
Also, please post reproducible examples, otherwise it is difficult to diagnose the issue.
This is a slightly modified but working version of the OP code.
package main
import (
"fmt"
"math"
"time"
)
func sineWave(value chan float64) {
defer close(value) // A channel must always be closed by the writer.
var div float64
sinMult := 6.2839
i := 0
fmt.Println("started")
for {
div = (float64(i+1) / sinMult)
time.Sleep(100 * time.Millisecond)
value <- math.Sin(div)
i++
if i == 4 {
// i = 0 // commented in order to quit the loop, thus close the channel, thus end the main for loop
break
}
}
}
func main() {
value := make(chan float64)
go sineWave(value) // start writing the values in a different routine
// drain the channel, it will end the loop whe nthe channel is closed
for v := range value {
fmt.Println(v)
}
}
Related
I'm starting out with Go and I'm now writing a simple program which reads out data from a sensor and puts that into a channel to do some calculations with it. I now have it working as follows:
package main
import (
"fmt"
"time"
"strconv"
)
func get_sensor_data(c chan float64) {
time.Sleep(1 * time.Second) // wait a second before sensor data starts pooring in
c <- 2.1 // Sensor data starts being generated
c <- 2.2
c <- 2.3
c <- 2.4
c <- 2.5
}
func main() {
s := 1.1
c := make(chan float64)
go get_sensor_data(c)
for {
select {
case s = <-c:
fmt.Println("the next value of s from the channel: " + strconv.FormatFloat(s, 'f', 1, 64))
default:
// no new values in the channel
}
fmt.Println(s)
time.Sleep(500 * time.Millisecond) // Do heavy "work"
}
}
This works fine, but the sensor generates a lot of data, and I'm always only interested in the latest data. With this setup however, it only reads out the next item with every loop, which means that if the channel at some point contains 20 values, the newest value only is read out after 10 seconds.
Is there a way for a channel to always only contain one value at a time, so that I always only get the data I'm interested in, and no unnecessary memory is used by the channel (although the memory is the least of my worries)?
Channels are best thought of as queues (FIFO). Therefore you can't really skip around. However there are libraries out there that do stuff like this: https://github.com/cloudfoundry/go-diodes is an atomic ring buffer that will overwrite old data. You can set a smaller size if you like.
All that being said, it doesn't sound like you need a queue (or ring buffer). You just need a mutex:
type SensorData struct{
mu sync.RWMutex
last float64
}
func (d *SensorData) Store(data float64) {
mu.Lock()
defer mu.Unlock()
d.last = data
}
func (d *SensorData) Get() float64 {
mu.RLock()
defer mu.RUnlock()
return d.last
}
This uses a RWMutex which means many things can read from it at the same time while only a single thing can write. It will store a single entry much like you said.
No. Channels are FIFO buffers, full stop. That is how channels work and their only purpose. If you only want the latest value, consider just using a single variable protected by a mutex; write to it whenever new data comes in, and whenever you read it, you will always be reading the latest value.
Channels serves a specific purpose. You might want to use a code that is inside a lock and update the variable whenever new value is to be set.
This way reciever will always get the latest value.
You cannot get that from one channel directly, but you can use one channel per value and get notified when there are new values:
package main
import (
"fmt"
"strconv"
"sync"
"time"
)
type LatestChannel struct {
n float64
next chan struct{}
mu sync.Mutex
}
func New() *LatestChannel {
return &LatestChannel{next: make(chan struct{})}
}
func (c *LatestChannel) Push(n float64) {
c.mu.Lock()
c.n = n
old := c.next
c.next = make(chan struct{})
c.mu.Unlock()
close(old)
}
func (c *LatestChannel) Get() (float64, <-chan struct{}) {
c.mu.Lock()
n := c.n
next := c.next
c.mu.Unlock()
return n, next
}
func getSensorData(c *LatestChannel) {
time.Sleep(1 * time.Second)
c.Push(2.1)
time.Sleep(100 * time.Millisecond)
c.Push(2.2)
time.Sleep(100 * time.Millisecond)
c.Push(2.3)
time.Sleep(100 * time.Millisecond)
c.Push(2.4)
time.Sleep(100 * time.Millisecond)
c.Push(2.5)
}
func main() {
s := 1.1
c := New()
_, hasNext := c.Get()
go getSensorData(c)
for {
select {
case <-hasNext:
s, hasNext = c.Get()
fmt.Println("the next value of s from the channel: " + strconv.FormatFloat(s, 'f', 1, 64))
default:
// no new values in the channel
}
fmt.Println(s)
time.Sleep(250 * time.Millisecond) // Do heavy "work"
}
}
If you do not need the notify about new value, you can try to read Channels inside channels pattern in Golang.
Try this package https://github.com/subbuv26/chanup
It allows the producer to update the channel with latest value, which replaces the latest value. And produces does not get blocked. (with this, stale values gets overridden).
So, on the consumer side, always only the latest item gets read.
import "github.com/subbuv26/chanup"
ch := chanup.GetChan()
_ := ch.Put(testType{
a: 10,
s: "Sample",
})
_ := ch.Update(testType{
a: 20,
s: "Sample2",
})
// Continue updating with latest values
...
...
// On consumer end
val := ch.Get()
// val contains latest value
There is another way to solve this problem (trick)
sender work faster: sender remove channel if channel_length > 1
go func() {
for {
msg:=strconv.Itoa(int(time.Now().Unix()))
fmt.Println("make: ",msg," at:",time.Now())
messages <- msg
if len(messages)>1{
//remove old message
<-messages
}
time.Sleep(2*time.Second)
}
}()
receiver work slower:
go func() {
for {
channLen :=len(messages)
fmt.Println("len is ",channLen)
fmt.Println("received",<-messages)
time.Sleep(10*time.Second)
}
}()
OR, we can delete old message from receiver side
(read message like delete it)
There is an elegant channel-only solution. If you're OK with adding one more channel and goroutine - you can introduce a buferless channel and a goroutine that tries to send the latest value from your channel to it:
package main
import (
"fmt"
"time"
)
func wrapLatest(ch <-chan int) <-chan int {
result := make(chan int) // important that this one i unbuffered
go func() {
defer close(result)
value, ok := <-ch
if !ok {
return
}
LOOP:
for {
select {
case value, ok = <-ch:
if !ok {
return
}
default:
break LOOP
}
}
for {
select {
case value, ok = <-ch:
if !ok {
return
}
case result <- value:
if value, ok = <-ch; !ok {
return
}
}
}
}()
return result
}
func main() {
sendChan := make(chan int, 10) // may be buffered or not
for i := 0; i < 10; i++ {
sendChan <- i
}
go func() {
for i := 10; i < 20; i++ {
sendChan <- i
time.Sleep(time.Second)
}
close(sendChan)
}()
recvChan := wrapLatest(sendChan)
for i := range recvChan {
fmt.Println(i)
time.Sleep(time.Second * 2)
}
}
i am new in golang recently. i have a question about goroutine when use time.sleep function. here the code.
package main
import (
"fmt"
"time"
)
func go1(msg_chan chan string) {
for {
msg_chan <- "go1"
}
}
func go2(msg_chan chan string) {
for {
msg_chan <- "go2"
}
}
func count(msg_chan chan string) {
for {
msg := <-msg_chan
fmt.Println(msg)
time.Sleep(time.Second * 1)
}
}
func main() {
var c chan string
c = make(chan string)
go go1(c)
go go2(c)
go count(c)
var input string
fmt.Scanln(&input)
}
and output is
go1
go2
go1
go2
go1
go2
i think when count function is execute sleep function, go1 and go2 will execute in random sequence. so the out put maybe like
go1
go1
go2
go2
go2
go1
when i delete the sleep code in count function. the result as i supposed , it's random.
i am stucked in this issue.
thanks.
First thing to notice is that there are three go routines and all of them are independent of each other. The only thing that combines the two go routines with count routine is the channel on which both go routines are sending the values.
time.Sleep is not making the go routines synchronous. On using time.Sleep you are actually letting the count go routine to wait for that long which let other go routine to send the value on the channel which is available for the count go routine to be able to receive.
One more thing that you can do to check it is increase the number of CPU's which will give you the random result.
func GOMAXPROCS(n int) int
GOMAXPROCS sets the maximum number of CPUs that can be executing
simultaneously and returns the previous setting. If n < 1, it does not
change the current setting. The number of logical CPUs on the local
machine can be queried with NumCPU. This call will go away when the
scheduler improves.
The number of CPUs available simultaneously to executing goroutines is
controlled by the GOMAXPROCS shell environment variable, whose default
value is the number of CPU cores available. Programs with the
potential for parallel execution should therefore achieve it by
default on a multiple-CPU machine. To change the number of parallel
CPUs to use, set the environment variable or use the similarly-named
function of the runtime package to configure the run-time support to
utilize a different number of threads. Setting it to 1 eliminates the
possibility of true parallelism, forcing independent goroutines to
take turns executing.
Considering the part where output of the go routine is random, it is always random. But channels most probably work in queue which is FIFO(first in first out) as it depends on which value is available on the channel to b received. So whichever be the value available on the channel to be sent is letting the count go routine to wait and print that value.
Take for an example even if I am using time.Sleep the output is random:
package main
import (
"fmt"
"time"
)
func go1(msg_chan chan string) {
for i := 0; i < 10; i++ {
msg_chan <- fmt.Sprintf("%s%d", "go1:", i)
}
}
func go2(msg_chan chan string) {
for i := 0; i < 10; i++ {
msg_chan <- fmt.Sprintf("%s%d", "go2:", i)
}
}
func count(msg_chan chan string) {
for {
msg := <-msg_chan
fmt.Println(msg)
time.Sleep(time.Second * 1)
}
}
func main() {
var c chan string
c = make(chan string)
go go1(c)
go go2(c)
go count(c)
time.Sleep(time.Second * 20)
fmt.Println("finished")
}
This sometimes leads to race condition which is why we use synchronization either using channels or wait.groups.
package main
import (
"fmt"
"sync"
"time"
)
var wg sync.WaitGroup
func go1(msg_chan chan string) {
defer wg.Done()
for {
msg_chan <- "go1"
}
}
func go2(msg_chan chan string) {
defer wg.Done()
for {
msg_chan <- "go2"
}
}
func count(msg_chan chan string) {
defer wg.Done()
for {
msg := <-msg_chan
fmt.Println(msg)
time.Sleep(time.Second * 1)
}
}
func main() {
var c chan string
c = make(chan string)
wg.Add(1)
go go1(c)
wg.Add(1)
go go2(c)
wg.Add(1)
go count(c)
wg.Wait()
fmt.Println("finished")
}
Now coming to the part where you are using never ending for loop to send the values on a channel. So if you remove the time.Sleep your process will hang since the loop will never stop to send the values on the channel.
I have a goroutine which can generate an infinite number of values (each more suitable than the last), but it takes progressively longer to find each values. I'm trying to find a way to add a time limit, say 10 seconds, after which my function does something with the best value received so far.
This is my current "solution", using a channel and timer:
// the goroutine which runs infinitely
// (or at least a very long time for high values of depth)
func runSearch(depth int, ch chan int) {
for i := 1; i <= depth; i++ {
fmt.Printf("Searching to depth %v\n", i)
ch <- search(i)
}
}
// consumes progressively better values until the channel is closed
func awaitBestResult(ch chan int) {
var result int
for result := range ch {
best = result
}
// do something with best result here
}
// run both consumer and producer
func main() {
timer := time.NewTimer(time.Millisecond * 2000)
ch := make(chan int)
go runSearch(1000, ch)
go awaitBestResult(ch)
<-timer.C
close(ch)
}
This mostly works - the best result is processed after the timer ends and the channel is closed. However, I then get a panic (panic: send on closed channel) from the runSearch goroutine, since the channel has been closed by the main function.
How can I stop the first goroutine running after the timer has completed? Any help is very appreciated.
You need to ensure that the goroutine knows when it is done processing, so that it doesn't attempt to write to a closed channel, and panic.
This sounds like a perfect case for the context package:
func runSearch(ctx context.Context, depth int, ch chan int) {
for i := 1; i <= depth; i++ {
select {
case <- ctx.Done()
// Context cancelled, return
return
default:
}
fmt.Printf("Searching to depth %v\n", i)
ch <- search(i)
}
}
Then in main():
// run both consumer and producer
func main() {
ctx := context.WithTimeout(context.Background, 2 * time.Second)
ch := make(chan int)
go runSearch(ctx, 1000, ch)
go awaitBestResult(ch)
close(ch)
}
You are getting a panic because your sending goroutine runSearch apparently outlives the timer and it is trying to send a value on the channel which is already closed by your main goroutine. You need to devise a way to signal the sending go routine not to send any values once your timer is lapsed and before you close the channel in main. On the other hand if your search gets over sooner you also need to communicate to main to move on. You can use one channel and synchronize so that there are no race conditions. And finally you need to know when your consumer has processed all the data before you can exit main.
Here's something which may help.
package main
import (
"fmt"
"sync"
"time"
)
var mu sync.Mutex //To protect the stopped variable which will decide if a value is to be sent on the signalling channel
var stopped bool
func search(i int) int {
time.Sleep(1 * time.Millisecond)
return (i + 1)
}
// (or at least a very long time for high values of depth)
func runSearch(depth int, ch chan int, stopSearch chan bool) {
for i := 1; i <= depth; i++ {
fmt.Printf("Searching to depth %v\n", i)
n := search(i)
select {
case <-stopSearch:
fmt.Println("Timer over! Searched till ", i)
return
default:
}
ch <- n
fmt.Printf("Sent depth %v result for processing\n", i)
}
mu.Lock() //To avoid race condition with timer also being
//completed at the same time as execution of this code
if stopped == false {
stopped = true
stopSearch <- true
fmt.Println("Search completed")
}
mu.Unlock()
}
// consumes progressively better values until the channel is closed
func awaitBestResult(ch chan int, doneProcessing chan bool) {
var best int
for result := range ch {
best = result
}
fmt.Println("Best result ", best)
// do something with best result here
//and communicate to main when you are done processing the result
doneProcessing <- true
}
func main() {
doneProcessing := make(chan bool)
stopSearch := make(chan bool)
// timer := time.NewTimer(time.Millisecond * 2000)
timer := time.NewTimer(time.Millisecond * 12)
ch := make(chan int)
go runSearch(1000, ch, stopSearch)
go awaitBestResult(ch, doneProcessing)
select {
case <-timer.C:
//If at the same time runsearch is also completed and trying to send a value !
//So we hold a lock before sending value on the channel
mu.Lock()
if stopped == false {
stopped = true
stopSearch <- true
fmt.Println("Timer expired")
}
mu.Unlock()
case <-stopSearch:
fmt.Println("runsearch goroutine completed")
}
close(ch)
//Wait for your consumer to complete processing
<-doneProcessing
//Safe to exit now
}
On playground. Change the value of timer to observe both the scenarios.
Short story:
I'm having an issue where a map that previously had data but should now be empty is reporting a len() of > 0 even though it appears to be empty, and I have no idea why.
Longer story:
I need to process a number of devices at a time. Each device can have a number of messages. The concurrency of Go seemed like an obvious place to begin, so I wrote up some code to handle it and it seems to be going mostly very well. However...
I started a single goroutine for each device. In the main() function I have a map that contains each of the devices. When a message comes in I check to see whether the device already exists and if not I create it, store it in the map, and then pass the message into the device's receiving buffered channel.
This works great, and each device is being processed nicely. However, I need the device (and its goroutine) to terminate when it doesn't receive any messages for a preset amount of time. I've done this by checking in the goroutine itself how much time has passed since the last message was received, and if the goroutine is considered stale then the receiving channel is closed. But how to remove from the map?
So I passed in a pointer to the map, and I have the goroutine delete the device from the map and close the receiving channel before returning. The problem though is that at the end I'm finding that the len() function returns a value > 0, but when I output the map itself I see that it's empty.
I've written up a toy example to try to replicate the fault, and indeed I'm seeing that len() is reporting > 0 when the map is apparently empty. The last time I tried it I saw 10. The time before that 14. The time before that one, 53.
So I can replicate the fault, but I'm not sure whether the fault is with me or with Go. How is len() reporting > 0 when there are apparently no items in it?
Here's an example of how I've been able to replicate. I'm using Go v1.5.1 windows/amd64
There are two things here, as far as I'm concerned:
Am I managing the goroutines properly (probably not) and
Why does len(m) report > 0 when there are no items in it?
Thanks all
Example Code:
package main
import (
"log"
"os"
"time"
)
const (
chBuffSize = 100 // How large the thing's channel buffer should be
thingIdleLifetime = time.Second * 5 // How long things can live for when idle
thingsToMake = 1000 // How many things and associated goroutines to make
thingMessageCount = 10 // How many messages to send to the thing
)
// The thing that we'll be passing into a goroutine to process -----------------
type thing struct {
id string
ch chan bool
}
// Go go gadget map test -------------------------------------------------------
func main() {
// Make all of the things!
things := make(map[string]thing)
for i := 0; i < thingsToMake; i++ {
t := thing{
id: string(i),
ch: make(chan bool, chBuffSize),
}
things[t.id] = t
// Pass the thing into it's own goroutine
go doSomething(t, &things)
// Send (thingMessageCount) messages to the thing
go func(t thing) {
for x := 0; x < thingMessageCount; x++ {
t.ch <- true
}
}(t)
}
// Check the map of things to see whether we're empty or not
size := 0
for {
if size == len(things) && size != thingsToMake {
log.Println("Same number of items in map as last time")
log.Println(things)
os.Exit(1)
}
size = len(things)
log.Printf("Map size: %d\n", size)
time.Sleep(time.Second)
}
}
// Func for each goroutine to run ----------------------------------------------
//
// Takes two arguments:
// 1) the thing that it is working with
// 2) a pointer to the map of things
//
// When this goroutine is ready to terminate, it should remove the associated
// thing from the map of things to clean up after itself
func doSomething(t thing, things *map[string]thing) {
lastAccessed := time.Now()
for {
select {
case <-t.ch:
// We received a message, so extend the lastAccessed time
lastAccessed = time.Now()
default:
// We haven't received a message, so check if we're allowed to continue
n := time.Now()
d := n.Sub(lastAccessed)
if d > thingIdleLifetime {
// We've run for >thingIdleLifetime, so close the channel, delete the
// associated thing from the map and return, terminating the goroutine
close(t.ch)
delete(*things, string(t.id))
return
}
}
// Just sleep for a second in each loop to prevent the CPU being eaten up
time.Sleep(time.Second)
}
}
Just to add; in my original code this is looping forever. The program is designed to listen for TCP connections and receive and process the data, so the function that is checking the map count is running in it's own goroutine. However, this example has exactly the same symptom even though the map len() check is in the main() function and it is designed to handle an initial burst of data and then break out of the loop.
UPDATE 2015/11/23 15:56 UTC
I've refactored my example below. I'm not sure if I've misunderstood #RobNapier or not but this works much better. However, if I change thingsToMake to a larger number, say 100000, then I get lots of errors like this:
goroutine 199734 [select]:
main.doSomething(0xc0d62e7680, 0x4, 0xc0d64efba0, 0xc082016240)
C:/Users/anttheknee/go/src/maptest/maptest.go:83 +0x144
created by main.main
C:/Users/anttheknee/go/src/maptest/maptest.go:46 +0x463
I'm not sure if the problem is that I'm asking Go to do too much, or if I've made a hash of understanding the solution. Any thoughts?
package main
import (
"log"
"os"
"time"
)
const (
chBuffSize = 100 // How large the thing's channel buffer should be
thingIdleLifetime = time.Second * 5 // How long things can live for when idle
thingsToMake = 10000 // How many things and associated goroutines to make
thingMessageCount = 10 // How many messages to send to the thing
)
// The thing that we'll be passing into a goroutine to process -----------------
type thing struct {
id string
ch chan bool
done chan string
}
// Go go gadget map test -------------------------------------------------------
func main() {
// Make all of the things!
things := make(map[string]thing)
// Make a channel to receive completion notification on
doneCh := make(chan string, chBuffSize)
log.Printf("Making %d things\n", thingsToMake)
for i := 0; i < thingsToMake; i++ {
t := thing{
id: string(i),
ch: make(chan bool, chBuffSize),
done: doneCh,
}
things[t.id] = t
// Pass the thing into it's own goroutine
go doSomething(t)
// Send (thingMessageCount) messages to the thing
go func(t thing) {
for x := 0; x < thingMessageCount; x++ {
t.ch <- true
time.Sleep(time.Millisecond * 10)
}
}(t)
}
log.Printf("All %d things made\n", thingsToMake)
// Receive on doneCh when the goroutine is complete and clean the map up
for {
id := <-doneCh
close(things[id].ch)
delete(things, id)
if len(things) == 0 {
log.Printf("Map: %v", things)
log.Println("All done. Exiting")
os.Exit(0)
}
}
}
// Func for each goroutine to run ----------------------------------------------
//
// Takes two arguments:
// 1) the thing that it is working with
// 2) the channel to report that we're done through
//
// When this goroutine is ready to terminate, it should remove the associated
// thing from the map of things to clean up after itself
func doSomething(t thing) {
timer := time.NewTimer(thingIdleLifetime)
for {
select {
case <-t.ch:
// We received a message, so extend the timer
timer.Reset(thingIdleLifetime)
case <-timer.C:
// Timer returned so we need to exit now
t.done <- t.id
return
}
}
}
UPDATE 2015/11/23 16:41 UTC
The completed code that appears to be working properly. Do feel free to let me know if there are any improvements that could be made, but this works (sleeps are deliberate to see progress as it's otherwise too fast!)
package main
import (
"log"
"os"
"strconv"
"time"
)
const (
chBuffSize = 100 // How large the thing's channel buffer should be
thingIdleLifetime = time.Second * 5 // How long things can live for when idle
thingsToMake = 100000 // How many things and associated goroutines to make
thingMessageCount = 10 // How many messages to send to the thing
)
// The thing that we'll be passing into a goroutine to process -----------------
type thing struct {
id string
receiver chan bool
done chan string
}
// Go go gadget map test -------------------------------------------------------
func main() {
// Make all of the things!
things := make(map[string]thing)
// Make a channel to receive completion notification on
doneCh := make(chan string, chBuffSize)
log.Printf("Making %d things\n", thingsToMake)
for i := 0; i < thingsToMake; i++ {
t := thing{
id: strconv.Itoa(i),
receiver: make(chan bool, chBuffSize),
done: doneCh,
}
things[t.id] = t
// Pass the thing into it's own goroutine
go doSomething(t)
// Send (thingMessageCount) messages to the thing
go func(t thing) {
for x := 0; x < thingMessageCount; x++ {
t.receiver <- true
time.Sleep(time.Millisecond * 100)
}
}(t)
}
log.Printf("All %d things made\n", thingsToMake)
// Check the `len()` of things every second and exit when empty
go func() {
for {
time.Sleep(time.Second)
m := things
log.Printf("Map length: %v", len(m))
if len(m) == 0 {
log.Printf("Confirming empty map: %v", things)
log.Println("All done. Exiting")
os.Exit(0)
}
}
}()
// Receive on doneCh when the goroutine is complete and clean the map up
for {
id := <-doneCh
close(things[id].receiver)
delete(things, id)
}
}
// Func for each goroutine to run ----------------------------------------------
//
// When this goroutine is ready to terminate it should respond through t.done to
// notify the caller that it has finished and can be cleaned up. It will wait
// for `thingIdleLifetime` until it times out and terminates on it's own
func doSomething(t thing) {
timer := time.NewTimer(thingIdleLifetime)
for {
select {
case <-t.receiver:
// We received a message, so extend the timer
timer.Reset(thingIdleLifetime)
case <-timer.C:
// Timer expired so we need to exit now
t.done <- t.id
return
}
}
}
map is not thread-safe. You cannot access a map on multiple goroutines safely. You can corrupt the map, as you're seeing in this case.
Rather than allow the goroutine to modify the map, the goroutine should write their identifier to a channel before returning. The main loop should watch that channel, and when an identifier comes back, should remove that element from the map.
You'll probably want to read up on Go concurrency patterns. In particular, you may want to look at Fan-out/Fan-in. Look at the links at the bottom. The Go blog has a lot of information on concurrency.
Note that your goroutine is busy waiting to check for timeout. There's no reason for that. The fact that you "sleep(1 second)") should be a clue that there's a mistake. Instead, look at time.Timer which will give you a chan that will receive a value after some time, which you can reset.
Your problem is how you're converting numbers to strings:
id: string(i),
That creates a string using i as a rune (int32). For example string(65) is A. Some unequal Runes resolve to equal strings. You get a collision and close the same channel twice. See http://play.golang.org/p/__KpnfQc1V
You meant this:
id: strconv.Itoa(i),
This is a good example of workers & controller mode in Go written by #Jimt, in answer to
"Is there some elegant way to pause & resume any other goroutine in golang?"
package main
import (
"fmt"
"runtime"
"sync"
"time"
)
// Possible worker states.
const (
Stopped = 0
Paused = 1
Running = 2
)
// Maximum number of workers.
const WorkerCount = 1000
func main() {
// Launch workers.
var wg sync.WaitGroup
wg.Add(WorkerCount + 1)
workers := make([]chan int, WorkerCount)
for i := range workers {
workers[i] = make(chan int)
go func(i int) {
worker(i, workers[i])
wg.Done()
}(i)
}
// Launch controller routine.
go func() {
controller(workers)
wg.Done()
}()
// Wait for all goroutines to finish.
wg.Wait()
}
func worker(id int, ws <-chan int) {
state := Paused // Begin in the paused state.
for {
select {
case state = <-ws:
switch state {
case Stopped:
fmt.Printf("Worker %d: Stopped\n", id)
return
case Running:
fmt.Printf("Worker %d: Running\n", id)
case Paused:
fmt.Printf("Worker %d: Paused\n", id)
}
default:
// We use runtime.Gosched() to prevent a deadlock in this case.
// It will not be needed of work is performed here which yields
// to the scheduler.
runtime.Gosched()
if state == Paused {
break
}
// Do actual work here.
}
}
}
// controller handles the current state of all workers. They can be
// instructed to be either running, paused or stopped entirely.
func controller(workers []chan int) {
// Start workers
for i := range workers {
workers[i] <- Running
}
// Pause workers.
<-time.After(1e9)
for i := range workers {
workers[i] <- Paused
}
// Unpause workers.
<-time.After(1e9)
for i := range workers {
workers[i] <- Running
}
// Shutdown workers.
<-time.After(1e9)
for i := range workers {
close(workers[i])
}
}
But this code also has an issue: If you want to remove a worker channel in workers when worker() exits, dead lock happens.
If you close(workers[i]), next time controller writes into it will cause a panic since go can't write into a closed channel. If you use some mutex to protect it, then it will be stuck on workers[i] <- Running since the worker is not reading anything from the channel and write will be blocked, and mutex will cause a dead lock. You can also give a bigger buffer to channel as a work-around, but it's not good enough.
So I think the best way to solve this is worker() close channel when exits, if the controller finds a channel closed, it will jump over it and do nothing. But I can't find how to check a channel is already closed or not in this situation. If I try to read the channel in controller, the controller might be blocked. So I'm very confused for now.
PS: Recovering the raised panic is what I have tried, but it will close goroutine which raised panic. In this case it will be controller so it's no use.
Still, I think it's useful for Go team to implement this function in next version of Go.
There's no way to write a safe application where you need to know whether a channel is open without interacting with it.
The best way to do what you're wanting to do is with two channels -- one for the work and one to indicate a desire to change state (as well as the completion of that state change if that's important).
Channels are cheap. Complex design overloading semantics isn't.
[also]
<-time.After(1e9)
is a really confusing and non-obvious way to write
time.Sleep(time.Second)
Keep things simple and everyone (including you) can understand them.
In a hacky way it can be done for channels which one attempts to write to by recovering the raised panic. But you cannot check if a read channel is closed without reading from it.
Either you will
eventually read the "true" value from it (v <- c)
read the "true" value and 'not closed' indicator (v, ok <- c)
read a zero value and the 'closed' indicator (v, ok <- c) (example)
will block in the channel read forever (v <- c)
Only the last one technically doesn't read from the channel, but that's of little use.
I know this answer is so late, I have wrote this solution, Hacking Go run-time, It's not safety, It may crashes:
import (
"unsafe"
"reflect"
)
func isChanClosed(ch interface{}) bool {
if reflect.TypeOf(ch).Kind() != reflect.Chan {
panic("only channels!")
}
// get interface value pointer, from cgo_export
// typedef struct { void *t; void *v; } GoInterface;
// then get channel real pointer
cptr := *(*uintptr)(unsafe.Pointer(
unsafe.Pointer(uintptr(unsafe.Pointer(&ch)) + unsafe.Sizeof(uint(0))),
))
// this function will return true if chan.closed > 0
// see hchan on https://github.com/golang/go/blob/master/src/runtime/chan.go
// type hchan struct {
// qcount uint // total data in the queue
// dataqsiz uint // size of the circular queue
// buf unsafe.Pointer // points to an array of dataqsiz elements
// elemsize uint16
// closed uint32
// **
cptr += unsafe.Sizeof(uint(0))*2
cptr += unsafe.Sizeof(unsafe.Pointer(uintptr(0)))
cptr += unsafe.Sizeof(uint16(0))
return *(*uint32)(unsafe.Pointer(cptr)) > 0
}
Well, you can use default branch to detect it, for a closed channel will be selected, for example: the following code will select default, channel, channel, the first select is not blocked.
func main() {
ch := make(chan int)
go func() {
select {
case <-ch:
log.Printf("1.channel")
default:
log.Printf("1.default")
}
select {
case <-ch:
log.Printf("2.channel")
}
close(ch)
select {
case <-ch:
log.Printf("3.channel")
default:
log.Printf("3.default")
}
}()
time.Sleep(time.Second)
ch <- 1
time.Sleep(time.Second)
}
Prints
2018/05/24 08:00:00 1.default
2018/05/24 08:00:01 2.channel
2018/05/24 08:00:01 3.channel
Note, refer to comment by #Angad under this answer:
It doesn't work if you're using a Buffered Channel and it contains
unread data
I have had this problem frequently with multiple concurrent goroutines.
It may or may not be a good pattern, but I define a a struct for my workers with a quit channel and field for the worker state:
type Worker struct {
data chan struct
quit chan bool
stopped bool
}
Then you can have a controller call a stop function for the worker:
func (w *Worker) Stop() {
w.quit <- true
w.stopped = true
}
func (w *Worker) eventloop() {
for {
if w.Stopped {
return
}
select {
case d := <-w.data:
//DO something
if w.Stopped {
return
}
case <-w.quit:
return
}
}
}
This gives you a pretty good way to get a clean stop on your workers without anything hanging or generating errors, which is especially good when running in a container.
You could set your channel to nil in addition to closing it. That way you can check if it is nil.
example in the playground:
https://play.golang.org/p/v0f3d4DisCz
edit:
This is actually a bad solution as demonstrated in the next example,
because setting the channel to nil in a function would break it:
https://play.golang.org/p/YVE2-LV9TOp
ch1 := make(chan int)
ch2 := make(chan int)
go func(){
for i:=0; i<10; i++{
ch1 <- i
}
close(ch1)
}()
go func(){
for i:=10; i<15; i++{
ch2 <- i
}
close(ch2)
}()
ok1, ok2 := false, false
v := 0
for{
ok1, ok2 = true, true
select{
case v,ok1 = <-ch1:
if ok1 {fmt.Println(v)}
default:
}
select{
case v,ok2 = <-ch2:
if ok2 {fmt.Println(v)}
default:
}
if !ok1 && !ok2{return}
}
}
From the documentation:
A channel may be closed with the built-in function close. The multi-valued assignment form of the receive operator reports whether a received value was sent before the channel was closed.
https://golang.org/ref/spec#Receive_operator
Example by Golang in Action shows this case:
// This sample program demonstrates how to use an unbuffered
// channel to simulate a game of tennis between two goroutines.
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
// wg is used to wait for the program to finish.
var wg sync.WaitGroup
func init() {
rand.Seed(time.Now().UnixNano())
}
// main is the entry point for all Go programs.
func main() {
// Create an unbuffered channel.
court := make(chan int)
// Add a count of two, one for each goroutine.
wg.Add(2)
// Launch two players.
go player("Nadal", court)
go player("Djokovic", court)
// Start the set.
court <- 1
// Wait for the game to finish.
wg.Wait()
}
// player simulates a person playing the game of tennis.
func player(name string, court chan int) {
// Schedule the call to Done to tell main we are done.
defer wg.Done()
for {
// Wait for the ball to be hit back to us.
ball, ok := <-court
fmt.Printf("ok %t\n", ok)
if !ok {
// If the channel was closed we won.
fmt.Printf("Player %s Won\n", name)
return
}
// Pick a random number and see if we miss the ball.
n := rand.Intn(100)
if n%13 == 0 {
fmt.Printf("Player %s Missed\n", name)
// Close the channel to signal we lost.
close(court)
return
}
// Display and then increment the hit count by one.
fmt.Printf("Player %s Hit %d\n", name, ball)
ball++
// Hit the ball back to the opposing player.
court <- ball
}
}
it's easier to check first if the channel has elements, that would ensure the channel is alive.
func isChanClosed(ch chan interface{}) bool {
if len(ch) == 0 {
select {
case _, ok := <-ch:
return !ok
}
}
return false
}
If you listen this channel you always can findout that channel was closed.
case state, opened := <-ws:
if !opened {
// channel was closed
// return or made some final work
}
switch state {
case Stopped:
But remember, you can not close one channel two times. This will raise panic.