I am trying to figure out whether I can reopen a channel after closing it.
Test case:
I have a channel with some things in it
I want to range over them therefore I need to close the channel beforehand
I want to put more stuff in the channel and iterate through it once more
go func() {
queue <- "1"
queue <- "2"
close(queue)
}()
for i := range queue {
go func(i string) {
fmt.Println("From queue: ", i)
}(i)
}
go func() {
open(queue)
queue <- "3"
queue <- "4"
close(queue)
}()
for i := range queue {
go func(i string) {
fmt.Println("From queue: ", i)
}(i)
}
Of course open does not exist. How can I implement something like what I need in Go?
I don't want to use the Sleep function
I want to range over them therefore I need to close the channel beforehand
Nope, no need to close the channel. It'll resume iteration when another item is pushed thru the channel.
The below code accepts console input and pushes it to the channel:
main.go
package main
import (
"log"
"bufio"
"os"
"fmt"
)
func process(c chan string) {
for s := range c {
log.Println("processed", s)
}
}
func main() {
c := make(chan string, 10)
go process(c)
// get from console and process
reader := bufio.NewReader(os.Stdin)
fmt.Println("INPUT STUFF. TYPE #quit TO EXIT.")
for {
input, _, _ := reader.ReadLine()
if string(input) == "#quit" {
break
}
c <- string(input)
}
log.Println("BYE!")
}
output
INPUT STUFF. TYPE #quit TO EXIT.
hello
2018/10/23 10:43:52 processed hello
world
2018/10/23 10:43:54 processed world
#quit
2018/10/23 10:43:57 BYE!
The below sample uses Sleep() and is runnable as a Go Playground snippet
package main
import (
"log"
"time"
)
func process(c chan string) {
for s := range c {
log.Println("processed", s)
}
}
func main() {
c := make(chan string, 10)
go process(c)
// push some data
c <- "barry allen"
c <- "iris west"
time.Sleep(time.Second * 2)
// push more data
c <- "joe west"
c <- "caitlin snow"
time.Sleep(time.Second * 3)
}
output
2009/11/10 23:00:00 processed barry allen
2009/11/10 23:00:00 processed iris west
2009/11/10 23:00:02 processed joe west
2009/11/10 23:00:02 processed caitlin snow
Hope this helps. Cheers,
You cannot reopen a closed channel but you can send a channel on a channel, perhaps this is what you're looking for?
package main
import (
"fmt"
"time"
)
func main() {
queue := make(chan chan int)
defer close(queue)
go func() { // reader
for {
ch := <-queue
for i := range ch {
fmt.Println(i)
}
fmt.Println("Done with this channel")
}
}()
go func() { // writer-1
ch := make(chan int)
defer close(ch)
queue <- ch
ch <- 4
ch <- 2
}()
go func() { // writer-2
ch := make(chan int)
defer close(ch)
queue <- ch
ch <- 4
ch <- 20
}()
time.Sleep(time.Second)
}
Tho you cannot reopen the channel, it is possible to create a new one and assign to the variable. The previously closed channel will be garbage collected. In this case I reused the queue variable, but you can also create a new queue2 variable and pass it to the goroutine.
package main
import (
"context"
"fmt"
"time"
)
func main() {
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
queue := make(chan int)
go printFormat(nil, queue)
queue <- 1
queue <- 2
close(queue)
// fake some actions in between
time.Sleep(2 * time.Second)
queue = make(chan int)
go printFormat(cancel, queue)
queue <- 3
queue <- 4
close(queue)
<-ctx.Done()
}
func printFormat(c context.CancelFunc, q chan int) {
for i := range q {
fmt.Printf("Data %d \n", i)
}
if c != nil {
c()
}
}
Related
The problem is that both the goOne and goTwo functions are sending values to the channels ch1 and ch2 respectively, but there is no corresponding receiver for these values in the main function. This means that the channels are blocked and the program is unable to proceed. As a result, the select statement in the main function is unable to read from the channels, so it always executes the default case.
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
ch1 := make(chan string)
ch2 := make(chan string)
wg.Add(2)
go goOne(&wg, ch1)
go goTwo(&wg, ch2)
select {
case <-ch1:
fmt.Println(<-ch1)
close(ch1)
case <-ch2:
fmt.Println(<-ch2)
close(ch2)
default:
fmt.Println("Default Case")
}
wg.Wait()
}
func goTwo(wg *sync.WaitGroup, ch2 chan string) {
ch2 <- "Channel 2"
wg.Done()
}
func goOne(wg *sync.WaitGroup, ch1 chan string) {
ch1 <- "Channel 1"
wg.Done()
}
Output:
Default Case
fatal error: all goroutines are asleep - deadlock!
goroutine 1 \[semacquire\]:
sync.runtime_Semacquire(0xc000108270?)
/usr/local/go/src/runtime/sema.go:62 +0x25
sync.(\*WaitGroup).Wait(0x4b9778?)
/usr/local/go/src/sync/waitgroup.go:139 +0x52
main.main()
/home/nidhey/Documents/Go_Learning/goroutines/select.go:29 +0x2af
goroutine 6 \[chan send\]:
main.goOne(0x0?, 0x0?)
/home/nidhey/Documents/Go_Learning/goroutines/select.go:39 +0x28
created by main.main
/home/nidhey/Documents/Go_Learning/goroutines/select.go:14 +0xc5
goroutine 7 \[chan send\]:
main.goTwo(0x0?, 0x0?)
/home/nidhey/Documents/Go_Learning/goroutines/select.go:33 +0x28
created by main.main
/home/nidhey/Documents/Go_Learning/goroutines/select.go:15 +0x119\```
I'm looking for a different pattern such as select to handle the case when the channels are blocked.
To fix the issue, I've added a <-ch1 or <-ch2 in the main function after wg.Wait() to receive the values sent to the channels and unblock them
It's not entirely clear what you want to do. If you want to wait for both goroutines to complete their work and get the result of their work into the channel, you don't need a weight group (because it won't be reached).
You can do something like this.
package main
import (
"fmt"
"time"
)
func main() {
ch1 := make(chan string)
ch2 := make(chan string)
go goOne(ch1)
go goTwo(ch2)
for {
select {
case v := <-ch1:
fmt.Println("Done ch1:", v)
ch1 = nil
case v := <-ch2:
fmt.Println("Done ch2:", v)
ch2 = nil
case <-time.After(time.Second):
fmt.Println("I didn't get result so lets skip it!")
ch1, ch2 = nil, nil
}
if ch1 == nil && ch2 == nil {
break
}
}
}
func goTwo(ch chan string) {
ch <- "Channel 2"
}
func goOne(_ chan string) {
//ch1 <- "Channel 1"
}
UPD:
Imagine if we are having two api end points, API1 & API2 which are returning same data but are hosted on different regions. So what I want to do, I need to make API calls for both apis in two different function ie goroutines and as soon as any one api sends us response back, I want to process the data received. So for that Im check whcih api is fetching data first using select block.
package main
import (
"context"
"fmt"
"math/rand"
"time"
)
func main() {
regions := []string{
"Europe",
"America",
"Asia",
}
// Just for different results for each run
rand.Seed(time.Now().UnixNano())
rand.Shuffle(len(regions), func(i, j int) { regions[i], regions[j] = regions[j], regions[i] })
output := make(chan string)
ctx, cancel := context.WithTimeout(context.Background(), 2 * time.Second)
for i, region := range regions {
go func(ctx context.Context, region string, output chan <- string, i int) {
// Do call with context
// If context will be cancelled just ignore it here
timeout := time.Duration(i)*time.Second
fmt.Printf("Call %s (with timeout %s)\n", region, timeout)
time.Sleep(timeout) // Simulate request timeout
select {
case <-ctx.Done():
fmt.Println("Cancel by context:", region)
case output <- fmt.Sprintf("Answer from `%s`", region):
fmt.Println("Done:", region)
}
}(ctx, region, output, i)
}
select {
case v := <-output:
cancel() // Cancel all requests in progress (if possible)
// Do not close output chan to avoid panics: When the channel is no longer used, it will be garbage collected.
fmt.Println("Result:", v)
case <-ctx.Done():
fmt.Println("Timeout by context done")
}
fmt.Println("There is we already have result or timeout, but wait a little bit to check all is okay")
time.Sleep(5*time.Second)
}
Firstly you have a race condition in that your channel publishing goroutines will probably not have been started by the time you enter the select statement, and it will immediately fall through to the default.
But assuming you resolve this (e.g. with another form of semaphore) you're on to the next issue - your select statement will either get chan1 or chan2 message, then wait at the bottom of the method, but since it is no longer in the select statement, one of your messages won't have arrived and you'll be waiting forever.
You'll need either a loop (twice in this case) or a receiver for both channels running in their own goroutines to pick up both messages and complete the waitgroup.
But in any case - as others have queried - what are you trying to achieve?
You can try something like iterating over a single channel (assuming both channels return the same type of data) and then count in the main method how many tasks are completed. Then close the channel once all the tasks are done. Example:
package main
import (
"fmt"
)
func main() {
ch := make(chan string)
go goOne(ch)
go goTwo(ch)
doneCount := 0
for v := range ch {
fmt.Println(v)
doneCount++
if doneCount == 2 {
close(ch)
}
}
}
func goTwo(ch chan string) {
ch <- "Channel 2"
}
func goOne(ch chan string) {
ch <- "Channel 1"
}
I was trying to debug this code but am stuck here. I wanted to access ch1, ch2 but found printed nothing.
package main
import (
"fmt"
)
type degen struct {
i, j string
}
func (x degen) CVIO(ch1, ch2 chan string, quit chan int, m, n string) {
for {
select {
case ch1 <- m:
fmt.Println(x.i)
case ch2 <- n:
fmt.Println("ok")
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
ch1 := make(chan string)
ch2 := make(chan string)
quit := make(chan int)
x := degen{"goosebump", "ok"}
go x.CVIO(ch1, ch2, quit, "goosebump", "ok")
}
Desired:
It should print the channel data as to be produced.
Its not really clear what you expect your code to do:
main() ends without waiting for the go routine to exit (its quite possible it the loop will not run at all).
in the select the sends will not proceed because there is no receiver (spec - "if the capacity is zero or absent, the channel is unbuffered and communication succeeds only when both a sender and receiver are ready.").
Nothing is sent to the quit channel.
I suspect that the following (playground) might do what you were expecting.
package main
import (
"fmt"
"sync"
)
type degen struct {
i, j string
}
func (x degen) CVIO(ch1, ch2 chan string, quit chan int, m, n string) {
for {
select {
case ch1 <- m:
fmt.Println(x.i)
case ch2 <- n:
fmt.Println("ok")
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
ch1 := make(chan string)
ch2 := make(chan string)
quit := make(chan int)
x := degen{"goosebump", "ok"}
var wg sync.WaitGroup
wg.Add(1)
go func() {
x.CVIO(ch1, ch2, quit, "goosebump", "ok")
wg.Done()
}()
<-ch1 // Receive from CH1 (allowing "ch1 <- m" in go routine to proceed)
<-ch2 // Receive from CH2 (allowing "ch2 <- n" in go routine to proceed)
quit <- 1
wg.Wait() // Wait for CVIO to end (which it should do due to above send)
}
I have made a simple code example to understand the usage of pipeline, here it is.
package main
import (
"fmt"
"sync"
"time"
)
func main() {
ch1 := make(chan int, 10) // Use buffered channel so as to avoid clogging
ch2 := make(chan string, 10)
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func1(i, ch1, &wg)
go func2(ch1, ch2)
}
wg.Wait()
close(ch1)
for val := range ch2 {
fmt.Println(val)
}
}
func func1(seconds int, ch chan<- int, wg *sync.WaitGroup) {
defer wg.Done()
time.Sleep(time.Duration(seconds) * time.Second)
fmt.Println(seconds)
ch <- seconds
}
func func2(ch1 chan int, ch2 chan string) {
for range ch1 {
ch2 <- "hello"
}
close(ch2)
}
Now, the problem is I don't get consistent output ( I understand it's some concurrency issue, which I haven't fully understood ).
Output
> go run pipeline-loop.go
0
1
2
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
panic: close of closed channel
goroutine 6 [running]:
main.func2(0xc00006c000, 0xc000056180)
/home/projects/go-tuts/pipeline-loop.go:36 +0x72
created by main.main
/home/projects/go-tuts/pipeline-loop.go:16 +0x10f
exit status 2
Another guy changed the code ( and it was working ) and put func2 outside the loop but I want func2 for each iteration of the func1.
Problem
So, I want to understand as to where should the WaitGroup and close(ch) be used ?
Thanks.
Temporarya
( A golang noobie )
Update
Based on a user's answer, I changed the code, now I get output as expected ( but not the solution to this question ) but still there's a deadlock.
https://play.golang.org/p/O_rp_FLvNh8
package main
import (
"fmt"
"sync"
"time"
)
func main() {
ch1 := make(chan int, 10) // Use buffered channel so as to avoid clogging
ch2 := make(chan string, 10)
var wg1 sync.WaitGroup
// var wg2 sync.WaitGroup
for i := 0; i < 3; i++ {
wg1.Add(1)
go func1(i, ch1)
go func2(ch1, ch2, &wg1)
}
for val := range ch2 {
fmt.Println(val)
}
wg1.Wait()
close(ch1)
close(ch2)
}
// func func1(seconds int, ch chan<- int, wg *sync.WaitGroup) {
func func1(seconds int, ch chan<- int) {
// defer wg.Done()
time.Sleep(time.Duration(seconds) * time.Second)
fmt.Println(seconds)
ch <- seconds
}
func func2(ch1 chan int, ch2 chan string, wg *sync.WaitGroup) {
defer wg.Done()
for range ch1 {
ch2 <- "hello"
}
}
There are multiple problems in your code.
In the loop, you are spawning multiple (3) goroutines that runs func2, and in func2, you send data to ch2 and call close(ch2). This is a problem. It can happen that when one goroutine is seding data to ch2, the other has closed that channel, which causes:
panic: close of closed channel
goroutine 6 [running]:
main.func2(0xc00006c000, 0xc000056180)
/home/projects/go-tuts/pipeline-loop.go:36 +0x72
created by main.main
/home/projects/go-tuts/pipeline-loop.go:16 +0x10f
exit status 2
In general, you don't need close a chanel multiple times - you only need to shut them once they are all finished. You need another WaitGroup for this; you need to pass both functions a WaitGroup.
Further reading: https://blog.golang.org/pipelines
UPDATE:
Personally I use a pattern for "works" that produce data into a same channel and the channel needs to be closed after all works are done:
for something {
wg.Add(1)
go func(i int) {
work(ch)
wg.Done()
}
}
go func() {
wg.Wait()
close()
}()
I think it is a good idea that keeps the API clean from WorkGroup as WorkGroup is about how you sync the work instead of how the work is done.
I have changed your code into this pattern: https://play.golang.org/p/vdCNsxWhgyQ
I suspect you only want one channel to read from ch1 and write to ch2. There is not much point in creating 3 go-routines to do the same thing (and you also end up closing the same channel mutliple time which causes a panic as leaf bebop pointed out)
func main() {
ch1 := make(chan int, 10) // Use buffered channel so as to avoid clogging
ch2 := make(chan string, 10)
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func1(i, ch1, &wg)
}
go func2(ch1, ch2)
wg.Wait()
close(ch1)
for val := range ch2 {
fmt.Println(val)
}
}
Say I want to accept an animal. The user can either set the type of animal at the What type of animal? prompt on a terminal, or she can go to http://localhost:1234/animal?type=kitten
Whichever she does, the terminal will read What type of animal? kitten (assuming she chose a kitten) and the program will then prompt the user on the terminal (and only the terminal) What is the kitten's name?
My thought was to use channels to go routines, but since both go routines will be stuck in a function call (Scan() for the terminal, ListenAndServe() for http) then I'm not clear how to stop the go routine that is still in a function call once the input is received. The normal method of selecting on channels won't work.
In C I'd just do a select(2) on the two relevant file descriptors (fd 0 and the fd from listen()) but I don't think that's how to do it in Go.
Note: I answered this down below. If you're down voting the question this I'd be curious to know why as I would have found the answer I came up with useful. If you're down voting the answer, I'd really be interested in knowing why. If it's non-idiomatic Go or if it has other issues I'm missing, I'd really like to fix them.
OK, I figured out a solution for what I was trying to do.
This might not be the most Go idiomatic way though. If the channels took a struct with input source and the string I could probably toss the select and the webinput channel.
package main
import (
"bufio"
"context"
"fmt"
"io"
"net/http"
"os"
"time"
)
func getHTTPAnimal(input chan string) *http.Server {
srv := &http.Server{Addr: ":1234"}
http.HandleFunc("/animal", func(w http.ResponseWriter, r *http.Request) {
animal, ok := r.URL.Query()["type"]
if !ok || len(animal[0]) <= 0 {
io.WriteString(w, "Animal not understood.\n")
return
}
io.WriteString(w, "Animal understood.\n")
input <- string(animal[0])
})
go func() {
if err := srv.ListenAndServe(); err != nil {
if err.Error() != "http: Server closed" {
fmt.Printf("getHTTPAnimal error with ListenAndServe: %s", err)
}
}
}()
return srv
}
func getTerminalInput(input chan string) {
scanner := bufio.NewScanner(os.Stdin)
for {
scanner.Scan()
input <- scanner.Text()
}
}
func main() {
input := make(chan string)
webinput := make(chan string)
srv := getHTTPAnimal(webinput)
fmt.Print("Enter animal type: ")
go getTerminalInput(input)
var animal string
select {
case ta := <-input:
animal = ta
case webta := <-webinput:
animal = webta
fmt.Printf("%s\n", animal)
}
ctx, _ := context.WithTimeout(context.Background(), 5*time.Second)
srv.Shutdown(ctx)
close(webinput)
fmt.Printf("Enter animal name: ")
name := <-input
fmt.Printf("Congrats on getting %s the half-%s\n", name, animal)
}
You can stop goroutine with close statement.
Example:
package main
import "sync"
func main() {
var wg sync.WaitGroup
wg.Add(1)
ch := make(chan int)
go func() {
for {
foo, ok := <- ch
if !ok {
println("done")
wg.Done()
return
}
println(foo)
}
}()
ch <- 1
ch <- 2
ch <- 3
close(ch)
wg.Wait()
}
There is couple of ways to stop goroutine:
1) by closing the read channel:
msgCh := make(chan string)
go func(){
for msg, ok := range msgCh {
if !ok {
return
}
}
}()
msgCh <- "String"
close(msgCh)
2) passing some sort of kill switch
msgCh := make(chan string)
killSwitch := make(chan struct{})
go func(){
for {
select{
case msg <- msgCh:
log.Println(msg)
case <-killSwitch:
return
}
}
}
msgCh <- "String"
close(killSwitch)
With approach NR2 you are avoiding writing to closed chan that would end up as panic. Also close signal is populated as fanout pattern (all places where that chan is used will receive that signal)
FATAL Error All go routines are asleep. Deadlock.
This is what I tried. I am calling wg.Done(). What is missing?
package main
import (
"fmt"
"strconv"
"sync"
)
func sender(wg *sync.WaitGroup, cs chan int) {
defer wg.Done()
for i := 0; i < 2; i++ {
fmt.Println(i)
cs <- i
}
}
func reciever(wg *sync.WaitGroup, cs chan int) {
x, ok := <-cs
for ok {
fmt.Println("Retrieved" + strconv.Itoa(x))
x, ok = <-cs
if !ok {
wg.Done()
break
}
}
}
func main() {
wg := &sync.WaitGroup{}
cs := make(chan int, 1000)
wg.Add(1)
go sender(wg, cs)
for i := 1; i < 30; i++ {
wg.Add(1)
go reciever(wg, cs)
}
wg.Wait()
close(cs)
}
You should to close channel before wg.Wait.
All your receivers are waiting for data from channel. That's why you have deadlock.
You can close channel at defer statement of sender function.
Also you need to wg.Done() if the first attempt of receiving from channel was unsuccessful (because channel already closed)
http://play.golang.org/p/qdEIEfY-kl
There are couple of things:
You need to close the channel once sender is completed.
In receiver, range over channel
Don't need to add 1 to wait group and call Done in sender
Please refer to http://play.golang.org/p/vz39RY6WA7