I try to write some goroutine with channel, but get deadlocked, why?
Am I doing wrong with WaitGroup, so confused...
package main
import (
"fmt"
"sync"
)
var wg sync.WaitGroup
func main() {
chan1 := make(chan string)
chan2 := make(chan string)
chan3 := make(chan string, 2)
wg.Add(1)
go makeChanStr("yeye", chan1, chan3)
wg.Add(1)
go makeChanStr("okok", chan2, chan3)
wg.Wait()
close(chan3)
println(<-chan1)
println(<-chan2)
for chs := range chan3 {
println(chs)
}
}
func makeChanStr(s string, c1 chan string, c2 chan string) {
defer wg.Done()
c1 <- "get " + s
c2 <- "same value"
fmt.Printf("execute ok %s", s)
}
Stackoverflow just don't let me submit the question.......so I just have to add some text.....
main block on wg.Wait(), which wait this two go routines to finish(because of wg.Add(1) and wg.Done())
go makeChanStr("yeye", chan1, chan3)
go makeChanStr("okok", chan2, chan3)
but they block on chan1 (or chan2) , because it's an unbuffer channel.
chan1 := make(chan string)
try change chan1 and chan2 to buffer channels:
chan1 := make(chan string,1)
chan2 := make(chan string,1)
This code blocks on wg.Wait() in main goroutine and writing to c1 in workers. To avoid it - read from c1 and c2 prior to wg.Wait() thus unblock workers and they will not block on writing to buffered c3. As a result wg.Done() will be called and wg.Wait() will not block main goroutine as well.
package main
import (
"fmt"
"sync"
)
var wg sync.WaitGroup
func main() {
chan1 := make(chan string)
chan2 := make(chan string)
chan3 := make(chan string, 2)
wg.Add(1)
go makeChanStr("yeye", chan1, chan3)
wg.Add(1)
go makeChanStr("okok", chan2, chan3)
println(<-chan1)
println(<-chan2)
wg.Wait()
close(chan3)
for chs := range chan3 {
println(chs)
}
}
func makeChanStr(s string, c1 chan string, c2 chan string) {
defer wg.Done()
c1 <- "get " + s
c2 <- "same value"
fmt.Printf("execute %s\n", s)
}
Related
I was trying to debug this code but am stuck here. I wanted to access ch1, ch2 but found printed nothing.
package main
import (
"fmt"
)
type degen struct {
i, j string
}
func (x degen) CVIO(ch1, ch2 chan string, quit chan int, m, n string) {
for {
select {
case ch1 <- m:
fmt.Println(x.i)
case ch2 <- n:
fmt.Println("ok")
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
ch1 := make(chan string)
ch2 := make(chan string)
quit := make(chan int)
x := degen{"goosebump", "ok"}
go x.CVIO(ch1, ch2, quit, "goosebump", "ok")
}
Desired:
It should print the channel data as to be produced.
Its not really clear what you expect your code to do:
main() ends without waiting for the go routine to exit (its quite possible it the loop will not run at all).
in the select the sends will not proceed because there is no receiver (spec - "if the capacity is zero or absent, the channel is unbuffered and communication succeeds only when both a sender and receiver are ready.").
Nothing is sent to the quit channel.
I suspect that the following (playground) might do what you were expecting.
package main
import (
"fmt"
"sync"
)
type degen struct {
i, j string
}
func (x degen) CVIO(ch1, ch2 chan string, quit chan int, m, n string) {
for {
select {
case ch1 <- m:
fmt.Println(x.i)
case ch2 <- n:
fmt.Println("ok")
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
ch1 := make(chan string)
ch2 := make(chan string)
quit := make(chan int)
x := degen{"goosebump", "ok"}
var wg sync.WaitGroup
wg.Add(1)
go func() {
x.CVIO(ch1, ch2, quit, "goosebump", "ok")
wg.Done()
}()
<-ch1 // Receive from CH1 (allowing "ch1 <- m" in go routine to proceed)
<-ch2 // Receive from CH2 (allowing "ch2 <- n" in go routine to proceed)
quit <- 1
wg.Wait() // Wait for CVIO to end (which it should do due to above send)
}
I am having 2 go-routines reading from a single channel. After 4 seconds, I cancel the context and terminate the select loop. Before terminating the loop I call close on the channel, since there are 2 go-routines the close gets called twice and causes a panic because one of the go-routines would have already closed the channel. Currently I am using a recover to recover from the panic, is there a better way of doing this?
package main
import (
"context"
"fmt"
"sync"
"time"
)
func numberGen(ctx context.Context, numChan chan int) {
num := 0
doneCh := ctx.Done()
defer func() {
if r := recover(); r != nil {
fmt.Println("recovered from ", r)
}
}()
for {
select {
case <-doneCh:
fmt.Println("done generating...")
close(numChan)
return
default:
num++
numChan <- num
}
}
}
func main() {
ctx, cancelFn := context.WithCancel(context.Background())
numChan := make(chan int)
var wg sync.WaitGroup
wg.Add(2)
go numberGen(ctx, numChan)
go numberGen(ctx, numChan)
go func(cfn context.CancelFunc) {
time.Sleep(10 * time.Millisecond)
cfn()
}(cancelFn)
for n := range numChan {
fmt.Println("received value ", n)
}
time.Sleep(2 * time.Second)
}
Close the channel after the goroutines are done sending values.
var wg sync.WaitGroup
wg.Add(2)
go numberGen(ctx, numChan, &wg)
go numberGen(ctx, numChan, &wg)
go func() {
wg.Wait()
close(numChan)
}()
Update numberGen to call Done() on the wait group. Also, remove the call to close.
func numberGen(ctx context.Context, numChan chan int, wg *sync.WaitGroup) {
defer wg.Done()
...
I am prototyping a series of go routines for a pipeline that each perform a transformation. The routines are terminating before all the data has passed through.
I have checked Donavan and Kernighan book and Googled for solutions.
Here is my code:
package main
import (
"fmt"
"sync"
)
func main() {
a1 := []string{"apple", "apricot"}
chan1 := make(chan string)
chan2 := make(chan string)
chan3 := make(chan string)
var wg sync.WaitGroup
go Pipe1(chan2, chan1, &wg)
go Pipe2(chan3, chan2, &wg)
go Pipe3(chan3, &wg)
func (data []string) {
defer wg.Done()
for _, s := range data {
wg.Add(1)
chan1 <- s
}
go func() {
wg.Wait()
close(chan1)
}()
}(a1)
}
func Pipe1(out chan<- string, in <-chan string, wg *sync.WaitGroup) {
defer wg.Done()
for s := range in {
wg.Add(1)
out <- s + "s are"
}
}
func Pipe2(out chan<- string, in <-chan string, wg *sync.WaitGroup) {
defer wg.Done()
for s := range in {
wg.Add(1)
out <- s + " good for you"
}
}
func Pipe3(in <-chan string, wg *sync.WaitGroup) {
defer wg.Done()
for s := range in {
wg.Add(1)
fmt.Println(s)
}
}
My expected output is:
apples are good for you
apricots are good for you
The results of running main are inconsistent. Sometimes I get both lines. Sometimes I just get the apples. Sometimes nothing is output.
As Adrian already pointed out, your WaitGroup.Add and WaitGroup.Done calls are mismatched. However, in cases like this the "I am done" signal is typically given by closing the output channel. WaitGroups are only necessary if work is shared between several goroutines (i.e. several goroutines consume the same channel), which isn't the case here.
package main
import (
"fmt"
)
func main() {
a1 := []string{"apple", "apricot"}
chan1 := make(chan string)
chan2 := make(chan string)
chan3 := make(chan string)
go func() {
for _, s := range a1 {
chan1 <- s
}
close(chan1)
}()
go Pipe1(chan2, chan1)
go Pipe2(chan3, chan2)
// This range loop terminates when chan3 is closed, which Pipe2 does after
// chan2 is closed, which Pipe1 does after chan1 is closed, which the
// anonymous goroutine above does after it sent all values.
for s := range chan3 {
fmt.Println(s)
}
}
func Pipe1(out chan<- string, in <-chan string) {
for s := range in {
out <- s + "s are"
}
close(out) // let caller know that we're done
}
func Pipe2(out chan<- string, in <-chan string) {
for s := range in {
out <- s + " good for you"
}
close(out) // let caller know that we're done
}
Try it on the playground: https://play.golang.org/p/d2J4APjs_lL
You're calling wg.Wait in a goroutine, so main is allowed to return (and therefore your program exits) before the other routines have finished. This would cause the behavior your see, but taking out of a goroutine alone isn't enough.
You're also misusing the WaitGroup in general; your Add and Done calls don't relate to one another, and you don't have as many Dones as you have Adds, so the WaitGroup will never finish. If you're calling Add in a loop, then every loop iteration must also result in a Done call; as you have it now, you defer wg.Done() before each of your loops, then call Add inside the loop, resulting in one Done and many Adds. This code would need to be significantly revised to work as intended.
I have made a simple code example to understand the usage of pipeline, here it is.
package main
import (
"fmt"
"sync"
"time"
)
func main() {
ch1 := make(chan int, 10) // Use buffered channel so as to avoid clogging
ch2 := make(chan string, 10)
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func1(i, ch1, &wg)
go func2(ch1, ch2)
}
wg.Wait()
close(ch1)
for val := range ch2 {
fmt.Println(val)
}
}
func func1(seconds int, ch chan<- int, wg *sync.WaitGroup) {
defer wg.Done()
time.Sleep(time.Duration(seconds) * time.Second)
fmt.Println(seconds)
ch <- seconds
}
func func2(ch1 chan int, ch2 chan string) {
for range ch1 {
ch2 <- "hello"
}
close(ch2)
}
Now, the problem is I don't get consistent output ( I understand it's some concurrency issue, which I haven't fully understood ).
Output
> go run pipeline-loop.go
0
1
2
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
panic: close of closed channel
goroutine 6 [running]:
main.func2(0xc00006c000, 0xc000056180)
/home/projects/go-tuts/pipeline-loop.go:36 +0x72
created by main.main
/home/projects/go-tuts/pipeline-loop.go:16 +0x10f
exit status 2
Another guy changed the code ( and it was working ) and put func2 outside the loop but I want func2 for each iteration of the func1.
Problem
So, I want to understand as to where should the WaitGroup and close(ch) be used ?
Thanks.
Temporarya
( A golang noobie )
Update
Based on a user's answer, I changed the code, now I get output as expected ( but not the solution to this question ) but still there's a deadlock.
https://play.golang.org/p/O_rp_FLvNh8
package main
import (
"fmt"
"sync"
"time"
)
func main() {
ch1 := make(chan int, 10) // Use buffered channel so as to avoid clogging
ch2 := make(chan string, 10)
var wg1 sync.WaitGroup
// var wg2 sync.WaitGroup
for i := 0; i < 3; i++ {
wg1.Add(1)
go func1(i, ch1)
go func2(ch1, ch2, &wg1)
}
for val := range ch2 {
fmt.Println(val)
}
wg1.Wait()
close(ch1)
close(ch2)
}
// func func1(seconds int, ch chan<- int, wg *sync.WaitGroup) {
func func1(seconds int, ch chan<- int) {
// defer wg.Done()
time.Sleep(time.Duration(seconds) * time.Second)
fmt.Println(seconds)
ch <- seconds
}
func func2(ch1 chan int, ch2 chan string, wg *sync.WaitGroup) {
defer wg.Done()
for range ch1 {
ch2 <- "hello"
}
}
There are multiple problems in your code.
In the loop, you are spawning multiple (3) goroutines that runs func2, and in func2, you send data to ch2 and call close(ch2). This is a problem. It can happen that when one goroutine is seding data to ch2, the other has closed that channel, which causes:
panic: close of closed channel
goroutine 6 [running]:
main.func2(0xc00006c000, 0xc000056180)
/home/projects/go-tuts/pipeline-loop.go:36 +0x72
created by main.main
/home/projects/go-tuts/pipeline-loop.go:16 +0x10f
exit status 2
In general, you don't need close a chanel multiple times - you only need to shut them once they are all finished. You need another WaitGroup for this; you need to pass both functions a WaitGroup.
Further reading: https://blog.golang.org/pipelines
UPDATE:
Personally I use a pattern for "works" that produce data into a same channel and the channel needs to be closed after all works are done:
for something {
wg.Add(1)
go func(i int) {
work(ch)
wg.Done()
}
}
go func() {
wg.Wait()
close()
}()
I think it is a good idea that keeps the API clean from WorkGroup as WorkGroup is about how you sync the work instead of how the work is done.
I have changed your code into this pattern: https://play.golang.org/p/vdCNsxWhgyQ
I suspect you only want one channel to read from ch1 and write to ch2. There is not much point in creating 3 go-routines to do the same thing (and you also end up closing the same channel mutliple time which causes a panic as leaf bebop pointed out)
func main() {
ch1 := make(chan int, 10) // Use buffered channel so as to avoid clogging
ch2 := make(chan string, 10)
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func1(i, ch1, &wg)
}
go func2(ch1, ch2)
wg.Wait()
close(ch1)
for val := range ch2 {
fmt.Println(val)
}
}
I am trying to figure out whether I can reopen a channel after closing it.
Test case:
I have a channel with some things in it
I want to range over them therefore I need to close the channel beforehand
I want to put more stuff in the channel and iterate through it once more
go func() {
queue <- "1"
queue <- "2"
close(queue)
}()
for i := range queue {
go func(i string) {
fmt.Println("From queue: ", i)
}(i)
}
go func() {
open(queue)
queue <- "3"
queue <- "4"
close(queue)
}()
for i := range queue {
go func(i string) {
fmt.Println("From queue: ", i)
}(i)
}
Of course open does not exist. How can I implement something like what I need in Go?
I don't want to use the Sleep function
I want to range over them therefore I need to close the channel beforehand
Nope, no need to close the channel. It'll resume iteration when another item is pushed thru the channel.
The below code accepts console input and pushes it to the channel:
main.go
package main
import (
"log"
"bufio"
"os"
"fmt"
)
func process(c chan string) {
for s := range c {
log.Println("processed", s)
}
}
func main() {
c := make(chan string, 10)
go process(c)
// get from console and process
reader := bufio.NewReader(os.Stdin)
fmt.Println("INPUT STUFF. TYPE #quit TO EXIT.")
for {
input, _, _ := reader.ReadLine()
if string(input) == "#quit" {
break
}
c <- string(input)
}
log.Println("BYE!")
}
output
INPUT STUFF. TYPE #quit TO EXIT.
hello
2018/10/23 10:43:52 processed hello
world
2018/10/23 10:43:54 processed world
#quit
2018/10/23 10:43:57 BYE!
The below sample uses Sleep() and is runnable as a Go Playground snippet
package main
import (
"log"
"time"
)
func process(c chan string) {
for s := range c {
log.Println("processed", s)
}
}
func main() {
c := make(chan string, 10)
go process(c)
// push some data
c <- "barry allen"
c <- "iris west"
time.Sleep(time.Second * 2)
// push more data
c <- "joe west"
c <- "caitlin snow"
time.Sleep(time.Second * 3)
}
output
2009/11/10 23:00:00 processed barry allen
2009/11/10 23:00:00 processed iris west
2009/11/10 23:00:02 processed joe west
2009/11/10 23:00:02 processed caitlin snow
Hope this helps. Cheers,
You cannot reopen a closed channel but you can send a channel on a channel, perhaps this is what you're looking for?
package main
import (
"fmt"
"time"
)
func main() {
queue := make(chan chan int)
defer close(queue)
go func() { // reader
for {
ch := <-queue
for i := range ch {
fmt.Println(i)
}
fmt.Println("Done with this channel")
}
}()
go func() { // writer-1
ch := make(chan int)
defer close(ch)
queue <- ch
ch <- 4
ch <- 2
}()
go func() { // writer-2
ch := make(chan int)
defer close(ch)
queue <- ch
ch <- 4
ch <- 20
}()
time.Sleep(time.Second)
}
Tho you cannot reopen the channel, it is possible to create a new one and assign to the variable. The previously closed channel will be garbage collected. In this case I reused the queue variable, but you can also create a new queue2 variable and pass it to the goroutine.
package main
import (
"context"
"fmt"
"time"
)
func main() {
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
queue := make(chan int)
go printFormat(nil, queue)
queue <- 1
queue <- 2
close(queue)
// fake some actions in between
time.Sleep(2 * time.Second)
queue = make(chan int)
go printFormat(cancel, queue)
queue <- 3
queue <- 4
close(queue)
<-ctx.Done()
}
func printFormat(c context.CancelFunc, q chan int) {
for i := range q {
fmt.Printf("Data %d \n", i)
}
if c != nil {
c()
}
}