I know by exchanging line 15 and line 17 gives no error, however, I don't understand why not exchange will gives deadlock
package main
import (
"fmt"
)
func greet(c chan string) {
fmt.Println("Hello " + <-c + "!")
}
func main() {
c := make(chan string)
//line15
c <- "John"
//line17
go greet(c)
}
fatal error: all goroutines are asleep - deadlock!
The channel c is unbuffered. Communication on an unbuffered channel does not proceed until the sender and receiver are both ready.
The program deadlocks because no receiver is ready when the main goroutine executes the send operation.
You can do something like this
package main
import (
"fmt"
"sync"
)
func greet(c chan string, wg *sync.WaitGroup) {
defer wg.Done()
fmt.Println("Hello " + <-c + "!")
}
func main() {
c := make(chan string, 10)
//line15
c <- "John"
//line17
var wg sync.WaitGroup
wg.Add(1)
go greet(c, &wg)
c <- "Alex"
wg.Add(1)
go greet(c, &wg)
wg.Wait()
}
Related
I want to print 1 to 100 use two goroutine:
package main
import (
"fmt"
"sync"
)
var my_chan chan int
var wg sync.WaitGroup
func worker() {
for true {
number := <-my_chan
fmt.Println(number)
number++
if number > 100 {
wg.Done()
return
}
my_chan <- number
}
}
func main() {
wg.Add(2)
my_chan := make(chan int)
init_num := 1
go worker()
go worker()
my_chan <- init_num
wg.Wait()
}
When I run the above code, I get the following error:
fatal error: all goroutines are asleep - deadlock!
Can anyone tell me where I am doing wrong?
Replace the channel creation with this:
my_chan = make(chan int)
Otherwise you are redeclaring my_chan in main, and all goroutines try to read from a nil channel. That will block.
Then it will count to 100 and deadlock. The check for number being larger than 100 will work for one of the goroutines, while the other one will be stuck waiting to read/write.
Thanks for Burak Serdar's answer, the following code works:
package main
import (
"fmt"
"sync"
)
var my_chan chan int
var wg sync.WaitGroup
func worker() {
for true {
number := <-my_chan
if number > 100 {
wg.Done()
my_chan <- 101
return
}
fmt.Println(number)
number++
my_chan <- number
}
}
func main() {
wg.Add(2)
my_chan = make(chan int)
init_num := 1
go worker()
go worker()
my_chan <- init_num
wg.Wait()
}
I was trying to debug this code but am stuck here. I wanted to access ch1, ch2 but found printed nothing.
package main
import (
"fmt"
)
type degen struct {
i, j string
}
func (x degen) CVIO(ch1, ch2 chan string, quit chan int, m, n string) {
for {
select {
case ch1 <- m:
fmt.Println(x.i)
case ch2 <- n:
fmt.Println("ok")
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
ch1 := make(chan string)
ch2 := make(chan string)
quit := make(chan int)
x := degen{"goosebump", "ok"}
go x.CVIO(ch1, ch2, quit, "goosebump", "ok")
}
Desired:
It should print the channel data as to be produced.
Its not really clear what you expect your code to do:
main() ends without waiting for the go routine to exit (its quite possible it the loop will not run at all).
in the select the sends will not proceed because there is no receiver (spec - "if the capacity is zero or absent, the channel is unbuffered and communication succeeds only when both a sender and receiver are ready.").
Nothing is sent to the quit channel.
I suspect that the following (playground) might do what you were expecting.
package main
import (
"fmt"
"sync"
)
type degen struct {
i, j string
}
func (x degen) CVIO(ch1, ch2 chan string, quit chan int, m, n string) {
for {
select {
case ch1 <- m:
fmt.Println(x.i)
case ch2 <- n:
fmt.Println("ok")
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
ch1 := make(chan string)
ch2 := make(chan string)
quit := make(chan int)
x := degen{"goosebump", "ok"}
var wg sync.WaitGroup
wg.Add(1)
go func() {
x.CVIO(ch1, ch2, quit, "goosebump", "ok")
wg.Done()
}()
<-ch1 // Receive from CH1 (allowing "ch1 <- m" in go routine to proceed)
<-ch2 // Receive from CH2 (allowing "ch2 <- n" in go routine to proceed)
quit <- 1
wg.Wait() // Wait for CVIO to end (which it should do due to above send)
}
I have made a simple code example to understand the usage of pipeline, here it is.
package main
import (
"fmt"
"sync"
"time"
)
func main() {
ch1 := make(chan int, 10) // Use buffered channel so as to avoid clogging
ch2 := make(chan string, 10)
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func1(i, ch1, &wg)
go func2(ch1, ch2)
}
wg.Wait()
close(ch1)
for val := range ch2 {
fmt.Println(val)
}
}
func func1(seconds int, ch chan<- int, wg *sync.WaitGroup) {
defer wg.Done()
time.Sleep(time.Duration(seconds) * time.Second)
fmt.Println(seconds)
ch <- seconds
}
func func2(ch1 chan int, ch2 chan string) {
for range ch1 {
ch2 <- "hello"
}
close(ch2)
}
Now, the problem is I don't get consistent output ( I understand it's some concurrency issue, which I haven't fully understood ).
Output
> go run pipeline-loop.go
0
1
2
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
> go run pipeline-loop.go
0
1
2
hello
hello
panic: close of closed channel
goroutine 6 [running]:
main.func2(0xc00006c000, 0xc000056180)
/home/projects/go-tuts/pipeline-loop.go:36 +0x72
created by main.main
/home/projects/go-tuts/pipeline-loop.go:16 +0x10f
exit status 2
Another guy changed the code ( and it was working ) and put func2 outside the loop but I want func2 for each iteration of the func1.
Problem
So, I want to understand as to where should the WaitGroup and close(ch) be used ?
Thanks.
Temporarya
( A golang noobie )
Update
Based on a user's answer, I changed the code, now I get output as expected ( but not the solution to this question ) but still there's a deadlock.
https://play.golang.org/p/O_rp_FLvNh8
package main
import (
"fmt"
"sync"
"time"
)
func main() {
ch1 := make(chan int, 10) // Use buffered channel so as to avoid clogging
ch2 := make(chan string, 10)
var wg1 sync.WaitGroup
// var wg2 sync.WaitGroup
for i := 0; i < 3; i++ {
wg1.Add(1)
go func1(i, ch1)
go func2(ch1, ch2, &wg1)
}
for val := range ch2 {
fmt.Println(val)
}
wg1.Wait()
close(ch1)
close(ch2)
}
// func func1(seconds int, ch chan<- int, wg *sync.WaitGroup) {
func func1(seconds int, ch chan<- int) {
// defer wg.Done()
time.Sleep(time.Duration(seconds) * time.Second)
fmt.Println(seconds)
ch <- seconds
}
func func2(ch1 chan int, ch2 chan string, wg *sync.WaitGroup) {
defer wg.Done()
for range ch1 {
ch2 <- "hello"
}
}
There are multiple problems in your code.
In the loop, you are spawning multiple (3) goroutines that runs func2, and in func2, you send data to ch2 and call close(ch2). This is a problem. It can happen that when one goroutine is seding data to ch2, the other has closed that channel, which causes:
panic: close of closed channel
goroutine 6 [running]:
main.func2(0xc00006c000, 0xc000056180)
/home/projects/go-tuts/pipeline-loop.go:36 +0x72
created by main.main
/home/projects/go-tuts/pipeline-loop.go:16 +0x10f
exit status 2
In general, you don't need close a chanel multiple times - you only need to shut them once they are all finished. You need another WaitGroup for this; you need to pass both functions a WaitGroup.
Further reading: https://blog.golang.org/pipelines
UPDATE:
Personally I use a pattern for "works" that produce data into a same channel and the channel needs to be closed after all works are done:
for something {
wg.Add(1)
go func(i int) {
work(ch)
wg.Done()
}
}
go func() {
wg.Wait()
close()
}()
I think it is a good idea that keeps the API clean from WorkGroup as WorkGroup is about how you sync the work instead of how the work is done.
I have changed your code into this pattern: https://play.golang.org/p/vdCNsxWhgyQ
I suspect you only want one channel to read from ch1 and write to ch2. There is not much point in creating 3 go-routines to do the same thing (and you also end up closing the same channel mutliple time which causes a panic as leaf bebop pointed out)
func main() {
ch1 := make(chan int, 10) // Use buffered channel so as to avoid clogging
ch2 := make(chan string, 10)
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func1(i, ch1, &wg)
}
go func2(ch1, ch2)
wg.Wait()
close(ch1)
for val := range ch2 {
fmt.Println(val)
}
}
I am trying to figure out whether I can reopen a channel after closing it.
Test case:
I have a channel with some things in it
I want to range over them therefore I need to close the channel beforehand
I want to put more stuff in the channel and iterate through it once more
go func() {
queue <- "1"
queue <- "2"
close(queue)
}()
for i := range queue {
go func(i string) {
fmt.Println("From queue: ", i)
}(i)
}
go func() {
open(queue)
queue <- "3"
queue <- "4"
close(queue)
}()
for i := range queue {
go func(i string) {
fmt.Println("From queue: ", i)
}(i)
}
Of course open does not exist. How can I implement something like what I need in Go?
I don't want to use the Sleep function
I want to range over them therefore I need to close the channel beforehand
Nope, no need to close the channel. It'll resume iteration when another item is pushed thru the channel.
The below code accepts console input and pushes it to the channel:
main.go
package main
import (
"log"
"bufio"
"os"
"fmt"
)
func process(c chan string) {
for s := range c {
log.Println("processed", s)
}
}
func main() {
c := make(chan string, 10)
go process(c)
// get from console and process
reader := bufio.NewReader(os.Stdin)
fmt.Println("INPUT STUFF. TYPE #quit TO EXIT.")
for {
input, _, _ := reader.ReadLine()
if string(input) == "#quit" {
break
}
c <- string(input)
}
log.Println("BYE!")
}
output
INPUT STUFF. TYPE #quit TO EXIT.
hello
2018/10/23 10:43:52 processed hello
world
2018/10/23 10:43:54 processed world
#quit
2018/10/23 10:43:57 BYE!
The below sample uses Sleep() and is runnable as a Go Playground snippet
package main
import (
"log"
"time"
)
func process(c chan string) {
for s := range c {
log.Println("processed", s)
}
}
func main() {
c := make(chan string, 10)
go process(c)
// push some data
c <- "barry allen"
c <- "iris west"
time.Sleep(time.Second * 2)
// push more data
c <- "joe west"
c <- "caitlin snow"
time.Sleep(time.Second * 3)
}
output
2009/11/10 23:00:00 processed barry allen
2009/11/10 23:00:00 processed iris west
2009/11/10 23:00:02 processed joe west
2009/11/10 23:00:02 processed caitlin snow
Hope this helps. Cheers,
You cannot reopen a closed channel but you can send a channel on a channel, perhaps this is what you're looking for?
package main
import (
"fmt"
"time"
)
func main() {
queue := make(chan chan int)
defer close(queue)
go func() { // reader
for {
ch := <-queue
for i := range ch {
fmt.Println(i)
}
fmt.Println("Done with this channel")
}
}()
go func() { // writer-1
ch := make(chan int)
defer close(ch)
queue <- ch
ch <- 4
ch <- 2
}()
go func() { // writer-2
ch := make(chan int)
defer close(ch)
queue <- ch
ch <- 4
ch <- 20
}()
time.Sleep(time.Second)
}
Tho you cannot reopen the channel, it is possible to create a new one and assign to the variable. The previously closed channel will be garbage collected. In this case I reused the queue variable, but you can also create a new queue2 variable and pass it to the goroutine.
package main
import (
"context"
"fmt"
"time"
)
func main() {
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
queue := make(chan int)
go printFormat(nil, queue)
queue <- 1
queue <- 2
close(queue)
// fake some actions in between
time.Sleep(2 * time.Second)
queue = make(chan int)
go printFormat(cancel, queue)
queue <- 3
queue <- 4
close(queue)
<-ctx.Done()
}
func printFormat(c context.CancelFunc, q chan int) {
for i := range q {
fmt.Printf("Data %d \n", i)
}
if c != nil {
c()
}
}
Code below works fine with hard coded JSON data however doesn't work when I read JSON data from a file. I'm getting fatal error: all goroutines are asleep - deadlock error when using sync.WaitGroup.
WORKING EXAMPLE WITH HARD-CODED JSON DATA:
package main
import (
"bytes"
"fmt"
"os/exec"
"time"
)
func connect(host string) {
cmd := exec.Command("ssh", host, "uptime")
var out bytes.Buffer
cmd.Stdout = &out
err := cmd.Run()
if err != nil {
fmt.Println(err)
}
fmt.Printf("%s: %q\n", host, out.String())
time.Sleep(time.Second * 2)
fmt.Printf("%s: DONE\n", host)
}
func listener(c chan string) {
for {
host := <-c
go connect(host)
}
}
func main() {
hosts := [2]string{"user1#111.79.154.111", "user2#111.79.190.222"}
var c chan string = make(chan string)
go listener(c)
for i := 0; i < len(hosts); i++ {
c <- hosts[i]
}
var input string
fmt.Scanln(&input)
}
OUTPUT:
user#user-VirtualBox:~/go$ go run channel.go
user1#111.79.154.111: " 09:46:40 up 86 days, 18:16, 0 users, load average: 5"
user2#111.79.190.222: " 09:46:40 up 86 days, 17:27, 1 user, load average: 9"
user1#111.79.154.111: DONE
user2#111.79.190.222: DONE
NOT WORKING - EXAMPLE WITH READING JSON DATA FILE:
package main
import (
"bytes"
"fmt"
"os/exec"
"time"
"encoding/json"
"os"
"sync"
)
func connect(host string) {
cmd := exec.Command("ssh", host, "uptime")
var out bytes.Buffer
cmd.Stdout = &out
err := cmd.Run()
if err != nil {
fmt.Println(err)
}
fmt.Printf("%s: %q\n", host, out.String())
time.Sleep(time.Second * 2)
fmt.Printf("%s: DONE\n", host)
}
func listener(c chan string) {
for {
host := <-c
go connect(host)
}
}
type Content struct {
Username string `json:"username"`
Ip string `json:"ip"`
}
func main() {
var wg sync.WaitGroup
var source []Content
var hosts []string
data := json.NewDecoder(os.Stdin)
data.Decode(&source)
for _, value := range source {
hosts = append(hosts, value.Username + "#" + value.Ip)
}
var c chan string = make(chan string)
go listener(c)
for i := 0; i < len(hosts); i++ {
wg.Add(1)
c <- hosts[i]
defer wg.Done()
}
var input string
fmt.Scanln(&input)
wg.Wait()
}
OUTPUT
user#user-VirtualBox:~/go$ go run deploy.go < hosts.txt
user1#111.79.154.111: " 09:46:40 up 86 days, 18:16, 0 users, load average: 5"
user2#111.79.190.222: " 09:46:40 up 86 days, 17:27, 1 user, load average: 9"
user1#111.79.154.111 : DONE
user2#111.79.190.222: DONE
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc210000068)
/usr/lib/go/src/pkg/runtime/sema.goc:199 +0x30
sync.(*WaitGroup).Wait(0xc210047020)
/usr/lib/go/src/pkg/sync/waitgroup.go:127 +0x14b
main.main()
/home/user/go/deploy.go:64 +0x45a
goroutine 3 [chan receive]:
main.listener(0xc210038060)
/home/user/go/deploy.go:28 +0x30
created by main.main
/home/user/go/deploy.go:53 +0x30b
exit status 2
user#user-VirtualBox:~/go$
HOSTS.TXT
[
{
"username":"user1",
"ip":"111.79.154.111"
},
{
"username":"user2",
"ip":"111.79.190.222"
}
]
Go program ends when the main function ends.
From the language specification
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
Therefore, you need to wait for your goroutines to finish. The common solution for this is to use sync.WaitGroup object.
The simplest possible code to synchronize goroutine:
package main
import "fmt"
import "sync"
var wg sync.WaitGroup // 1
func routine() {
defer wg.Done() // 3
fmt.Println("routine finished")
}
func main() {
wg.Add(1) // 2
go routine() // *
wg.Wait() // 4
fmt.Println("main finished")
}
And for synchronizing multiple goroutines
package main
import "fmt"
import "sync"
var wg sync.WaitGroup // 1
func routine(i int) {
defer wg.Done() // 3
fmt.Printf("routine %v finished\n", i)
}
func main() {
for i := 0; i < 10; i++ {
wg.Add(1) // 2
go routine(i) // *
}
wg.Wait() // 4
fmt.Println("main finished")
}
WaitGroup usage in order of execution.
Declaration of global variable. Making it global is the easiest way to make it visible to all functions and methods.
Increasing the counter. This must be done in main goroutine because there is no guarantee that newly started goroutine will execute before 4 due to memory model guarantees.
Decreasing the counter. This must be done at the exit of goroutine. Using deferred call, we make sure that it will be called whenever function ends no matter but no matter how it ends.
Waiting for the counter to reach 0. This must be done in main goroutine to prevent program exit.
* The actual parameters are evaluated before starting new gouroutine. Thus it is needed to evaluate them explicitly before wg.Add(1) so the possibly panicking code would not leave increased counter.
Use
param := f(x)
wg.Add(1)
go g(param)
instead of
wg.Add(1)
go g(f(x))
Thanks for the very nice and detailed explanation Grzegorz Żur.
One thing that I want to point it out that typically the func that needs to be threaded wont be in main(), so we would have something like this:
package main
import (
"bufio"
"fmt"
"io"
"io/ioutil"
"math/rand"
"os"
"reflect"
"regexp"
"strings"
"sync"
"time"
)
var wg sync.WaitGroup // VERY IMP to declare this globally, other wise one //would hit "fatal error: all goroutines are asleep - deadlock!"
func doSomething(arg1 arg1Type) {
// cured cancer
}
func main() {
r := rand.New(rand.NewSource(time.Now().UnixNano()))
randTime := r.Intn(10)
wg.Add(1)
go doSomething(randTime)
wg.Wait()
fmt.Println("Waiting for all threads to finish")
}
The thing that I want to point it out is that global declaration of wg is very crucial for all threads to finish before main()
try this code snippest
package main
import (
"bytes"
"fmt"
"os/exec"
"time"
"sync"
)
func connect(host string, wg *sync.WaitGroup) {
defer wg.Done()
cmd := exec.Command("ssh", host, "uptime")
var out bytes.Buffer
cmd.Stdout = &out
err := cmd.Run()
if err != nil {
fmt.Println(err)
}
fmt.Printf("%s: %q\n", host, out.String())
time.Sleep(time.Second * 2)
fmt.Printf("%s: DONE\n", host)
}
func listener(c chan string，wg *sync.WaitGroup) {
for {
host，ok := <-c
// check channel is closed or not
if !ok{
break
}
go connect(host)
}
}
func main() {
var wg sync.WaitGroup
hosts := [2]string{"user1#111.79.154.111", "user2#111.79.190.222"}
var c chan string = make(chan string)
go listener(c)
for i := 0; i < len(hosts); i++ {
wg.Add(1)
c <- hosts[i]
}
close(c)
var input string
fmt.Scanln(&input)
wg.Wait()
}