I am working on the tree exercise of tour.golang. I have tried to implement the same function as written below.
func Same(t1, t2 *tree.Tree) bool {
ch1 := make(chan int)
ch2 := make(chan int)
go Walk(t1, ch1);
go Walk(t2, ch2);
for c := range ch1 {
d := <- ch2
if c-d !=0 {
return false
}
}
return true
}
Using the forever loop, I would like to compare if an output from ch1 is different from that of ch2. But the following is throwing this error:
fatal error: all goroutines are asleep - deadlock!
live version
You are seeing a deadlock for a very simple reason: you are ranging over ch1, but never closing it, so the for loop never terminates.
You could fix this by manually iterating over each tree only a certain number of times like your 0..10 loop in main():
// Same determines whether the trees
// t1 and t2 contain the same values.
func Same(t1, t2 *tree.Tree) bool {
ch1 := make(chan int)
ch2 := make(chan int)
go Walk(t1, ch1)
go Walk(t2, ch2)
for i := 0; i < 10; i++ {
c := <-ch1
d := <-ch2
if c-d != 0 {
return false
}
}
return true
}
Playground
Alternatively, you can alter the signature of Walk to accept a waitgroup argument that is incremented by the caller of Walk and decremented when each Walk returns along with a goroutine to close the channel once you're done walking:
// Walk walks the tree t sending all values
// from the tree to the channel ch.
func Walk(t *tree.Tree, ch chan int, wg *sync.WaitGroup) {
defer wg.Done()
if t.Left != nil {
wg.Add(1)
Walk(t.Left, ch, wg)
}
ch <- t.Value
if t.Right != nil {
wg.Add(1)
Walk(t.Right, ch, wg)
}
}
// Same determines whether the trees
// t1 and t2 contain the same values.
func Same(t1, t2 *tree.Tree) bool {
ch1 := make(chan int)
ch2 := make(chan int)
var wg1 sync.WaitGroup
wg1.Add(1)
go Walk(t1, ch1, &wg1)
go func() {
wg1.Wait()
close(ch1)
}()
var wg2 sync.WaitGroup
wg2.Add(1)
go Walk(t2, ch2, &wg2)
go func() {
// not strictly necessary, since we're not ranging over ch2, but here for completeness
wg2.Wait()
close(ch2)
}()
for c := range ch1 {
d := <-ch2
if c-d != 0 {
return false
}
}
return true
}
Playground
You should close the channel after walking the tree to terminate the range loop in case the trees are equal (to be aware of: Same returns true when the infix traversal of the trees is equal, their structure is not required to be equal).
func WalkTreeAndThenCloseChannel(t *tree.Tree, ch chan int) {
Walk(t, ch)
close(ch)
}
func Same(t1, t2 *tree.Tree) bool {
ch1 := make(chan int)
ch2 := make(chan int)
go WalkTreeAndThenCloseChannel(t1, ch1);
go WalkTreeAndThenCloseChannel(t2, ch2);
Note: You should check if the second channel has been closed in case the trees have a different number of items and a difference is not found ("thread starvation" would be a more appropriate term here than "deadlock").
There is a problem here you are not sending a value to a channel for right subtree in walk function. But receiving it on other side that's why deadlock error. Because you are receiving a value from channel in case of right subtree which is never sent.
Related
func New(k int) *Tree
// New() returns a random binary tree holding the values k, 2k, ..., 10k.
I'm just trying traversal binary tree in goroutine and add values to channel. Then print them in main goroutine
Code
func binary(t *tree.Tree, ch chan int) {
if t != nil {
binary(t.Left, ch)
ch <- t.Value
binary(t.Right, ch)
}
}
func Walk(t *tree.Tree, ch chan int) {
defer close(ch)
binary(t, ch)
}
func main() {
ch := make(chan int)
go Walk(tree.New(1), ch)
for i := range ch {
fmt.Printf("%d ", <-ch)
_ = i
}
}
Expected output = 1 2 3 4 5 6 7 8 9 10
Result = 2 4 6 8 10
The for statement with a range clause over a channel receives values from the channel and stores them in the loop variable.
Meaning the i variable will hold values received from ch, you do not need to receive from ch.
Yet, you're not using i, and you do receive from ch. So you'll skip every second element (and you'll also risk getting blocked if there are odd number of elements delivered on the channel).
Do it like this:
for v := range ch {
fmt.Printf("%d ", v)
}
Based on the suggestion of icza:
func binary(t *tree.Tree, ch chan int) {
if t != nil {
binary(t.Left, ch)
ch <- t.Value
binary(t.Right, ch)
}
}
func Walk(t *tree.Tree, ch chan int) {
defer close(ch)
binary(t, ch)
}
func main() {
ch := make(chan int)
go Walk(tree.New(1), ch)
for v := range ch {
fmt.Printf("%d ", v)
}
}
I'm learning to work with Go channels, and I'm always getting deadlocks. What might be wrong with this code? Printer randomly stops working when array sizes are unequal; I guess it would help to somehow notify printer that receiver stopped working. Any ideas how to fix it? My code is pasted below.
package main
import (
"fmt"
"sync"
)
var wg = sync.WaitGroup{}
var wgs = sync.WaitGroup{}
var sg = make(chan int, 50)
var gp1 = make(chan int, 50)
var gp2 = make(chan int, 50)
func main(){
wgs.Add(2)
go Sender(0)
go Sender(11)
wg.Add(3)
go Receiver()
go Printer()
go Printer2()
wg.Wait()
}
func Sender(start int){
defer wgs.Done()
for i := start; i < 20; i++ {
sg <- i
}
}
func Receiver(){
defer wg.Done()
for i := 0; i < 20; i++{
nr := <- sg
if nr % 2 == 0{
gp1 <- nr
} else{
gp2 <- nr
}
}
}
func Printer(){
defer wg.Done()
var m [10]int
for i := 0; i < 10; i++ {
m[i] = <- gp1
}
wgs.Wait()
fmt.Println(m)
}
func Printer2(){
defer wg.Done()
var m [10]int
for i := 0; i < 10; i++ {
m[i] = <- gp2
}
wgs.Wait()
fmt.Println(m)
}
// Better to use this one
// func Receiver(senderChannel <-chan int, printerChannel1 chan<- int, printerChannel2 chan<- int, wg *sync.WaitGroup) {
The Sender generates (I think 28 messages) . Roughly half the first 20 of these go to one of gp1 and gp2. Printer and Printer2 then unload the messages
Trouble is, the way that Receiver splits the messages depends on if the number received is odd or even. But you aren't controlling for this. If one of the Printers has less than 10 items in it's queue it will hang
That's one potential problem
Your core problem is that everything in this is "dead reckoning": they expect to see a fixed number of messages, but this doesn't necessarily match up with reality. You should set up the channels so that they get closed once all of their data gets produced.
This probably means setting up an intermediate function to manage the sending:
func Sender(from, to int, c chan<- int) {
for i := from; i < to; i++ {
c <- i
}
}
func SendEverything(c chan<- int) {
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
Sender(0, 20, c)
}()
go func() {
defer wg.Done()
Sender(11, 20, c)
}()
wg.Wait()
close(c)
}
Make the dispatcher function work on everything in the channel:
func Receive(c <-chan int, odds, evens chan<- int) {
for n := range c {
if n%2 == 0 {
evens <- n
} else {
odds <- n
}
}
close(odds)
close(evens)
}
And then you can share a single print function:
func Printer(prefix string, c <-chan int) {
for n := range c {
fmt.Printf("%s: %d\n", prefix, n)
}
}
Finally, you have a main function that stitches it all together:
func main() {
var wg sync.WaitGroup
inputs := make(chan int)
odds := make(chan int)
evens := make(chan int)
wg.Add(4)
go func() {
defer wg.Done()
SendEverything(inputs)
}()
go func() {
defer wg.Done()
Receive(inputs, odds, evens)
}()
go func() {
defer wg.Done()
Printer("odd number", odds)
}()
go func() {
defer wg.Done()
Printer("even number", evens)
}()
wg.Wait()
}
The complete example is at https://play.golang.org/p/qTUqlt-uaWH.
Note that I've completely refrained from using any global variables, and either things have a hopefully self-explanatory very short name (i and n are simple integers, c is a channel) or are complete words (odds, evens). I've tended to keep sync.WaitGroup objects local to where they're created. Since everything is passed as parameters, I don't need two copies of the same function to act on different global variables, and if I chose to write test code for this, I could create my own local channels.
I have been working in Golang for a long time. But still I am facing this problem though I know the solution to my problem. But never figured out why is it happening.
For example If I have a pipeline situation for inbound and outbound channels like below:
package main
import (
"fmt"
)
func main() {
for n := range sq(sq(gen(3, 4))) {
fmt.Println(n)
}
fmt.Println("Process completed")
}
func gen(nums ...int) <-chan int {
out := make(chan int)
go func() {
for _, n := range nums {
out <- n
}
close(out)
}()
return out
}
func sq(in <-chan int) <-chan int {
out := make(chan int)
go func() {
for n := range in {
out <- n * n
}
close(out)
}()
return out
}
It does not give me a deadlock situation. But if I remove the go routine inside the outbound code as below:
func sq(in <-chan int) <-chan int {
out := make(chan int)
for n := range in {
out <- n * n
}
close(out)
return out
}
I received a deadlock error. Why is it so that looping over channels using range without go routine gives a deadlock.
This situation caused of output channel of sq function is not buffered. So sq is waiting until next function will read from output, but if sq is not async, it will not happen (Playground link):
package main
import (
"fmt"
"sync"
)
var wg sync.WaitGroup
func main() {
numsCh := gen(3, 4)
sqCh := sq(numsCh) // if there is no sq in body - we are locked here until input channel will be closed
result := sq(sqCh) // but if output channel is not buffered, so `sq` is locked, until next function will read from output channel
for n := range result {
fmt.Println(n)
}
fmt.Println("Process completed")
}
func gen(nums ...int) <-chan int {
out := make(chan int)
go func() {
for _, n := range nums {
out <- n
}
close(out)
}()
return out
}
func sq(in <-chan int) <-chan int {
out := make(chan int, 100)
for n := range in {
out <- n * n
}
close(out)
return out
}
Your function creates a channel, writes to it, then returns it. The writing will block until somebody can read the corresponding value, but that's impossible because nobody outside this function has the channel yet.
func sq(in <-chan int) <-chan int {
// Nobody else has this channel yet...
out := make(chan int)
for n := range in {
// ...but this line will block until somebody reads the value...
out <- n * n
}
close(out)
// ...and nobody else can possibly read it until after this return.
return out
}
If you wrap the loop in a goroutine then both the loop and the sq function are allowed to continue; even if the loop blocks, the return out statement can still go and eventually you'll be able to connect up a reader to the channel.
(There's nothing intrinsically bad about looping over channels outside of goroutines; your main function does it harmlessly and correctly.)
The reason of the deadlock is because the main is waiting for the sq return and finish, but the sq is waiting for someone read the chan then it can continue.
I simplified your code by removing layer of sq call, and split one sentence into 2 :
func main() {
result := sq(gen(3, 4)) // <-- block here, because sq doesn't return
for n := range result {
fmt.Println(n)
}
fmt.Println("Process completed")
}
func gen(nums ...int) <-chan int {
out := make(chan int)
go func() {
for _, n := range nums {
out <- n
}
close(out)
}()
return out
}
func sq(in <-chan int) <-chan int {
out := make(chan int)
for n := range in {
out <- n * n // <-- block here, because no one is reading from the chan
}
close(out)
return out
}
In sq method, if you put code in goroutine, then the sq will returned, and main func will not block, and consume the result queue, and the goroutine will continue, then there is no block any more.
func main() {
result := sq(gen(3, 4)) // will not blcok here, because the sq just start a goroutine and return
for n := range result {
fmt.Println(n)
}
fmt.Println("Process completed")
}
func gen(nums ...int) <-chan int {
out := make(chan int)
go func() {
for _, n := range nums {
out <- n
}
close(out)
}()
return out
}
func sq(in <-chan int) <-chan int {
out := make(chan int)
go func() {
for n := range in {
out <- n * n // will not block here, because main will continue and read the out chan
}
close(out)
}()
return out
}
The code is a bit complicated,
Let's simplify
First eq below, not has deadLock
func main() {
send := make(chan int)
receive := make(chan int)
go func() {
send<-3
send<-4
close(send)
}()
go func() {
receive<- <-send
receive<- <-send
close(receive)
}()
for v := range receive{
fmt.Println(v)
}
}
Second eq below,remove "go" has deadLock
func main() {
send := make(chan int)
receive := make(chan int)
go func() {
send<-3
send<-4
close(send)
}()
receive<- <-send
receive<- <-send
close(receive)
for v := range receive{
fmt.Println(v)
}
}
Let's simplify second code again
func main() {
ch := make(chan int)
ch <- 3
ch <- 4
close(ch)
for v := range ch{
fmt.Println(v)
}
}
The reason of the deadlock is no buffer channel waiting in main goroutine.
Two Solutions
// add more cap then "channel<-" time
func main() {
ch := make(chan int,2)
ch <- 3
ch <- 4
close(ch)
for v := range ch{
fmt.Println(v)
}
}
//async "<-channel"
func main() {
ch := make(chan int)
go func() {
for v := range ch {
fmt.Println(v)
}
}()
ch <- 3
ch <- 4
close(ch)
}
My understanding is
when the main thread is blocked for waiting for the chan to be writen or read, Go will detect if any other Go routine is running. If there is no any other Go routine running, it will have "fatal error: all goroutines are asleep - deadlock!"
I tested if by using the below simple case
func main() {
c := make(chan int)
go func() {
time.Sleep(10 * time.Second)
}()
c <- 1
}
The deadlock error is reported after 10 seconds.
Let's say I have an int channel in Go:
theint := make(chan int)
I want to wrap this channel in a new channel called incremented
incremented := make(chan int)
Such that:
go func() { theint <- 1 }
<- incremented // 2
appended can be assumed to be the only one that reads from the int.
It will work if a run a goroutine in the background
go func() {
for num := range theint {
incremented <- num + 1
}
}
However, I prefer to do it without a goroutine since I can't control it in my context.
Is there a simpler way to do it?
One thing that came to mind is python's yield:
for num in theint:
yield num + 1
Is something like this possible in go?
Generator pattern
What you are trying to implement is generator pattern. To use channels and goroutines for implementation of this pattern is totally common practice.
However, I prefer to do it without a goroutine since I can't control it in my context.
I believe the problem is deadlock
fatal error: all goroutines are asleep - deadlock!
To avoid deadlocks and orphaned (not closed) channels use sync.WaitGroup. This is an idiomatic way to control goroutines.
Playground
package main
import (
"fmt"
"sync"
)
func incGenerator(n []int) chan int {
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(len(n))
for _, i := range n {
incremented := i + 1
go func() {
wg.Done()
ch <- incremented
}()
}
go func() {
wg.Wait()
close(ch)
}()
return ch
}
func main() {
n := []int{1, 2, 3, 4, 5}
for x := range incGenerator(n) {
fmt.Println(x)
}
}
One thing you can also consider is having a select on the int channel and an exit channel - in an infinite for loop. You can choose a variable increment value too. Please see code below:
package main
import (
"fmt"
"sync"
"time"
)
func main() {
var accum int //accumulator of incremented values
var wg sync.WaitGroup
c1 := make(chan int)
exChan := make(chan bool)
wg.Add(1)
go func() {
time.Sleep(time.Second * 1)
c1 <- 1
wg.Done()
}()
wg.Add(1)
go func() {
time.Sleep(time.Second * 2)
c1 <- 2
wg.Done()
}()
wg.Add(1)
go func() {
time.Sleep(time.Second * 2)
c1 <- 5
wg.Done()
}()
go func() {
wg.Wait()
close(exChan)
}()
for {
var done bool
select {
case incBy := <-c1: //Increment by value in channel
accum += incBy
fmt.Println("Received value to increment:", incBy, "; Accumulated value is", accum)
case d := <-exChan:
done = !(d)
}
if done == true {
break
}
}
fmt.Println("Final accumulated value is", accum)
}
Playground: https://play.golang.org/p/HmdRmMCN7U
Exit channel not needed, if we are having non-zero increments always. I like #I159 's approach too!
Anyways, hope this helps.
I am learning Go and am working on this lesson from the GoTours. Here's what I have so far.
package main
import (
"fmt"
"code.google.com/p/go-tour/tree"
)
// Walk walks the tree t sending all values
// from the tree to the channel ch.
func Walk(t *tree.Tree, ch chan int) {
if t != nil {
Walk(t.Left, ch)
ch <- t.Value
Walk(t.Right, ch)
}
}
func main() {
var ch chan int = make(chan int)
go Walk(tree.New(1), ch)
for c := range ch {
fmt.Printf("%d ", c)
}
}
As you see, I try to test out my Walk function by printing out the values I wrote into a channel. However, I get the following error.
1 2 3 4 5 6 7 8 9 10 throw: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
main.go:25 +0x85
goroutine 2 [syscall]:
created by runtime.main
/usr/local/go/src/pkg/runtime/proc.c:221
exit status 2
This error should be expected I think because I never close the channel. However, is there a way I could "catch" this deadlock error and deal with it programmatically?
Deadlock is similar to a nil pointer deference in that is represents a BUG in your program. This class of error is usually not recoverable for this reason.
As lbonn mentioned, the problem here is you need to "close(myChan)" your channel. If you do not do this the for-range loop, that the loop will wait for the next element forever.
You can try something like this:
func main() {
var ch chan int = make(chan int)
go func() {
Walk(tree.New(1), ch)
close(ch)
}()
for c := range ch {
fmt.Printf("%d ", c)
}
}
If you want to traverse the tree in parallel you will need to make further changes:
package main
import (
"code.google.com/p/go-tour/tree"
"fmt"
"sync"
)
// Walk walks the tree t sending all values
// from the tree to the channel ch.
func Walk(t *tree.Tree, ch chan int, done *sync.WaitGroup) {
if t != nil {
done.Add(2)
go Walk(t.Left, ch, done) //look at each branch in parallel
go Walk(t.Right, ch, done)
ch <- t.Value
}
done.Done()
}
func main() {
var ch chan int = make(chan int, 64) //note the buffer size
go func() {
done := new(sync.WaitGroup)
done.Add(1)
Walk(tree.New(1), ch, done)
done.Wait()
close(ch)
}()
for c := range ch {
fmt.Printf("%d ", c)
}
}
This deadlocks because, the range construct iterates until the channel is closed.
http://golang.org/ref/spec#For_statements
Here, you need to either close the channel when the tree is fully explored or use another construct.
For this example, you know that the trees are of size 10, so you can simply do a for loop from 1 to 10 and read from the channel once at each iteration.
No, you cannot recover from a deadlock.
The channel deadlock error is:
fatal error: all goroutines are asleep - deadlock!
channel deadlock is not a panic error, it's a fatal error, see https://golang.org/pkg/log/#Fatal
Fatal is equivalent to Print() followed by a call to os.Exit(1).
As you can see, Fatal will call os.Exit(1) at last, so it's totally different with panic, and that's why it can't be recover
I came up with this solution, It basically uses 2 channels and concludes the trees are same if both the channels are closed.
package main
import "golang.org/x/tour/tree"
import "fmt"
// Walk walks the tree t sending all values
// from the tree to the channel ch.
func Walk(t *tree.Tree, ch chan int) {
if t == nil {
return
}
Walk(t.Left, ch)
ch <- t.Value
Walk(t.Right, ch)
}
// Same determines whether the trees
// t1 and t2 contain the same values.
func Same(t1, t2 *tree.Tree) bool {
tc1 := make(chan int)
tc2 := make(chan int)
go func() {
Walk(t1, tc1)
close(tc1)
}()
go func() {
Walk(t2, tc2)
close(tc2)
}()
for {
x1, y1 := <-tc1;
x2, y2 := <-tc2;
if x1 != x2 {
return false
}
if !y1 || !y2 {
return true
}
}
}
func main() {
t1, t2 := tree.New(123), tree.New(1)
fmt.Println(Same(t1, t2))
}
I'm following go-tour guide and only used resources taught so far (Unlike using sync package in the above solution.)