Golang Channels dealing with send != receive - go

Consider the toy example below. The code works perfectly but when you interchange the 2 lines marked as replace, there will be a deadlock. Is there a better way to deal with such a situation when you have different number of sends and receives?
package main
import "fmt"
import "strconv"
func main() {
a := make(chan string)
b := make(chan string)
go func() {
for i := 0; i < 2; i++ {
go func(i int) {
fmt.Println(<-a)
b <- strconv.Itoa(i) + "b" // replace
a <- strconv.Itoa(i) + "a" // replace
}(i)
}
}()
a <- "0"
for i := 0; i < 2; i++ {
fmt.Println(<-b)
}
}
EDIT: using a select statement, there's a chance that a gets picked up by the select and there's still no way to prevent a deadlock because the goroutines can't execute
package main
import "fmt"
import "strconv"
func main() {
a := make(chan string)
b := make(chan string)
c := make(chan bool)
cancel := make(chan bool)
go func() {
for i := 0; i < 2; i++ {
go func(i int) {
fmt.Println(<-a)
b <- strconv.Itoa(i) + "b" // replace
a <- strconv.Itoa(i) + "a" // replace
c <- true
}(i)
}
}()
go func() {
<-c
<-c
cancel <- true
}()
a <- "0"
loop:
for {
select {
case ain := <-a:
fmt.Println("select", ain)
case bin := <-b:
fmt.Println("select", bin)
case <-cancel:
break loop
}
}
}

Use select:
package main
import "fmt"
import "strconv"
func main() {
a := make(chan string)
b := make(chan string)
go func() {
for i := 0; i < 2; i++ {
go func(i int) {
fmt.Println(<-a)
b <- strconv.Itoa(i) + "b" // replace
a <- strconv.Itoa(i) + "a" // replace
}(i)
}
}()
// regardless of which comes in first, this will handle it
select {
case ain <- a:
fmt.Println("sent a", ain)
case bin <- b:
fmt.Println("sent b", bin)
case <- cancel:
break
}
}
That example will sit and block for an item sent on either a or b channels.
Optionally I usually set a cancel token or a timeout.
Your original code deadlocks on the switch because you sent on B, where you were only listening on A. Golang requires you to be listening on the channel BEFORE you ever send to it. This is the pattern for multiple channels, not knowing which you are going to get first.

Related

Use channel to limit the number of active go routines

I'm reading the "The Go Programming Language"
One way to limit the number of running go routines is to use a "counting semaphore".
The other way is Limiting number of go routines running
I am allowing 2 more go routines in the case. I'm getting deadlock error.
What causes the deadlock in my code?
package main
import (
"bytes"
//"context"
"fmt"
"runtime"
"strconv"
"sync"
"time"
)
func main() {
max := 2
var wg sync.WaitGroup
squares := make(chan int)
tokens := make(chan struct{}, max)
for i := 20; i >= 1; i-- {
tokens <- struct{}{}
wg.Add(1)
go func(n int) {
defer func() { <-tokens }()
defer wg.Done()
fmt.Println("run go routine ", getGID())
squares <- Square(n)
}(i)
}
go func() {
wg.Wait()
close(squares)
}()
for s := range squares {
fmt.Println("Get square: ", s)
}
}
func Square(num int) int {
time.Sleep(time.Second * time.Duration(num))
fmt.Println(num * num)
return num * num
}
func getGID() uint64 {
b := make([]byte, 64)
b = b[:runtime.Stack(b, false)]
b = bytes.TrimPrefix(b, []byte("goroutine "))
b = b[:bytes.IndexByte(b, ' ')]
n, _ := strconv.ParseUint(string(b), 10, 64)
return n
}
The goroutines block on sending to squares. Main is not receiving on squares because it blocks on starting the the goroutines.
Fix by moving the code that starts the goroutines to a goroutine. This allows main to continue executing to the receive on squares.
squares := make(chan int)
go func() {
max := 2
var wg sync.WaitGroup
tokens := make(chan struct{}, max)
for i := 20; i >= 1; i-- {
tokens <- struct{}{}
wg.Add(1)
go func(n int) {
defer func() { <-tokens }()
defer wg.Done()
fmt.Println("run go routine ", getGID())
squares <- Square(n)
}(i)
}
wg.Wait()
close(squares)
}()
fmt.Println("About to receive")
for s := range squares {
fmt.Println("Get square: ", s)
}
Run it on the playground.

Passing info from channel to channel while altering it in go

I'm new to the go programming language.
I just learned about channels from their website, and tried to create
The following program:
1) I want to create 100,000 channels.
2) when the first channel receives a msg (value) he adds 1 to it
and pass it on to the next channel (by order).
3) when we'll get to the last channel it Will print 100,001.
I have created the channels with a loop, but I find it difficult to find
how to pass and alter the data in a "domino" fashion like I described.
Any solution or reference would be appreciated.
Thanks!
Here is my code:
package main
func addOneAndPass(c1 chan int, c2 chan int) {
c := make(chan int)
c <- 1
val := <- c
}
func main() {
const n = 100000
var channels [n]chan int
for i := 0; i < n; i++ {
channels[i] = make(chan int)
}
}
https://play.golang.org/p/ku-Dretm8EA
package main
import (
"fmt"
)
func add1(in chan int) (chan int) {
i := <-in
out := make(chan int, 1)
out <- (i+1)
return out
}
func main() {
ch := make(chan int, 1)
ch <- 1
for i := 0; i < 100000; i++ {
ch = add1(ch)
}
fmt.Println(<-ch)
}
another solution: https://play.golang.org/p/uWVxSG0xgqU
package main
import (
"fmt"
)
func add1(in, out chan int) {
i := <-in
out <- (i+1)
}
func main() {
start := make(chan int)
var in = start
var ch chan int
for i := 0; i < 100000; i++ {
ch = make(chan int)
go add1(in, ch)
in = ch
}
start <- 1
fmt.Println(<-ch)
}

Go: transformed channel

Let's say I have an int channel in Go:
theint := make(chan int)
I want to wrap this channel in a new channel called incremented
incremented := make(chan int)
Such that:
go func() { theint <- 1 }
<- incremented // 2
appended can be assumed to be the only one that reads from the int.
It will work if a run a goroutine in the background
go func() {
for num := range theint {
incremented <- num + 1
}
}
However, I prefer to do it without a goroutine since I can't control it in my context.
Is there a simpler way to do it?
One thing that came to mind is python's yield:
for num in theint:
yield num + 1
Is something like this possible in go?
Generator pattern
What you are trying to implement is generator pattern. To use channels and goroutines for implementation of this pattern is totally common practice.
However, I prefer to do it without a goroutine since I can't control it in my context.
I believe the problem is deadlock
fatal error: all goroutines are asleep - deadlock!
To avoid deadlocks and orphaned (not closed) channels use sync.WaitGroup. This is an idiomatic way to control goroutines.
Playground
package main
import (
"fmt"
"sync"
)
func incGenerator(n []int) chan int {
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(len(n))
for _, i := range n {
incremented := i + 1
go func() {
wg.Done()
ch <- incremented
}()
}
go func() {
wg.Wait()
close(ch)
}()
return ch
}
func main() {
n := []int{1, 2, 3, 4, 5}
for x := range incGenerator(n) {
fmt.Println(x)
}
}
One thing you can also consider is having a select on the int channel and an exit channel - in an infinite for loop. You can choose a variable increment value too. Please see code below:
package main
import (
"fmt"
"sync"
"time"
)
func main() {
var accum int //accumulator of incremented values
var wg sync.WaitGroup
c1 := make(chan int)
exChan := make(chan bool)
wg.Add(1)
go func() {
time.Sleep(time.Second * 1)
c1 <- 1
wg.Done()
}()
wg.Add(1)
go func() {
time.Sleep(time.Second * 2)
c1 <- 2
wg.Done()
}()
wg.Add(1)
go func() {
time.Sleep(time.Second * 2)
c1 <- 5
wg.Done()
}()
go func() {
wg.Wait()
close(exChan)
}()
for {
var done bool
select {
case incBy := <-c1: //Increment by value in channel
accum += incBy
fmt.Println("Received value to increment:", incBy, "; Accumulated value is", accum)
case d := <-exChan:
done = !(d)
}
if done == true {
break
}
}
fmt.Println("Final accumulated value is", accum)
}
Playground: https://play.golang.org/p/HmdRmMCN7U
Exit channel not needed, if we are having non-zero increments always. I like #I159 's approach too!
Anyways, hope this helps.

all goroutines are asleep deadlock

I am trying to play around with goroutines and channel
package main
import (
"fmt"
"math/rand"
"time"
)
func boring(msg string) <-chan string {
c := make(chan string)
go func() {
for i := 0; ; i++ {
c <- fmt.Sprintf("%s %d", msg, i)
time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond)
}
}()
return c
}
func main() {
c := fanInNew(boring("joe"), boring("anh"))
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
fmt.Println("You both are boring, I am leaving")
}
func fanInNew(input1, input2 <-chan string) <-chan string {
c := make(chan string)
for {
select {
case s := <-input1:
c <- s
case s := <-input2:
c <- s
}
}
return c
}
If i run this program it is giving me error, all goroutines are asleep, deadlock.
But If I put select inside anonymous goroutine, it works just fine. Working example:
package main
import (
"fmt"
"math/rand"
"time"
)
func boring(msg string) <-chan string {
c := make(chan string)
go func() {
for i := 0; ; i++ {
c <- fmt.Sprintf("%s %d", msg, i)
time.Sleep(time.Duration(rand.Intn(1e3)) * time.Millisecond)
}
}()
return c
}
func main() {
c := fanInNew(boring("joe"), boring("anh"))
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
fmt.Println("You both are boring, I am leaving")
}
func fanInNew(input1, input2 <-chan string) <-chan string {
c := make(chan string)
go func() {
for {
select {
case s := <-input1:
c <- s
case s := <-input2:
c <- s
}
}
}()
return c
}
Can you please help me to understand reasoning behind it.
The for statements loops forever, and so the <-c chan is never passed along and the chans get filled, but the main thread gets stuck waiting for c := fanInNew(a, b).
fanInNew() never returns because for loops forever (and blocks on select btw):
func fanInNew(input1, input2 <-chan string) <-chan string {
c := make(chan string)
for { // Loop forever and read from inputs 1 and 2
select {
case s := <-input1:
c <- s
case s := <-input2:
c <- s
}
}
return c
}
Then in the main thread this function never returns the c chan.
func main() {
// Never gets passed the next line
c := fanInNew(boring("joe"), boring("anh"))
}
So you can put the for loops themselves in goroutines, as you did in the second example. Also typically goroutines should return, either because you pass a message in (such as by close()ing), or because they reach a return statement.
In any case, what you have in the 2nd example is great for demonstrating the use of anonymous closures. The chan passed into the goroutine can be returned elsewhere and enable safe message passing between threads:
c := make(chan string)
go func() {
for {
select {
case s := <-input1:
c <- s
case s := <-input2:
c <- s
}
}
}()
return c
There are a few ways of ending a for loop in an anonymous goroutine, including select on a second chan, a closing channel, which when close()ed you can return. Also, typically WaitGroups can achieve that.

What is the correct way to write a distinct channel in Go?

I am a beginner in go.
I am trying to figure out an easy way to implement a channel that only output distinct values.
What I want to do is this:
package example
import (
"fmt"
"testing"
)
func TestShouldReturnDistinctValues(t *testing.T) {
var c := make([]chan int)
c <- 1
c <- 1
c <- 2
c <- 2
c <- 3
for e := range c {
// only print 1, 2 and 3.
fmt.println(e)
}
}
Should I be concern about memory leak here if I were to use a map to remember previous values?
You really can't do that, you'd have to keep a track of the values somehow, a map[int]struct{} is probably the most memory efficient way.
A simple example:
func UniqueGen(min, max int) <-chan int {
m := make(map[int]struct{}, max-min)
ch := make(chan int)
go func() {
for i := 0; i < 1000; i++ {
v := min + rand.Intn(max)
if _, ok := m[v]; !ok {
ch <- v
m[v] = struct{}{}
}
}
close(ch)
}()
return ch
}
I have done similar things before, except my problem was output inputs in ascending order. You can do this by adding a middle go routine. Here is an example:
package main
func main() {
input, output := distinct()
go func() {
input <- 1
input <- 1
input <- 2
input <- 2
input <- 3
close(input)
}()
for i := range output {
println(i)
}
}
func distinct() (input chan int, output chan int) {
input = make(chan int)
output = make(chan int)
go func() {
set := make(map[int]struct{})
for i := range input {
if _, ok := set[i]; !ok {
set[i] = struct{}{}
output <- i
}
}
close(output)
}()
return
}

Resources