In Go, is there a more idiomatic way to add all of the elements of an array/slice into a channel than the following?
ch := make(chan string)
values := []string{"lol", "cat", "lolcat"}
go func() {
for _, v := range values {
ch <- v
}
}()
I was looking for something like ch <- values... but that is rejected by the compiler.
Until iterators will come along, yes, the code you wrote is as idiomatic as it gets. I have it packaged for reuse as something like this in codebases I work on:
// ToChan returns a channel containing all elements in the slice s.
// The channel is closed when all elements are consumed from the channel.
func ToChan[T any](s []T) <-chan T {
ch := make(chan T, len(s))
for _, e := range s {
ch <- e
}
close(ch)
return ch
}
https://go.dev/play/p/c5v4df_M1IG
A for/range loop is the idiomatic way to send all of the elements of a slice to a channel:
for _, v := range values {
ch <- v
}
It is not necessary to run the for loop in a goroutine, as shown in the question.
You can declare a chan of string arrays, unless you absolutely want to keep a chan of strings :
package main
import "fmt"
func main() {
ch := make(chan []string)
values := []string{"lol", "cat", "lolcat"}
go func() {
ch <- values
}()
fmt.Printf("Values : %+v\n", <-ch)
}
Related
I am working on the tree exercise of tour.golang. I have tried to implement the same function as written below.
func Same(t1, t2 *tree.Tree) bool {
ch1 := make(chan int)
ch2 := make(chan int)
go Walk(t1, ch1);
go Walk(t2, ch2);
for c := range ch1 {
d := <- ch2
if c-d !=0 {
return false
}
}
return true
}
Using the forever loop, I would like to compare if an output from ch1 is different from that of ch2. But the following is throwing this error:
fatal error: all goroutines are asleep - deadlock!
live version
You are seeing a deadlock for a very simple reason: you are ranging over ch1, but never closing it, so the for loop never terminates.
You could fix this by manually iterating over each tree only a certain number of times like your 0..10 loop in main():
// Same determines whether the trees
// t1 and t2 contain the same values.
func Same(t1, t2 *tree.Tree) bool {
ch1 := make(chan int)
ch2 := make(chan int)
go Walk(t1, ch1)
go Walk(t2, ch2)
for i := 0; i < 10; i++ {
c := <-ch1
d := <-ch2
if c-d != 0 {
return false
}
}
return true
}
Playground
Alternatively, you can alter the signature of Walk to accept a waitgroup argument that is incremented by the caller of Walk and decremented when each Walk returns along with a goroutine to close the channel once you're done walking:
// Walk walks the tree t sending all values
// from the tree to the channel ch.
func Walk(t *tree.Tree, ch chan int, wg *sync.WaitGroup) {
defer wg.Done()
if t.Left != nil {
wg.Add(1)
Walk(t.Left, ch, wg)
}
ch <- t.Value
if t.Right != nil {
wg.Add(1)
Walk(t.Right, ch, wg)
}
}
// Same determines whether the trees
// t1 and t2 contain the same values.
func Same(t1, t2 *tree.Tree) bool {
ch1 := make(chan int)
ch2 := make(chan int)
var wg1 sync.WaitGroup
wg1.Add(1)
go Walk(t1, ch1, &wg1)
go func() {
wg1.Wait()
close(ch1)
}()
var wg2 sync.WaitGroup
wg2.Add(1)
go Walk(t2, ch2, &wg2)
go func() {
// not strictly necessary, since we're not ranging over ch2, but here for completeness
wg2.Wait()
close(ch2)
}()
for c := range ch1 {
d := <-ch2
if c-d != 0 {
return false
}
}
return true
}
Playground
You should close the channel after walking the tree to terminate the range loop in case the trees are equal (to be aware of: Same returns true when the infix traversal of the trees is equal, their structure is not required to be equal).
func WalkTreeAndThenCloseChannel(t *tree.Tree, ch chan int) {
Walk(t, ch)
close(ch)
}
func Same(t1, t2 *tree.Tree) bool {
ch1 := make(chan int)
ch2 := make(chan int)
go WalkTreeAndThenCloseChannel(t1, ch1);
go WalkTreeAndThenCloseChannel(t2, ch2);
Note: You should check if the second channel has been closed in case the trees have a different number of items and a difference is not found ("thread starvation" would be a more appropriate term here than "deadlock").
There is a problem here you are not sending a value to a channel for right subtree in walk function. But receiving it on other side that's why deadlock error. Because you are receiving a value from channel in case of right subtree which is never sent.
I want to create a slice that is a channel and contains integers.
test := make(chan []int)
test <- 5
This is the way I initialized it but I have no idea how to now pass a value since for slices we would use append but for channels we send data using <-
I have tried with both just <- and append and both combined like below and can't get it to work
test <- append(test, 5)
You defined a channel of []int but are trying to send it an int. You have to send it a slice of ints and then have the receiver use that slice.
A working example is here: https://play.golang.org/p/TmcUKU8G-1
Notice that I'm appending the 4 to the things slice and not the channel itself
package main
import (
"fmt"
)
func main() {
c := make(chan []int)
things := []int{1, 2, 3}
go func() {
c <- things
}()
for _, i := range <-c {
fmt.Println(i)
}
go func() {
c <- append(things, 4)
}()
for _, i := range <-c {
fmt.Println(i)
}
}
Output:
1
2
3
1
2
3
4
This is called
buffered channel
the right syntaxis is
test := make(chan int, 5)
test <- 1
test <- 2
Golang tour has an example:
https://tour.golang.org/concurrency/3
Let's say I have an int channel in Go:
theint := make(chan int)
I want to wrap this channel in a new channel called incremented
incremented := make(chan int)
Such that:
go func() { theint <- 1 }
<- incremented // 2
appended can be assumed to be the only one that reads from the int.
It will work if a run a goroutine in the background
go func() {
for num := range theint {
incremented <- num + 1
}
}
However, I prefer to do it without a goroutine since I can't control it in my context.
Is there a simpler way to do it?
One thing that came to mind is python's yield:
for num in theint:
yield num + 1
Is something like this possible in go?
Generator pattern
What you are trying to implement is generator pattern. To use channels and goroutines for implementation of this pattern is totally common practice.
However, I prefer to do it without a goroutine since I can't control it in my context.
I believe the problem is deadlock
fatal error: all goroutines are asleep - deadlock!
To avoid deadlocks and orphaned (not closed) channels use sync.WaitGroup. This is an idiomatic way to control goroutines.
Playground
package main
import (
"fmt"
"sync"
)
func incGenerator(n []int) chan int {
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(len(n))
for _, i := range n {
incremented := i + 1
go func() {
wg.Done()
ch <- incremented
}()
}
go func() {
wg.Wait()
close(ch)
}()
return ch
}
func main() {
n := []int{1, 2, 3, 4, 5}
for x := range incGenerator(n) {
fmt.Println(x)
}
}
One thing you can also consider is having a select on the int channel and an exit channel - in an infinite for loop. You can choose a variable increment value too. Please see code below:
package main
import (
"fmt"
"sync"
"time"
)
func main() {
var accum int //accumulator of incremented values
var wg sync.WaitGroup
c1 := make(chan int)
exChan := make(chan bool)
wg.Add(1)
go func() {
time.Sleep(time.Second * 1)
c1 <- 1
wg.Done()
}()
wg.Add(1)
go func() {
time.Sleep(time.Second * 2)
c1 <- 2
wg.Done()
}()
wg.Add(1)
go func() {
time.Sleep(time.Second * 2)
c1 <- 5
wg.Done()
}()
go func() {
wg.Wait()
close(exChan)
}()
for {
var done bool
select {
case incBy := <-c1: //Increment by value in channel
accum += incBy
fmt.Println("Received value to increment:", incBy, "; Accumulated value is", accum)
case d := <-exChan:
done = !(d)
}
if done == true {
break
}
}
fmt.Println("Final accumulated value is", accum)
}
Playground: https://play.golang.org/p/HmdRmMCN7U
Exit channel not needed, if we are having non-zero increments always. I like #I159 's approach too!
Anyways, hope this helps.
I'm working on a concurrent Go library, and I stumbled upon two distinct patterns of synchronization between goroutines whose results are similar:
Waitgroup
package main
import (
"fmt"
"sync"
"time"
)
var wg sync.WaitGroup
func main() {
words := []string{"foo", "bar", "baz"}
for _, word := range words {
wg.Add(1)
go func(word string) {
time.Sleep(1 * time.Second)
defer wg.Done()
fmt.Println(word)
}(word)
}
// do concurrent things here
// blocks/waits for waitgroup
wg.Wait()
}
Channel
package main
import (
"fmt"
"time"
)
func main() {
words := []string{"foo", "bar", "baz"}
done := make(chan bool)
// defer close(done)
for _, word := range words {
// fmt.Println(len(done), cap(done))
go func(word string) {
time.Sleep(1 * time.Second)
fmt.Println(word)
done <- true
}(word)
}
// Do concurrent things here
// This blocks and waits for signal from channel
for range words {
<-done
}
}
I was advised that sync.WaitGroup is slightly more performant, and I have seen it being used commonly. However, I find channels more idiomatic. What is the real advantage of using sync.WaitGroup over channels and/or what might be the situation when it is better?
Independently of the correctness of your second example (as explained in the comments, you aren't doing what you think, but it's easily fixable), I tend to think that the first example is easier to grasp.
Now, I wouldn't even say that channels are more idiomatic. Channels being a signature feature of the Go language shouldn't mean that it is idiomatic to use them whenever possible. What is idiomatic in Go is to use the simplest and easiest to understand solution: here, the WaitGroup convey both the meaning (your main function is Waiting for workers to be done) and the mechanic (the workers notify when they are Done).
Unless you're in a very specific case, I don't recommend using the channel solution here.
For your simple example (signalling the completion of jobs), the WaitGroup is the obvious choice. And the Go compiler is very kind and won't blame you for using a channel for the simple signalling of the completion task, but some code reviewer do.
"A WaitGroup waits for a collection of goroutines to finish.
The main goroutine calls Add(n) to set the number of
goroutines to wait for. Then each of the goroutines
runs and calls Done() when finished. At the same time,
Wait can be used to block until all goroutines have finished."
words := []string{"foo", "bar", "baz"}
var wg sync.WaitGroup
for _, word := range words {
wg.Add(1)
go func(word string) {
defer wg.Done()
time.Sleep(100 * time.Millisecond) // a job
fmt.Println(word)
}(word)
}
wg.Wait()
The possibilities are limited only by your imagination:
Channels can be buffered:
words := []string{"foo", "bar", "baz"}
done := make(chan struct{}, len(words))
for _, word := range words {
go func(word string) {
time.Sleep(100 * time.Millisecond) // a job
fmt.Println(word)
done <- struct{}{} // not blocking
}(word)
}
for range words {
<-done
}
Channels can be unbuffered, and you may use just a signalling channel (e.g. chan struct{}):
words := []string{"foo", "bar", "baz"}
done := make(chan struct{})
for _, word := range words {
go func(word string) {
time.Sleep(100 * time.Millisecond) // a job
fmt.Println(word)
done <- struct{}{} // blocking
}(word)
}
for range words {
<-done
}
You may limit the number of concurrent jobs with buffered channel capacity:
t0 := time.Now()
var wg sync.WaitGroup
words := []string{"foo", "bar", "baz"}
done := make(chan struct{}, 1) // set the number of concurrent job here
for _, word := range words {
wg.Add(1)
go func(word string) {
done <- struct{}{}
time.Sleep(100 * time.Millisecond) // job
fmt.Println(word, time.Since(t0))
<-done
wg.Done()
}(word)
}
wg.Wait()
You may send a message using a channel:
done := make(chan string)
go func() {
for _, word := range []string{"foo", "bar", "baz"} {
done <- word
}
close(done)
}()
for word := range done {
fmt.Println(word)
}
Benchmark:
go test -benchmem -bench . -args -n 0
# BenchmarkEvenWaitgroup-8 1827517 652 ns/op 0 B/op 0 allocs/op
# BenchmarkEvenChannel-8 1000000 2373 ns/op 520 B/op 1 allocs/op
go test -benchmem -bench .
# BenchmarkEvenWaitgroup-8 1770260 678 ns/op 0 B/op 0 allocs/op
# BenchmarkEvenChannel-8 1560124 1249 ns/op 158 B/op 0 allocs/op
Code(main_test.go):
package main
import (
"flag"
"fmt"
"os"
"sync"
"testing"
)
func BenchmarkEvenWaitgroup(b *testing.B) {
evenWaitgroup(b.N)
}
func BenchmarkEvenChannel(b *testing.B) {
evenChannel(b.N)
}
func evenWaitgroup(n int) {
if n%2 == 1 { // make it even:
n++
}
for i := 0; i < n; i++ {
wg.Add(1)
go func(n int) {
select {
case ch <- n: // tx if channel is empty
case i := <-ch: // rx if channel is not empty
// fmt.Println(n, i)
_ = i
}
wg.Done()
}(i)
}
wg.Wait()
}
func evenChannel(n int) {
if n%2 == 1 { // make it even:
n++
}
for i := 0; i < n; i++ {
go func(n int) {
select {
case ch <- n: // tx if channel is empty
case i := <-ch: // rx if channel is not empty
// fmt.Println(n, i)
_ = i
}
done <- struct{}{}
}(i)
}
for i := 0; i < n; i++ {
<-done
}
}
func TestMain(m *testing.M) {
var n int // We use TestMain to set up the done channel.
flag.IntVar(&n, "n", 1_000_000, "chan cap")
flag.Parse()
done = make(chan struct{}, n)
fmt.Println("n=", n)
os.Exit(m.Run())
}
var (
done chan struct{}
ch = make(chan int)
wg sync.WaitGroup
)
It depends on the use case. If you are dispatching one-off jobs to be run in parallel without needing to know the results of each job, then you can use a WaitGroup. But if you need to collect the results from the goroutines then you should use a channel.
Since a channel works both ways, I almost always use a channel.
On another note, as pointed out in the comment your channel example isn't implemented correctly. You would need a separate channel to indicate there are no more jobs to do (one example is here). In your case, since you know the number of words in advance, you could just use one buffered channel and receive a fixed number of times to avoid declaring a close channel.
If you are particularly sticky about using only channels, then it needs to be done differently (if we use your example does, as #Not_a_Golfer points out, it'll produce incorrect results).
One way is to make a channel of type int. In the worker process send a number each time it completes the job (this can be the unique job id too, if you want you can track this in the receiver).
In the receiver main go routine (which will know the exact number of jobs submitted) - do a range loop over a channel, count on till the number of jobs submitted are not done, and break out of the loop when all jobs are completed. This is a good way if you want to track each of the jobs completion (and maybe do something if needed).
Here's the code for your reference. Decrementing totalJobsLeft will be safe as it'll ever be done only in the range loop of the channel!
//This is just an illustration of how to sync completion of multiple jobs using a channel
//A better way many a times might be to use wait groups
package main
import (
"fmt"
"math/rand"
"time"
)
func main() {
comChannel := make(chan int)
words := []string{"foo", "bar", "baz"}
totalJobsLeft := len(words)
//We know how many jobs are being sent
for j, word := range words {
jobId := j + 1
go func(word string, jobId int) {
fmt.Println("Job ID:", jobId, "Word:", word)
//Do some work here, maybe call functions that you need
//For emulating this - Sleep for a random time upto 5 seconds
randInt := rand.Intn(5)
//fmt.Println("Got random number", randInt)
time.Sleep(time.Duration(randInt) * time.Second)
comChannel <- jobId
}(word, jobId)
}
for j := range comChannel {
fmt.Println("Got job ID", j)
totalJobsLeft--
fmt.Println("Total jobs left", totalJobsLeft)
if totalJobsLeft == 0 {
break
}
}
fmt.Println("Closing communication channel. All jobs completed!")
close(comChannel)
}
I often use channels to collect error messages from goroutines that could produce an error. Here is a simple example:
func couldGoWrong() (err error) {
errorChannel := make(chan error, 3)
// start a go routine
go func() (err error) {
defer func() { errorChannel <- err }()
for c := 0; c < 10; c++ {
_, err = fmt.Println(c)
if err != nil {
return
}
}
return
}()
// start another go routine
go func() (err error) {
defer func() { errorChannel <- err }()
for c := 10; c < 100; c++ {
_, err = fmt.Println(c)
if err != nil {
return
}
}
return
}()
// start yet another go routine
go func() (err error) {
defer func() { errorChannel <- err }()
for c := 100; c < 1000; c++ {
_, err = fmt.Println(c)
if err != nil {
return
}
}
return
}()
// synchronize go routines and collect errors here
for c := 0; c < cap(errorChannel); c++ {
err = <-errorChannel
if err != nil {
return
}
}
return
}
Also suggest to use waitgroup but still you want to do it with channel then below i mention a simple use of channel
package main
import (
"fmt"
"time"
)
func main() {
c := make(chan string)
words := []string{"foo", "bar", "baz"}
go printWordrs(words, c)
for j := range c {
fmt.Println(j)
}
}
func printWordrs(words []string, c chan string) {
defer close(c)
for _, word := range words {
time.Sleep(1 * time.Second)
c <- word
}
}
I am a beginner in go.
I am trying to figure out an easy way to implement a channel that only output distinct values.
What I want to do is this:
package example
import (
"fmt"
"testing"
)
func TestShouldReturnDistinctValues(t *testing.T) {
var c := make([]chan int)
c <- 1
c <- 1
c <- 2
c <- 2
c <- 3
for e := range c {
// only print 1, 2 and 3.
fmt.println(e)
}
}
Should I be concern about memory leak here if I were to use a map to remember previous values?
You really can't do that, you'd have to keep a track of the values somehow, a map[int]struct{} is probably the most memory efficient way.
A simple example:
func UniqueGen(min, max int) <-chan int {
m := make(map[int]struct{}, max-min)
ch := make(chan int)
go func() {
for i := 0; i < 1000; i++ {
v := min + rand.Intn(max)
if _, ok := m[v]; !ok {
ch <- v
m[v] = struct{}{}
}
}
close(ch)
}()
return ch
}
I have done similar things before, except my problem was output inputs in ascending order. You can do this by adding a middle go routine. Here is an example:
package main
func main() {
input, output := distinct()
go func() {
input <- 1
input <- 1
input <- 2
input <- 2
input <- 3
close(input)
}()
for i := range output {
println(i)
}
}
func distinct() (input chan int, output chan int) {
input = make(chan int)
output = make(chan int)
go func() {
set := make(map[int]struct{})
for i := range input {
if _, ok := set[i]; !ok {
set[i] = struct{}{}
output <- i
}
}
close(output)
}()
return
}