I must have missed something because I haven't found the answer online for this pretty basic problem. I am using a buffered channel capable of holding three int values.
I am then using three goroutines to fill it up and I want to do an operation once the buffered channel is full.
Here is a snippet explaining the problem:
func main() {
// Initialization of the slice a and 0 < n < len(a) - 1.
difs := make(chan int, 3)
go routine(a[:n], difs)
go routine(a[n + 1:], difs)
go routine(a[n - 1:n + 1], difs)
fmt.Println(<-difs) // Display the first result returned by one of the routine.
}
func routine(a []int, out chan<- int) {
// Long computation.
out <- result
}
I would like to update my code so that fmt.Println(<-difs) displays an array of int when all the values have been computed. I could use three times <-difs but I wonder if Go offers something cleaner to do that.
Wait using channel itself, like this working sample code:
package main
import "fmt"
func main() {
a := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12} // Initialization of the slice a and 0 < n < len(a) - 1.
difs := make(chan int, 3)
go routine(a[0:4], difs)
go routine(a[4:8], difs)
go routine(a[8:12], difs)
result := []int{<-difs, <-difs, <-difs}
fmt.Println(result) // Display the first result returned by one of the routine.
}
func routine(a []int, out chan<- int) {
result := 0 // Long computation.
for _, v := range a {
result += v
}
out <- result
}
output:
[10 42 26]
The function len() supports channels, returning the number of unread queued elements within the channel. However, you'll have to run a loop to periodically check it.
The traditional method of handling this is using a sync.WaitGroup, as so:
func main() {
// Initialization of the slice a and 0 < n < len(a) - 1.
var wg sync.WaitGroup
wg.Add(3)
difs := make(chan int, 3)
go routine(a[:n], difs, &wg)
go routine(a[n + 1:], difs, &wg)
go routine(n - 1:n + 1], difs, &wg)
wg.Wait() // blocks until wg.Done() is called 3 times
fmt.Println(<-difs) // Display the first result returned by one of the routine.
}
// NOTE: This MUST accept a POINTER to the waitgroup. They are not copy-safe.
func routine(a []int, out chan<- int, wg *sync.WaitGroup) {
defer wg.Done()
// Long computation.
out <- result
}
Related
Hello I have the following simple program:
I have a function printNums(nums chan int) which should take 3 numbers from a channel and print them concurrently. Each printing is done in a new goroutine inside the range statement.
However when I run this program, instead of expected output 4, 12, 32, i get 12, 12, 32. Can you help me find out where is the problem and why i don't receive from channel same values that have been sent to channel? Thank you.
package main
import ("fmt")
func printNums(nums chan int){
c := make(chan struct{}, 100)
for num := range(nums){
go func(){
fmt.Println(num)
c <- struct{}{}
}()
}
<-c
<-c
<-c
}
func main(){
nums := make(chan int)
go func() {
nums <- 4
nums <- 12
nums <- 32
close(nums)
}()
printNums(nums)
}
You are printing the current value of num instead of the value num was when the goroutine was created. The loop variable is overwritten at every turn, and the goroutine may see the updated value.
for num := range(nums){
go func(x int){
fmt.Println(x)
c <- struct{}{}
}(num)
}
While I've seen similar issues, none of the answers found on SO have helped me, but I hope a kind soul with an eagle eye can help me pinpoint the issue here. I know it's not the best use of goroutines but I wanted to do it this way as an exercise, which has obviously failed.
my code
package main
import (
"fmt"
"sort"
"sync"
)
func main() {
X := []int{1, 2, 3, 4, 0}
Y := []int{2, 3, 6, 8, 4}
solution := Solution(X, Y)
fmt.Println(solution)
}
//Solution solution
func Solution(X []int, Y []int) int {
size := len(X)
resultChan := make(chan int)
results := make(map[int]int)
ParseDivision(size, X, Y, resultChan)
for val := range resultChan {
results[val] = results[val] + 1
}
close(resultChan)
return FindGreatest(results)
}
//Divide divide
func Divide(a int, b int, resultChan chan<- int, wg *sync.WaitGroup) {
defer wg.Done()
result := float64(a) / float64(b)
resultChan <- int(result * 1000)
}
//FindGreatest find greatest in map
func FindGreatest(myMap map[int]int) int {
values := make([]int, 0, len(myMap))
for _, val := range myMap {
values = append(values, val)
}
sort.Ints(values)
return values[len(values)-1]
}
//ParseDivision int
func ParseDivision(lenth int, X []int, Y []int, resultChan chan<- int) {
var wg sync.WaitGroup
wg.Add(lenth)
for i := 0; i < lenth; i++ {
go Divide(X[i], Y[i], resultChan, &wg)
}
wg.Wait()
}
Result: fatal error: all goroutines are asleep - deadlock!
I am not at all sure why as I've followed a number of examples as well as answers from SO, with respect to passing the waitGroup by reference, as well as using a channel to get the result of the operations performed in goroutines.
Some things to note:
the go-routine that writes to a channel should typically close the channel;
and following from above, the "reader" go-routine should wait for the channel to close (via range et.c) - i.e. it should never have to close the channel its reading from.
since your results channel is unbuffered (buffer size of 0), the main go-routine blocks waiting as the workers try to write, and the main go-routine is not (yet) reading from the results channel (as the pool manager go-routine must complete for this to happen - DEADLOCK!). Solution:
ensure the work pool go-routine runs independent of the main (reader) go-routine.
3-line fix: https://play.golang.org/p/9hYuyDgMjGi
I'm trying to understand this piece of code, not sure why the 2nd go is executed before the 1st one. It'd be great if someone can really help me out with this!
func sum(a []int, c chan int) {
fmt.Println("summing: ", a)
total := 0
for _, v := range a {
total += v
}
//fmt.Println("send to c",total)
c <- total // send total to c
}
func main() {
//a := []int{7, 2, 8,134,23,23,1,23,1234,143, -9, 4, 0, 1234}
c := make(chan int)
go sum([]int{1,2,3}, c)
go sum([]int{4,5,6}, c)
x := <-c
fmt.Println(x)
x = <-c
fmt.Println(x)
}
OUTPUT:
summing: [4 5 6]
15
summing: [1 2 3]
6
You have nothing explicitly synchronizing the order of the two goroutines. If you run this enough times, you will see the calls to fmt.Println print in different sequences. When executing goroutines, as they are concurrent operations, you have no guarantees when they will execute and/or complete. You need to use various standard library packages, or channels themselves to synchronize the execution of concurrently running goroutines.
For example (by leveraging the blocking nature of channels, you could do something like):
func main() {
c := make(chan int)
go sum([]int{1, 2, 3}, c)
//use the channel to block until it receives a send
x := <-c
fmt.Println(x)
//then execute the next routine
go sum([]int{4, 5, 6}, c)
x = <-c
fmt.Println(x)
}
Another example (significantly less practical, but here to look at other common go synchronization features) you could introduce a wait group, and a range over a channel:
func sum(a []int, c chan int, wg *sync.WaitGroup) {
defer wg.Done()
fmt.Println("summing: ", a)
total := 0
for _, v := range a {
total += v
}
//fmt.Println("send to c",total)
c <- total // send total to c
}
func main() {
c := make(chan int)
wg := new(sync.WaitGroup)
//concurrently call the concurrent calls to sum, allowing execution to continue to the range of the channel
go func() {
//increment the wait group, and pass it to the sum func to decrement it when it is complete
wg.Add(1)
go sum([]int{1, 2, 3}, c, wg)
//wait for the above call to sum to complete
wg.Wait()
//and repeat...
wg.Add(1)
go sum([]int{4, 5, 6}, c, wg)
wg.Wait()
//all calls are complete, close the channel to allow the program to exit cleanly
close(c)
}()
//range of the channel
for theSum := range c {
x := theSum
fmt.Println(x)
}
}
Below is a struct of type Stuff. It has three ints. A Number, its Double and its Power. Let's pretend that calculating the double and power of a given list of ints is an expensive computation.
type Stuff struct {
Number int
Double int
Power int
}
func main() {
nums := []int{2, 3, 4} // given numbers
stuff := []Stuff{} // struct of stuff with transformed ints
double := make(chan int)
power := make(chan int)
for _, i := range nums {
go doubleNumber(i, double)
go powerNumber(i, power)
}
// How do I get the values back in the right order?
fmt.Println(stuff)
}
func doubleNumber(i int, c chan int) {
c <- i + i
}
func powerNumber(i int, c chan int) {
c <- i * i
}
The result of fmt.Println(stuff) should be the same as if stuff was initialized like:
stuff := []Stuff{
{Number: 2, Double: 4, Power: 4}
{Number: 3, Double: 6, Power: 9}
{Number: 4, Double: 8, Power: 16}
}
I know I can use <- double and <- power to collect values from the channels, but I don't know what double / powers belong to what numbers.
Goroutines run concurrently, independently, so without explicit synchronization you can't predict execution and completion order. So as it is, you can't pair returned numbers with the input numbers.
You can either return more data (e.g. the input number and the output, wrapped in a struct for example), or pass pointers to the worker functions (launched as new goroutines), e.g. *Stuff and have the goroutines fill the calculated data in the Stuff itself.
Returning more data
I will use a channel type chan Pair where Pair is:
type Pair struct{ Number, Result int }
So calculation will look like this:
func doubleNumber(i int, c chan Pair) { c <- Pair{i, i + i} }
func powerNumber(i int, c chan Pair) { c <- Pair{i, i * i} }
And I will use a map[int]*Stuff because collectable data comes from multiple channels (double and power), and I want to find the appropriate Stuff easily and fast (pointer is required so I can also modify it "in the map").
So the main function:
nums := []int{2, 3, 4} // given numbers
stuffs := map[int]*Stuff{}
double := make(chan Pair)
power := make(chan Pair)
for _, i := range nums {
go doubleNumber(i, double)
go powerNumber(i, power)
}
// How do I get the values back in the right order?
for i := 0; i < len(nums)*2; i++ {
getStuff := func(number int) *Stuff {
s := stuffs[number]
if s == nil {
s = &Stuff{Number: number}
stuffs[number] = s
}
return s
}
select {
case p := <-double:
getStuff(p.Number).Double = p.Result
case p := <-power:
getStuff(p.Number).Power = p.Result
}
}
for _, v := range nums {
fmt.Printf("%+v\n", stuffs[v])
}
Output (try it on the Go Playground):
&{Number:2 Double:4 Power:4}
&{Number:3 Double:6 Power:9}
&{Number:4 Double:8 Power:16}
Using pointers
Since now we're passing *Stuff values, we can "pre-fill" the input number in the Stuff itself.
But care must be taken, you can only read/write values with proper synchronization. Easiest is to wait for all "worker" goroutines to finish their jobs.
var wg = &sync.WaitGroup{}
func main() {
nums := []int{2, 3, 4} // given numbers
stuffs := make([]Stuff, len(nums))
for i, n := range nums {
stuffs[i].Number = n
wg.Add(2)
go doubleNumber(&stuffs[i])
go powerNumber(&stuffs[i])
}
wg.Wait()
fmt.Printf("%+v", stuffs)
}
func doubleNumber(s *Stuff) {
defer wg.Done()
s.Double = s.Number + s.Number
}
func powerNumber(s *Stuff) {
defer wg.Done()
s.Power = s.Number * s.Number
}
Output (try it on the Go Playground):
[{Number:2 Double:4 Power:4} {Number:3 Double:6 Power:9} {Number:4 Double:8 Power:16}]
Writing different slice elements concurrently
Also note that since you can write different array or slice elements concurrently (for details see Can I concurrently write different slice elements), you can write the results directly in a slice without channels. See Refactor code to use a single channel in an idiomatic way how this can be done.
Personally, I would use a chan Stuff to pass the results back on, then spin up goroutines computing a full Stuff and pass it back. If you need the various part of a single Stuff computed concurrently, you can spawn goroutines from each goroutine, using dedicated channels. Once you've collected all the results, you can then (optionally) sort the slice with the accumulated values.
Example of what I mean below (you could, in principle, use a sync.WaitGroup to coordinate things, but if the input count is known, you don't strictly speaking need it).
type Stuff struct {
number int64
double int64
square int64
}
// Compute a Stuff with individual computations in-line, send it out
func computeStuff(n int64, out chan<- Stuff) {
rv := Stuff{number: n}
rv.double = n * 2
rv.square = n * n
out <- rv
}
// Compute a Stuff with individual computations concurrent
func computeStuffConcurrent(n int64, out chan<- Stuff) {
rv := Stuff{number: n}
dc := make(chan int64)
sc := make(chan int64)
defer close(dc)
defer close(sc)
go double(n, dc)
go square(n, sc)
rv.double = <-dc
rv.square = <-sc
out <- rv
}
func double(n int64, result chan<- int) {
result <- n * 2
}
func square(n int64, result chan<- int) {
result <- n * n
}
func main() {
inputs := []int64{1, 2, 3}
results := []Stuff{}
resultChannel := make(chan Stuff)
for _, input := range inputs {
go computeStuff(input, resultChannel)
// Or the concurrent version, if the extra performance is needed
}
for c := 0; c < len(inputs); c++ {
results = append(results, <- resultChannel)
}
// We now have all results, sort them if you need them sorted
}
I want to create slice and add values returned from channel.
Below is the code I tried but could not able to solve it.
I have to send address of the slice, but I am not able to figure out how :(
package main
import "fmt"
import "time"
func sendvalues(cs chan int){
for i:=0;i<10;i++{
cs<-i
}
}
func appendInt(cs chan int, aINt []int)[]*int{
for {
select {
case i := <-cs:
aINt = append(aINt,i)//append returns new type right ?
fmt.Println("slice",aINt)
}
}
}
func main() {
cs := make(chan int)
intSlice := make([]int, 0,10)
fmt.Println("Before",intSlice)
go sendvalues(cs)
go appendInt(cs,intSlice)// I have to pass address here
time.Sleep(999*999999)
fmt.Println("After",intSlice)
}
Your code won't work for two (in fact three) reasons:
append returns a new slice as soon as the capacity is reached.
Thus, the assignment in appendInt will do nothing.
appendInt runs concurrently, therefore:
As long as appendInt does not message main that it is finished,
main does not know when the intSlice has all the values you want.
You have to wait for all goroutines to return at the end of main
Problem 1: Modifying slices in functions
You may know that in Go every value you pass to a function is copied. Reference values,
such as slices, are copied too, but have pointers internally which then point to the original memory location. That means you can modify the elements of a slice in a function. What you
can't do is reassigning this value with a new slice as the internal pointer would point to somewhere different. You need pointers for that. Example (Play):
func modify(s *[]int) {
for i:=0; i < 10; i++ {
*s = append(*s, i)
}
}
func main() {
s := []int{1,2,3}
modify(&s)
fmt.Println(s)
}
Problem 2: Synchronizing goroutines
To wait for started goroutines, you can use a sync.WaitGroup. Example (Play):
func modify(wg *sync.WaitGroup, s *[]int) {
defer wg.Done()
for i:=0; i < 10; i++ {
*s = append(*s, i)
}
}
func main() {
wg := &sync.WaitGroup{}
s := []int{1,2,3}
wg.Add(1)
go modify(wg, &s)
wg.Wait()
fmt.Println(s)
}
The example above waits (using wg.Wait()) for modify to finish
(modify calls wg.Done() when finished). If you remove the wg.Wait() call, you will
see why not synchronizing is a problem. Comparison of outputs:
With wg.Wait(): [1 2 3 0 1 2 3 4 5 6 7 8 9]
Without wg.Wait(): [1 2 3]
The main goroutine returns earlier than the modify goroutine which is why you will never
see the modified results. Therefore synchronizing is absolutely necessary.
A good way to communicate the new slice would be to use a channel. You would not need to
use pointers and you would have synchronization. Example (Play):
func modify(res chan []int) {
s := []int{}
for i:=0; i < 10; i++ {
s = append(s, i)
}
res <- s
}
func main() {
c := make(chan []int)
go modify(c)
s := <-c
fmt.Println(s)
}