I am working on a sample program to print sum of odd and sum of even number between 1 to 100 using goroutine with multiple channels.
you can find my code
here
output
sum of even number = 2550
sum of odd number = 2500
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.print(0x434100, 0x11db7c)
/tmp/sandbox052575152/main.go:18 +0xc0
main.main()
/tmp/sandbox052575152/main.go:14 +0x120
The code works but with deadlock.
I am not sure what is wrong in my code
We can iterate through values sent over a channel. To break such iteration channel needs to be closed explicitly. Otherwise range would block forever in the same way as for nil channel. In your code you did't close the sum(for print function sumValues channel) channel. That's why following function will be blocked for forever.
func print(sumValues <-chan string ){
for val := range sumValues {
fmt.Println(val)
}
}
So you have to close the sum channel in the doSum function after all the go routine in the doSum function are complete (otherwise sum channel might be closed before go routines are complete). You can use sync.WaitGroup to do that. See the updated doSum function below:
func doSum(sum chan<- string, oddChan <-chan int, evenChan <-chan int) {
var waitGroup sync.WaitGroup
waitGroup.Add(2) // Must wait for 2 calls to 'done' before moving on
go func(sum chan<- string) {
s1 := 0
for val := range oddChan {
s1 += val
}
sum <- fmt.Sprint("sum of odd number = ", s1)
waitGroup.Done()
}(sum)
go func(sum chan<- string) {
s1 := 0
for val := range evenChan {
s1 += val
}
sum <- fmt.Sprint("sum of even number = ", s1)
waitGroup.Done()
}(sum)
// Waiting for all goroutines to exit
waitGroup.Wait()
// all goroutines are complete now close the sum channel
close(sum)
}
Related
I am reading values that are put into a channel ch via an infinite for. I would like some way to signal if a value has been read and operated upon (via the sq result) and add it to some sort of counter variable upon success. That way I have a way to check if my channel has been exhausted so that I can properly exit my infinite for loop.
Currently it is incrementing regardless if a value was read, thus causing it to exit early when the counter == num. I only want it to count when the value has been squared.
EDIT: Another approach I have tested is to receive the ok val out of the channel upon reading and setting val and then check if !ok { break }. However I receive a deadlock panic since the for did has not properly break. Example here: https://go.dev/play/p/RYNtTix2nm2
package main
import "fmt"
func main() {
num := 5
// Buffered channel with 5 values.
ch := make(chan int, num)
defer close(ch)
for i := 0; i < num; i++ {
go func(val int) {
fmt.Printf("Added value: %d to the channel\n", val)
ch <- val
}(i)
}
// Read from our channel infinitely and increment each time a value has been read and operated upon
counter := 0
for {
// Check our counter and if its == num then break the infinite loop
if counter == num {
break
}
val := <-ch
counter++
go func(i int) {
// I'd like to verify a value was read from ch & it was processed before I increment the counter
sq := i * i
fmt.Println(sq)
}(val)
}
}
let me try to help you in figuring out the issue.
Reading issue
The latest version of the code you put in the question is working except when you're about to read values from the ch channel. I mean with the following code snippet:
go func(i int) {
// I'd like to verify a value was read from ch & it was processed before I increment the counter
sq := i * I
fmt.Println(sq)
}(val)
In fact, it's not needed to spawn a new goroutine for each read. You can consume the messages as soon as they arrived in the ch channel. This is possible due to writing done inside goroutines. Thanks to them, the code can go ahead and reach the reading phase without being blocked.
Buffered vs unbuffered
In this scenario, you used a buffered channel with 5 slots for data. However, if you're relying on the buffered channel you should signal when you finish sending data to it. This is done with a close(ch) invocation after all of the Go routines finish their job. If you use an unbuffered channel it's fine to invoke defer close(ch) next to the channel initialization. In fact, this is done for cleanup and resource optimization tasks. Back to your example, you can change the implementation to use unbuffered channels.
Final Code
Just to recap, the two small changes that you've to do are:
Use an unbuffered channel instead of a buffered one.
Do Not use a Go routine when reading the messages from the channel.
Please be sure to understand exactly what's going on. Another tip can be to issue the statement: fmt.Println("NumGoroutine:", runtime.NumGoroutine()) to print the exact number of Go routines running in that specific moment.
The final code:
package main
import (
"fmt"
"runtime"
)
func main() {
num := 5
// Buffered channel with 5 values.
ch := make(chan int)
defer close(ch)
for i := 0; i < num; i++ {
go func(val int) {
fmt.Printf("Added value: %d to the channel\n", val)
ch <- val
}(i)
}
fmt.Println("NumGoroutine:", runtime.NumGoroutine())
// Read from our channel infinitely and increment each time a value has been read and operated upon
counter := 0
for {
// Check our counter and if its == num then break the infinite loop
if counter == num {
break
}
val := <-ch
counter++
func(i int) {
// I'd like to verify a value was read from ch & it was processed before I increment the counter
sq := i * i
fmt.Println(sq)
}(val)
}
}
Let me know if this helps you, thanks!
package main
import "fmt"
func main() {
c := make(chan int)
done := make(chan bool)
go func() {
for i := 0; i < 10; i++ {
c <- i
}
close(c)
}()
go func() {
for i := range c {
fmt.Println(i)
done <- true
}
close(done)
}()
for i := 0; i < 10; i++ {
<-done
}
}
In this example, the done channel is used to signal that a value has been read from the c channel. After each value is read from c, a signal is sent on the done channel. The main function blocks on the done channel, waiting for a signal before continuing. This ensures that all values from c have been processed before the program terminates.
I am new to Golang concurrency and have been working to understand this piece of code mentioned below.
I witness few things which I am unable to explain why it happens:
when using i smaller than equal to 100000 for i <= 100000 { in main function, it sometimes prints different values for nResults and countWrites (in last two statements)
fmt.Printf("number of result writes %d\n", nResults) fmt.Printf("Number of job writes %d\n", jobWrites)
when using i more than 1000000 it gives panic: send on closed channel
How can I make sure that the values send to jobs is not on closed channel and later after all values are received in results we can close the channel without deadlock?
package main
import (
"fmt"
"sync"
)
func worker(wg *sync.WaitGroup, id int, jobs <-chan int, results chan<- int, countWrites *int64) {
defer wg.Done()
for j := range jobs {
*countWrites += 1
go func(j int) {
if j%2 == 0 {
results <- j * 2
} else {
results <- j
}
}(j)
}
}
func main() {
wg := &sync.WaitGroup{}
jobs := make(chan int)
results := make(chan int)
var i int = 1
var jobWrites int64 = 0
for i <= 10000000 {
go func(j int) {
if j%2 == 0 {
i += 99
j += 99
}
jobWrites += 1
jobs <- j
}(i)
i += 1
}
var nResults int64 = 0
for w := 1; w < 1000; w++ {
wg.Add(1)
go worker(wg, w, jobs, results, &nResults)
}
close(jobs)
wg.Wait()
var sum int32 = 0
var count int64 = 0
for r := range results {
count += 1
sum += int32(r)
if count == nResults {
close(results)
}
}
fmt.Println(sum)
fmt.Printf("number of result writes %d\n", nResults)
fmt.Printf("Number of job writes %d\n", jobWrites)
}
Quite a few problems in your code.
Sending on closed channel
One general principle of using Go channels is
don't close a channel from the receiver side and don't close a channel if the channel has multiple concurrent senders
(https://go101.org/article/channel-closing.html)
The solution for you is simple: don't have multiple concurrent senders, and then you can close the channel from the sender side.
Instead of starting millions of separate goroutine for each job you add to the channel, run one goroutine that executes the whole loop to add all jobs to the channel. And close the channel after the loop. The workers will consume the channel as fast as they can.
Data races by modifying shared variables in multiple goroutines
You're modifying two shared variables without taking special steps:
nResults, which you pass to the countWrites *int64 in the worker.
i in the loop that writes to the jobs channel: you're adding 99 to it from multiple goroutines, making it unpredictable how many values you actually write to the jobs channel
To solve 1, there are many options, including using sync.Mutex. However since you're just adding to it, the easiest solution is to use atomic.AddInt64(countWrites, 1) instead of *countWrites += 1
To solve 2, don't use one goroutine per write to the channel, but one goroutine for the entire loop (see above)
Here is my program.
package main
import "fmt"
func init() {
fmt.Println("init function")
}
func main() {
// gen number to naturals channel
naturals := gen(1, 2, 3)
// write the squars in a channel
squar := dosquar(naturals)
for p := range squar {
fmt.Println("printer shows----- ", p)
}
}
func gen(nums ...int) chan int {
naturals := make(chan int)
go func() {
for _, n := range nums {
fmt.Println("generating number ", n)
naturals <- n
fmt.Println("generated number ", n)
}
//close(naturals)
}()
return naturals
}
func dosquar(naturals chan int) chan int {
// write the squars the values
squar := make(chan int)
go func() {
for number := range naturals {
fmt.Println("recieved number ", number)
squar <- number * number
fmt.Println("sent squar of number ", number*number)
}
//close(squar)
}()
return squar
}
It gives following error.
init function
generating number 1
generated number 1
generating number 2
recieved number 1
sent squar of number 1
recieved number 2
generated number 2
generating number 3
printer shows----- 1
printer shows----- 4
sent squar of number 4
recieved number 3
sent squar of number 9
printer shows----- 9
generated number 3
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
/Users/siyaram/go/src/github.com/SrmHitter9062/go-pipeline/main.go:15 +0x127
goroutine 19 [chan receive]:
I expect it to be running as channels are being ranged.That leads me to a question that do we need to close the channel every time, when we are finished with them ?
Please somebody help clarifying why program is not working as expected.
You can do whatever you want, but a range loop over a channel won't exit until the channel is closed.
I'm reading/working through Go Concurrency Patterns: Pipelines and cancellation, but i'm having trouble understanding the Stopping short section. We have the following functions:
func sq(in <-chan int) <-chan int {
out := make(chan int)
go func() {
for n := range in {
out <- n * n
}
close(out)
}()
return out
}
func gen(nums ...int) <-chan int {
out := make(chan int)
go func() {
for _, n := range nums {
out <- n
}
close(out)
}()
return out
}
func merge(cs ...<-chan int) <-chan int {
var wg sync.WaitGroup
out := make(chan int, 1) // enough space for the unread inputs
// Start an output goroutine for each input channel in cs. output
// copies values from c to out until c is closed, then calls wg.Done.
output := func(c <-chan int) {
for n := range c {
out <- n
}
wg.Done()
}
wg.Add(len(cs))
for _, c := range cs {
go output(c)
}
// Start a goroutine to close out once all the output goroutines are
// done. This must start after the wg.Add call.
go func() {
wg.Wait()
close(out)
}()
return out
}
func main() {
in := gen(2, 3)
// Distribute the sq work across two goroutines that both read from in.
c1 := sq(in)
c2 := sq(in)
// Consume the first value from output.
out := merge(c1, c2)
fmt.Println(<-out) // 4 or 9
return
// Apparently if we had not set the merge out buffer size to 1
// then we would have a hanging go routine.
}
Now, if you notice line 2 in merge, it says we make the out chan with buffer size 1, because this is enough space for the unread inputs. However, I'm almost positive that we should allocate a chan with buffer size 2. In accordance with this code sample:
c := make(chan int, 2) // buffer size 2
c <- 1 // succeeds immediately
c <- 2 // succeeds immediately
c <- 3 // blocks until another goroutine does <-c and receives 1
Because this section implies that a chan of buffer size 3 would not block. Can anyone please clarify/assist my understanding?
The program sends two values to the channel out and reads one value from the channel out. One of the values is not received.
If the channel is unbuffered (capacity 0), then one of the sending goroutines will block until the program exits. This is a leak.
If the channel is created with a capacity of 1, then both goroutines can send to the channel and exit. The first value sent to the channel is received by main. The second value remains in the channel.
If the main function does not receive a value from the channel out, then a channel of capacity 2 is required to prevent the goroutines from blocking indefinitely.
I am trying to understand, how golang channel works. I read a book about the go language and found the following example.
package main
import (
"fmt"
)
// Send the sequence 2, 3, 4, ... to returned channel
func generate() chan int {
ch := make(chan int)
go func() {
for i := 2; i <= 100 ; i++ {
ch <- i
}
}()
return ch
}
// Filter out input values divisible by 'prime', send rest to returned channel
func filter(in chan int, prime int) chan int {
out := make(chan int)
go func() {
for {
if i := <-in; i%prime != 0 {
out <- i
}
}
}()
return out
}
func sieve() chan int {
out := make(chan int)
go func() {
ch := generate()
for {
prime := <-ch
ch = filter(ch, prime)
out <- prime
}
}()
return out
}
func main() {
primes := sieve()
for {
fmt.Println(<-primes)
}
}
When I run this programm, I've got a deadlock, but when I change the generate function to
// Send the sequence 2, 3, 4, ... to returned channel
func generate() chan int {
ch := make(chan int)
go func() {
for i := 2; ; i++ {
ch <- i
}
}()
return ch
}
Then the programm will run the infinite loop, but not deadlock. Why do I get deadlock, when I remove the condition in for loop?
What do you mean with blocking principle?
You can see it illustrated in the blog post "The Nature Of Channels In Go "
for an unbuffered channel:
(Illustration from blog post "The Nature Of Channels In Go ", written by William Kennedy, Feb. 2014)
Unbuffered channels have no capacity and therefore require both goroutines to be ready to make any exchange.
When a goroutine attempts to write a resource to an unbuffered channel and there is no goroutine waiting to receive the resource, the channel will lock the goroutine and make it wait.
When a goroutine attempts to read from an unbuffered channel, and there is no goroutine waiting to send a resource, the channel will lock the goroutine and make it wait.
That is what happens in your case with your reader:
func main() {
primes := sieve()
for {
fmt.Println(<-primes)
}
}
since primes is never closed, main remains blocked.
It (main) is in step 3:
in step 3, the goroutine on the right places his hand into the channel or performs a read.
That goroutine is also locked in the channel until the exchange is complete.
The sender never calls close(primes).
Let's consider a simpler example:
func generate() chan int {
ch := make(chan int)
go func() {
for i := 2; /*i < 100*/; i++ {
ch <- i
}
}()
return ch
}
func main() {
for i := range generate() {
fmt.Println(i)
}
}
With the condition i < 100 uncommented, the goroutine spawned by generate stops after sending 98 numbers. However, it does not close the channel, so main has no way of knowing that no more numbers are going to be sent, and it just keeps blocking on the channel. Since main is now the only goroutine still in existence (the other one has returned), and it's blocking, you have a deadlock.