I am working with Go concurrency and have the following code:
package main
import (
"fmt"
"runtime"
"sync"
)
func main() {
runtime.GOMAXPROCS(1)
var wg sync.WaitGroup
wg.Add(2)
fmt.Println("Starting Goroutines")
go func() {
defer wg.Done()
for count := 0; count < 3; count++ {
for char := 'a'; char < 'a'+26; char++ {
fmt.Printf("%c", char)
}
}
fmt.Println()
}()
go func() {
defer wg.Done()
for count := 0; count < 3; count++ {
for char := 'A'; char < 'A'+26; char++ {
fmt.Printf("%c", char)
}
}
fmt.Println()
}()
fmt.Println("Waiting to Finish")
wg.Wait()
fmt.Println("Terminating")
}
My output is:
Starting Goroutines
Waiting to Finish
ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZ
abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz
Terminating
My problem is I declare the first goroutine to display lower case letters and the second goroutine to display upper case letters. Shouldn't the output be lowercase first then uppercase?
Any explanation would be helpful.
NOTE: This code came from the Go In Action ebook and i didnt fully understand their explanation.
Your output can vary each time you execute the program. The execution order of go routines is not guaranteed. Therefore your output is not deterministic.
Related
When writing data into same channel using multiple go routines with waitgroup after waiting wg.Wait() getting exception saying all go routines are asleep or deedlock.
package main
import (
"fmt"
"runtime"
"sync"
)
var wg sync.WaitGroup
func CreateMultipleRoutines() {
ch := make(chan int)
for i := 0; i < 10; i++ { // creates 10 go routines and adds to waitgroup
wg.Add(1)
go func() {
for j := 0; j < 10; j++ {
ch <- j
}
wg.Done() // indication of go routine is done to main routine
}()
}
fmt.Println(runtime.NumGoroutine())
wg.Wait() //wait for all go routines to complete
close(ch) // closing channel after completion of wait fo go routines
for v := range ch { // range can be used since channel is closed
fmt.Println(v)
}
fmt.Println("About to exit program ...")
}
When tried to implement this without waitgroup I am able to read data from channel by looping exact number of times data pushed to channel but i cant range since there will be panic when we close channel. here is the example code
package main
import (
"fmt"
"runtime"
)
func main() {
ch := make(chan int)
for i := 0; i < 10; i++ { // creates 10 go routines and adds to waitgroup
go func(i int) {
for j := 0; j < 10; j++ {
ch <- j * i
}
}(i)
}
fmt.Println(runtime.NumGoroutine())
for v := 0; v < 100; v++ {
fmt.Println(<-ch)
}
fmt.Println("About to exit program ...")
}
I want to understand why waitgroup in wait state is still waiting even though all go routines are signalled Done() which inturn makes number of go routines to zero
I think your original code has some problems.
You are closing the channel before reading from it.
You are not getting the advantage of using 10 goroutines because of your channel is 1 "sized". So one goroutine is producing one result per once.
My solution would be to spawn a new goroutine to monitor if the 10 goroutines finished its jobs. There you will use your WaitGroup.
Then the code would be like:
package main
import (
"fmt"
"runtime"
"sync"
)
var wg sync.WaitGroup
func main() {
ch := make(chan int, 10)
for i := 0; i < 10; i++ { // creates 10 go routines and adds to waitgroup
wg.Add(1)
go func() {
for j := 0; j < 10; j++ {
ch <- j
}
wg.Done() // indication of go routine is done to main routine
}()
}
go func(){
wg.Wait()
close(ch)
}()
fmt.Println(runtime.NumGoroutine())
for v := range ch { // range can be used since channel is closed
fmt.Println(v)
}
fmt.Println("About to exit program ...")
}
By default a chan holds no items, so all go routines are blocked on sending, until something reads from it. They never actually reach the wg.Done() statement.
A solution would be to close the channel in it's own go routine. Wrap your wg.Wait() and close(ch) lines like this:
go func() {
wg.Wait() //wait for all go routines to complete
close(ch) // closing channel after completion of wait fo go routines
}()
Then you can range over the channel, which will only close after all of the sending go routines have finished (and implicitly all values have been received).
I'm thinking start 1000 goroutines at the same time using for loop in Golang.
The problem is: I have to make sure that every goroutine has been executed.
Is it possible using channels to help me make sure of that?
The structure is kinda like this:
func main {
for i ... {
go ...
ch?
ch?
}
As #Andy mentioned You can use sync.WaitGroup to achieve this. Below is an example. Hope the code is self-explanatory.
package main
import (
"fmt"
"sync"
"time"
)
func dosomething(millisecs int64, wg *sync.WaitGroup) {
defer wg.Done()
duration := time.Duration(millisecs) * time.Millisecond
time.Sleep(duration)
fmt.Println("Function in background, duration:", duration)
}
func main() {
arr := []int64{200, 400, 150, 600}
var wg sync.WaitGroup
for _, n := range arr {
wg.Add(1)
go dosomething(n, &wg)
}
wg.Wait()
fmt.Println("Done")
}
To make sure the goroutines are done and collect the results, try this example:
package main
import (
"fmt"
)
const max = 1000
func main() {
for i := 1; i <= max; i++ {
go f(i)
}
sum := 0
for i := 1; i <= max; i++ {
sum += <-ch
}
fmt.Println(sum) // 500500
}
func f(n int) {
// do some job here and return the result:
ch <- n
}
var ch = make(chan int, max)
In order to wait for 1000 goroutines to finish, try this example:
package main
import (
"fmt"
"sync"
)
func main() {
wg := &sync.WaitGroup{}
for i := 0; i < 1000; i++ {
wg.Add(1)
go f(wg, i)
}
wg.Wait()
fmt.Println("Done.")
}
func f(wg *sync.WaitGroup, n int) {
defer wg.Done()
fmt.Print(n, " ")
}
I would suggest that you follow a pattern. Concurrency and Channel is Good but if you use it in a bad way, your program might became even slower than expected. The simple way to handle multiple go-routine and channel is by a worker pool pattern.
Take a close look at the code below
// In this example we'll look at how to implement
// a _worker pool_ using goroutines and channels.
package main
import "fmt"
import "time"
// Here's the worker, of which we'll run several
// concurrent instances. These workers will receive
// work on the `jobs` channel and send the corresponding
// results on `results`. We'll sleep a second per job to
// simulate an expensive task.
func worker(id int, jobs <-chan int, results chan<- int) {
for j := range jobs {
fmt.Println("worker", id, "started job", j)
time.Sleep(time.Second)
fmt.Println("worker", id, "finished job", j)
results <- j * 2
}
}
func main() {
// In order to use our pool of workers we need to send
// them work and collect their results. We make 2
// channels for this.
jobs := make(chan int, 100)
results := make(chan int, 100)
// This starts up 3 workers, initially blocked
// because there are no jobs yet.
for w := 1; w <= 3; w++ {
go worker(w, jobs, results)
}
// Here we send 5 `jobs` and then `close` that
// channel to indicate that's all the work we have.
for j := 1; j <= 5; j++ {
jobs <- j
}
close(jobs)
// Finally we collect all the results of the work.
for a := 1; a <= 5; a++ {
<-results
}
}
This simple example is taken from here . Also the results channel can help you keep track of all the go routines executing the jobs including failure notice.
I am trying to understand concurrency and goroutines, and had a couple questions about the following experimental code:
Why does it create a memory leak? I thought that a return at the end of the goroutine would allow memory associated with it to get cleaned up.
Why do my loops almost never reach 999? In fact, when I output to a file and study the output, I notice that it rarely prints integers in double digits; the first time it prints "99" is line 2461, and for "999" line 6120. This behavior is unexpected to me, which clearly means I don't really understand what is going on with goroutine scheduling.
Disclaimer:
Be careful running the code below, it can crash your system if you don't stop it after a few seconds!
CODE
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
for {
// spawn four worker goroutines
spawnWorkers(4, wg)
// wait for the workers to finish
wg.Wait()
}
}
func spawnWorkers(max int, wg sync.WaitGroup) {
for n := 0; n < max; n++ {
wg.Add(1)
go func() {
defer wg.Done()
f(n)
return
}()
}
}
func f(n int) {
for i := 0; i < 1000; i++ {
fmt.Println(n, ":", i)
}
}
Thanks to Tim Cooper, JimB, and Greg for their helpful comments. The corrected version of the code is posted below for reference.
The two fixes were to pass in the WaitGroup by reference, which fixed the memory leak, and to pass n correctly into the anonymous goroutine, and
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
for {
// spawn four worker goroutines
spawnWorkers(4,&wg)
// wait for the workers to finish
wg.Wait()
}
}
func spawnWorkers(max int, wg *sync.WaitGroup) {
for n := 0; n < max; n++ {
wg.Add(1)
go func(n int) {
defer wg.Done()
f(n)
return
}(n)
}
}
func f(n int) {
for i := 0; i < 1000; i++ {
fmt.Println(n, ":", i)
}
}
I am using an infinite for loop with a label. Outside the scope of the for loop, I have a scheduled function running as a go routine. I want to break the for loop from the scheduled function when a certain condition is met. How can I accomplish the same ? This is what I am trying, which obviously won't work due to scope issue.
package main
import (
"fmt"
"time"
"sync"
)
func main() {
count := 0
var wg sync.WaitGroup
wg.Add(1)
t := time.NewTicker(time.Second*1)
go func (){
for {
fmt.Println("I will print every second", count)
count++
if count > 5 {
break myLoop;
wg.Done()
}
<-t.C
}
}()
i := 1
myLoop:
for {
fmt.Println("iteration", i)
i++
}
wg.Wait()
fmt.Println("I will execute at the end")
}
Make a signalling channel.
quit := make(chan struct{}{})
Close it when you want to break a loop.
go func (){
for {
fmt.Println("I will print every second", count)
count++
if count > 5 {
close(quit)
wg.Done()
return
}
<-t.C
}
}()
Read on closed channel returns immediately with zero value (but we dont need it in this case). Otherwise reading from it blocks and select pass execution to "default" case.
myLoop:
for {
select {
case <- quit:
break myLoop
default:
fmt.Println("iteration", i)
i++
}
}
Darigaaz's answer works for a single goroutine, but closing a closed channel panics (and you also don't need a waitgroup in that instance). If you have multiple goroutines, and want the loop to exit after all of them have finished, use a waitgroup with a closer routine:
https://play.golang.org/p/RhmUzWhneT
package main
import (
"fmt"
"sync"
"time"
)
func main() {
var wg sync.WaitGroup
quitCh := make(chan struct{})
for i := 1; i <= 5; i++ {
wg.Add(1)
go func(i int) {
count := 1
t := time.NewTicker(time.Millisecond)
for count <= 5 {
fmt.Printf("Goroutine %v iteration %v\n", i, count)
count++
<-t.C
}
wg.Done()
}(i)
}
// This is the closer routine.
go func() {
wg.Wait()
close(quitCh)
}()
t := time.NewTicker(500 * time.Microsecond)
loop:
for i := 1; ; i++ { // this is still infinite
select {
case <-quitCh:
break loop // has to be named, because "break" applies to the select otherwise
case <-t.C:
fmt.Println("Main iteration", i)
}
}
fmt.Println("End!")
}
As an alternative to the named loop style, you can instead use a fallthrough break in that select:
for i := 1; ; i++ { // this is still infinite
select {
case <-quitCh:
// fallthrough
case <-t.C:
fmt.Println("Main iteration", i)
continue
}
break // only reached if the quitCh case happens
}
why in that script http://play.golang.org/p/Q5VMfVB67-
goroutine shower doesn't work ?
package main
import "fmt"
func main() {
ch := make(chan int)
go producer(ch)
go shower(ch)
for i := 0; i < 10; i++ {
fmt.Printf("main: %d\n", i)
}
}
func shower(c chan int) {
for {
j := <-c
fmt.Printf("worker: %d\n", j)
}
}
func producer(c chan int) {
for i := 0; i < 10; i++ {
c <- i
}
}
Your main function exit way before the goroutines have a chance to complete their own work.
You need to wait for them to finish before ending main() (which stops the all program), with for instance sync.WaitGroup, as seen in "Wait for the termination of n goroutines".
In your case, you need to wait for goroutine shower() to end: pass a wg *sync.WaitGroup instance, for shower() to signal wg.Done() when it finishes processing.