I'm trying launch goroutines in an infinite loop until I get the response I'm looking for but the select is unreachable if I change for i := 0; i < 10; i++ {} to for {}. What's a pattern for solving this?
package main
import (
"fmt"
"math/rand"
"time"
)
func myFunc() float64 {
c := make(chan float64)
for i := 0; i < 10; i++ {
go func() {
var value float64
value = someOp()
if value > .9 {
c <- value
}
}()
}
// unreachable code the if the for loop above is infinite
for {
select {
case x := <-c:
return x
default:
}
}
}
func someOp() float64 {
rand.Seed(time.Now().UnixNano())
return rand.Float64()
}
func main() {
fmt.Println(myFunc())
}
Starting an unlimited number of goroutines is usually not a good idea. A better approach is to start a fixed number of goroutines that loop looking for an answer. Return from these goroutines when an answer is found.
func myFunc() float64 {
c := make(chan float64, 1) // Size 1 prevents race between main goroutine and workers
done := make(chan struct{})
defer close(done)
// Start a fixed number of goroutines
for i := 0; i < 10; i++ {
go func() {
for {
select {
case <-done:
// myfunc exited with result, return from this goroutine
return
default:
var value float64
value = someOp()
if value > .9 {
select {
case c <- value:
// This is first goroutine to send a value
default:
// Another goroutine sent a value
}
return
}
}
}
}()
}
return <-c
}
https://play.golang.org/p/SRpeT8k34eA
Related
Im studying Golang now on my freetime and I am trying sample exams online to test what i learned,
I came about this coding exam task but I cant seem to make it work/run without a crash,
im getting fatal error: all goroutines are asleep - deadlock! error, can anybody help what I am doing wrong here?
func executeParallel(ch chan<- int, done chan<- bool, functions ...func() int) {
ch <- functions[1]()
done <- true
}
func exampleFunction(counter int) int {
sum := 0
for i := 0; i < counter; i++ {
sum += 1
}
return sum
}
func main() {
expensiveFunction := func() int {
return exampleFunction(200000000)
}
cheapFunction := func() int {return exampleFunction(10000000)}
ch := make(chan int)
done := make(chan bool)
go executeParallel(ch, done, expensiveFunction, cheapFunction)
var isDone = <-done
for result := range ch {
fmt.Printf("Result: %d\n", result)
if isDone {
break;
}
}
}
Your executeParallel function will panic if less than 2 functions are provided - and will only run the 2nd function:
ch <- functions[1]() // runtime panic if less then 2 functions
I think it should look more like this: running all input functions in parallel and grabbing the first result.
for _, fn := range functions {
fn := fn // so each iteration/goroutine gets the proper value
go func() {
select {
case ch <- fn():
// first (fastest worker) wins
default:
// other workers results are discarded (if reader has not read results yet)
// this ensure we don't leak goroutines - since reader only reads one result from channel
}
}()
}
As such there's no need for a done channel - as we just need to read the one and only (quickest) result:
ch := make(chan int, 1) // big enough to capture one result - even if reader is not reading yet
executeParallel(ch, expensiveFunction, cheapFunction)
fmt.Printf("Result: %d\n", <-ch)
https://play.golang.org/p/skXc3gZZmRn
package main
import "fmt"
func executeParallel(ch chan<- int, done chan<- struct{}, functions ...func() int) {
// Only execute the second function [1], if available.
if len(functions) > 1 {
ch <- functions[1]()
}
// Close the done channel to signal the for-select to break and the main returns.
close(done)
}
// example returns the number of iterations for [0..counter-1]
func example(counter int) int {
sum := 0
for i := 0; i < counter; i++ {
sum += 1
}
return sum
// NOTE(SS): This function could just return "counter-1"
// to avoid the unnecessary calculation done above.
}
func main() {
var (
cheap = func() int { return example(10000000) }
expensive = func() int { return example(200000000) }
ch = make(chan int)
done = make(chan struct{})
)
// executeParallel takes ch, done channel followed by variable
// number of functions where on the second i.e., indexed 1
// function is executed on a separated goroutine which is then
// sent to ch channel which is then received by the for-select
// reciever below i.e., <-ch is the receiver.
go executeParallel(ch, done, expensive, cheap)
for {
select {
// Wait for something to be sent to done or the done channel
// to be closed.
case <-done:
return
// Keep receiving from ch (if something is sent to it)
case result := <-ch:
fmt.Println("Result:", result)
}
}
}
I have commented on the code so that it's understandable. As you didn't the actual question the logic could be still wrong.
in below example, the race detector will trigger an error. I am fine with it, though, as it does not change keys, the map header (if i might say), i struggle figuring out what is the reason of the race. I simply dont understand what is going on under hood so that a race detection is emitted.
package main
import (
"fmt"
"sync"
)
// scores holds values incremented by multiple goroutines.
var scores = make(map[string]int)
func main() {
var wg sync.WaitGroup
wg.Add(2)
scores["A"] = 0
scores["B"] = 0
go func() {
for i := 0; i < 1000; i++ {
// if _, ok := scores["A"]; !ok {
// scores["A"] = 1
// } else {
scores["A"]++
// }
}
wg.Done()
}()
go func() {
for i := 0; i < 1000; i++ {
scores["B"]++ // Line 28
}
wg.Done()
}()
wg.Wait()
fmt.Println("Final scores:", scores)
}
Map values are not addressable, so incrementing the integer values requires writing them back to the map itself.
The line
scores["A"]++
is equivalent to
tmp := scores["A"]
scores["A"] = tmp + 1
If you use a pointer to make the integer values addressable, and assign all the keys before the goroutines are dispatched, you can see there is no longer a race on the map itself:
var scores = make(map[string]*int)
func main() {
var wg sync.WaitGroup
wg.Add(2)
scores["A"] = new(int)
scores["B"] = new(int)
go func() {
for i := 0; i < 1000; i++ {
(*scores["A"])++
}
wg.Done()
}()
go func() {
for i := 0; i < 1000; i++ {
(*scores["B"])++
}
wg.Done()
}()
wg.Wait()
fmt.Println("Final scores:", scores)
}
Why the result is not as expected with flag "-race" ?
It expected the same result: 1000000 - with flag "-race" and without this
https://gist.github.com/romanitalian/f403ceb6e492eaf6ba953cf67d5a22ff
package main
import (
"fmt"
"runtime"
"sync/atomic"
"time"
)
//$ go run -race main_atomic.go
//954203
//
//$ go run main_atomic.go
//1000000
type atomicCounter struct {
val int64
}
func (c *atomicCounter) Add(x int64) {
atomic.AddInt64(&c.val, x)
runtime.Gosched()
}
func (c *atomicCounter) Value() int64 {
return atomic.LoadInt64(&c.val)
}
func main() {
counter := atomicCounter{}
for i := 0; i < 100; i++ {
go func(no int) {
for i := 0; i < 10000; i++ {
counter.Add(1)
}
}(i)
}
time.Sleep(time.Second)
fmt.Println(counter.Value())
}
The reason why the result is not the same is because time.Sleep(time.Second) does not guarantee that all of your goroutines are going to be executed in the timespan of one second. Even if you execute go run main.go, it's not guaranteed that you will get the same result every time. You can test this out if you put time.Milisecond instead of time.Second, you will see much more inconsistent results.
Whatever value you put in the time.Sleep method, it does not guarantee that all of your goroutines will be executed, it just means that it's less likely that all of your goroutines won't finish in time.
For consistent results, you would want to synchronise your goroutines a bit. You can use WaitGroup or channels.
With WaitGroup:
//rest of the code above is the same
func main() {
counter := atomicCounter{}
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func(no int) {
for i := 0; i < 10000; i++ {
counter.Add(1)
}
wg.Done()
}(i)
}
wg.Wait()
fmt.Println(counter.Value())
}
With channels:
func main() {
valStream := make(chan int)
doneStream := make(chan int)
result := 0
for i := 0; i < 100; i++ {
go func() {
for i := 0; i < 10000; i++ {
valStream <- 1
}
doneStream <- 1
}()
}
go func() {
counter := 0
for count := range doneStream {
counter += count
if counter == 100 {
close(doneStream)
}
}
close(valStream)
}()
for val := range valStream {
result += val
}
fmt.Println(result)
}
I am trying the fan in - fan out pattern with a factorial problem. But I am getting:
fatal error: all goroutines are asleep - deadlock!
and unable to identify the reason for deadlock.
I am trying to concurrently calculate factorial for 100 numbers using the fan-in fan-out pattern.
package main
import (
"fmt"
)
func main() {
_inChannel := _inListener(generator())
for val := range _inChannel {
fmt.Print(val, " -- ")
}
}
func generator() chan int { // NEED TO CALCULATE FACTORIAL FOR 100 NUMBERS
ch := make(chan int) // CREATE CHANNEL TO INPUT NUMBERS
go func() {
for i := 1; i <= 100; i++ {
ch <- i
}
close(ch) // CLOSE CHANNEL WHEN ALL NUMBERS HAVE BEEN WRITTEM
}()
return ch
}
func _inListener(ch chan int) chan int {
rec := make(chan int) // CHANNEL RECEIVED FROM GENERATOR
go func() {
for num := range ch { // RECEIVE THE INPUT NUMBERS FROM GENERATOR
result := factorial(num) // RESULT IS A NEW CHANNEL CREATED
rec <- <-result // MERGE INTO A SINGLE CHANNEL; rec
close(result)
}
close(rec)
}()
return rec // RETURN THE DEDICATED CHANNEL TO RECEIVE ALL OUTPUTS
}
func factorial(n int) chan int {
ch := make(chan int) // MAKE A NEW CHANNEL TO OUTPUT THE RESULT
// OF FACTORIAL
total := 1
for i := n; i > 0; i-- {
total *= i
}
ch <- total
return ch // RETURN THE CHANNEL HAVING THE FACTORIAL CALCULATED
}
I have put in comments, so that it becomes easier to follow the code.
I'm no expert in channels. I've taking on this to try and get more familiar with go.
Another issue is the int isn't large enough to take all factorials over 20 or so.
As you can see, I added a defer close as well as a logical channel called done in the generator func. The rest of the changes probably aren't needed. With channels you need to make sure something is ready to take off a value on the channel when you put something on a channel. Otherwise deadlock. Also, using
go run -race main.go
helps at least see which line(s) are causing problems.
I hope this helps and isn't removed for being off topic.
I was able to remove the deadlock by doing this:
package main
import (
"fmt"
)
func main() {
_gen := generator()
_inChannel := _inListener(_gen)
for val := range _inChannel {
fmt.Print(val, " -- \n")
}
}
func generator() chan int { // NEED TO CALCULATE FACTORIAL FOR 100 NUMBERS
ch := make(chan int) // CREATE CHANNEL TO INPUT NUMBERS
done := make(chan bool)
go func() {
defer close(ch)
for i := 1; i <= 100; i++ {
ch <- i
}
//close(ch) // CLOSE CHANNEL WHEN ALL NUMBERS HAVE BEEN WRITTEM
done <- true
}()
// this function will pull off the done for each function call above.
go func() {
for i := 1; i < 100; i++ {
<-done
}
}()
return ch
}
func _inListener(ch chan int) chan int {
rec := make(chan int) // CHANNEL RECEIVED FROM GENERATOR
go func() {
for num := range ch { // RECEIVE THE INPUT NUMBERS FROM GENERATOR
result := factorial(num) // RESULT IS A NEW CHANNEL CREATED
rec <- result // MERGE INTO A SINGLE CHANNEL; rec
}
close(rec)
}()
return rec // RETURN THE DEDICATED CHANNEL TO RECEIVE ALL OUTPUTS
}
func factorial(n int) int {
// OF FACTORIAL
total := 1
for i := n; i > 0; i-- {
total *= i
}
return total // RETURN THE CHANNEL HAVING THE FACTORIAL CALCULATED
}
I am using an infinite for loop with a label. Outside the scope of the for loop, I have a scheduled function running as a go routine. I want to break the for loop from the scheduled function when a certain condition is met. How can I accomplish the same ? This is what I am trying, which obviously won't work due to scope issue.
package main
import (
"fmt"
"time"
"sync"
)
func main() {
count := 0
var wg sync.WaitGroup
wg.Add(1)
t := time.NewTicker(time.Second*1)
go func (){
for {
fmt.Println("I will print every second", count)
count++
if count > 5 {
break myLoop;
wg.Done()
}
<-t.C
}
}()
i := 1
myLoop:
for {
fmt.Println("iteration", i)
i++
}
wg.Wait()
fmt.Println("I will execute at the end")
}
Make a signalling channel.
quit := make(chan struct{}{})
Close it when you want to break a loop.
go func (){
for {
fmt.Println("I will print every second", count)
count++
if count > 5 {
close(quit)
wg.Done()
return
}
<-t.C
}
}()
Read on closed channel returns immediately with zero value (but we dont need it in this case). Otherwise reading from it blocks and select pass execution to "default" case.
myLoop:
for {
select {
case <- quit:
break myLoop
default:
fmt.Println("iteration", i)
i++
}
}
Darigaaz's answer works for a single goroutine, but closing a closed channel panics (and you also don't need a waitgroup in that instance). If you have multiple goroutines, and want the loop to exit after all of them have finished, use a waitgroup with a closer routine:
https://play.golang.org/p/RhmUzWhneT
package main
import (
"fmt"
"sync"
"time"
)
func main() {
var wg sync.WaitGroup
quitCh := make(chan struct{})
for i := 1; i <= 5; i++ {
wg.Add(1)
go func(i int) {
count := 1
t := time.NewTicker(time.Millisecond)
for count <= 5 {
fmt.Printf("Goroutine %v iteration %v\n", i, count)
count++
<-t.C
}
wg.Done()
}(i)
}
// This is the closer routine.
go func() {
wg.Wait()
close(quitCh)
}()
t := time.NewTicker(500 * time.Microsecond)
loop:
for i := 1; ; i++ { // this is still infinite
select {
case <-quitCh:
break loop // has to be named, because "break" applies to the select otherwise
case <-t.C:
fmt.Println("Main iteration", i)
}
}
fmt.Println("End!")
}
As an alternative to the named loop style, you can instead use a fallthrough break in that select:
for i := 1; ; i++ { // this is still infinite
select {
case <-quitCh:
// fallthrough
case <-t.C:
fmt.Println("Main iteration", i)
continue
}
break // only reached if the quitCh case happens
}