I have the following program which serves as a proof of concept. I'm attempting to aggregate the results from chann, that is, too merge each instance of chann in to a common slice. Is this possible with my approach?
So my output for the following example would be a slice containing the following (in any order): []int{0,1,2}, thanks.
func DoStuff(i int, chann chan[]int, wg *sync.WaitGroup) {
defer wg.Done()
chann <-[]int{i}
}
func main() {
var wg sync.WaitGroup
chann := make(chan int[], 3)
defer close(chann)
for i := 0; i < count; 3 {
wg.Add(1)
go DoStuff(i, chann, &wg)
}
wg.Wait()
for {
select {
case result := <-chann:
fmt.Println(result)
os.Exit(1)
}
}
return nil
}
what you want to do is possible, but your program will not run because you are reading from the channel after wg.Wait(), so all goroutines will stop waiting to write, because you will never read from the channel.
You can read from the channel in a goroutine:
for i := 0; i < count; 3 {
wg.Add(1)
go DoStuff(i, chann, &wg)
}
}
go func() {
for data:=range chann {
// Process data
}
}()
wg.Wait()
// Close here, so the reading goroutine can terminate
close(chann)
Related
My task is to sync 2 goroutines so the output should look like that:
foobarfoobarfoobarfoobar
.The issue is that when I call them they come out completely randomized. This is my code:
package main
import (
"fmt"
"sync"
"time"
)
type ConcurrentPrinter struct {
sync.WaitGroup
sync.Mutex
}
func (cp *ConcurrentPrinter) printFoo(times int) {
cp.WaitGroup.Add(times)
go func() {
cp.Lock()
fmt.Print("foo")
cp.Unlock()
}()
}
func (cp *ConcurrentPrinter) printBar(times int) {
cp.WaitGroup.Add(times)
go func() {
cp.Lock()
fmt.Print("bar")
cp.Unlock()
}()
}
func main() {
times := 10
cp := &ConcurrentPrinter{}
for i := 0; i <= times; i++ {
cp.printFoo(i)
cp.printBar(i)
}
time.Sleep(10 * time.Millisecond)
}
As outlined in the comments, using goroutines may not be the best use case for what you are trying to achieve - and thus this may be an XY problem.
Having said that, if you want to ensure two independent goroutines interleave their work in an alternating sequence, you can implement a set of "ping-pong" mutexs:
var ping, pong sync.Mutex
pong.Lock() // ensure the 2nd goroutine waits & the 1st goes first
go func() {
for {
ping.Lock()
foo()
pong.Unlock()
}
}()
go func() {
for {
pong.Lock()
bar()
ping.Unlock()
}
}()
https://go.dev/play/p/VO2LoMJ8fek
Using channel:
func printFoo(i int, ch chan<- bool, wg *sync.WaitGroup) {
wg.Add(1)
go func() {
defer wg.Done()
fmt.Print("foo")
ch <- true
}()
}
func printBar(i int, ch chan<- bool, wg *sync.WaitGroup) {
wg.Add(1)
go func() {
defer wg.Done()
fmt.Print("bar")
ch <- true
}()
}
func main() {
times := 4
firstchan := make(chan bool)
secondchan := make(chan bool)
var wg sync.WaitGroup
for i := 0; i <= times; i++ {
printFoo(i, firstchan, &wg)
<-firstchan
printBar(i, secondchan, &wg)
<-secondchan
}
wg.Wait()
}
https://go.dev/play/p/MlZ9dHkUXGb
I am newbie in Golang - While trying to rewrite the following to a single thread implementation.
.....
run := func(handler func(chan<- modify), threads int) <-chan modify {
result := make(chan modify)
go func() {
var wg sync.WaitGroup
for i := 0; i < threads; i++ {
wg.Add(1)
go func() {
defer wg.Done()
handler(result)
}()
}
wg.Wait()
close(result)
}()
return result
}
modifyAgg := run(func(result chan<- modify) {
aggre := run(func(result chan<- modify) {
u.addAgg(slices, result) // returns result channel
}, u.threads.GrpTxns)
....
In the above code the variable addAgg is of type chan<- modify. The following is not - get error while ranging over the variable aggre "cannot range over addAgg(type func())"
aggre := func() {
result:= make(chan modify)
defer close(result)
u.addAgg(slices, result) // returns result channel
}
How to change the second implementation to mimic the first one? Thank you !
I was able to implement this in single thread...
aggre := func() <-chan modify{
result:= make(chan modify, 50) // make it non blocking
u.addAgg(slices, result)
defer close(result)
return result
}()
I encounter a problem about using defer wg.Done and channel.
If I coded like below, there is no problem.
for i := 0; i < ntasks; i++ {
wg.Add(1)
go func(args DoTaskArgs) {
// defer wg.Done()
for {
worker := <-registerChan
ok := call(worker, "Worker.DoTask", &args, nil)
if ok {
wg.Done()
// go func() {
registerChan <- worker
// }()
break
}
}
}(DoTaskArgs{jobName, mapFiles[i], phase, i, n_other})
}
wg.Wait()
But if I use defer wg.Done(), the code would be stucked, unless wrap the registerChan <- worker with go func.
for i := 0; i < ntasks; i++ {
wg.Add(1)
go func(args DoTaskArgs) {
defer wg.Done()
for {
worker := <-registerChan
ok := call(worker, "Worker.DoTask", &args, nil)
if ok {
// go func() {
registerChan <- worker
// }()
break
}
}
}(DoTaskArgs{jobName, mapFiles[i], phase, i, n_other})
}
wg.Wait()
What is the problem here?
Well, first off, your channel use is confused and will block. In the goroutine it reads from the channel. But nothing wrote into it.
I don't think your problem has anything to do with defer.
I started learning go recently and I am stuck on a problem.
I have a simple go routine which either returns or pushes value to a channel.
And my main fn delegates work to this routine till it meets condition or data is exhausted.
This code seem to deadlock on "found" channel. What am I doing wrong?
There are multiple workers
Item can be found in more than one worker at the same time
Once item is found, all workers should be stopped.
.
func workerRoutine(data Data, found chan bool, wg *sync.WaitGroup){
defer (*wg).Done()
// data processing
// return on false
// multiple routines can set this at the same time
found <-true
}
func main {
// ....
found:=make(chan bool)
var wg sync.WaitGroup
itemFound:=false
Loop:
for i:=0; i<limit; i++ {
select {
case <-found:
itemFound = true
break Loop
default:
if(some_check) {
wg.Add(1)
go workerRoutine(mdata,found,&wg)
}
}
}
wg.Wait()
// use itemFound
}
One possible solution is to avoid select statement and use separate goroutine for receiver (or sender, or both).
Example:
package main
import "sync"
func worker(res chan bool, wg *sync.WaitGroup) {
res <- true
wg.Done()
}
func receiver(res chan bool, wg *sync.WaitGroup) {
for range res {
}
wg.Done()
}
func main() {
var wg, wg2 sync.WaitGroup
wg.Add(1)
wg2.Add(10)
found := make(chan bool)
go receiver(found, &wg)
for i := 0; i < 10; i++ {
go worker(found, &wg2)
}
wg2.Wait()
close(found)
wg.Done()
}
I'm trying to follow along the bounded goroutine example that is posted at http://blog.golang.org/pipelines/bounded.go. The problem that I'm having is that if there are more workers spun up then the amount of work to do, the extra workers never get cancelled. Everything else seems to work, the values get computed and logged, but when I close the groups channel, the workers just hang at the range statement.
I guess what I don't understand (in both my code and the example code) is how do the workers know when there is no more work to do and that they should exit?
Update
A working (i.e. non-working) example is posted at http://play.golang.org/p/T7zBCYLECp. It shows the deadlock on the workers since they are all asleep and there is no work to do. What I'm confused about is that I think the example code would have the same problem.
Here is the code that I'm currently using:
// Creates a pool of workers to do a bunch of computations
func computeAll() error {
done := make(chan struct{})
defer close(done)
groups, errc := findGroups(done)
// start a fixed number of goroutines to schedule with
const numComputers = 20
c := make(chan result)
var wg sync.WaitGroup
wg.Add(numComputers)
for i := 0; i < numComputers; i++ {
go func() {
compute(done, groups, c)
wg.Done()
}()
}
go func() {
wg.Wait()
close(c)
}()
// log the results of the computation
for r := range c { // log the results }
if err := <-errc; err != nil {
return err
}
return nil
}
Here is the code that fills up the channel with data:
// Retrieves the groups of data the must be computed
func findGroups(done <-chan struct{}) (<-chan model, <-chan error) {
groups := make(chan model)
errc := make(chan error, 1)
go func() {
// close the groups channel after find returns
defer close(groups)
group, err := //... code to get the group ...
if err == nil {
// add the group to the channel
select {
case groups <- group:
}
}
}()
return groups, errc
}
And here is the code that reads the channel to do the computations.
// Computes the results for the groups of data
func compute(done <-chan struct{}, groups <-chan model, c chan<- result) {
for group := range groups {
value := compute(group)
select {
case c <- result{value}:
case <-done:
return
}
}
}
Because you're trying to read from errc and it's empty unless there's an error.
//edit
computeAll() will always block on <- errc if there are no errors, another approach is to use something like:
func computeAll() (err error) {
.........
select {
case err = <-errc:
default: //don't block
}
return
}
Try to close the errc as OneOfOne says
go func() {
wg.Wait()
close(c)
close(errc)
}()
// log the results of the computation
for r := range c { // log the results }
if err := range errc {
if err != nil {
return err
}
}