With the following code:
package main
import (
"github.com/davecgh/go-spew/spew"
"sync"
"time"
)
func callbackWithTimeout(cbFunc func() ([]byte, error), timeout time.Duration) {
defer wg.Done() //I don't want this function to know about sync.WaitGroup
time.Sleep(timeout)
d, e := cbFunc()
spew.Dump(d)
spew.Dump(e)
}
var wg sync.WaitGroup
func main() {
wg.Add(1)
go func() {
cbFunc := func() ([]byte, error) {
//I feel like I should be able to defer here instead
return nil, nil
}
callbackWithTimeout(cbFunc, time.Duration(4*time.Second))
}()
println("some line")
wg.Wait()
}
In function callbackWithTimeout, I don't want to use defer wg.Done() because it's not callbackWithTimeout()'s concern to wg.Done(). How do I go about implementing such a thing? i.e., remove any sync.WaitGroup in callbackWithTimeout? I have a bit of problem understanding the separation of concerns here as a callback'er function should not have to know about waitgroups but in this case it seems, I have no other choice?
I feel like it should be a caller's responsibility to wg.Done() (which in this case is the cbFunc) but lack any concise reference to documentation or ideas on how to implement it in Go because by definition, all a callback function does is call the function back. So, where I am doing it wrong?
Silly assumptions were made by yours truly during refactoring. Working code below. Many thanks.
package main
import (
"errors"
"github.com/davecgh/go-spew/spew"
"sync"
"time"
)
func callbackWithTimeout(cbFunc func() ([]byte, error), timeout time.Duration) {
time.Sleep(timeout)
d, e := cbFunc()
spew.Dump(d)
spew.Dump(e)
}
func main() {
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
callbackWithTimeout(func() ([]byte, error) {
b := []byte{1, 2, 3, 4}
e := errors.New("error123")
return b, e
}, time.Duration(2*time.Second))
}()
println("some line")
wg.Wait()
}
May be like this?
package main
import (
"sync"
"time"
"github.com/davecgh/go-spew/spew"
)
func callbackWithTimeout(cbFunc func() ([]byte, error), timeout time.Duration) {
time.Sleep(timeout)
d, e := cbFunc()
spew.Dump(d)
spew.Dump(e)
}
func main() {
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done() // move it here
cbFunc := func() ([]byte, error) {
//I feel like I should be able to defer here instead
return nil, nil
}
callbackWithTimeout(cbFunc, time.Duration(4*time.Second))
}()
println("some line")
wg.Wait()
}
Related
My task is to sync 2 goroutines so the output should look like that:
foobarfoobarfoobarfoobar
.The issue is that when I call them they come out completely randomized. This is my code:
package main
import (
"fmt"
"sync"
"time"
)
type ConcurrentPrinter struct {
sync.WaitGroup
sync.Mutex
}
func (cp *ConcurrentPrinter) printFoo(times int) {
cp.WaitGroup.Add(times)
go func() {
cp.Lock()
fmt.Print("foo")
cp.Unlock()
}()
}
func (cp *ConcurrentPrinter) printBar(times int) {
cp.WaitGroup.Add(times)
go func() {
cp.Lock()
fmt.Print("bar")
cp.Unlock()
}()
}
func main() {
times := 10
cp := &ConcurrentPrinter{}
for i := 0; i <= times; i++ {
cp.printFoo(i)
cp.printBar(i)
}
time.Sleep(10 * time.Millisecond)
}
As outlined in the comments, using goroutines may not be the best use case for what you are trying to achieve - and thus this may be an XY problem.
Having said that, if you want to ensure two independent goroutines interleave their work in an alternating sequence, you can implement a set of "ping-pong" mutexs:
var ping, pong sync.Mutex
pong.Lock() // ensure the 2nd goroutine waits & the 1st goes first
go func() {
for {
ping.Lock()
foo()
pong.Unlock()
}
}()
go func() {
for {
pong.Lock()
bar()
ping.Unlock()
}
}()
https://go.dev/play/p/VO2LoMJ8fek
Using channel:
func printFoo(i int, ch chan<- bool, wg *sync.WaitGroup) {
wg.Add(1)
go func() {
defer wg.Done()
fmt.Print("foo")
ch <- true
}()
}
func printBar(i int, ch chan<- bool, wg *sync.WaitGroup) {
wg.Add(1)
go func() {
defer wg.Done()
fmt.Print("bar")
ch <- true
}()
}
func main() {
times := 4
firstchan := make(chan bool)
secondchan := make(chan bool)
var wg sync.WaitGroup
for i := 0; i <= times; i++ {
printFoo(i, firstchan, &wg)
<-firstchan
printBar(i, secondchan, &wg)
<-secondchan
}
wg.Wait()
}
https://go.dev/play/p/MlZ9dHkUXGb
I try to send an error in the channel on recovery
Why this error is not sent to the channel?
package main
import (
"fmt"
"sync"
"errors"
)
func main() {
var wg sync.WaitGroup
wg.Add(1)
batchErrChan := make(chan error)
go func(errchan chan error) {
defer func() {
if r := recover(); r != nil {
errchan <- errors.New("recover err")
}
close(batchErrChan)
wg.Done()
}()
panic("ddd")
}(batchErrChan)
go func() {
for _ = range batchErrChan {
fmt.Println("err in range")
}
}()
wg.Wait()
}
https://play.golang.org/p/0ytunuYDWZU
I expect "err in range" to be printed, but it is not. Why?
Your program ends before the goroutine gets a chance to print the message. Try waiting to it:
...
done:=make(chan struct{})
go func() {
for _ = range batchErrChan {
fmt.Println("err in range")
}
close(done)
}()
wg.Wait()
<-done
}
Say I have the following code (playground):
package main
import (
"fmt"
"sync"
)
func createStr() *string {
tmp := "foo"
return &tmp
}
func main() {
var (
s *string
wg sync.WaitGroup
)
go func() {
wg.Add(1)
defer wg.Done()
s = createStr()
}()
wg.Wait()
fmt.Printf("s after: %v", s)
}
I would have expected s to not equal nil.
However, if I add a small wait, I get s != nil (playground):
package main
import (
"fmt"
"sync"
"time"
)
func createStr() *string {
tmp := "foo"
return &tmp
}
func main() {
var (
s *string
wg sync.WaitGroup
)
go func() {
wg.Add(1)
defer wg.Done()
s = createStr()
}()
wg.Wait()
time.Sleep(time.Second)
fmt.Printf("s after: %v", s)
}
This just caused a bug in a program I wrote. What is happening with sync.WaitGroup that's not causing my program to wait for s to be assigned a string pointer in my go func?
Your placement of Add method for sync.WaitGroup is wrong. Do not use Add inside a goroutine (anonymous goroutine here) but use it in the goroutine (main goroutine here) that's going to wait for it.
A probable situation that was happening in your code was that wg.Wait() didn't wait as the goroutine's wg.Add(1) wasn't called yet and hence s == nil. The following code fixes the issue:
Go Playground
package main
import (
"fmt"
"sync"
)
func createStr() *string {
tmp := "foo"
return &tmp
}
func main() {
var (
s *string
wg sync.WaitGroup
)
// Use wg.Add() here
wg.Add(1)
go func() {
defer wg.Done()
s = createStr()
}()
wg.Wait()
fmt.Printf("s after: %v", *s)
}
I have a go function processing that use two distinct goroutines. produce will push some data into a channel and consume will read these data. Here is an example:
type MyObject struct{
...
}
func processing() {
var wg sync.WaitGroup
dataChannel := make(chan MyObject, 5)
wg.Add(2)
go produce(wg, dataChannel)
go consume(wg, dataChannel)
wg.Wait()
}
func produce (wg *sync.WaitGroup, dataChannel chan MyObject){
for{
// Produce data in dataChannel
}
}
func consume (wg *sync.WaitGroup, dataChannel chan MyObject){
for{
// Consume data from dataChannel
}
}
I want my processing function to be started and stoped by an HTTP call. So I am looking to do something as follow:
func main() {
// echo instance
e := echo.New()
e.GET("/", startProcessing)
e.Logger.Fatal(e.Start(":8099"))
}
func startProcessing(c echo.Context) error{
command := c.QueryParam("command")
if(command == "start"){
processing()
}else if(command == "stop"){
if (/* ? processing is running ? */){
/* ? stop processing process? */
}
}
}
What is the correct way to do this with Go?
Here how to start and stop a function using context, try this:
package main
import (
"context"
"fmt"
"sync"
"time"
)
func main() {
ctx, cancel := context.WithCancel(context.Background())
var wg sync.WaitGroup
dataChannel := make(chan MyObject, 5)
wg.Add(2)
go produce(ctx, &wg, dataChannel)
go consume(&wg, dataChannel)
time.Sleep(1 * time.Second)
cancel() // cancel when we are finished consuming data
wg.Wait()
}
func produce(ctx context.Context, wg *sync.WaitGroup, dataChannel chan MyObject) {
defer wg.Done()
i := 1
for {
select {
case <-ctx.Done():
close(dataChannel)
return // returning not to leak the goroutine
case dataChannel <- MyObject{i}:
i++
time.Sleep(250 * time.Millisecond)
}
}
}
func consume(wg *sync.WaitGroup, dataChannel chan MyObject) {
defer wg.Done()
for v := range dataChannel {
fmt.Println(v)
}
}
type MyObject struct {
i int
}
For HTTP you need to do it yourself!
It needs to have some concurrent safe ID or map or something to keep track of how many functions you called and then call a cancel() to stop it.
This recovery works:
func TestSomeTest(t *testing.T) {
defer func() {
r := recover()
fmt.Println("recovery")
fmt.Println(r)
}()
panic("panic here")
}
But this does not:
func TestSomeTest(t *testing.T) {
panic("panic here")
}
func TestMain(m *testing.M) {
defer func() {
r := recover()
fmt.Println("recovery")
fmt.Println(r)
}()
ret := m.Run()
os.Exit(ret)
}
Why?
I expect that panic here will be recovered by code in func TestMain(m *testing.M). Why not? I just have panic without any recovery in this case.
Full code:
package main
import (
"fmt"
"os"
"testing"
)
func TestSomeTest(t *testing.T) {
// defer func() {
// r := recover()
// fmt.Println("recovery")
// fmt.Println(r)
// }()
panic("panic here")
}
func TestMain(m *testing.M) {
defer func() {
r := recover()
fmt.Println("recovery")
fmt.Println(r)
}()
ret := m.Run()
os.Exit(ret)
}
To run this code I used go test command.
It's because the tests are run in separate goroutines.
It's like if your fist example sent off a goroutine, which can't be recovered.
func TestSomeTest(t *testing.T) {
defer func() {
r := recover()
fmt.Println("recovery")
fmt.Println(r)
}()
go func() {
// won't recover
panic("panic here")
}()
time.Sleep(time.Second)
}