This recovery works:
func TestSomeTest(t *testing.T) {
defer func() {
r := recover()
fmt.Println("recovery")
fmt.Println(r)
}()
panic("panic here")
}
But this does not:
func TestSomeTest(t *testing.T) {
panic("panic here")
}
func TestMain(m *testing.M) {
defer func() {
r := recover()
fmt.Println("recovery")
fmt.Println(r)
}()
ret := m.Run()
os.Exit(ret)
}
Why?
I expect that panic here will be recovered by code in func TestMain(m *testing.M). Why not? I just have panic without any recovery in this case.
Full code:
package main
import (
"fmt"
"os"
"testing"
)
func TestSomeTest(t *testing.T) {
// defer func() {
// r := recover()
// fmt.Println("recovery")
// fmt.Println(r)
// }()
panic("panic here")
}
func TestMain(m *testing.M) {
defer func() {
r := recover()
fmt.Println("recovery")
fmt.Println(r)
}()
ret := m.Run()
os.Exit(ret)
}
To run this code I used go test command.
It's because the tests are run in separate goroutines.
It's like if your fist example sent off a goroutine, which can't be recovered.
func TestSomeTest(t *testing.T) {
defer func() {
r := recover()
fmt.Println("recovery")
fmt.Println(r)
}()
go func() {
// won't recover
panic("panic here")
}()
time.Sleep(time.Second)
}
Related
I try to send an error in the channel on recovery
Why this error is not sent to the channel?
package main
import (
"fmt"
"sync"
"errors"
)
func main() {
var wg sync.WaitGroup
wg.Add(1)
batchErrChan := make(chan error)
go func(errchan chan error) {
defer func() {
if r := recover(); r != nil {
errchan <- errors.New("recover err")
}
close(batchErrChan)
wg.Done()
}()
panic("ddd")
}(batchErrChan)
go func() {
for _ = range batchErrChan {
fmt.Println("err in range")
}
}()
wg.Wait()
}
https://play.golang.org/p/0ytunuYDWZU
I expect "err in range" to be printed, but it is not. Why?
Your program ends before the goroutine gets a chance to print the message. Try waiting to it:
...
done:=make(chan struct{})
go func() {
for _ = range batchErrChan {
fmt.Println("err in range")
}
close(done)
}()
wg.Wait()
<-done
}
say I have a case of reader, manipulator, consumer in different routines:
package main
import (
"context"
"fmt"
"golang.org/x/sync/errgroup"
"github.com/pkg/errors"
)
func Reader(ctx context.Context, chanFromReader chan int) error {
defer close(chanFromReader)
for i := 0; i < 100; i++ {
select {
case <-ctx.Done():
return nil
case chanFromReader <- i:
}
}
return nil
}
func Manipulate(ctx context.Context, chanFromReader chan int, chanToWriter chan int) error {
defer close(chanToWriter)
for {
select {
case <-ctx.Done():
return nil
case x, ok := <-chanFromReader:
if !ok {
return nil
}
chanToWriter <- 2 * x
}
}
}
func Writer(ctx context.Context, chanToWriter chan int) error {
for {
select {
case <-ctx.Done():
return nil
case x, ok := <-chanToWriter:
if !ok {
return nil
}
fmt.Println("Writer: ", x)
if x == 10 {
return errors.New("Generate some error in writer")
}
}
}
}
func main() {
g, ctx := errgroup.WithContext(context.Background())
chanFromReader := make(chan int)
chanToWriter := make(chan int)
func(ctx context.Context, chanToWriter chan int) {
g.Go(func() error {
return Writer(ctx, chanToWriter)
})
}(ctx, chanToWriter)
func(ctx context.Context, chanFromReader chan int, chanToWriter chan int) {
g.Go(func() error {
return Manipulate(ctx, chanFromReader, chanToWriter)
})
}(ctx, chanFromReader, chanToWriter)
func(ctx context.Context, chanFromReader chan int) {
g.Go(func() error {
return Reader(ctx, chanFromReader)
})
}(ctx, chanFromReader)
g.Wait()
fmt.Println("Main wait done")
}
https://play.golang.org/p/whslVE3rzel
In case the writer fails for some reason, I'm having trouble aborting the rest of the routines.
In the example above for instance, though they listen on ctx for cancellation they still deadlock on case of fail in writer, is there a workaround this?
I thought of adding this:
func Manipulate(ctx context.Context, chanFromReader chan int, chanToWriter chan int) error {
defer close(chanToWriter)
for {
select {
case <-ctx.Done():
return nil
case x, ok := <-chanFromReader:
if !ok {
return nil
}
select {
case <-ctx.Done():
return nil
case chanToWriter <- 2 * x:
}
}
}
}
which solves it, but it looks so unclean...
I would propose a solution where each channel gets closed only by the code that creates it. This can be enforced by returning a receive-only channel from the function that creates the channel and is responsible for closing it:
(kudos to mh-cbon for further refining this:)
https://play.golang.org/p/Tq4OVW5sSP4
package main
import (
"context"
"fmt"
"log"
"sync"
)
func read(ctx context.Context) (<-chan int, <-chan error) {
ch := make(chan int)
e := make(chan error)
go func() {
defer close(e)
defer close(ch)
for i := 0; i < 12; i++ {
select {
case <-ctx.Done():
return
case ch <- i:
}
}
}()
return ch, e
}
func manipulate(in <-chan int) (<-chan int, <-chan error) {
ch := make(chan int)
e := make(chan error)
go func() {
defer close(e)
defer close(ch)
for n := range in {
ch <- 2 * n
}
}()
return ch, e
}
func write(in <-chan int) <-chan error {
e := make(chan error)
go func() {
defer close(e)
for n := range in {
fmt.Println("written: ", n)
if n == 10 {
e <- fmt.Errorf("output error during write")
}
}
}()
return e
}
func collectErrors(errs ...<-chan error) {
var wg sync.WaitGroup
for i := 0; i < len(errs); i++ {
wg.Add(1)
go func(errs <-chan error) {
defer wg.Done()
for err := range errs {
log.Printf("%v", err)
}
}(errs[i])
}
wg.Wait()
}
func main() {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
ch1, err1 := read(ctx)
ch2, err2 := manipulate(ch1)
err3 := write(ch2)
collectErrors(err1, err2, err3)
fmt.Println("main wait complete")
}
This way, each channel gets closed reliably, and the I/O errors from write will cause the child context to be cancelled, shutting down the other goroutines.
I have a go function processing that use two distinct goroutines. produce will push some data into a channel and consume will read these data. Here is an example:
type MyObject struct{
...
}
func processing() {
var wg sync.WaitGroup
dataChannel := make(chan MyObject, 5)
wg.Add(2)
go produce(wg, dataChannel)
go consume(wg, dataChannel)
wg.Wait()
}
func produce (wg *sync.WaitGroup, dataChannel chan MyObject){
for{
// Produce data in dataChannel
}
}
func consume (wg *sync.WaitGroup, dataChannel chan MyObject){
for{
// Consume data from dataChannel
}
}
I want my processing function to be started and stoped by an HTTP call. So I am looking to do something as follow:
func main() {
// echo instance
e := echo.New()
e.GET("/", startProcessing)
e.Logger.Fatal(e.Start(":8099"))
}
func startProcessing(c echo.Context) error{
command := c.QueryParam("command")
if(command == "start"){
processing()
}else if(command == "stop"){
if (/* ? processing is running ? */){
/* ? stop processing process? */
}
}
}
What is the correct way to do this with Go?
Here how to start and stop a function using context, try this:
package main
import (
"context"
"fmt"
"sync"
"time"
)
func main() {
ctx, cancel := context.WithCancel(context.Background())
var wg sync.WaitGroup
dataChannel := make(chan MyObject, 5)
wg.Add(2)
go produce(ctx, &wg, dataChannel)
go consume(&wg, dataChannel)
time.Sleep(1 * time.Second)
cancel() // cancel when we are finished consuming data
wg.Wait()
}
func produce(ctx context.Context, wg *sync.WaitGroup, dataChannel chan MyObject) {
defer wg.Done()
i := 1
for {
select {
case <-ctx.Done():
close(dataChannel)
return // returning not to leak the goroutine
case dataChannel <- MyObject{i}:
i++
time.Sleep(250 * time.Millisecond)
}
}
}
func consume(wg *sync.WaitGroup, dataChannel chan MyObject) {
defer wg.Done()
for v := range dataChannel {
fmt.Println(v)
}
}
type MyObject struct {
i int
}
For HTTP you need to do it yourself!
It needs to have some concurrent safe ID or map or something to keep track of how many functions you called and then call a cancel() to stop it.
With the following code:
package main
import (
"github.com/davecgh/go-spew/spew"
"sync"
"time"
)
func callbackWithTimeout(cbFunc func() ([]byte, error), timeout time.Duration) {
defer wg.Done() //I don't want this function to know about sync.WaitGroup
time.Sleep(timeout)
d, e := cbFunc()
spew.Dump(d)
spew.Dump(e)
}
var wg sync.WaitGroup
func main() {
wg.Add(1)
go func() {
cbFunc := func() ([]byte, error) {
//I feel like I should be able to defer here instead
return nil, nil
}
callbackWithTimeout(cbFunc, time.Duration(4*time.Second))
}()
println("some line")
wg.Wait()
}
In function callbackWithTimeout, I don't want to use defer wg.Done() because it's not callbackWithTimeout()'s concern to wg.Done(). How do I go about implementing such a thing? i.e., remove any sync.WaitGroup in callbackWithTimeout? I have a bit of problem understanding the separation of concerns here as a callback'er function should not have to know about waitgroups but in this case it seems, I have no other choice?
I feel like it should be a caller's responsibility to wg.Done() (which in this case is the cbFunc) but lack any concise reference to documentation or ideas on how to implement it in Go because by definition, all a callback function does is call the function back. So, where I am doing it wrong?
Silly assumptions were made by yours truly during refactoring. Working code below. Many thanks.
package main
import (
"errors"
"github.com/davecgh/go-spew/spew"
"sync"
"time"
)
func callbackWithTimeout(cbFunc func() ([]byte, error), timeout time.Duration) {
time.Sleep(timeout)
d, e := cbFunc()
spew.Dump(d)
spew.Dump(e)
}
func main() {
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
callbackWithTimeout(func() ([]byte, error) {
b := []byte{1, 2, 3, 4}
e := errors.New("error123")
return b, e
}, time.Duration(2*time.Second))
}()
println("some line")
wg.Wait()
}
May be like this?
package main
import (
"sync"
"time"
"github.com/davecgh/go-spew/spew"
)
func callbackWithTimeout(cbFunc func() ([]byte, error), timeout time.Duration) {
time.Sleep(timeout)
d, e := cbFunc()
spew.Dump(d)
spew.Dump(e)
}
func main() {
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done() // move it here
cbFunc := func() ([]byte, error) {
//I feel like I should be able to defer here instead
return nil, nil
}
callbackWithTimeout(cbFunc, time.Duration(4*time.Second))
}()
println("some line")
wg.Wait()
}
If I do this:
func main() {
foo := 1
go func() {
fmt.Println(foo)
}()
}
is referencing foo inside that func wrong?
It is fine, only in changing context needs some attention(in case of local pointer variables):
package main
import (
"errors"
"fmt"
)
func test() {
defer func() { fmt.Println(1) }()
defer func() { fmt.Println(2) }()
defer func() { fmt.Println(3) }()
}
func main() {
test()
err := errors.New("error 1")
defer func() { fmt.Println(err) }()
err = errors.New("error 2")
}
and also see:
https://www.goinggo.net/2014/06/pitfalls-with-closures-in-go.html