I’m just getting started with Golang and the net/rpc package. I’m trying to understand when you might use the asynchronous client.Go() call over the client.Call() method most examples online use. Would calling client.Call asynchronously via something like
go client.Call(...)
essentially be the same as using the client.Go call? I’ve seen this example online too (for example when calling multiple simultaneous RPCs).
As documented:
Go invokes the function asynchronously. It returns the Call structure representing the invocation. The done channel will signal when the call is complete by returning the same Call object. If done is nil, Go will allocate a new channel. If non-nil, done must be buffered or Go will deliberately crash.
What this means is that it issues the command, but does not wait for it to finish.
By contrast:
Call invokes the named function, waits for it to complete, and returns its error status.
Neither method executes in a goroutine directly*--this is left as an exercise for the caller (so an argument might be made that Go is a misnomer).
If you look at the source to Call, perhaps it is more clear:
func (client *Client) Call(serviceMethod string, args interface{}, reply
interface{}) error {
call := <-client.Go(serviceMethod, args, reply, make(chan *Call, 1)).Done
return call.Error
}
So in actuality, Call is a wrapper around Go, which waits for the operation to complete, whereas Go is the underlying function, which leaves waiting up to the caller.
*Clearly, in the background, a goroutine is involved somewhere, since this is a non-blocking operation.
Go provides two ways of handling errors, but I'm not sure which one to use.
Assuming I'm implementing a classic ForEach function which accepts a slice or a map as an argument. To check whether an iterable is passed in, I could do:
func ForEach(iterable interface{}, f interface{}) {
if isNotIterable(iterable) {
panic("Should pass in a slice or map!")
}
}
or
func ForEach(iterable interface{}, f interface{}) error {
if isNotIterable(iterable) {
return fmt.Errorf("Should pass in a slice or map!")
}
}
I saw some discussions saying panic() should be avoided, but people also say that if program cannot recover from error, you should panic().
Which one should I use? And what's the main principle for picking the right one?
You should assume that a panic will be immediately fatal, for the entire program, or at the very least for the current goroutine. Ask yourself "when this happens, should the application immediately crash?" If yes, use a panic; otherwise, use an error.
Use panic.
Because your use case is to catch a bad use of your API. This should never happen at runtime if the program is calling your API properly.
In fact, any program calling your API with correct arguments will behave in the same way if the test is removed. The test is there only to fail early with an error message helpful to the programmer that did the mistake. Ideally, the panic might be reached once during development when running the testsuite and the programmer would fix the call even before committing the bad code, and that incorrect use would never reach production.
See also this reponse to question Is function parameter validation using errors a good pattern in Go?.
I like the way it's done in some libraries where on top of a regular method DoSomething, its "panicky" version is added with MustDoSomething. I'm relatively new to go, but I've already seen it in several places, notably sqlx.
In general, if you want to expose your code to someone else, you should either have Must- and a regular version of the method, or your methods/functions should give the client a chance to recover the way they want and so error should be available to them in a go-idiomatic way.
Having said that, I agree that if your API/library is used inappropriately, it's Ok to panic as well. As a matter of fact, I've also seen methods like MustGetenv() that will panic if a critical env.var is missing. Fail-fast mechanism basically.
If some mandatory requirement is not provided or not there while starting the service (eg. database connection, some service configuration which is required) then you should use panic.
There should be return error for any user response or server side error.
Ask yourself these questions:
Do you expect the exceptional situation to occur, regardless how well would you code your app? Do you think it should be useful to make the user aware of such condition as part of the normal usage of your app? Handle it as an error, because it concerns the application as working normally.
Should that exceptional situation NOT occur if you code appropriately (and somewhat defensively)? (example: dividing by zero, or accessing an array element out of bounds) Is your app totally clueless under that error? Panic.
Do you have your API and want to ensure users use it appropriately? Panic. Your API will seldom recover if used incorrectly.
Use error whenever possible
Only use panic when your code could end up in a bad state that would be prone to crashing; something truly unexpected. The example above with ForEach() is an exported func that accepts an interface so it should expect someone will improperly call it. And if it is improperly called, you know why you cannot continue and you know how to handle that error. isNotIterable is literally binary and easy to control.
But error is not like a try/catch
Even if you try to justify panic/recover by looking at throw/catch from other languages, you still use errors. We know you are trying the function because you are calling it, we know there was an error because err != nil, and just like checking the type of exception thrown you can check the type of error returned with errors.Is(err, ErrNotIterable)
So should you use panic for errors in concurrency?
The answer is still most likely no. Errors are still the preferred way in Go and you can use a wait group to shut down the goroutines:
ctx, cancel := context.WithTimeout(context.Background(), time.Minute*5)
// automatically cancel in 5 min
defer cancel()
errGroup, ctx := errgroup.WithContext(ctx)
errGroup.Go(func() error {
// do crazy stuff; you can still check for errors
if ... {
return fmt.Errorf("critical error, stopping all goroutines now")
}
// code completed without issues
return nil
})
err = errGroup.Wait()
Even using the structure of the original example, you still have better control with errors than panics:
func ForEach(iterable interface{}, f interface{}) error {
if isNotIterable(iterable) {
return fmt.Errorf("expected something iterable but got %v", reflect.ValueOf(iterable).String())
}
switch v.Kind() {
case reflect.Map:
...
case reflect.Array, reflect.Slice:
...
default:
return fmt.Errorf("isNotIterable is false but I do not know how to iterate through %v", reflect.ValueOf(iterable).String())
}
But error feels very verbose
Yes, that is the point. When an error is returned, it is at that point to do something about it. You are giving the calling code options rather than making the decision to start shutting down and killing the application unless you recover(). If you are just returning the same error all the way up the call stack then error will seem inferior to panic, but this is due to not addressing issues when they happen.
So when to use panic?
When your code is on a collision course to crash and you cannot assume your way out of it. Another is when the code assumes something that is no longer true and having to check the integrity in every function from here on out would be tedious (and might impact performance). Still, you would use panic() only to get out of the layers of uncertainty... then still handle errors:
func ForEach(iterable interface{}, f interface{}) error {
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("cannot iterate due to unexpected runtime error %v", r)
return
}
}()
...
// perhaps a broken pipe in a global var
// or an included module threw a panic at you!
}
But if you are still not convinced... Here is the Go FAQ
We believe that coupling exceptions to a control structure, as in the try-catch-finally idiom, results in convoluted code. It also tends to encourage programmers to label too many ordinary errors, such as failing to open a file, as exceptional.
Go takes a different approach. For plain error handling, Go's multi-value returns make it easy to report an error without overloading the return value. A canonical error type, coupled with Go's other features, makes error handling pleasant but quite different from that in other languages.
A panic typically means something went unexpectedly wrong. Mostly used to fail fast on errors that shouldn’t occur during normal operation, or that we aren’t prepared to handle gracefully. So in this case just return the error, you don't want your program to panic.
I think none of the previous answers are correct:
By default, if we don't know what to do with the "error" code must panic following best programming patterns:
https://en.wikipedia.org/wiki/Fail-fast
Putting it more formally, our "Turing Machine" is broken and we need to come back to an "stable state" or "reset state". More info at https://en.wikipedia.org/wiki/Reset_(computing)
For example in web (micro)services that means returning a 40X error (panic caused by input from user) or 50X error (panic caused by something else - hardware, network, assert error, ...)
If we know what to do with the "error", then we do not have an error in first place, but an uncomfortable return value. This is a normal execution condition and probably not an error. Normally this correspond to the happy vs non-happy path modeling.
In a summary, the err return value is mostly a wrong idea, even if the GO community has adopted it as a religion. Using error return values is just a patchy way to speed up program execution since it require fewer CPU instructions to be implemented, but most of the time, except for low-level services, it is useless and promote dirty code. (note that GO was designed to implement those low-level services as an "easy-C", but it was adopted for high-level (Level 7) application programs when an error must fail fast to avoid continuing with undefined states that can potentially cause money being lost of fatal casualties. In case of doubt, default to panic.
Don't use panic for normal error handling. Use error and multiple return values. See https://golang.org/doc/effective_go.html#errors.
Will context.Done() unblock when a context variable goes out of scope and cancel is not explicitly called?
Let's say I have the following code:
func DoStuff() {
ctx, _ := context.WithCancel(context.Background())
go DoWork(ctx)
return
}
Will ctx.Done() unblock in DoWork after the return in DoStuff()?
I found this thread, https://groups.google.com/forum/#!topic/golang-nuts/BbvTlaQwhjw, where the person asking how to use Context.Done() claims that context.Done() will unblock when the context variable leaves scope but no one validated this, and I didn't see anything in the docs.
No, it doesn't cancel automatically when the context leaves scope. Typically one calls defer cancel() (using the callback from ctx.WithCancel()) oneself to make sure that the context is cancelled.
https://blog.golang.org/context provides a good overview of how to use contexts correctly (including the defer pattern above). Also, the source code https://golang.org/src/context/context.go is quite readable and you can see there's no magic that would provide automatic cancellation.
"Unblocking" is not the clearest terminology. Done() returns a channel (or nil) that will receive a struct{} and/or close when the context is "cancelled". What exactly that chan is, or when it is sent on, is up to the individual implementation. It may be sent/closed at some fixed time as with WithDeadline, or manually done as with WithCancel.
The key though, is that this is never "automatic" or guaranteed to happen. If you make a context with WithCancel and read from the Done() channel, that read will block indefinitely until the Cancel() method is called. If that never happens, then you have a wasted goroutine and your application's memory will increase each time you do it.
Once the context is completely out of scope (no executing goroutine is listening to it or has a reference to the parent context), it will get garbage collected and everything will go away.
EDIT: After reading the source though, it looks like WithCancel and friends spawn goroutines to propigate the cancellation. Therefore you must make sure Cancel gets called at some point to avoid goroutine leaks.
In Go, go is the keyword to spawn a goroutine. However, it seems the keyword doesn't return any value, so how could I detect if the goroutine is spawned correctly?
It seems Go uses the following ways to check whether a func is successful or not.
value, err = myFunc(value)
Is there a similar usage for the go keyword to detect a creation error? It seems go will throw a runtime exception if it failed.
I want to make a test to find out the maximum number of goroutine I could create for a CPU.
As you already know:
value, err = myFunc(value)
is the idiomatic way to handle exceptions by returning the built-in error type. In a way you can compare it to a checked exception, I guess. In your case though, failing to spawn a new goroutine is more of a runtime exception. How golang handles those is by using panics. You can handle them in your code with the built-in recover() function, which will try to regain control of the execution flow. Without that the panic will go up the stack until it crashes the program.
Notice that recover() has to be called in a function which is being defered, those functions are pushed into a list and are always called at the end of the function in which they were defered - so even when the panic occurs they will be called, allowing you to call recover(). If you just try to call recover() at the end of your function (or well anywhere after you panicking subfunction) the execution will never reach it. If you can handle the panic (recover() doesn't return an err) so that your program can actually continue it will execute from the point where the function that threw the panic was.
Think the above blog post is enough but if you need more examples just comment here.
Also your system will most probably be bounded by RAM memory rather than CPU.
A goroutine creation is (more or less) just a memory allocation. You cannot in general catch memory allocation exceptions and it's the same with goroutines.
If your program runs out of memory, there's usually nothing you can do about it beyond quitting and restarting.
If the function you are using a go routine that returns an error, you can instead call go on an anonymous function and handle the error in the anonymous function.
go func() {
value, err := myFunc()
}()
Speaking as a new go enthusiast trying to work with the go way of error handling. To be clear - I like exceptions.
I have a server that accepts a connection , processes a set of requests and replies to them. I found that I can do
if err != nil{
panic(err)
}
in the deep down processing code
and have
defer func() {
if err := recover(); err != nil {
log.Printf("%s: %s", err, debug.Stack()) // line 20
}
}()
in the client connection code (each connection is in a goroutine). This nicely wraps everything up, forcefully closes the connection (other defers fire) and my server continues to hum along.
But this feels an awful lot like a throw/catch scenario - which golang states it doesn't support. Questions
is this stable. ie recovering a panic is an OK thing to do as an
ongoing way of life. Its not intended to just slightly defer an
immediate shutdown
I looked for a discussion on this topic and did not find it anywhere - any pointers?
I feel that the answer is 'yes it works' and can be used inside you own code, but panic should NOT be used by a library intended for wider use. The standard and polite way for a library to behave is by error returns
Yes, you can do what you suggest. There are some situations within the standard packages where panic/recover is used for handling errors. The official Go blog states:
For a real-world example of panic and recover, see the json package
from the Go standard library. It decodes JSON-encoded data with a set
of recursive functions. When malformed JSON is encountered, the parser
calls panic to unwind the stack to the top-level function call, which
recovers from the panic and returns an appropriate error value (see
the 'error' and 'unmarshal' methods of the decodeState type in
decode.go).
Some pointers:
Use error for your normal use cases. This should be your default.
If your code would get clearer and simpler by using a panic/recover (such as with a recursive call stack), then use it for that particular case.
Never let a package leak panics. Panics used within a package should be recovered within the package and returned as an error.
Recovering from a panic is stable. Don't worry about continuing execution after a recover. You can see such behavior in standard library such as with the net/http package which recovers from panics within handlers to prevent the entire http server to go crash when panicing on a single request.
Generally most methods won't panic, they will return an error instead, and there's a bit of an overhead of using defer.
So yes, it does work, but the "proper" / "go" way is to return an error instead of using panic / recover.