What is the use of defer in Go? The language documentation says it is executed when the surrounding function returns. Why not just put the code at end of given function?
We usually use defer to close or deallocate resources.
A surrounding function executes all deferred function calls before it returns, even if it panics. If you just place a function call at the end of a surrounding function, it is skipped when panic happens.
Moreover a deferred function call can handle panic by calling the recover built-in function. This cannot be done by an ordinary function call at the end of a function.
Each deferred call is put on stack, and executed in reverse order when the surrounding function ends. The reversed order helps deallocate resources correctly.
The defer statement must be reached for a function to be called.
You can think of it as another way to implement try-catch-finally blocks.
Closing like try-finally:
func main() {
f, err := os.Create("file")
if err != nil {
panic("cannot create file")
}
defer f.Close()
// no matter what happens here file will be closed
// for sake of simplicity I skip checking close result
fmt.Fprintf(f,"hello")
}
Closing and panic handling like try-catch-finally
func main() {
defer func() {
msg := recover()
fmt.Println(msg)
}()
f, err := os.Create(".") // . is a current directory
if err != nil {
panic("cannot create file")
}
defer f.Close()
// no matter what happens here file will be closed
// for sake of simplicity I skip checking close result
fmt.Fprintf(f,"hello")
}
The benefit over try-catch-finally is that there is no nesting of blocks and variable scopes. This simplifies the structure of the surrounding function.
Just like finally blocks, deferred function calls can also modify the return value if they can reach the returned data.
func yes() (text string) {
defer func() {
text = "no"
}()
return "yes"
}
func main() {
fmt.Println(yes())
}
There are already good answers here. I would like to mention one more use case.
func BillCustomer(c *Customer) error {
c.mutex.Lock()
defer c.mutex.Unlock()
if err := c.Bill(); err != nil {
return err
}
if err := c.Notify(); err != nil {
return err
}
// ... do more stuff ...
return nil
}
The defer in this example ensures that no matter how BillCustomer returns, the mutex will be unlocked immediately prior to BillCustomer returning. This is extremely useful because without defer you would have to remember to unlock the mutex in every place that the function could possibly return.
ref.
Well, it's not always guaranteed that your code may reach the end of the function (e.g. an error or some other condition may force you to return well ahead of the end of a function). The defer statement makes sure that whatever function is assigned to it gets executed for sure even if the function panics or the code returns well before the end of the function.
The defer statement also helps keep the code clean esp. in cases when there are multiple return statements in a function esp. when one needs to free resources before return (e.g. imagine you have an open call for accessing a resource at the beginning of the function - for which a corresponding close must be called before the function returns for avoiding a resource leak. And say your function has multiple return statements, maybe for different conditions including error checking. In such a case, without defer, you normally would call close for that resource before each return statement). The defer statement makes sure the function you pass to it is always called irrespective of where the function returns, and thus saves you from extraenous housekeeping work.
Also defer can be called multiple times in the same function. E.g.: In case you have different resources being allocated through your function which need to be eventually freed before returning, then you can call defer for each of them after allocation and these functions are executed in the reverse order of the sequence in which they were called when the function exits.
Key benefit of using defer - it will be called any way no matter how function will return. If an extraordinary situation would occur deferred function will be called.
So it gives nice things:
Recover after panic. This allows yes realize try ... catch behavior.
Not to forget clean up (close files, free memory, etc) before normal exit. You may open some resource and you have to close it before exit. But function can have several exit points - so you have to add freeing in every return point. That’s very tedious in maintenance. Or you can put only one deferred statement - and resources will be released automatically.
Summary:
When we do certain operations that need cleanup, we can "schedule" the cleanup operations which would be run when the function returns no matter which path that happens, including due to panic.
Detailed answer:
Programming languages strive to provide constructs that facilitate simpler and less error-prone development. (E.g. why should Golang support garbage collection when we can free the memory ourselves)
A function can return at multiple points. The user might overlook doing certain cleanup operations in some paths
Some cleanup operations are not relevant in all return paths
Also, it is better to keep the cleanup code closer to the original operation which needed the cleanup
When we do certain operations that need cleanup, we can "schedule" the cleanup operations which would be run when the function returns no matter which path that happens.
A defer statement defers the execution of a function until the
surrounding function returns.
This example demonstrates defer functionality:
func elapsed(what string) func() {
start := time.Now()
fmt.Println("start")
return func() {
fmt.Printf("%s took %v\n", what, time.Since(start))
}
}
func main() {
defer elapsed("page")()
time.Sleep(time.Second * 3)
}
Out:
start
page took 3s
Related
I'm having an argument with a friend, over possible wasted resources in Go.
Does a Reader have to be closed, if it's operating over an in-memory byte array?
func readJar(zipBytes []byte, readMeta bool) (m jar.Manifest, err error) {
reader, err := zip.NewReader(bytes.NewReader(zipBytes), int64(len(zipBytes)))
if err != nil {
return
}
for _, file := range reader.File {
switch file.Name {
case jar.ManifestPath:
m, err = readManifest(file)
if err != nil {
return
}
}
}
return
}
func readManifest(file *zip.File) (jar.Manifest, error) {
reader, err := file.Open()
if err != nil {
return nil, err
}
defer reader.Close()
return jar.ReadManifest(reader)
}
Whilst originally it was thought to be a source of File Handle Leaks, something else got the blame.
Will this leak memory, or does Go have sufficient escape analysis / garbage collection that it will be fine?
Golang compiler takes care for unreachable variables:-
The storage location does have an effect on writing efficient
programs. When possible, the Go compilers will allocate variables that
are local to a function in that function's stack frame. However, if
the compiler cannot prove that the variable is not referenced after
the function returns, then the compiler must allocate the variable on
the garbage-collected heap to avoid dangling pointer errors. Also, if
a local variable is very large, it might make more sense to store it
on the heap rather than the stack.
Although Golang contains garbage collection. It would be better to use clean up functions. You can use defer function to close the file at the end of function.
defer f.close()
Check Documentation for SetFinalizer to get more understanding on these concepts:
func SetFinalizer(obj interface{}, finalizer interface{})
SetFinalizer sets the finalizer associated with obj to the provided
finalizer function. When the garbage collector finds an unreachable
block with an associated finalizer, it clears the association and runs
finalizer(obj) in a separate goroutine. This makes obj reachable
again, but now without an associated finalizer. Assuming that
SetFinalizer is not called again, the next time the garbage collector
sees that obj is unreachable, it will free obj.
The finalizer runs for an object to check if it is unreachable from the source. It can be used for file descriptors but it would be a mistake to depend on a finalizer to flush an in-memory I/O buffer such as a bufio.Writer, because the buffer would not be flushed at program exit.
I have two goroutines go doProcess_A() and go doProcess_B(). Both can call saveData(), a non goroutine method.
should I use go saveData() instead of saveData() ?
Which one is safe?
var waitGroup sync.WaitGroup
func main() {
for i:=0; i<4; i++{
waitGroup.Add(2)
go doProcess_A(i)
go doProcess_B(i)
}
waitGroup.Wait()
}
func doProcess_A(i int) {
// do process
// the result will be stored in data variable
data := "processed data-A as string"
uniqueFileName := "file_A_"+strconv.Itoa(i)+".txt"
saveData(uniqueFileName, data)
waitGroup.Done()
}
func doProcess_B(i int) {
// do some process
// the result will be stored in data variable
data := "processed data-B as string"
uniqueFileName := "file_B_"+strconv.Itoa(i)+".txt"
saveData(uniqueFileName, data)
waitGroup.Done()
}
// write text file
func saveData(fileName ,dataStr string) {
// file name will be unique.
// there is no chance to be same file name
err := ioutil.WriteFile("out/"+fileName, []byte(dataStr), 0644)
if err != nil {
panic(err)
}
}
here, does one goroutine wait for disk file operation when other goroutine is doing?
or, are two goroutine make there own copy of saveData() ?
Goroutines typically don't wait for anything except you explicitly tell them to or if an operation is waiting on a channel or other blocking operation. In your code there is a possibility of a race condition with unwanted results if multiple goroutines call the saveData() function with same filename. It appears that the two goroutines are writing to different files, therefore as long as the filenames are unique, the saveData operation will be safe in a goroutine. It doesn't make sense to use a go routine to call saveData(), don't unnecessarily complicate your life, just call it directly in the doProcess_X functions.
Read more about goroutines and make sure you are using it where it is absolutely necessary. - https://gobyexample.com/goroutines
Note: Just because you are writing a Go application doesn't mean you
should litter it with goroutines. Read and understand what problem it
solves so as to know the best time to use it.
http.Serve either returns an error as soon as it is called or blocks if successfully executing.
How can I make it so that if it blocks it does so in its own goroutine? I currently have the following code:
func serveOrErr(l net.Listener, handler http.Handler) error {
starting := make(chan struct{})
serveErr := make(chan error)
go func() {
starting <- struct{}{}
if err := http.Serve(l, handler); err != nil {
serveErr <- err
}
}()
<-starting
select {
case err := <-serveErr:
return err
default:
return nil
}
}
This seemed like a good start and works on my test machine but I believe that there are no guarantees that serveErr <- err would be called before case err := <-serveErr therefore leading to inconsistent results due to a data race if http.Serve were to produce an error.
http.Serve either returns an error as soon as it is called or blocks if successfully executing
This assumption is not correct. And I believe it rarely occurs. http.Serve calls net.Listener.Accept in the loop – an error can occur any time (socket closed, too many open file descriptors etc.). It's http.ListenAndServe, usually being used for running http servers, which often fails early while binding listening socket (no permissions, address already in use).
In my opinion what you're trying to do is wrong, unless really your net.Listener.Accept is failing on the first call for some reason. Is it? If you want to be 100% sure your server is working, you could try to connect to it (and maybe actually transmit something), but once you successfully bound the socket I don't see it really necessary.
You could use a timeout on your select statement, e.g.
timeout := time.After(5 * time.Millisecond) // TODO: ajust the value
select {
case err := <-serveErr:
return err
case _ := <- timeout:
return nil
}
This way your select will block until serveErr has a value or the specified timeout has elapsed. Note that the execution of your function will therefore block the calling goroutine for up to the duration of the specified timeout.
Rob Pike's excellent talk on go concurrency patterns might be helpful.
Why does a call to defer func() { recover() }() successfully recover a panicking goroutine, but a call to defer recover() not?
As an minimalistic example, this code doesn't panic
package main
func main() {
defer func() { recover() }()
panic("panic")
}
However, replacing the anonymous function with recover directly panics
package main
func main() {
defer recover()
panic("panic")
}
Quoting from the documentation of the built-in function recover():
If recover is called outside the deferred function it will not stop a panicking sequence.
In your second case recover() itself is the deferred function, and obviously recover() does not call itself. So this will not stop the panicking sequence.
If recover() would call recover() in itself, it would stop the panicking sequence (but why would it do that?).
Another Interesting Example:
The following code also doesn't panic (try it on the Go Playground):
package main
func main() {
var recover = func() { recover() }
defer recover()
panic("panic")
}
What happens here is we create a recover variable of function type which has a value of an anonymous function calling the built-in recover() function. And we specify calling the value of the recover variable to be the deferred function, so calling the builtin recover() from that stops the panicing sequence.
The Handling panic section mentions that
Two built-in functions, panic and recover, assist in reporting and handling run-time panics
The recover function allows a program to manage behavior of a panicking goroutine.
Suppose a function G defers a function D that calls recover and a panic occurs in a function on the same goroutine in which G is executing.
When the running of deferred functions reaches D, the return value of D's call to recover will be the value passed to the call of panic.
If D returns normally, without starting a new panic, the panicking sequence stops.
That illustrates that recover is meant to be called in a deferred function, not directly.
When it panic, the "deferred function" cannot be the built-in recover() one, but one specified in a defer statement.
DeferStmt = "defer" Expression .
The expression must be a function or method call; it cannot be parenthesized.
Calls of built-in functions are restricted as for expression statements.
With the exception of specific built-in functions, function and method calls and receive operations can appear in statement context.
An observation is that the real problem here is the design of defer and thus the answer should say that.
Motivating this answer, defer currently needs to take exactly one level of nested stack from a lambda, and the runtime uses a particular side effect of this constraint to make a determination on whether recover() returns nil or not.
Here's an example of this:
func b() {
defer func() { if recover() != nil { fmt.Printf("bad") } }()
}
func a() {
defer func() {
b()
if recover() != nil {
fmt.Printf("good")
}
}()
panic("error")
}
The recover() in b() should return nil.
In my opinion, a better choice would have been to say that defer takes a function BODY, or block scope (rather than a function call,) as its argument. At that point, panic and the recover() return value could be tied to a particular stack frame, and any inner stack frame would have a nil pancing context. Thus, it would look like this:
func b() {
defer { if recover() != nil { fmt.Printf("bad") } }
}
func a() {
defer {
b()
if recover() != nil {
fmt.Printf("good")
}
}
panic("error")
}
At this point, it's obvious that a() is in a panicking state, but b() is not, and any side effects like "being in the first stack frame of a deferred lambda" aren't necessary to correctly implement the runtime.
So, going against the grain here: The reason this doesn't work as might be expected, is a mistake in the design of the defer keyword in the go language, that was worked around using non-obvious implementation detail side effects and then codified as such.
A range on a channel
for s := range cs {
fmt.Println("Recieved Cake: ", s)
}
should keep a function open until the channel closes at which point the function/goroutine should terminate. When a function terminates the defer function should run just prior. This doesn't seem to be the case and I can't find any reasons why.
Sample code at http://play.golang.org/p/ADu1MzAe9P produces defer statements as expected except for the function that is recieving from the channel. Any reasons as to why this would be? thanks!
The reason why the defer function is not executing is that the application reaches the end of the main function causing the entire program to terminate without waiting for goroutines.
Go Specification says:
When the function main returns, the program exits. It does not wait for other (non-main) goroutines to complete.
Since your recieveCakeAndPack is still waiting for the channel to close (which never happens) it will never defer before the termination of the program.
Edit
On a side note - putting the defer statements last in a function is not meaningful. Instead put them directly after the statement you want to defer such as:
fmt.Println("Entering function")
defer fmt.Println("Leaving function")
or
file, err := os.Open("file.txt")
if err != nil {
return err
}
defer file.Close()
The defer function/method calls will be executed when leaving the function in a Last-In-First-Out order.