I am a complete beginner in golang, in fact I am debugging someone else's program to find out the cause of an unexpected exit.
I want to know how can I set a breakpoint in gdb at the "exit" routine called just before the program shuts down?
I have so far tried
gdb <program name>
run
<...wait for program to quit>
break 'runtime.goexit'
run
<...wait for program to break>
But it does not break, instead it just exits.
If you have access to source code you can add a defer to handle exit from arbitrary point like:
https://play.golang.org/p/uliAc3j7f-
package main
import (
"fmt"
)
func main() {
defer func() {
fmt.Println("Place breakpoint here")
if recovered := recover(); recovered != nil {
fmt.Println("Handled panic:", recovered)
}
}()
fmt.Println("Hello, playground")
panic("Something went wrong")
}
Also Go is case sensitive so try break 'runtime.Goexit' - see https://golang.org/pkg/runtime/#Goexit
But it does not break, instead it just exits.
You can use catch syscall exit (or catch sycall exit_group if you are
on Linux).
That is guaranteed to stop the program if it really exists (as opposed to being terminated by a signal).
Related
What if an imported library spawns a goroutine that can panic? In this case, there is nothing a developer can do to stop the program from exiting.
Like in this code, calling a bad library with deferred recover does not help as the library is spawning a goroutine that panics, and it cannot be caught by the main's recover.
Do I understand it correct that the only remedy is to be very careful when choosing dependencies for one's project and hope that the authors do not do something similar?
package main
import (
"fmt"
"time"
)
func main() {
defer func() {
r := recover()
if r != nil {
fmt.Println("panic caught")
}
}()
badLibrary()
}
func badLibrary() {
go recklessFunction()
time.Sleep(time.Second)
}
func recklessFunction() {
panic("something went wrong")
}
You're right, you can't do anything about that. Neither can you if a dependency calls os.Exit() for example.
Launching goroutines as a library is often bad practice anyway, let the client (user of the library) choose if concurrent execution (a goroutine) is wanted, adding the go keyword is trivial. There are exceptions of course.
package main
import (
"fmt"
"time"
evtx "github.com/0xrawsec/golang-evtx/evtx"
)
func main() {
fd, err := evtx.Open("D:\\ForwardedEvents\\Logs\\ForwardedEvents.evtx")
if err != nil {
fmt.Println(err)
}
stopchan := make(chan bool)
mychan := fd.MonitorEvents(stopchan, time.Duration(100))
x := <- mychan
fmt.Println(x)
}
The code I wrote; Windows Event Viewer Dan gets logs and outputs it, but when I run the code it says "File is flagged as a dirty." I am getting the error. How can I fix it?
The library you are using returns the error if the file you are opening is flagged as dirty (it has nothing to do with your IDE). You can choose to ignore the error if you want (or use the OpenDirty function that attempts to repair the file if its dirty but this will not work if something else has it open).
Why is this error arising? Probably because the file was not closed properly (or something is still writing to it). The Microsoft docs say:
The ELF_LOGFILE_HEADER_DIRTY flag can be used by the event-logging service to detect if the event log was not properly closed.
I need to call lots of short-lived (and occasionally some long-lived) external processes in rapid succession and process both stdout and stderr in realtime. I've found numerous solutions for this using StdoutPipe and StderrPipe with a bufio.Scanner for each, packaged into goroutines. This works most of the time, but it swallows the external command's output occasionally, and I can't figure out why.
Here's a minimal example displaying that behaviour on MacOS X (Mojave) and on Linux:
package main
import (
"bufio"
"log"
"os/exec"
"sync"
)
func main() {
for i := 0; i < 50000; i++ {
log.Println("Loop")
var wg sync.WaitGroup
cmd := exec.Command("echo", "1")
stdout, err := cmd.StdoutPipe()
if err != nil {
panic(err)
}
cmd.Start()
stdoutScanner := bufio.NewScanner(stdout)
stdoutScanner.Split(bufio.ScanLines)
wg.Add(1)
go func() {
for stdoutScanner.Scan() {
line := stdoutScanner.Text()
log.Printf("[stdout] %s\n", line)
}
wg.Done()
}()
cmd.Wait()
wg.Wait()
}
}
I've left out the stderr handling for this. When running this, I get only about 49,900 [stdout] 1 lines (the actual number varies with each run), though there should be 50,000. I'm seeing 50,000 loop lines, so it doesn't seem to die prematurely. This smells like a race condition somewhere, but I can't figure out where.
It works just fine if I don't put the scanning loop in a goroutine, but then I lose the ability to simultaneously read stderr, which I need.
I've tried running this with -race, Go reports no data races.
I'm out of ideas, what am I getting wrong?
You're not checking for errors in several places.
In some, this is not actually causing problems, but it's still a good idea to check:
cmd.Start()
may return an error, in which case the command was never run. (This is not the actual problem.)
When stdoutScanner.Scan() returns false, stdoutScanner.Err() may show an error. If you start checking this, you'll find some errors:
2020/02/19 15:38:17 [stdout err] read |0: file already closed
This isn't the actual problem, but—aha—this matches the symptoms you see: not all of the output got seen. Now, why would reading stdout claim that the file is closed? Well, where did stdout come from? It's from here:
stdout, err := cmd.StdoutPipe()
Take a look at the source code for this function, which ends with these lines:
c.closeAfterStart = append(c.closeAfterStart, pw)
c.closeAfterWait = append(c.closeAfterWait, pr)
return pr, nil
(and pr is the pipe-read return value). Hmm: what could closeAfterWait mean?
Now, here are your last two lines in your loop:
cmd.Wait()
wg.Wait()
That is, first we wait for cmd to finish. (When cmd finishes, what gets closed?) Then we wait for the goroutine that's reading cmd's stdout to finish. (Hm, what could still be reading from the pr pipe?)
The fix is now obvious: swap the wg.Wait(), which waits for the consumer of the stdout pipe to finish reading it, with the cmd.Wait(), which waits for echo ... to exit and then closes the read end of the pipe. If you close while the readers are still reading, they may never read what you expected.
How to obtain the stack trace of the last (ideally of all) go routine (the app has multiple go routines) which panicked and recovered and logged only a not much descriptive error message? I don't know which routine recovered. Also, please keep in mind that I will not alter the code of any imported package. This panic happened in some of the imported packages which creates multiple go routines so I need a way to grab the stack trace of the last recovered routine in order to find where it panic.
The short answer is: Not possible but there are exceptions.
Golang has a few stack control methods and types.
You can control the stack levels with runtime/debug/SetTraceback
func SetTraceback(level string)
SetTraceback sets the amount of detail printed by the runtime
inthe traceback it prints before exiting due to an
unrecovered panic or an internal runtime error.
The level argument takes the same values as the GOTRACEBACK
environment variable. For example, SetTraceback("all") ensure
that the program prints all goroutines when it crashes.
See the package runtime documentation for details. If
SetTraceback is called with a level lower than that of the
environment variable, the call is ignored.
You can also print the stack strace with runtime/debug/Stack
func Stack() []byte
Stack returns a formatted stack trace of the goroutine that calls it. It calls runtime.Stack with a large enough buffer to capture the entire trace.
Also you need to understand how the Built-in funct recover works.
The recover built-in function allows a program to manage behavior of a
panicking goroutine. Executing a call to recover inside a deferred
function (but not any function called by it) stops the panicking sequence
by restoring normal execution and retrieves the error value passed to the
call of panic. If recover is called outside the deferred function it will
not stop a panicking sequence. In this case, or when the goroutine is not
panicking, or if the argument supplied to panic was nil, recover returns
nil. Thus the return value from recover reports whether the goroutine is
panicking.
func recover() interface{}
Working Example
This example assumes that the package does not call recover (detailed in another section).
Golang Playground Link
package main
import (
"log"
"errors"
"runtime/debug"
"time"
)
func f2() {
panic(errors.New("oops")) // line 11
}
func f1() {
f2() // line 15
}
func main() {
defer func() {
if e := recover(); e != nil {
log.Printf("%s: %s", e, debug.Stack()) // line 20
}
}()
go f1() // line 25
time.Sleep(time.Second * 1)
}
If package calls recover
If the code is recovering from the panic you need to use a debugger or remove the recover to understand what is going on as seen on the example below which demonstrate that recovered panics can not be "recovered" again.
Golang Playground Link
package main
import (
"log"
"errors"
"runtime/debug"
"time"
)
func f2() {
panic(errors.New("oops")) // line 11
}
func f1() {
defer func() {
if e := recover(); e != nil {
log.Printf("internal %s: %s", e, debug.Stack()) // line 20
}
}()
f2() // line 15
}
func main() {
defer func() {
if e := recover(); e != nil {
log.Printf("external %s: %s", e, debug.Stack()) // line 20
} else {
log.Println("Nothing to print")
}
}()
go f1() // line 25
time.Sleep(time.Second * 1)
}
Lesser then two evils
Debug with Delve Or temporaly edit the package so it logs the full message (once understood you can revert the changes).
Also if you find the problem let the package author know so it can be fixed.
It's easy to redirect a golang panic to a file, just use recover() to capture that and use syscall.Dup2() to redirect. But when it comes to a C panic, it seems to be useless, just like the image, the console will show the error message like
"fatal error: unexpected signal during runtime execution" and some stack message. How to redirect these error message to a file
package main
/*
#include <stdio.h>
void sayHi(int a, int b) {
int c = a/b;
}
*/
import "C"
import (
"runtime/debug"
"syscall"
"os"
"log"
)
func main() {
logFile, logErr := os.OpenFile("/home/error.log", os.O_CREATE|os.O_RDWR|os.O_APPEND, 0666)
if logErr != nil {
log.Println("Fail to find", *logFile)
os.Exit(1)
}
log.SetOutput(logFile)
defer func() {
if r := recover(); r != nil {
syscall.Dup2(int(logFile.fd()), 2)
debug.PrintStack()
}
}()
C.sayHi(1, 0)
}
ps:The key point is how to redirect the error message on the terminal screen to a file?
Integer division by zero at runtime in C does not result in a traditional panic() that you can recover() from. It turns out the C standard does not define a specific action to be taken in this case; this is called "undefined behavior", and you should be working to avoid it (for instance, by checking your denominators).
But I assume you are only using that as an example; it just turns out to not be a very good example.
What can happen in C, and what you're getting in your case, is that the operating system can throw a signal. Signals are a way for the operating system to notify a traditional C program that something went wrong, for instance that the user hit control-C (asking to terminate the process) or that a floating-point exception occurred. You can catch these from Go with the os/signal package, but note that only the Interrupt and Kill signals are available on all OSs (and only Interrupt can be caught; Kill cannot). See the package documentation for details.