Investigating Segmentation Faults and Debugging Tools - bash

After compiling a C++ program with make and gcc I experienced a segmentation fault while running. The program just exited without any error message.
Even though I didn't compile the program in debug mode, by running it with gdb I actually got an error message to see where the segfault happened.
What I would like to know is:
Why did gdb display the line causing the error, but the regular bash did not?
How can the gdb display the line when the program was not compiled in debug mode?
In the past I always re-compiled in debug mode (to attatch the symbolic table to the binaries) and investigted the backtrace of the core dump with ddd. Is this the proper way to fix segmentation faults, or what is the common way to do it?

Why did gdb display the line causing the error, but the regular bash did not?
Because gdb is built to do that. Trapping the segfault and reporting about it to you is part of its job, to help you debug. For example, you can then obtain a backtrace, examine the various stack frames, etc. to determine the nature and perhaps the cause of the error. Bash is not built to do that, and no such behavior is provided for free.
How can the gdb display the line when the program was not compiled in debug mode?
Clearly some kind of debugging information is inserted into the binary by default, even when you do not ask for it. Enough, at least, to provide line numbers for code. If you're using GCC, then that might correspond to -g1, whereas -g is equivalent to -g2. If you're curious, you could compile with -g0 to see whether that eliminates the line number information.
In the past I always re-compiled in debug mode (to attatch the symbolic table to the binaries) and investigted the backtrace of the core dump with ddd. Is this the proper way to fix segmentation faults, or what is the common way to do it?
There's no "proper" here beyond "whatever works".
I do find that its easier to debug programs compiled with optimization disabled, and of course debug information is most helpful -- all provided that you can reproduce the error with such a binary. I also tend to engage valgrind whenever a program I'm working on segfaults. That, too, is more informative when debug information is available, and if there is a memory problem (which a segfault almost invariably indicates) then valgrind will likely identify it even if the debug version of the program doesn't crash. As for ddd, that's just one of several UI choices, including supported tools' native ones. Use what works for you in that regard.
Oh, and in decades of programming, I've never yet had to resort to analyzing a core dump. My time may come eventually, of course, but I'm content to defer it.

Related

debugging a simulated processor

I have an embedded project which runs on a 68332 processor target (68k family). There is no OS on the target. We have a custom simulator that will allow our code to execute within Windows. The simulator is completely without our control to modify. Basically the simulator is executing the machine code which isn't very good when you need to debug. What I would really like to do is interface a debugger to allow us to debug at the source level rather than at the machine/assembly level. Has anyone ever done such a thing? Is there a spec that debuggers support? Perhaps would something like gdb work for this? Any advice is appreciated.
This is not necessarily an answer to your question - I'm not familiar with hooking up an existing 3rd-party debugger to a program executing inside a VM so I can't advise about that.
However, you control the source of your simulator so you can try implementing an interface (maybe a local socket, etc.) where your simulator keeps reporting status information about the code that's executing and links it up with source files by reading debug information from some generated debugging database. You'd likely have to support reading the debugging format of the compiler that compiles your 68k code and then use that information to link back assembly instructions to source code lines.
This way you're effectively implementing a debugger, but since you already have the simulator (a VM really), that's probably not too much of extra work - the simulator already has all state information about the executing 68k code, you just need a way to temporarily pause execution and extract state information during pause. Stepping through code after that is probably a trivial repeat of these steps.

LiteIDE GDB with Golang

I installed LiteIDE and GDB. I opened my Go project in LiteIDE and added a breakpoint to some point in the code. Then i switched back to terminal and ran the project binary that was supposed to envoke the breakpoint and nothing happened. What am i doing wrong?
You have to actually launch the executable from the IDE for breakpoints to mean anything.
Also keep in mind that gdb is mostly meaningless with Go 1.3.x and even more so with 1.4 (dev).
From https://golang.org/doc/gdb:
GDB does not understand Go programs well. The stack management, threading, and runtime contain aspects that differ enough from the execution model GDB expects that they can confuse the debugger, even when the program is compiled with gccgo. As a consequence, although GDB can be useful in some situations, it is not a reliable debugger for Go programs, particularly heavily concurrent ones. Moreover, it is not a priority for the Go project to address these issues, which are difficult. In short, the instructions below should be taken only as a guide to how to use GDB when it works, not as a guarantee of success.
In time, a more Go-centric debugging architecture may be required.
I use this package https://github.com/gostart/debug/ and so far it is the best solution that I have found.
Hope this helps.

how to obtain VC++ compiler-style help on GCC G++ compile/link errors (on linux)

I'm using VC++ as professional developer for more than 10 years and it has been good to me, now I'm trying to broaden my horizons and learn C++ development on Linux.
On Windows things are simple, VC++ does it all (editing, project management, help, debugging), but on linux things are different, you have assemble your development environment from different tools.
I'm still trying to tie things together, and one thing I still haven't figured out is how to decipher GCC (G++) errors when compiling/linking C++ apps on Linux (although I realize GCC is multi-platform, I'll refer to my linux experience here only).
In VC++, things are very clear: If during compilation VC++'s compiler encounters error in program, it will create new entry in 'output' window with the 'compiler error ID'. Example:
c:\projectA\fileB.cpp(38) : error C2228: left of '.cout' must have class/struct/union
From here, you can click on the line in question in 'output' window, press F1, and 'Microsoft Document Browser' app will start (if it wasn't started already), which will load MSDN help file describing compile error connected to the compiler error ID (in example it's C2228), usually with sample you can check out to figure out what's wrong with your code. If you don't have MDB installed, you can always search on the web for C2228 and get the same help page, optionally finding other people's web pages describing their experience with this error.
The same thing is with linking, you'll get 'linker error ID' (e.g. LNK1123), which you can use to find help either locally or on web.
Try as I might, I can't find this kind of functionality in GCC's G++. All I can see is bunch of less experienced GCC developers asking another bunch of more experienced GCC developers to analyze their code based on descriptive compiler/linker errors with no associated error IDs.
Is there tool(set) that provides VC++ compiler-style help on GCC G++ compile/link errors for linux?
You may try to use qtcreator. At least it can show the errors in a more comprehensive way comparable to the VC++, that is, it can locate the error position and highlight the error line and variables.
If you can an alternative might be to use Clang instead. It gives much better error messages than g++. It compiles most code these days (but it still a work in progress). Highly recommended.
Alternatively (as another poster has mentioned) you could use an IDE such as Eclipse to capture the error messages, though I don't think that adds anything beyond taking you to the line number on double-click.

Programmer's Debugging toolkit pack

What kind of debugging tools you've been using to debug working binaries?
Is there are debugging toolkits in addition to GDB?
The only reason is that I'm quite new for system debugging and I've been debugging my system service.
Sultan
I'd recommend Valgrind [1]. It's quite useful when dealing with memory leaks and segfaults.
The segfaults can be tracked by letting GDB run (without any breakpoint) and check the backtrace ('bt' command), after the crash.
P.S.: I don't remember if Valgrind is avaliable for other systems, but since you asked about alternatives to GDB, I'm assuming you're on a *nix box.
Have a nice debugging.
[1] http://valgrind.org/
Assuming you are on Linux systems, one of the most valuable tool is valgrind.
I don't really use anything else except for precondition/postcondition check in the code itself (i.e. assertion on methods' input values).

Using GDB without debugging symbols on x86?

How do I use GDB to debug a program which do not have debugging symbols on a 32-bit x86 processor? Inspecting the function arguments, local variables, resolving pointers would be useful to know how to do.
The intention is not really to use this for reverse engineering, as I'm sometimes just too lazy to install the debugging symbols and would be great to know how to get some basic information out of gdb.
To start out, you can do;
gdb "whatever"
break __libc_start_main
r
that will setup a breakpoint in libc's crt0 code and allow you to break before main, even if the target binary is totally stripped.
That will get you to a running state at a breakpoint before most user code. You can then single step, dissasemble, dump memory etc... to your heart's content.
This works on all platforms, the fact your asking about IA-32 / x86 does not matter.
Without debugging symbols, you can only debug at the ASM level. Ok you get a bit more information, but you're not going to get very far unless you understand a bit of ASM and the code the compiler generates. This will let you do a simple inspection of local variables etc if you know what you're doing.
If you have the source, it's going to be far easier just to recompile it.
All you can do is look at registers and the contents of the stack - you'll have to do everything by inferring what things are used for, as Draemon mentions.
Well, the absolutely most important thing is that you be able to unwind the stack. There are three ways this can be ensured:
Build debugging symbols with -g
On systems that do C++ exception unwinding via tables (probably anything ELF these days?), the -funwind-tables flag will tell it to generate such tables regardless of language, and GDB can use these tables (at least, with x86 linux it can).
Or, failing those, at least make sure that -fomit-frame-pointer isn't enabled

Resources