Trying to debug an app running on freebsd using gdb - gdbserver. Somehow if I do a local debugging (using gdb on bsd) everything is going well however using the gdbserver (I run gdb and gdbserver on the same bsd machine) seems to fail in the initialization process before main.
my workflow is:
echo "int main(int argc, char** argv){return 0;}">main.cpp
clang++ -ggdb main.cpp
gdbserver localhost:2222 ./a.out
Same machine or different doesn't matter:
gdb
file a.out
target remote localhost:2222
break main
continue -> here everything is crashing (not gdbserver but app).
Thank you in advance :)
The output of my console is:
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd".
(gdb) file a.out
Reading symbols from a.out...done.
(gdb) target remote localhost:2222
Remote debugging using localhost:2222
0x0000000800602110 in ?? ()
(gdb) break main
Breakpoint 1 at 0x400770: file main.cpp, line 1.
(gdb) continue
Continuing.
Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb)
Some additional information:
(gdb) backtrace
#0 0x0000000000000000 in ?? ()
#1 0x0000000200000000 in ?? ()
#2 0x0000000000400200 in ?? ()
#3 0x0000000000400510 in ?? ()
#4 0x00007fffffffeba0 in ?? ()
#5 0x00007fffffffeba8 in ?? ()
#6 0x00007fffffffebc0 in ?? ()
#7 0x000000000040023c in crt_noinit_tag ()
#8 0x000000080061e000 in ?? ()
#9 0x0000000000400248 in crt_noinit_tag ()
#10 0x00007fffffffedd8 in ?? ()
#11 0x00007fffffffebd0 in ?? ()
#12 0x0000000000000000 in ?? ()
(gdb) info sharedlibrary
From To Syms Read Shared Object Library
0x00000008008599c0 0x00000008008b2f58 Yes /usr/lib/libc++.so.1
0x0000000800ae4a20 0x0000000800af0a78 Yes /lib/libcxxrt.so.1
0x0000000800cfdcf0 0x0000000800d158a8 Yes /lib/libm.so.5
0x0000000800f5b780 0x00000008010673a8 Yes /lib/libc.so.7
0x00000008012cc150 0x00000008012d58e8 Yes /lib/libgcc_s.so.1
0x0000000800602110 0x0000000800615639 Yes /libexec/ld-elf.so.1
This issue is reproducible using FreeBSD 10.1 and the described steps.
For those who are looking to solve this problem:
Had a conversation with a maintainer of gdb in FreeBSD. The result is that in FreeBSD there is no gdbserver which can be used alone. Instead there is a remote debug server called ds2 which can be used instead. Unfortunately ds2 is not available in FreeBSD 10.1 but it is working well in the newer versions of FreeBSD. Tested it with gdb and it is working!!!.
Related
I wrote a simple C program to print hello world. Then I ran it through
aarch64-linux-gnu-gcc -ohello hello.c -static -g3
gdb-multiarch hello
After this, I run and gdb encounters an internal error:
Reading symbols from hello...done.
(gdb) r
Starting program: /home/gt/hello
/build/gdb-GT4MLW/gdb-8.1/gdb/i387-tdep.c:592: internal-error: void i387_supply_fxsave(regcache*, int, const void*): Assertion `tdep->st0_regnum >= I386_ST0_REGNUM' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n)
Here's the output of file hello:
hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.7.0, BuildID[sha1]=a...b, with debug_info, not stripped
This is my hello.c:
#include<stdio.h>
int main(){
printf("hello world");
return 0;
}
What am I doing wrong? What else do I need to do? I am running Ubuntu 18.04 on an x86_64 machine.
When I use gdb hello, I am unable to use breakpoints, I get this error:
Reading symbols from hello...done.
(gdb) break 4
Breakpoint 1 at 0x400404: file hello.c, line 4.
(gdb) r
Starting program: /home/gt/hello
Warning:
Cannot insert breakpoint 1.
Cannot access memory at address 0x400404
(gdb)
I am following the guide given on this page under the first section.
In order to run and debug your AArch64 executable, you (in general) need to run it on an AArch64 machine, or in an AArch64 emulator.
You might have some setup where qemu more or less transparently emulates aarch64 binaries for you - but that doesn't work quite as transparently for the debugger. In general you can run the debugger on one machine, connected over a network to a debugging server on a different machine, allowing you to debug a process running on the machine with the debugging server.
The guide you linked shows how to set up qemu to allow it to transparently emulate binaries as you execute them. That guide only shows executing, not debugging, but it has got a link "Debugging using GDB" that points to https://ubuntuforums.org/showthread.php?t=2010979&s=096fb05dbd59acbfc8542b71f4b590db&p=12061325#post12061325, where it is explained how to debug a process which executes within qemu emulation. This essentially amounts to the same remote debugging with a debugging server as I mentioned above.
The essential bits of this post is this:
# In a terminal
$ qemu-arm-static -g 10101 ./hello
# In a new terminal
$ gdb-multiarch
(gdb) target remote :10101
Remote debugging using :10101
[New Remote target]
[Switching to Remote target]
I have been trying to debug and step into docker daemon code using gdb or cgdb on a Ubuntu 14.04 container running on host OS (ubuntu 14.04 OS). I have used -O0 -g flags while building the debug binaries within a container.
Version of go used in the container:
# go version
go version go1.4.3 linux/amd64
I am unable to step through the code using cgdb:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"... Reading symbols from bundles/1.9.0-dev/dynbinary/docker...done. (
gdb) b main.handleGlobalDaemonFlag
Breakpoint 1 at 0x497b30
(gdb) r daemon -D
Starting program: /go/src/github.com/docker/docker/bundles/1.9.0-dev/dynbinary/docker daemon -D
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff5a00700 (LWP 2775)]
[New Thread 0x7ffff51ff700 (LWP 2776)]
[New Thread 0x7ffff49fe700 (LWP 2777)]
Breakpoint 1, 0x0000000000497b30 in main.handleGlobalDaemonFlag ()
(gdb) n Single stepping until exit from function main.handleGlobalDaemonFlag, which has no line number information.
Breakpoint 1, 0x0000000000497b30 in main.handleGlobalDaemonFlag ()
(gdb) l 1 in /tmp/go-build212795923/github.com/docker/docker/pkg/term/_obj/_cgo_export.c
(gdb)
How to confirm that the built binaries are debug ? Can I use a tool readelf to read through the headers to confirm this ?
Should I be setting a GOPATH or GOROOT to a specific directory ?
root#6511af5b06c1:/go/src/github.com/docker/docker# echo $GOROOT
root#6511af5b06c1:/go/src/github.com/docker/docker# echo $GOPATH
/go:/go/src/github.com/docker/docker/vendor
Are there any other dependencies which could be missing on my environment ?
I have gdb installed on my machine. Today I have compiled another version of gdb that is running fine. Now I want to debug this new gdb using my older gdb. Please guide me in this regard. How can I know that how gdb reads symbols from the provided executable, how it inserts break points, handles function calls and other things.
Thanks.
Think easily; when you want to debug some program, you probably compile it with -g or -ggdb and run gdb, don't you?
Download gdb source.
Compile it with -ggdb
./configure --prefix=<where-to-install>
make CFLAGS="-ggdb" CXXFLAGS="-ggdb"
make install
Debug it!
gdb <where-to-install>/bin/gdb
I've never tried it (and never thought it), but it may work. (And it looks very interesting; I'm about to try it!)
Um, I've just tested it in cygwin, and figure out the problem that the debugger gdb's output and the debuggee gdb's output are mixed; I solved it by using gdbserver to debug.
# On terminal 1..
$ gdbserver localhost:1234 gdb-gdb/prefix/bin/gdb
Process gdb-gdb/prefix/bin/gdb created; pid = 972
Listening on port 1234
Remote debugging from host 127.0.0.1
GNU gdb (GDB) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-mingw32".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
(gdb) q
Child exited with status 0
GDBserver exiting
and
# On terminal 2..
$ gdb gdb-gdb/prefix/bin/gdb
GNU gdb (GDB) 7.8
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-cygwin".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from gdb-gdb/prefix/bin/gdb...done.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
0x7c93120f in ntdll!DbgBreakPoint ()
from /cygdrive/c/WINDOWS/system32/ntdll.dll
(gdb) c
Continuing.
[Inferior 1 (Remote target) exited normally]
(gdb)
Once the first gdb starts running after taking the new gdb as an input file it will become paused after showing the info message. At this point you can put a break point on the function of new gdb which you want to execute.
e.g break insert_breakpoints // the function used to insert break points.
Now execute: run
This will start the execution of the new loaded gdb. Use file command to provide any executable HelloWorld.c comiled with -g option (for building debugging symbols) to the new gdb.
Now insert break point any where in the HelloWorld executable i.e
break main
This break command will call the insert_breakpoints function of gdb used for the insertion of breakpoints at which we have previously placed a break point.
Now you can use backtrace or other commands for examining the function calls and other stuff like that.
Hope that will solve your problem.
#ikh I think that gdb by default is compiled with debugging symbols because issuing :
file /path/to/compiled/gdb gives:
ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=0xd1c553318661f8b557f4c3640b02cee1ef512ac0, not stripped
Which means that it has debug info available in it.
Please correct me if I am wrong.
I'm using gdb on OSX, which seems to have neither the gcore nor generate-core-file commands:
$ gdb
GNU gdb 6.3.50-20050815 (Apple version gdb-1705) (Fri Jul 1 10:50:06 UTC 2011)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin".
(gdb) gcore
Undefined command: "gcore". Try "help".
(gdb) generate-core-file
Undefined command: "generate-core-file". Try "help".
(gdb)
Given this, how might I go about generating a core dump, or something approximating one, via GDB?
(I suspect I can use dump memory, but that requires an address range, and I'm struggling to find the right info invocation to get the right memory range...)
Run lldb --attach-pid, then use the process save-core command to save the core. Note that the process will be paused right from when you attach to it, so be careful if it’s an important process.
$ lldb --attach-pid <pid>
(lldb) process attach --pid 76669
Process 76669 stopped
Executable module set to "/bin/bash".
Architecture set to: x86_64h-apple-macosx.
(lldb) process save-core "core"
mach_header: 0xfeedfacf 0x01000007 0x00000008 0x00000004 0x00000030 0x00000e08 0x00000000 0x00000000
...
Saving data for segment at 0x7fd455200000
...
See How to generate a core file for a crashed app in XCode + gdb?
Also, maybe a newer gdb has a gcore that works on MacOS. I don't know, but you could search around and find out.
I wrote a very simple Qt program here:
int main(int argc, char* argv[])
{
QApplication app(argc, argv);
QTableView table(&frame);
table.resize(100, 100);
table.show();
return app.exec();
}
And when I try to set a breakpoint where the table gets clicked, I get this error from gdb:
(gdb) symbol-file /usr/lib/libQtGui.so.4.4.3.debug
Load new symbol table from "/usr/lib/libQtGui.so.4.4.3.debug"? (y or n) y
Reading symbols from /usr/lib/libQtGui.so.4.4.3.debug...done.
(gdb) br 'QAbstractItemView::clicked(QModelIndex const&)'
Breakpoint 1 at 0x5fc660: file .moc/release-shared/moc_qabstractitemview.cpp, line 313.
(gdb) run
Starting program: ./qt-test
Warning:
Cannot insert breakpoint 1.
Error accessing memory address 0x5fc660: Input/output error.
Does anyone know why the breakpoint can't be inserted?
Don't use the gdb command symbol-file to load external symbols. The breakpoint addresses will be wrong since they're not relocated.
Instead, put a breakpoint in main, run the program, and then set your breakpoint:
gdb ./program
GNU gdb 6.8-debian blah blah blah
(gdb) br main
Breakpoint 1 at 0x80489c1
(gdb) run
Starting program: ./program
Breakpoint 1, 0x080489c1 in main ()
(gdb) br 'QAbstractItemView::clicked(QModelIndex const&)'
Breakpoint 2 at 0xb7d24664
(gdb) continue
Continuing.
Then make your breakpoint happen.
Make sure to specify the parameter list in the function you want to set a breakpoint in, without the names of those parameters, just their types.
The actual error:
Error accessing memory address 0x5fc660: Input/output error.
Can be caused by 32/64 bit mixups. Check, for example, that you didn't attach to a 32-bit binary with a 64-bit process' ID, or vice versa.
If you want to automatically break in main without setting a breakpoint you can also use the start command.
If you need to provide any arguments to the program you can use:
start argument1 argument2
OK for me I got this when building with mingw-w64 (native or cross compiler).
I'm not sure what the exact problem was, but if I build it using gcc mingw-w64 i686-5.1.0-posix-sjlj-rt_v4-rev0 then it creates (finally) debuggable builds. Otherwise
(gdb) break main
...
(gdb) r
...
Cannot insert breakpoint 1.
Cannot access memory at address 0x42445c
<process basically hangs>
message 19 times out of 20, though sometimes it did actually work (very rarely).
gdb 7.8.1 and 7.9.1 seemed to be able to debug the created exe. So it's probably not the version of gdb that makes a difference.
My current theory/suspect is either it was the version of gcc or possibly the sljl vs. dwarf2 "aspect" to the compiler [?] (i686-492-posix-dwarf-rt_v3-rev1 didn't work, and cross compiling with some form of gcc 4.9.2 didn't either). Didn't try other versions of gcc.
update: newer gcc (5.1.0) but cross compiling I still got this failure. The cause in this case turned out to be a dependency library that my build (FFmpeg) was using by linking against (libgme in this case) which is exporting a few errant "shared" symbols (when I am building a static executable). Because of this, "shared" builds brake (https://trac.ffmpeg.org/ticket/282) and somehow it screws up gdb as well. For instance possibly linking against SDL can do this to you as well. My thought is possibly an ld bug [?]