Remote kernel debug on ARMv7 using Olimex ARM-USB-OCD adapter - linux-kernel

I'm currently trying to debug my kernel. My goal is to put a breakpoint in a new syscall that I am implementing. The kernel runs on a remote Imx6q board. I've setup the JTAG debugger and I can connect GDB to it and pause the execution.
My issue is whith debug symbols.
I've added those properties to my defconfig :
CONFIG_GDB_SCRIPTS=y
CONFIG_DEBUG_KERNEL=y
CONFIG_RANDOMIZE_BASE=n
CONFIG_FRAME_POINTER=y
CONFIG_KGDB=y
CONFIG_DEBUG_INFO=y
When I start a session :
(gdb) tar ext :3333
Remote debugging using :3333
warning: No executable has been specified and target does not support
determining executable automatically. Try using the "file" command.
0xa7780ef0 in ?? ()
(gdb) c
Continuing.
I can attach a symbol file by hand, but it is required to provide an address to attach it.
(gdb) add-symbol-file /home/tlavocat/development/android/out/target/product/wandboard_qca/kernel-imx/kernel/sys.o
The address where /home/tlavocat/development/android/out/target/product/wandboard_qca/kernel-imx/kernel/sys.o has been loaded is missing
The function I want to stop in is this one :
wandboard_qca:/ # cat /proc/kallsyms | grep sys_keeper_get_state
c003e0ac T sys_keeper_get_state
And it is implemented in kernel/sys.c.
My question is, how can I attach my symbols correctly to the right address ?
Thank's for your answers

I simply needed to load le kernel binary.
file .out/target/product/wandboard_qca/kernel-imx/vmlinux
And then connect to the remote target.

Related

select subset of debug info files in gdb session

On my fedora box I have installed a lot of separate debug infos.
sudo dnf debuginfo-install <list of packets>
Now, if I debug some simple code it needs very long until some symbol is displayed or some values are printed. It is quite clear that is absolutly needed to evaluate all the installed symbol files to get all information.
But if I have a problem, say on a lib like goocanvas I only want to have my local debug smbols generated with my own compiled code with -g option and the only the debug infos for goocanvas libs.
How can that kind of selection be achieved? Only by renaming the folder of debug info files and generate a copy of needed ones? Maybe as a symlink? Or is there a common selection option anywhere?
You can skip all debug info from shared libraries and only load goocanvas lib symbols. Here is a sample of how to do it in gdb session:
[ ~]$ gdb -q /your/binary
(gdb) set auto-solib-add off
(gdb) start
Temporary breakpoint 1, 0x000055555564edd0 in main ()
(gdb) sharedlibrary goocanvas
From gdb doc:
If your program uses lots of shared libraries with debug info that
takes large amounts of memory, you can decrease the gdb memory
footprint by preventing it from automatically loading the symbols from
shared libraries. To that end, type set auto-solib-add off before
running the inferior, then load each library whose debug symbols you
do need with sharedlibrary regexp, where regexp is a regular
expression that matches the libraries whose symbols you want to be
loaded.
See also this related question: How to prevent GDB from loading debugging symbol for a (large) library?

Eclipse gdb won't debug Cygwin executable

I created a simple Hello world executable using Cygwin on my Windows 7 PC, building it with: gcc hello.c -g
I can use gdb from the command line on the executable with no problem.
Then in Eclipse Kepler I created a debug configuration with these settings:
- The C/C++ Application has the full path to the executable
- It is not connected to a project; the Project box is blank. The workspace doesn't have any projects in it.
- "Disable auto build" is selected
- "Stop on startup at: main" is selected
- The GDB debugger is set to C:\cygwin64\bin\gdb.exe
- I added the full path to the directory containing hello.c and the executable to the "Source Lookup Path"
My original problem is with using TI's Code Composer Studio, which is based on Eclipse Kepler, to try to debug a unit test executable that is created with a hand written makefile from the shell. Downloading Eclipse Kepler and using a simple hello world program is to strip the problem down to basics. I run into the same problems with CCS or regular Eclipse.
The problem is, when I try to run this debug configuration, this is what the gdb traces console shows me:
290,997 2-gdb-set breakpoint pending on
290,999 2^done
290,999 (gdb)
290,999 3-gdb-set detach-on-fork on
291,000 3^done
291,000 (gdb)
291,000 4-enable-pretty-printing
291,001 4^done
291,001 (gdb)
291,001 5-gdb-set python print-stack none
291,002 5^done
291,002 (gdb)
291,002 6-gdb-set print object on
291,003 6^done
291,003 (gdb)
291,003 7-gdb-set print sevenbit-strings on
291,004 7^done
291,004 (gdb)
291,004 8-gdb-set host-charset UTF-8
291,005 8^done
291,005 (gdb)
291,005 9-gdb-set target-charset WINDOWS-1252
291,006 9^done
291,006 (gdb)
291,006 10-gdb-set target-wide-charset UTF-16
291,007 10^done
291,007 (gdb)
291,007 11source .gdbinit
291,008 &"source .gdbinit\n"
291,008 &".gdbinit: No such file or directory.\n"
291,008 11^error,msg=".gdbinit: No such file or directory."
291,009 (gdb)
291,009 12-gdb-set target-async off
291,010 12^done
291,010 (gdb)
291,010 13-gdb-set auto-solib-add on
291,011 13^done
291,011 (gdb)
291,016 14-file-exec-and-symbols --thread-group i1 C:/IntelligentD/scratch/a.exe
291,051 14^done
291,051 (gdb)
291,051 15-gdb-show --thread-group i1 language
291,052 15^done,value="auto"
291,052 (gdb)
291,052 16-gdb-set --thread-group i1 language c
291,053 16^done
291,053 (gdb)
291,053 17-data-evaluate-expression --thread-group i1 "sizeof (void*)"
291,054 17^done,value="8"
291,054 (gdb)
291,054 18-gdb-set --thread-group i1 language auto
291,055 18^done
291,055 (gdb)
291,055 19-interpreter-exec --thread-group i1 console "show endian"
291,056 ~"The target endianness is set automatically (currently little endian)\n"
291,056 19^done
291,056 (gdb)
291,057 20-break-insert --thread-group i1 -t -f main
291,058 20^done,bkpt=
{number="1",type="breakpoint",disp="del",enabled="y",addr="0x00000001004010dd",\
func="main",file="hello.c",fullname="/cygdrive/c/IntelligentD/scratch/hello.c",line="5",thread-group\
s=["i1"],times="0",original-location="main"}
291,059 (gdb)
291,060 21-exec-run --thread-group i1
291,075 =thread-group-started,id="i1",pid="10376"
291,077 22-list-thread-groups --available
291,077 =thread-created,id="1",group-id="i1"
291,077 ~"[New Thread 10376.0x22c8]\n"
291,077 21^running
291,077 *running,thread-id="all"
291,077 (gdb)
291,080 =thread-exited,id="1",group-id="i1"
291,080 =thread-group-exited,id="i1"
291,080 21^error,msg="During startup program exited with code 0xc0000135."
291,080 (gdb)
291,080 22^error,msg="Can not fetch data now."
291,080 (gdb)
291,088 23-gdb-exit
291,090 23^exit
Following some advice I read, at the command line I started gdb with:
gdb --interpreter=mi a.exe
and entered the same commands that show up in the log. When I get to command 21 I get different output:
(gdb)
21-exec-run --thread-group i1
=thread-group-started,id="i1",pid="7244"
=thread-created,id="1",group-id="i1"
~"[New Thread 7244.0xca4]\n"
21^running
*running,thread-id="all"
(gdb)
=library-loaded,id="/cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll",target-name="/cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll",host-name="/cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll",symbols-loaded="0",thread-group="i1"
=library-loaded,id="/cygdrive/c/WINDOWS/system32/kernel32.dll",target-name="/cygdrive/c/WINDOWS/system32/kernel32.dll",host-name="/cygdrive/c/WINDOWS/system32/kernel32.dll",symbols-loaded="0",thread-group="i1"
=library-loaded,id="/cygdrive/c/WINDOWS/system32/KERNELBASE.dll",target-name="/cygdrive/c/WINDOWS/system32/KERNELBASE.dll",host-name="/cygdrive/c/WINDOWS/system32/KERNELBASE.dll",symbols-loaded="0",thread-group="i1"
=library-loaded,id="/cygdrive/c/WINDOWS/System32/SYSFER.DLL",target-name="/cygdrive/c/WINDOWS/System32/SYSFER.DLL",host-name="/cygdrive/c/WINDOWS/System32/SYSFER.DLL",symbols-loaded="0",thread-group="i1"
=library-loaded,id="/usr/bin/cygwin1.dll",target-name="/usr/bin/cygwin1.dll",host-name="/usr/bin/cygwin1.dll",symbols-loaded="0",thread-group="i1"
=library-loaded,id="/cygdrive/c/WINDOWS/system32/psapi.dll",target-name="/cygdrive/c/WINDOWS/system32/psapi.dll",host-name="/cygdrive/c/WINDOWS/system32/psapi.dll",symbols-loaded="0",thread-group="i1"
=thread-created,id="2",group-id="i1"
~"[New Thread 7244.0x1ba8]\n"
*running,thread-id="all"
=breakpoint-modified,bkpt={number="1",type="breakpoint",disp="del",enabled="y",addr="0x00000001004010dd",func="main",file="hello.c",fullname="/cygdrive/c/scratch/hello.c",line="5",thread-groups=["i1"],times="1",original-location="main"}
~"\nTemporary breakpoint "
~"1, main () at hello.c:5\n"
~"5\t printf(\"Hello world\n\");\n"
*stopped,reason="breakpoint-hit",disp="del",bkptno="1",frame={addr="0x00000001004010dd",func="main",args=[],file="hello.c",fullname="/cygdrive/c/scratch/hello.c",line="5"},thread-id="1",stopped-threads="all"
=breakpoint-deleted,id="1"
(gdb)
Does this mean there's a bug in how Eclipse Kepler interacts with gdb? I don't know what this means, or where to go from here.
Answering my own question because I figured it out through much trial and error, and it didn't involve mucking with gdb-mi.
In short, I found no way to have Eclipse debug an executable that is not associated with an existing project. I had to create a new project of type "Makefile project with existing code" with this new project's home directory somewhere that does not already have the .project and .cproject files, in my case a subdirectory of my original cross-compiled project which is a CCS project. Then, in the new Eclipse project's debug configuration, point it to where to find the source code.
Eventually I got this working in Code Composer Studio. The full explanation is the last post I made in this thread: http://e2e.ti.com/support/development_tools/code_composer_studio/f/81/p/375037/1323811.aspx#1323811
I'm not including the full explanation in this answer because my question here is about regular Eclipse, while the answer is about the CCS IDE.

Linux debugging with Jtag - [ARM9][AT91SAM9G25] - Amontec, openocd, gdb, eclipse

I'm trying to start kernel debugging with this sytem:
Amontec JTAGkey2, openocd, gdb, eclipse.
At the end I would like to debug kernel and application that is running within.
I have few problems, and it seems that I need to solve them sequently.
Now I have CPU suspend/resume, read/write RAM
What is missing: Step into, Step over, C/C++ Level debugging.
I do following:
- Connect JTAG, Power up board, start uImage with Debug messages via Uboot
- start openocd:
# openocd -f /usr/share/openocd/scripts/interface/jtagkey2.cfg -f /usr/share/openocd/scripts/board/at91sam9g20-ek.cfg
Output:
jtag_nsrst_delay: 200
jtag_ntrst_delay: 200
RCLK - adaptive
TapName | Enabled | IdCode Expected IrLen IrCap IrMask Instr
---|--------------------|---------|------------|------------|------|------|------|---------
0 | at91sam9g20.cpu | Y | 0x00000000 | 0x0792603f | 0x04 | 0x01 | 0x0f | 0x0f
Info : max TCK change to: 30000 kHz
Info : RCLK (adaptive clock speed)
Info : JTAG tap: at91sam9g20.cpu tap/device found: 0x0792603f (mfg: 0x01f, part: 0x7926, ver: 0x0)
Info : Embedded ICE version 6
And problems starts here:
openocd:
Warn : acknowledgment received, but no packet pending
undefined debug reason 6 - target needs reset
Warn : target not halted
eclipse:
symbol-file /opt/Tixi_Repos/KiwiG6v2/buildroot-2011.05/package_tixi/linux-2.6.39/arch/arm/boot/compressed/vmlinux
target remote localhost:3333
start () at arch/arm/boot/compressed/head.S:108
108 kphex r5, 8 /* end of kernel */
It seems also that JTAG is trying to load the code into 0x0, what is incorrect I suppose:
Update 1:
After analyzing some online tutorials for ARM:
Eclipse Reset and Halt commands doesn't work perfect. It is better to uncheck them and write into command window. Also load address can be add:
monitor halt
load arch/arm/boot/compressed/vmlinux 0x22000000
I don't use
monitor reset
I let Uboot start and initialize RAM and other peripherals. Then I stop Uboot by getting into shell. Then I let eclipse write linux into RAM, and start it. It takes very long, but works bit better. Kernel starts and stopps on RPC initialization without giving console back.
would it be possible to load kernel into RAM within Uboot console, and start JTAG session afterwards ?
What is the difference between [load ...] and [monitor load...] commands
Why do I need to load /compressed/vmlinux instead of uImage ?
in eclipse window I have two load fields: load image i load symbol. I disable both options but write only load arch/arm/boot/compressed/vmlinux 0x22000000. Is it maybe the reason for next problems ?
Update 2:
Ok. Thank you for hints.
I've made some progress. Could you give me some advices, maybe I'm still doing something wrong.
Now my kernel runs under JTAG control, but I still can't debug on source code level.
I do as follows:
Power up the board, go into uboot shell.
start openOCD session
Set Uboot breakpoint in bootm.c on theKernel call:
cleanup_before_linux ();
theKernel (0, machid, bd->bi_boot_params);
start eclipse debug session :
monitor halt
load uboot-a without offset
load u-boot-2010.06/u-boot
Loading section .text, size 0x349ec lma 0x26f00000
start uboot and let it run
uboot stopps on "theKernel" call
I know that kernel is located on address 0x20008000.
restart openOCD session
start ecipse debugger once more with kernel configuration:
monitor halt
load kernel on address 0x20008000
load arch/arm/boot/compressed/vmlinux 0x20008000
Loading section .text, size 0x8bdc7c lma 0x20008000
start debugg session
Everything works fine now, and kernel starts, but I still can't debug on source code level.
"symbol is not available"
DEBUG and DEBUG_INFO are on for kernel.
vmlinux screenshot
What seems starnge for me that there are around 50 function symbols in this file.

Debug Linux kernel pre-decompression stage

I am trying to use GDB to debug a Linux kernel zImage before it is decompressed. The kernel is running on an ARM target and I have a JTAG debugger connected to it with a GDB server stub. The target has to load a boot loader. The boot loader reads the kernel image from flash and puts it in RAM at 0x20008000, then branches to that location.
I have started GDB and connected to the remote target, then I use GDB's add-symbol-file command like so:
add-symbol-file arch/arm/boot/compressed/vmlinux 0x20008000 -readnow
When I set a breakpoint for that address, it does trap at the correct place - right when it branches to the kernel. However, GDB shows the wrong line from the source of arch/arm/boot/compressed/head.S. It's 4 lines behind. How can I fix this?
I also have tried adding the -s section addr option to add-symbol-file with -s .start 0x20008000; this results in exactly the same problem.
There are assembler macros that print out stuff when compiling with low level debug. You have to make sure the macros are appropriate for your board.
linux-latest/arch/arm$ find . -name debug-macro.S | wc
56 56 2306
Find the file for your board and ensure the correct serial port registers are hit. You can instrument the code with out using JTAG. These macros are used in the decompress code. Of course configure with *CONFIG_DEBUG_LL*.
Most likely the ATAGs are not correct or one of the other requirements. Checkout Documentation/arm/Booting to make sure you have registers set properly. Note there is a new requirement with recent kernels to send a dt list.

How to 'reload' source files in GDB

Is there a command in gdb that I can use to (re)load / "refresh" source files? (As far as I can see, gdb works only with source directories, according to Debugging with GDB: Source - and there is no specific command to "refresh")
Background about my problem:
I use a virtual machine with a debug kernel, so I can connect to a local instance of gdb, and can debug kernel modules. The modules are compiled with debug info on, and this specifies folders where the source of the modules is kept (Instruct GDB 6.5 to use source embedded in object file - Stack Overflow). I have the source directories in the same path(s) in both VM and local machine.
The problem is this - I need to do quite a bit of steps in order to get the module to segfault - and the remote gdb to go into the stack. Then I do a backtrace, and I can see the source files referenced, i.e.
#0 0xc0132a13 in ?? ()
#1 0xc056e551 in ?? ()
#2 0xc056e506 in ?? ()
#3 0xd8be53f3 in mymodule_func1 (var1=0xd79f9b44, var2=0x0, var3=825269148)
at /media/src/mymodule.h:954
#4 0xd8be53d0 in mymodule_func2 (data=3617561412)
at /media/src/mymodule.h:936
#5 0xc014fe87 in ?? ()
#6 0xc0151478 in ?? ()
Then I try to do say, list /media/src/mymodule.h:954 - and I realize that I have changed stuff on the local version of mymodule.h file!!
So I undo the changes - but unfortunately, GDB does not see these changes! And, of course, I don't want to restart GDB - because that means I have to restart the VM, and go through the entire procedure in order to get the kernel module to segfault again :( !!
Then I try to do something like this:
(gdb) show symbol-reloading
Dynamic symbol table reloading multiple times in one run is off.
(gdb) set symbol-reloading on
(gdb) add-symbol-file ~/mymodule.o 0xd8be4000
add symbol table from file "/media/src/mymodule.o" at
.text_addr = 0xd8be4000
(y or n) y
Reading symbols from /media/src/mymodule.o...done.
... in hope that it will somehow "reload" the source files - but unfortunately, list /media/src/mymodule.h:954 shows that it doesn't, nothing is changed - even though gdb does recognize that something has changed, as in warning: Source file is more recent than executable.... (so, for the time being, I have to restart entire VM and gdb as well :( :( )
Resetting the directory list using the directory command appears to have the desired effect.
From https://www.cs.rochester.edu/~nelson/courses/csc_173/review/gdb.html:
After changing program, reload executable with file command
(gdb) file gdbprog
A program is being debugged already. Kill it? (y or n) y
Load new symbol table from "gdbprog"? (y or n) y
Reading symbols from gdbprog...
done.
Breakpoint 1 at 0x2298: file gdbprog.cc, line 10.
(gdb) run
Starting program: gdbprog
Breakpoint 1, InitArrays (array=0x18be8)
at gdbprog.cc:10
10 for(i = 0;i < 10;i++)
This warning means source files from which binary was made are updated with new changes.
To remove this warning just rebuild the binary you are debugging with new and modified files.

Resources