Breakpoints not being hit in remote linux kernel debugging using gdb - linux-kernel

I am trying to remotely debug a linux kernel running on an arm cortex-a9 target using jtag probe and gdb.
I can connect to the kernel and halt it with gdb. I am able to set breakpoints in the kernel code and gdb confirms there placement as well but the problem is that once I start the execution and issue a continue command, the breakpoints are never hit and the kernel continues to run....
Please help me out in this regard.
Thanks.

As pointed in this thread, you should set the breakpoints as hardware breakpoints, namely - using the hbreak command. Only then the breakpoints will be hit.

For anyone reading this, the debugger will not break with software breakpoints by default, see the relevant doc:
"If the architecture that you are using supports the kernel option CONFIG_STRICT_KERNEL_RWX, you should consider turning it off. This option will prevent the use of software breakpoints because it marks certain regions of the kernel’s memory space as read-only. If kgdb supports it for the architecture you are using, you can use hardware breakpoints if you desire to run with the CONFIG_STRICT_KERNEL_RWX option turned on, else you need to turn off this option."
at https://www.kernel.org/doc/html/v4.14/dev-tools/kgdb.html
Disable RWX and recompile, then software breakpoints should work (they started working normally here after this)

Is some cases, KASLR (Kernel Address Space Layout Randomization) can be the culprit.
Even though you setup hbreak, the actual code location can be different from the address seen from the .elf file when using KASLR, so either pass --append "nokaslr" to the kernel boot argument, or configure the kernel with RANDOMIZE_BASE=n. This applies for arm64 and x86_64. (maybe other architectures too).

Related

Any way to set a breakpoint in GDB without stopping the target?

I'm using GDB (through an IDE) to debug an arm cortex microcontroller, and I've encountered an issue where the micro is halted briefly (for 10-20ms) whenever a breakpoint is set or cleared (not hit, but set, as in the code has not yet reached the breakpoint). This long of a pause can cause significant problems when driving an electric motor, for example.
The IDE has a debug console which shows that the GDB client is sending a SIGINT to the GDB server whenever I add or remove a breakpoint. I know that in the command-line client you have to use ctrl+c to interrupt the process to issue any command, but for modern microcontrollers (ARM cortex-m, etc.) it is not necessary to interrupt the processor to insert breakpoints, read memory, and in some cases to trace program execution. I am wondering if this is something that is being imposed by the GDB interface artificially.
Is there any way to create a new breakpoint without halting the target?
I have tried using "async" mode in GDB, but it informs me that I must halt the program to insert breakpoints. I have also verified that breakpoints can be set with the underlying debug server (OpenOCD) without halting, so in this case GDB is incorrect.
Any input is appreciated, and thanks in advance.

how to set a memory breakpoint in windbg kernel mode?

I would like to set a memory breakpoints on access in windbg in the kernel mode debugger
I want the debugger breaks everytime a specific module in usermode is hit with the kernel debugger.
but I've read somewhere its impossible to set it, in order to make a memory breakpoints I have to write a plugin to make it
I tried to use SDbgExt plugin with the !vprotect command, but it fails to set memory bp
If I have to write a plugin to allow memory bp in kernel mode It has to be a driver?
I've read some chapters in windows internals book, but it doesn't help me at all.
I couldn't find too much info how to start deal with it
You can set breakpoints on user mode addresses from kernel mode. The only thing you should take care is to switch to the right process with ".process /i " command
If it is a one-off breakpoint -- that is, you are content with process being destroyed by debugging -- zero out the entire module using e command (edit memory). Set the whole thing to cc (which is int 3 as far as I remember)... zeros will do as well. You will break as soon as you touch any of the module's code.
Next step, remember where you were (relative to the module) and set a proper breakpoint.
Hope that helps.
(editing) Do you have full symbols? If you do, did you try bm module!*
Sounds like you want to set a "breakpoint on access" but instead of specifying an address you want to specify a range? I have never seen it done in windbg. The BA breakpoints uses HW debug registers instead of inserting INTs like SW breakpoints so this is definitely HW platform specific.
I have done this on an ARM chipset once using a HW debugger. ETM on ARM allows you to set triggers on address ranges.

GDB does not break on a read access to an address caught by PIN

I've captured a read memory trace (i.e. addresses of all read accesses) of a program-run by Intel PIN, ASLR was off. I can capture it several times and the trace is still exactly the same.
Then I take an address from the trace (a specific address I am interested in), set a watchpoint in GDB by rwatch *0x7fffe309e643 but GDB does not notice the access. I'm doing this in the same terminal where the trace has been taken so it should access the same addresses; when I capture it then again, the trace is still very the same.
Do you have any ideas or hints why GDB does not catch it?
I have to note that the access very likely happens within a (C++) structure loaded from a Boost serialization and that the code has been compiled with -g3 -O1 flags (-O1 is really crucial for other reasons but a minimal example showed me that GDB breaks correctly even with -O1).
UPDATE:
As suggested, I tried PIN debugger (which actually utilizes GDB) but there is some problem with watchpoints. To present this problem, I made a minimal example.
int main()
{
unsigned char i = 4;
i++;
i = 3;
return 0;
}
I acquired a read memory trace with PIN and chose 0x7fffffffd89f (or similar) as the address of interest. Then I started PIN debugger and GDB in parallel terminals and connected GDB to PIN ((gdb) target remote :57946, as suggested by PIN). In GDB I set watchpoint (gdb) rwatch *0x7fffffffd89f and started execution but I get
Continuing.
Warning:
Could not insert hardware watchpoint 1.
Could not insert hardware breakpoints:
You may have requested too many hardware breakpoints/watchpoints.
even though I only inserted a single watchpoint. I also tried to explicitly disable hardware breakpoints by set can-use-hw-watchpoints 0 but then it ran for an hour with no result and no processor load so I killed it.
Is there any other way how to set a functional watchpoint at a specific address when GDB is connected to PIN debugger? Note that there is no such problem with standalone GDB.
I use GDB 7.7.1, PIN 2.14-71313 and GCC 4.4.7, running Ubuntu 14.04.
The original problem would be likely solved then.
From pin user guide:
The "pin" Executable (Launcher)
The kit's root directory contains a "pin" executable. This is a 32-bit
launcher, used for launching Pin in 32 and 64 bit modes. The launcher
sets up the environment to find the libraries supplied with the kit.
It is likely that the environment is different when running pinbin, and therefore stack addresses will also be different (0x7fffe309e643 looks like a possible stack address) even without ASLR.
Your best bet is probably to use Pin application debugging interface.

.ASM file debugging tool

I am wondering which debugging tool I can use for an assembly program and how to use it.
I have written a simple bootloader in assembly. However, it is not quite working properly as I wished, even though I think the logic is correct. So, I am trying to use a debugger so that I can step through the bootloader, checking the register status and etc.
I tried GDB on Ubuntu, compiling my .asm to .elf and .o (Do I need to do it? If yes, what is the next step?) Also, I read that there is an internal debugger in Bochs simulator, but I can't quite find any document how to use it. I also have Visual Studio 2010, windbg, but I don't know how to use it for .asm file debugging.
If you have done this before, it would be an easy answer. Any help would be really appreciated.
Sincerely
If you want to debug bootloader code, you obviously need to run it in the same environment that the code itself is going to run in. As I'm sure you already know, bootloader code is executed in real mode once the BIOS finishes doing the POST. The bootloader is then loaded into memory at 7c00h and a jump to that address is executed.
Obviously, this kind of environment cannot be reliably emulated once you've got your computer running and a "real" operating system already loaded, since by that time your CPU is in protected mode (or long mode, if it's AMD64). Your only option at this point is to use QEMU or Bochs in order to emulate a real PC inside your operating system. I've used Bochs to debug some bootloader code I've written in the past and it worked quite well. Check the manual pages for more detailed instructions.

Tool to Debug Guest OS in Virtual Box

I'm just cross posting the same question I did on virtualbox.org. http://forums.virtualbox.org/viewtopic.php?f=9&t=26702&p=119139#p119139
If not breaking any rule, I'd appreciate to kwon more about it since stackoverflow promisses to be more dynamic!
"Hi,
I did some search and could not find any tool to debug a guest system from the early boot in virtual box. Although, I came across JCP, a x86 emulator in java that is not so powerful and beautyful but has a debug mode where one can view the Physical Memory, the CPU registers along other things. It also makes it possible to execute CPU instructions step by step and set break points, watchpoints and conditional ones. There's such thing in Virtual Box?
I think would be amazing to have it and be able to inspect the system while its running. For learn about PC architecture or as a tool to develop a kernel as well.
In the case you think its good idea (I think it is) how can it be achieved? I'm interested in develop such sort o things and would like to know if it is feasible if not already implemented somewhere."
EDT: Are modern x86 able to interrupt its execution just after a cpu cycle and pass execution addres to another code to just do this? Yes, the trap flag can be set to put the processor in step by step execution mode. x86 will execute one instruction and call INT 3.
Contrary to what is stated above, VirtualBox now contains a (limited) debugger. Add --dbg to the command line when starting the VM. For more information consult:
12.1.3. The built-in VM debugger
The OSDev wiki has some useful information on debugging a guest operating system, though according to this page VirtualBox doesn't have a debugger at present. I've been using QEmu with the GDB stub and it works quite nicely, so you might like to give that a go instead.

Resources