How to debug debugging using Clion -> armgdb -> J-link gdb server - debugging

I am trying to use Clion IDE to debug various NXP ARM M MCUs using J-link.
In case when program is only in RAM it works fine but but in case of flash targets after program load IDE often slows down for a while and then I will get timeout. Some click on pause button in right moment pauses program somewhere and allows me to debug...
My question is there any way how to see interactions of IDE with armgdb client or armgdb and j-link gdb server?
I tried to capture communication with gdb server using wire shark but it seems to be a binary protocol...
Thank you.

I found there is possibility to let gdb echo commands which showed me what is going on.
Unfortunately in case of Clion the only way is to put it into .gdbinit file in the root of project.
set trace-commands on
It showed me warning
warning: A handler for the OS ABI "Windows" is not built into this configuration
of GDB.
Which leads me to comment of this question gdb-multiarch (MINGW64) cannot determine architecture from executable?
Well I am not sure what caused Clion to hang but having .gdbinit with just set osabi none in it solved it.

Related

ESP32 JTAG debugging with VSCode

recently I have bougth a ESP32 devkit and a low budget FT4232H jtag adapter which I managed to setup in VSCode using the Espressif-idf plugin.
Basically it seems to work so far. I can build, flash and monitor the ESP32 from VSCode. Also also managed to setup the debug configuration, but I am still missing some features in while debugging.
I can step through the code, watch variables and see the call stack.
But I can't see periphals, registers or memory
I guess I have to set some more options in GDB, OpenOCd or even ESP32 config but I don't know which ones.
Any ideas were i have to dig?
You appear to be almost there.
I have a very similar interface with the memory loading afterwards but only after stepping into certain operations.
https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/jtag-debugging/debugging-examples.html
Following this document produced by espressif. Memory can be accessed and stepped into using the keyboard shortcut (x) when using openOCD.
Give that a try.

How to debug bootloader code running on RAM for powerPC

I'm developing a bootloader for the MPC5748G in C.
The bootloader will be running on RAM so i want to know how can i debug the bootloader code running on the MPC Target.
Thanks
Without further information on the platform you are using, the general guideline is that you compile your Bootloader software with Debug symbols, and once loaded, start a debugging session which should be configured to attach to the process (instead of the classical load-then-break-at-main). Then, hitting pause should interrupt the target, and from here on it is a normal debugging session.

Issue with qemu and gdb

I have a device driver that is freezing the OS. The mouse wont even move. I am trying to debug this issue and I believe one good approach is to use gdb with qemu, two things I have never used before. Is there a better approach?
So first I need to compile the kernel with debug symbols which I have done already.
Now, there is a new file that is generated called vmlinux that is located in that same folder as the source. It seems that I also need a bzImage file according to this so I can run the newly compiled kernel using:
qemu-system-i386 -kernel bzImage
or in debug mode
qemu-system-i386 -s -S -kernel bzImage
I cannot locate the bzImage file. Where do I find it or what is missing here? Is the bzImage referring to the OS Image I created using qemu-img create?
Also, what I do not understand is that now the kernel is compiled (vmlinux) how does I run it with qemu? So my question is when I run it with qemu or the debugger is the kernel running as an app in my main OS?
also how can I install my device driver? My understanding the kernel is not Ubuntu so there is no UI?
Also, I installed qemu and when I type qemu I get command not found. I am guessing I have to pick a specific processor emulator as in qemu-system-i386, qemu-system-x86_64, or qemu-x86_64?
How is qemu different or similar to the kvm command?
Thanks.
So, if I understand the problem correctly, you have a kernel module that needs no specific hardware. When you are working with the module, the system freezes but the kernel log contains nothing special.
The following may be helpful.
Getting the log
The symptoms you described may still be a result of a kernel oops or panic. The logging facilities sometimes die before they can output the information about the error to the log file. You may try to output the log via a serial port, this should be more reliable.
As your kernel module does not need any specific hardware, the easiest way is probably to install the same Linux distro as you use to a virtual machine and connect the virtual serial port (COM) of that machine to a pipe on your host system.
This is usually quite easy to do. For example, this blog post contains the detailed instructions in case the host OS and the guest OS are Ubuntu 11.10.
VirtualBox is used there to manage the virtual machines. If you prefer QEMU, this should be possible as well. I suppose it is a bit easier to go with VirtualBox though but it is a matter of personal preference.
Basically, you need to perform the following steps.
Create a virtual machine and install the Linux distro you need as a guest OS there.
Enable a serial port (COM1, ...) in the configuration of the virtual machine and configure it to connect to a special file on the host ("host pipe"), say /tmp/vbox_serial.
Start the guest OS and adjust its boot options: at least, add console=ttyS0,115200 or something like that to the kernel options in the boot loader menu.
On the host, start minicom, socat or whatever else to read from /tmp/vbox_serial.
That is it. Now you should get the kernel log of the guest OS pouring to your host system via /tmp/vbox_serial. If the guest system crashes then, you will get the log even if it is not saved into a file on the guest itself.
To make things easier, you may use socat on your host system rather than minicom that the author of that blog post suggests. The power of minicom is probably not needed here.
This way, you can use socat and tee to save the log to guest.log file while still outputting it to the console:
socat /tmp/vbox_serial - | tee guest.log
If there was a kernel oops or panic, the backtrace in the log usually helps to find out what
has gone wrong.
Detecting Deadlocks
If you have obtained the full log via a serial connection or some other means and still there is nothing suspicious there and you suspect there has been a deadlock in the kernel,
lockdep tool may help. It is included into the kernel (but you may need to rebuild the kernel with CONFIG_LOCKDEP_SUPPORT=y).
Lockdep detects the potential deadlocks and outputs the results to the kernel log. This presentation may help you analyse its output.
Tracing Facilities
If you need tracing of some events in the kernel to debug your system, there are some tools that could be handy.
Kprobes - a kind of breakpoints you can set in almost arbitrary place in the kernel. Can be used to trace function calls among other things, with a moderate performance impact.
SystemTap - a powerful system to analyze what is going on in the kernel. Part of it is based on Kprobes.
Ftrace - a tracing system included into the kernel, incurs less overhead than Kprobes if that matters.

Developing and Debugging KEXT on mac

I'm developing a KEXT on mac using Xcode, After every compile I'm changing permissions through terminal and load the KEXT, then reading results from console app. Some of the mistakes in development giving system a kernel panic and I have to restart my mac, this is so annoying. I was wondering if there is a better way to develop and debug a KEXT?
This is too big a topic for an answer, but it is at least well documented, look at these documents from Apple:
When Things Go Wrong: Debugging the Kernel
Debugging a Kernel Extension with GDB
Technical Note TN2063: Understanding and Debugging Kernel Panics
Also note that you can get the output from kprintf() logging calls via Firewire (using the fwkpfv command-line utility on the other Mac) or Serial Port (mainly useful for testing in VMs, as modern Macs don't have serial ports). kprintf is synchronous, so unlike the kernel.log you will see the debug output even if it occurs immediately before a crash.

Low level qemu based debugging

I've to test some low level code on an ARM architecture. Typically experimentation is quite complicated on the real board, so I was thinking about QEMU.
What I'd like to get is some kind of debugging information like printfs or gdb. I know that this is simple with linux since it implements both the device driver for the QEMU Integrator and the gdb feature, but I'm not working with Linux. Also I suspect that extracting this kind of functionality from the Linux kernel source code would be complicated.
I'm searching from some simple operating system that already implements one of those features. Do you have some advice?
You don't need a target OS to debug code that's running inside QEMU -- QEMU already does that for you.
Specifically, QEMU supports remote debugging from GDB -- you can run QEMU with the appropriate command-line options and it will export an interface that a copy of GDB (running on the host machine) can connect to. At that point, you can debug the program in GDB pretty much just as if you were running it on the host machine.
http://wiki.osdev.org/GDB appears to have a bit more basic information; possibly not enough to completely get you started, but at least give you the basic idea and some terms to look for in the QEMU and GDB documentation. Skip over the bit about "Implementing GDB Stubs", which doesn't apply here since QEMU has one already, and start at the section on "Using Emulator Stubs". The short form is simply that you start QEMU with the -s option (export a GDB connection on localhost:1234) and the -S option (wait for a GDB "continue" command before starting execution), and then in GDB on your host you say target remote :1234 instead of run. Also, of course, you need to be using an ARM version of GDB rather than a native-x86 one.
(In addition, if you're willing to pay for a commercial solution, CodeSourcery's ARM toolchain has the IDE integration to set all of this up automatically, including support for "printf" to print into the debugger console. That works on a physical board, too, if you've got a hardware debugger. Usual disclaimer about me being a CodeSourcery employee applies -- but I do find it very easy to use.)
Update, 2012: CodeSourcery's toolchain is now called Mentor Graphics Sourcery CodeBench, but all the above still applies.
I realise that I am addressing your original problem here rather than your proposed solution (perhaps that's better?), but to use GDB (or Insight/GDB) directly on the target, use a low-cost JTAG tool and OpenOCD. An example of such a set-up and how to implement it can be found here.
If you have a larger budget, a more fully featured JTAG debugger may be useful, such as the Abatron BDI3000 with bdiGDB firmware which allows remote debugging and device programming over Ethernet with GDB and no special drivers or target debug agent.
Maybe a microkernel like OKL4 would suit your needs?

Resources