I'm having a larger OpenCL project here. Currently I'm printing debug messages to the console.
It would be nice to have this Debug messages in a file.
Anyone has an idea how to get the printstream of the OpenCL kernel and deflect it to a file?
Or maybe a better idea for handling debug messages?
The OpenCL specification states that the output of printf is sent to an implementation-defined output stream. There is no way to programatically control where this output goes.
If you are running the program from a terminal/console environment, you can achieve the effect you want by including a unique string inside your OpenCL printf calls and filtering the output when you run the program. For example, in your kernel code, you could have printf calls like this:
kernel void foo(...)
{
...
printf("OCL: ...", ...);
...
}
When you run the program, you can then redirect just the OpenCL printf calls to a file with a command like this:
(Unix)
./foo | grep "^OCL: " >ocl_debug.txt
(Windows)
foo | findstr "OCL: " >ocl_debug.txt
As output stream of OpenCL is implementation-defined, you have 2 options to get it into file. First is already proposed by jprice, second is to redirect all output into file
./opencl_app &>./file.txt
Let me ask a question: order of printf outputs isn't guaranteed to be same to order of it's calls. Is it really helpful in big project debugging? If you're using printf, my guess is that your platform is AMD or Intel. Both vendors provide handy debuggers, which can be used inside the kernel.
Related
Consider the following Linux kernel dump stack trace; e.g., you can trigger a panic from the kernel source code by calling panic("debugging a Linux kernel panic");:
[<001360ac>] (unwind_backtrace+0x0/0xf8) from [<00147b7c>] (warn_slowpath_common+0x50/0x60)
[<00147b7c>] (warn_slowpath_common+0x50/0x60) from [<00147c40>] (warn_slowpath_null+0x1c/0x24)
[<00147c40>] (warn_slowpath_null+0x1c/0x24) from [<0014de44>] (local_bh_enable_ip+0xa0/0xac)
[<0014de44>] (local_bh_enable_ip+0xa0/0xac) from [<0019594c>] (bdi_register+0xec/0x150)
In unwind_backtrace+0x0/0xf8 what does +0x0/0xf8 stand for?
How can I see the C code of unwind_backtrace+0x0/0xf8?
How to interpret the panic's content?
It's just an ordinary backtrace, those functions are called in reverse order (first one called was called by the previous one and so on):
unwind_backtrace+0x0/0xf8
warn_slowpath_common+0x50/0x60
warn_slowpath_null+0x1c/0x24
ocal_bh_enable_ip+0xa0/0xac
bdi_register+0xec/0x150
The bdi_register+0xec/0x150 is the symbol + the offset/length there's more information about that in Understanding a Kernel Oops and how you can debug a kernel oops. Also there's this excellent tutorial on Debugging the Kernel
Note: as suggested below by Eugene, you may want to try addr2line first, it still needs an image with debugging symbols though, for example
addr2line -e vmlinux_with_debug_info 0019594c(+offset)
Here are two alternatives for addr2line. Assuming you have the proper target's toolchain, you can do one of the following:
Use objdump:
locate your vmlinux or the .ko file under the kernel root directory, then disassemble the object file :
objdump -dS vmlinux > /tmp/kernel.s
Open the generated assembly file, /tmp/kernel.s. with a text editor such as vim. Go to
unwind_backtrace+0x0/0xf8, i.e. search for the address of unwind_backtrace + the offset. Finally, you have located the problematic part in your source code.
Use gdb:
IMO, an even more elegant option is to use the one and only gdb. Assuming you have the suitable toolchain on your host machine:
Run gdb <path-to-vmlinux>.
Execute in gdb's prompt: list *(unwind_backtrace+0x10).
For additional information, you may checkout the following resources:
Kernel Debugging Tricks.
Debugging The Linux Kernel Using Gdb
In unwind_backtrace+0x0/0xf8 what the +0x0/0xf8 stands for?
The first number (+0x0) is the offset from the beginning of the function (unwind_backtrace in this case). The second number (0xf8) is the total length of the function. Given these two pieces of information, if you already have a hunch about where the fault occurred this might be enough to confirm your suspicion (you can tell (roughly) how far along in the function you were).
To get the exact source line of the corresponding instruction (generally better than hunches), use addr2line or the other methods in other answers.
When I was debugging my .c file using lldb on terminal for Mac, I some how cannot find the location of the segmentation fault. I have debugged the code numerous of times and it is still producing the same error. Can someone help me on why I can find the location of segmentation fault. enter image description here
Use the bt command in lldb to see the call stack. You've called a libc function like scanf() and are most likely passing an invalid argument to it. When you see the call stack, you will see a stack frame with your own code on it, say it is frame #3. You can select that frame with f 3, and you can look at variables with the v command to understand what arguments were passed to the libc function that led to a crash.
Without knowing what your code is doing, I would suggest using a tool like valgrind instead of just a normal debugger. It's designed to look for memory issues for lower-level languages like C/C++/FORTRAN. For example, it will tell you if you're trying to use an index that is too large for an array.
From the quick start guide, try valgrind --leak-check=yes myprog arg1 arg2
Idea is to use argdist to measure latency duration of napi_poll() which returns number of packet processed (called work). Ratio of execution latency of napi_poll() to number of packets processed would give me average amount of time it took to process each packet in form of histogram.
I am using following command
argdist -H 'r:c:napi_poll():u64:$latency/$retval#avg time per packet (ns)'
which end up giving me error Failed to attach BPF to kprobe and in dmesg I get message like Could not insert probe at napi_poll+0: -2
I am just curios why I can not attach kretprobes to napi_poll() when similar trick works with net_rx_action() ?
Most of the time the Failed to attach BPF to kprobe error is caused by an inlined function. As explained in the Kprobes documentation (section Kprobes Features and Limitations), Kprobes will fail to attach if the target function was inlined. Since napi_poll is static, it might have been inlined at compile time.
You can check in kernel symbols if napi_poll was inlined or not:
$ cat /boot/System.map-`uname -r` | grep " napi_poll"
$
$ cat /boot/System.map-`uname -r` | grep " net_rx_action"
ffffffff817d8110 t net_rx_action
On my system, napi_poll is inlined while net_rx_action is not.
There are several workarounds for this problem, depending on your goal.
If you don't mind recompiling your kernel, you can use the Linux inline attribute to ensure napi_poll is not inlined.
If you can't change your kernel, the usual workaround is to find a calling function of napi_poll that provides the same information. A function called by napi_poll can also work if it provides enough information and is not inlined itself.
During bootup of the target board we see uboot (bootloader) printing some information such as Image name, Image type, Load Address, Verifying checksum etc on the console. Which printing mechanism does it use? Does it use something like printk or it has its own definition for printing info even before kernel boots up? Where can I find the code for its printing implementation?
In the normal U-boot boot process,a limited amount of information is printed to the console. It use the the same kind of functions to print information as we use during the C programming.
u-boot use printf and puts to print the information on the console. you can find the same function implementations in the u-boot source code (u-boot boardfile and drivers).
There are a lot of commands which you can try from u-boot command prompts for more information.
To enable more messages you can either:
Using debug_cond (cond, fmt, args...): if you define some cond, once it is met, the U-boot will print out this message.
Using debug(fmt, args...): you can define DEBUG in the file u-boot-include/configs/<boardfile>_common.h (like in my case mx6_common.h), once do that and recompile the code, the U-boot will print out all debug message
Note: If you put too much debug into the code, it might make u-boot hang up.
You can enter uBoot when you startup (interrupting the startup of the kernel) if you want information about uBoot for example where it prints to or what the values are of it's variables you can use the 'printenv' command and change them with the 'setenv' command.
We've been using CygWin (/usr/bin/x86_64-w64-mingw32-gcc) to generate Windows 64-bit executable files and it has been working fine through yesterday. Today it stopped working in a bizarre way--it "caches" standard output until the program ends. I wrote a six line example
that did the same thing. Since we use the code in batch, I wouldn't worry except when I run a test case on the now-strangely-caching executable, it opens the output files, ends early, and does not fill them with data. (The same code on Linux works fine, but these guys are using Windows.) I know it's not gorgeous code, but it demonstrates my problem, printing the numbers "1 2 3 4 5 6 7 8 9 10" only after I press the key.
#include <stdio.h>
main ()
{
char q[256];
int i;
for (i = 1; i <= 10; i++)
printf ("%d ", i);
gets (q);
printf ("\n");
}
Does anybody know enough CygWin to help me out here? What do I try? (I don't know how to get version numbers--I did try to get them.) I found a 64-bit cygwin1.dll in /cygdrive/c/cygwin64/bin and that didn't help a bit. The 32-bit gcc compilation works fine, but I need 64-bit to work. Any suggestions will be appreciated.
Edit: we found and corrected an unexpected error in the original code that caused the program not to populate the output files. At this point, the remaining problem is that cygwin won't show the output of the program.
For months, the 64-bit executable has properly generated the expected output, just as the 32-bit version did. Just today, it has started exhibiting the "caching" behavior described above. The program sends many hundreds of lines with many newline characters through stdout. Now, when the 64-bit executable is created as above, none of these lines are shown until the program completes and the entire output it printed at once. Can anybody provide insight into this problem?
This is quite normal. printf outputs to stdout which is a FILE* and is normally line buffered when connected to a terminal. This means you will not see any output until you write a newline, or the internal buffer of the stdout FILE* is full (A common buffer size is 4096 bytes).
If you write to a file or pipe, output might be fully buffered, in which case output is flushed when the internal buffer is full and not when you write a newline.
In all cases the buffers of a FILE* are flushed when: you call fflush(..). You call fclose(..) or the program ends normally.
Your program will behave the same on windows/cygwin as on linux.
You can add a call to fflush(stdout) to see the output immediately.
for (i = 1; i <= 10; i++) {
printf ("%d ", i);
fflush(stdout);
}
Also, do not use the gets() function.
If your real programs "ends early" and does not write data in text files that it's supposed to, it may be it crashes due to a bug of yours before it finishes, in which case the buffered output will not be flushed out. Or, more unlikely, you call the _exit() function, which will terminate the program without flushing FILE* buffers (in contrast to the exit() function)