Breakpoints in .cu files in Visual Studio 2013 work fine when using the "Local Windows Debugger". But when using nsight's "Start CUDA debugging" the breakpoints are neglected. How is this possible? At nsight's site they state: "Use the familiar Visual Studio Locals, Watches, Memory and Breakpoints windows". So I guess the normal breakpoints can be used?
Edit:
Enable CUDA Memory Checker: On/Off makes no difference
Generate GPU Debug Information: No/Yes (-G0) makes no difference
Start CUDA/Graphics debugging: breakpoints neglected
"Start CUDA debugging" debugs device (kernel) code, i.e. stuff compiled with nvcc -> bunch of preprocessing -> cudafe++ -> cicc toolchain path.
"Local Windows Debugger" debugs host code, a stuff compiled with either nvcc -> bunch of preprocessing -> cl or just cl.
It does not matter in which file,.cpp, .cu or .h your code is. The only thing that matters is if your code is annotated as __device__ or __global__ or not.
As of CUDA 7.5 RC (Aug 2015), on Windows you can only debug one of those at a time. On Linux and OSX you can debug both at the same time with cuda-gdb.
See also: NVIDIA CUDA Compiler Driver NVCC
Other things that could lead to frustration during debugging on Windows:
You are setting up properties for one configuration/platform pair, but running another one
Something went wrong with .pdb files for host and device modules. Check nvcc, cl, nvlink and link options. For example host and device debug info could be written in the same file, overwriting each other.
Aggressive optimizations: inlining, optimizing out locals, etc. Release code is almost impossible to debug for a human. Debugger can be fooled as well.
Presence of undefined behavior and/or of memory access violations. They can easily crash debugger leading to unexpected results, such as breakpoints not being hit.
You forgot to check errors for one of the CUDA API or kernel calls, there was error, and CUDA context is dead and kernels will not run anymore. But you don't know this yet. Your host code continues to run, and you expect kernel breakpoints to hit, but it will never happen, because kernel will just not be called.
All bugs described above could be in a library. Don't expect libraries to be bug-free.
Compilers, debuggers and drivers have bugs too. But you should always assume it's something wrong with your code first, and if nothing helps, investigate and file a bug report to a vendor.
Related
I'd like to know what sets the "Debug with Nsight" option apart from simply executing the binary through Visual Studio or the OS's command line.
The reason I ask is because my program works fine if I run it by "Debugging with Nsight", but I get a few unspecified cudaErrors with some cudaMemcpys following a driver crash when launching it with Visual Studio's launch button (or simply launching the executable), which leads me to believe that Nsight must have some kind of specific launch parameters necessary for the program to run correctly.
The driver crash followed by API errors occurs when your app hits a windows TDR event due to kernel execution taking too long. You can work around this by modifying the system registry, or putting a Quadro or Tesla GPU in TCC mode, or reducing the run-time of your kernel(s).
When you debug with nsight, your kernel execution may get halted for various reasons (single step, breakpoints, and other reasons), and then restarted, depending on what you are doing exactly in your debug session. The halting of the kernel execution allows the windows watchdog to be satisfied without a TDR event.
CUDA nSight debugger allows you to debug the CUDA kernels line by line, you can't do this with the standard Visual Studio debugger.
Presumably nSight performs some code injection to enable it to detect the runtime of kernels, its also possible on your settings that when debugging with nSight your kernels may not be executing on the GPU. These could be the cause of errors coming and going between debuggers. I know when I used them I had similar inconsistencies.
If you run your program through the nSight profiler it should be able to clearly log the memCpy errors for you.
I am wondering which debugging tool I can use for an assembly program and how to use it.
I have written a simple bootloader in assembly. However, it is not quite working properly as I wished, even though I think the logic is correct. So, I am trying to use a debugger so that I can step through the bootloader, checking the register status and etc.
I tried GDB on Ubuntu, compiling my .asm to .elf and .o (Do I need to do it? If yes, what is the next step?) Also, I read that there is an internal debugger in Bochs simulator, but I can't quite find any document how to use it. I also have Visual Studio 2010, windbg, but I don't know how to use it for .asm file debugging.
If you have done this before, it would be an easy answer. Any help would be really appreciated.
Sincerely
If you want to debug bootloader code, you obviously need to run it in the same environment that the code itself is going to run in. As I'm sure you already know, bootloader code is executed in real mode once the BIOS finishes doing the POST. The bootloader is then loaded into memory at 7c00h and a jump to that address is executed.
Obviously, this kind of environment cannot be reliably emulated once you've got your computer running and a "real" operating system already loaded, since by that time your CPU is in protected mode (or long mode, if it's AMD64). Your only option at this point is to use QEMU or Bochs in order to emulate a real PC inside your operating system. I've used Bochs to debug some bootloader code I've written in the past and it worked quite well. Check the manual pages for more detailed instructions.
I want to run the Visual Studio debugger on a .Net program that has been obfuscated. (It's my own program - I'm not trying to crack someone else's program.)
When I compile the program with "System.Diagnostics.Debugger.Break()", or if I attach the debugger to the running process, then the debugger shows "No Source Available". This I understand - there is no source that corresponds to the obfuscated version of the program. But when I click "Show Disassembly" it shows a Disassembly window with "No disassembly available."
Why? Any suggestions as to what I can do to get the disassembly to work?
Under Debug Options I do have "Enable address-level debugging" and "Show disassembly if source is not available" checked.
Thanks.
EDIT
Just to try to explain a bit more ...
The program in question is my own program, and the obfuscation program being used is also my own program. The obfuscation program runs ILDAsm.exe, modifies the ILAsm code, and runs ILAsm.exe.
My obfuscation is apparently introducing problems so the program no longer works correctly. To understand how/why it isn't working I'd like to trace it. But for some reason Visual Studio debugger says "No disassembly available", and then I can't do anything at all. (When the non-obfuscated program is run under Visual Studio the Disassembly window can be opened and shows the expected information. It's only the obfuscated version that produces this problem with VS debugger.)
EDIT 2
Haven't been able to find an answer to my question, but I do have a sort of workaround now.
I've installed WinDbg and confirmed that it can attach to my obfuscated program and can at least single-step it and show the current execution location in a disassembly window. But I've never used WinDbg before, and it looks a bit daunting.
I've also followed the advice seen several places to load sos.dll into WinDbg. Maybe that will help.
But I'm guessing that what I'll actually end up doing is throwing lots of temporary logging statements into my program around the places where it's not working, and hopefully gain some understanding that way.
OK, I happened to stumble on what I was doing wrong.
As I mention in the first edit of my question, my obfuscation is being done by running ILDAsm.exe, modifying the ILAsm statements, and then running ILAsm.exe. It turns out I was missing the /Debug option on the ILAsm.exe run. So even though my Visual Studio builds were Debug builds, the JIT conversions at run time were Release/Optimize conversions.
With /Debug specified on ILAsm.exe I'm getting a PDB file, and when Visual Studio debugger connects to my program I'm getting IL source code display and, if I request it, disassembly display too! So now all is well!
What is the difference between Start Debugging (F5) and Start without Debugging (CTRL-F5) when the code is compiled in Release mode?
I am seeing that CTRL-F5 is 10x faster than F5 for some C++ code. If I am not wrong, the debugger is attached to the executing process for F5 and it is not for CTRL-F5. Since this is Release mode, the compiled code does not have any debugging information. So, if I do not have any breakpoints, the execution times should be the same across the two, isn't it?!
(Assume that the Release and Debug modes are the typical configurations you get when you create a new Visual C++ project.)
The problem is that Windows drops in a special Debug Heap, if it detects that your program is running under a Debugger. This appears to happen at the OS level, and is independent of any Debug/Release mode settings for your compilation.
You can work around this 'feature' by setting an environment variable: _NO_DEBUG_HEAP=1
This same issue has been driving me nuts for a while; today I found the following, from whence this post is derived:
http://blogs.msdn.com/b/larryosterman/archive/2008/09/03/anatomy-of-a-heisenbug.aspx
"Start without debugging" just tells Windows to launch the app as it would normally run.
"Start with debugging" starts the VS debugger and has it run the app within the debugger.
This really doesn't have much to do with the debug/release build settings.
When you build the default 'debug' configuration of your app, you'll have the following main differences to the release build:
The emitted code won't be optimised, so is easier to debug because it more closely matches your source
The compiler & linker will output a .PDB file containing lots of extra information to help a debugger - the presence or absence of this information makes no difference to the performance of the code, just the ease of debugging.
Conditional macros like ASSERT and VERIFY will be no-ops in a release build but active in a debug build.
Each one of these items is independent and optional! You can turn any or all of them on or off and still run the code under the debugger, you just won't find life so easy.
When you run 'with debugging' things perform differently for several reasons:
The VS debugger is very inefficient at starting, partly because everything in VS is slow - on versions prior to VS2010 every pixel of the screen will be repainted about 30 times as the IDE staggers into debug mode with much flashing and flickering.
Depending on how things are configured, the debugger might spend a lot of time at startup trying to load symbols (i.e. PDB files) for lots and lots of OS components which are part of your process - it might try fetching these files over the web, which can take an age in some circumstances.
A number of activities your application normally does (loading DLLs, starting threads, handling exceptions) all cause the debugger to be alerted. This has the effect both of slowing them down and of making them tend to run sequentially.
IsDebuggerPresent() and OutputDebugString() behave differently depending on whether a debugger is attached.
IsDebuggerPresent() simply returns another value, so your program can react to this value and behave differently on purpose. OutputDebugString() returns much faster when there's no debugger attached, so if it's called lots of times you'll see that the program runs much faster without the debugger.
When running 'with debugging' the debug heap is used, even for release mode. This causes severe slowdowns in code using a lot of malloc/free or new/delete, which can happen in C++ code without you noticing it because libraries/classes tend to hide this memory management stuff from you.
c++ with visual studio 2008
if i use assert() from assert.h and compile in debug mode, the application crashes if the assert condition doesn't hold and it prints me in the console on what line in what file this happened. that's quite useful, but i'd prefere to trap into the debugger at this position instead, if the condition doesn't hold
how can I do that?
thanks!
Try running your program under a debugger. Also, if you have a JIT debugger registered with the OS, then this should actually invoke said debugger. If you are on a Windows machine, take a look at this MSDN article or this post.