Performance test between debug & release mode - debugging

I'm testing performance. The program receive huge bytes from other other side through TCP connection and parse into target structure.
If I try it in Debug (F5), delayed very seriously. But if I try it in Release (Ctrl+F5), it works very smoothly.
I check the situation using WireShark, TCP window update statement let me know the status of receive side.
I wonder if I want to make end-user program. The program should be work in debug mode as well as release mode?
The data size really big and the transmit interval is very short. In this case, the difference of performance between debug and release will be very cleary? I couldn't see that kind of difference ever.

This question conflates two different issues. One is Debug versus Release builds. Debug builds are slower for numerous reasons, all aimed at making debugging easier.
The second issue is that launching an application under the debugger (F5) versus not under the debugger (Ctrl+F5) can affect performance. The Windows heap does additional checks when launched under the debugger, processing of OutputDebugString and module loads may take longer, etc.
These two issues are completely orthogonal. You can launch a Release build under the debugger, or you can launch a Debug build not under the debugger.
If the problem is that Debug builds are slower then, well, yes, they are. You can control this somewhat by enabling optimizations in some source files or by turning off debug iterators, but Debug builds are supposed to be somewhat slower.
If the problem is that builds launched under the debugger are slower then don't do that when you care about performance. You can always attach the debugger after, which will avoid some of the slowdown, or don't attach the debugger at all.

When you publish, your application, it should be compiled in "Release" mode only. Debug mode is used to really "debug" while your development.

You should only program with Release mode in mind. It's really the only version of your app that should reach the end-user. Debug mode is only for "housekeeping" and well... debugging. Not really the user's concern.

Related

Live vs. offline debugging

I've been trying to find the difference between these 2 types of debugging, but couldn't find it anywhere (been googling almost 30 minutes), so I'm asking here: What's the difference between live vs. offline debugging? What do people mean when they say a debugger is "live" vs. "offline"?
Debugging types
There are several ways of debugging that can be distinguished:
live debugging vs. post mortem debugging (what you call "offline" debugging, also called "dump debugging")
kernel debugging vs. user mode debugging
local debugging vs. remote debugging
which give 8 combinations in total.
For live debugging, you can distinguish between invasive debugging vs. noninvasive debugging.
Live debugging vs. offline debugging
In live debugging, the program is running and the debugger is attached to it. This means you can still interact with the program. You can set breakpoints, handle exceptions that would normally cause the program to terminate, modify the memory etc.
The downside of live debugging is its temporal/fluent nature. If you enter a wrong command or step too far, the situation is gone and might not be repeatable.
I mentioned that there are 2 sub-modes for live debugging: invasive and noninvasive debugging: in noninvasive debugging, the debugger does not attach to the target application. It suspends all of the program's threads and has access to the memory, registers, and other such information. However, the debugger cannot control the target.
In post mortem debugging, someone has captured a memory dump of a running program at a certain point in time. In many cases this is done upon a specific event, e.g. an unhandled exception that causes the program to terminate. Since the memory dump is a file on disk, you can analyze it as often as you want and you get the exact same situation.
The downside if post mortem debugging is, of course, that the program is not running, you can't interact with it and it's very hard to find out what happens next.
"Online" debugging is the normal process:
Tell the debugger to tell the program to step forwards;
Look at what the program state is at the moment;
Set a breakpoint for the future;
Tell the debugger to simply run the program;
If the breakpoint 'fires', have a look at the program state now.
There are two ways to "offline" debug:
You can take your source code and manually step through what the processor ought to be doing, watching for unexpected program paths.
Note if you do this, you need to diligently not "know" what the processor is "supposed" to do and just do that: you need to honestly obey the code as though you were the computer. Often you get other people, who don't know the code, to do this instead of you.
You take the result of a run-log, usually captured by a hardware probe, and use the debugger to "post mortem" the run.
The latter usually requires a processor that will transmit what it is doing out a "Trace" port (not all have this), and a hardware device (like a probe) connected to the Trace port to capture the data. That probe then communicates with a debugger, which takes the data and presents it to the programmer. The programmer can work backwards and forwards through this Trace log, and see the execution path that the code actually took, rather than the code the programmer thought it should take.
Some processors not only transmit what instruction they're currently processing, but also what data they read or wrote while doing this. A more sophisticated debugger can take this extra data and provide a 'snapshot' of the system at any time during the run, allowing the programmer to analyse why the code behaved the way it did.
The reason that it is called "offline" is because once the log has been captured, you can disconnect and power down the target, and look at the saved log at any time in the future without still being connected to the probe or processor.

Why do some programs work when using the debugger, but not when running a normal debug execution?

I've had several times where I was writing a program using Visual Studio and something, somewhere along the line breaks. Naturally, my first thought is to set a breakpoint early on in the program and then step through the code slowly, line by line until I reach the error.
But often times I find that everything works correctly when slowly stepping through code in the debugger. However, when I just try to run the program without breakpoints something goes wrong.
Why does this happen? Is stepping through the code line by line actually different from just running the program in dubug mode?
(I'm not talking about debug vs release, i'm talking about debug vs line-by-line debugging)
One reason could be that you have timing issues between multiple executing threads that occur differently when the program runs "at full speed" and when you pause a thread in a debugger.
It could be due to a some thread unsafe operations that seem to happen correctly when stepping through the code but not at run time. Compiler optimizations are ruled out because you are sure that you are running a debug build.
Running inside the debugger materially alters the runtime options for eg. heap operations so that errors can be detected and surfaced to the user.
See here.
When a process is run under any debugger, certain heap debug options
are automatically enabled for all heaps in the process.
This is not for sure your issue, but a likely candidate.

Debugger issues with Visual Studio and WP7 emulator

I have a complex C# project that I ported over from C++ and now I'm in the middle of debugging. Things are working great most of the time but more often than not I have huge problems with Visual Studio and debugger attached to WP7 emulator. For some strange reasons, my debug session is often abruptly terminated while stepping through the code without any indication from VS or any trace left in the Output window.
There are even some cases when a breakpoint is hit and then when I hover over a particular variable, VS simply exits the current debugging session. If I refrain from inspecting the variable contents, nothing happens and VS waits happily forever.
As the app is a memory hog by definition, I am wondering whether I am hitting any debugger / WP7 / emulator limits of any kind. Why would a mouse over variable terminate debugging session? Most of all, why is there no trace of what happened? I am left to wonder whether this is a VS issue or an emulator issue or even an app issue.
What are your computer specs?
I have seen similar problems on computers with low specs, especially computers with low memory.
Try clearing out memory hogs from your PC (CCleaner is a good tool) and running Visual Studio in administrator mode.
I have found this post which has helped me immensely. It appears that having ToString() overrides can sometimes crash the debugging session. I have implemented mine for the sole purpose of having customized value representation of variables/values in the debugger.
After removing all the ToString() overrides I am able to debug normally again. The thing that still puzzles me is the fact that no exceptions are leaked from my ToString() overrides so I wonder why debugger behaves the way it does but at least the problem is solved for now.
I hope this helps someone.

GPU code gives different time when run from VS2008 and when running only .exe

I have cuda events in my code to record the time of execution. When I click "Start Debuggin" from VS 2008, the timer gives a value of 1.5 seconds. However, when I run the program from .exe file, it gives time of .4 seconds. Why this difference?
There's no inherent reason that running attached to the debugger should introduce a performance difference, other than (just a few possibilities):
Do you have any conditional breakpoints set? Depending on the condition, these can have a dramatic impact on execution time.
Are you explicitly writing large amounts of data to the Debug or Trace listeners? (Edit: that's relevant for C#, probably not for C++.)
Is the EXE compiled in Release mode? By default, the Release config turns on optimizations that aren't present when building in Debug mode.
Is your timing code really only timing the relevant section? If you're starting the timer at the start of program execution instead of around the GPU calls that you're really interested in, you may be accidentally timing some startup tasks that are tied to running with the debugger that won't be active in a standalone app.

Visual C++: Difference between Start with/without debugging in Release mode

What is the difference between Start Debugging (F5) and Start without Debugging (CTRL-F5) when the code is compiled in Release mode?
I am seeing that CTRL-F5 is 10x faster than F5 for some C++ code. If I am not wrong, the debugger is attached to the executing process for F5 and it is not for CTRL-F5. Since this is Release mode, the compiled code does not have any debugging information. So, if I do not have any breakpoints, the execution times should be the same across the two, isn't it?!
(Assume that the Release and Debug modes are the typical configurations you get when you create a new Visual C++ project.)
The problem is that Windows drops in a special Debug Heap, if it detects that your program is running under a Debugger. This appears to happen at the OS level, and is independent of any Debug/Release mode settings for your compilation.
You can work around this 'feature' by setting an environment variable: _NO_DEBUG_HEAP=1
This same issue has been driving me nuts for a while; today I found the following, from whence this post is derived:
http://blogs.msdn.com/b/larryosterman/archive/2008/09/03/anatomy-of-a-heisenbug.aspx
"Start without debugging" just tells Windows to launch the app as it would normally run.
"Start with debugging" starts the VS debugger and has it run the app within the debugger.
This really doesn't have much to do with the debug/release build settings.
When you build the default 'debug' configuration of your app, you'll have the following main differences to the release build:
The emitted code won't be optimised, so is easier to debug because it more closely matches your source
The compiler & linker will output a .PDB file containing lots of extra information to help a debugger - the presence or absence of this information makes no difference to the performance of the code, just the ease of debugging.
Conditional macros like ASSERT and VERIFY will be no-ops in a release build but active in a debug build.
Each one of these items is independent and optional! You can turn any or all of them on or off and still run the code under the debugger, you just won't find life so easy.
When you run 'with debugging' things perform differently for several reasons:
The VS debugger is very inefficient at starting, partly because everything in VS is slow - on versions prior to VS2010 every pixel of the screen will be repainted about 30 times as the IDE staggers into debug mode with much flashing and flickering.
Depending on how things are configured, the debugger might spend a lot of time at startup trying to load symbols (i.e. PDB files) for lots and lots of OS components which are part of your process - it might try fetching these files over the web, which can take an age in some circumstances.
A number of activities your application normally does (loading DLLs, starting threads, handling exceptions) all cause the debugger to be alerted. This has the effect both of slowing them down and of making them tend to run sequentially.
IsDebuggerPresent() and OutputDebugString() behave differently depending on whether a debugger is attached.
IsDebuggerPresent() simply returns another value, so your program can react to this value and behave differently on purpose. OutputDebugString() returns much faster when there's no debugger attached, so if it's called lots of times you'll see that the program runs much faster without the debugger.
When running 'with debugging' the debug heap is used, even for release mode. This causes severe slowdowns in code using a lot of malloc/free or new/delete, which can happen in C++ code without you noticing it because libraries/classes tend to hide this memory management stuff from you.

Resources