Debugging Shaders on Visual Studio 2012, can't find the Symbols - debugging

I don't know if many of you tried the new excellent feature of Visual Studio 2012 to debug Direct3D based apps.
I successfully capture a frame of my app, then I want to debug the execution of a Vertex Shader:
I click on the green triangle to debug a given vertex, but I got a "No Symbol Found" message which prevent me to debug it.
Someone knows what to do for Visual Studio to find the symbols?
Thanks.

You need to compile your shaders with debug information. And in order to reliably debug, you likely will want to disable optimizations as well. Depending on how you compile your shaders, this will be one of two ways:
D3DCompile / D3DCompileFromFile: Pass the D3DCOMPILE_DEBUG and D3DCOMPILE_SKIP_OPTIMIZATION flags to D3DCompile. Note that depending on D3D version and whether you are using D3DX, these flags might have different prefixes.
fxc.exe: The compile flags have switch equivalents. For D3DCOMPILE_DEBUG, pass /Zi to fxc. For D3DCOMPILE_SKIP_OPTIMIZATION, pass /Od.
Visual Studio 2012 is fairly smart about finding the source for your shaders based on the embedded debug information, but should it fail to do this, it will prompt you to point to the appropriate file.

From msdn:
It's not possible to debug an app and its shader code at the same time. However, you can alternate between them
It's possible that you are debugging the application and trying to debug shader at the same time.

When using SlimDX you can pass ShaderFlags to the shader compiler. Passing ShaderFlags.Debug causes debug symbols to be included.
var bytecode = ShaderBytecode.CompileFromFile("shader.fx", "VShader", "vs_4_0", ShaderFlags.Debug, EffectFlags.None)
Hopefully it is something similar in native Direct3D.

I was able to debug shaders in VS2012.
I used this command line:
fxc /TTargetProfile /EShaderMainFunctionName /Od /Zi /FdMyShader.pdb MyShaderFile
In code, I used:
D3DX11CompileFromFile(filename, NULL, NULL, "ShaderMainFunctionName, "TargetProfile", D3D10_SHADER_DEBUG | D3D10_SHADER_SKIP_OPTIMIZATION, ...);
TargetProfile is something like vs_5_0 or ps_5_0.
I'm pretty sure it would work with D3DCompile.
When I forgot to put D3D10_SHADER_DEBUG | D3D10_SHADER_SKIP_OPTIMIZATION, and I tried to load the pdb file manually, it said the symbols didn't match.
When I got it right, the Pixel Shader in the Pixel History view was a blue link. When I clicked it, it automatically loaded the correct file and I could start debugging it.
I couldn't find an in-code equivalend to /Fd that generated the pdb file, which is why I had to go through command line.

Related

How to generate shader debug information (pdb) in VS2012

I'm trying to debug my shaders in Directx 11 SDK application using graphics diagnostics tool in VS2012, however when I click Start Debugging on one of the shaders in Graphics Pixel History panel I'm getting Pixel Shader.pdb not loaded and I'm not able to find the pdb file anywhere.
I tried compiling shaders at runtime using D3DX11CompileFromFile with D3DCOMPILE_DEBUG flag as well as using the HLSL compiler with debugging information turned on (/Od /Zi) but none of these options is producing the pdb file I could use in graphics diagnostics tool.
How do I generate these files?
I think this "Pixel Shader.pdb not loaded" message is misleading. The is no any .pdb files generated by HLSL-compiler. All debug info are integrated into binary (either memory blob or .cso file).
Does your shader file named "Pixel Shader"? Maybe it says not about a shader, but some kind of Visual Studio's internal source files (shader debugger in VS2012 was somewhat unstable sometimes)
Some ideas that, probably, can help you to solve issue:
Make sure that your debug and release output binaries are not messed up. Check both debug and release configuration of project's properties and each shader's properties.
Make sure that you are loading right shader file. Check ten times all file paths.
Clean your project, eliminate all binaries by hand. Rebuild again.
IIRC, in graphics debugger you can only debug shaders, compiled offline (i.e. not by D3DX11CompileFromFile).
Try to compile using fxc.exe directly.
Use filenames that does not contains spaces and special characters.
Make sure your test shader is simple enough, so debugger will not crash. Thy to use default template.
If it's still doesn't works, write a minimal example project, check that it doesn't works, post it here so we can test it.
Try VS2013
Try your videocard vendor's debugging tool (such as NVIDIA nsight or AMD GPU PerfStudio), to see if it is problem with Microsoft tools or not.
Hope it helps somehow.

Disable or fix #ifdef-sensitive colouring and intellisense in Visual Studio

The problem: My syntax highlighting and IntelliSense are broken. I have a C++ source file like this:
#include "stdafx.hpp"
#ifdef SOMETHING
do_some_stuff;
#endif
where stdafx.hpp (the precompiled header for the project) includes a .h file that says:
#ifdef DEFINE_SOMETHING
#define SOMETHING
#endif
and DEFINE_SOMETHING is defined in the project properties for the project (under C++ / Preprocessor).
Visual Studio is losing track, and displaying do_some_stuff; (which is actually lots of lines of code) in plain grey - I have neither syntax colouring nor IntelliSense.
The question: How can I either make Visual Studio get this right (unlikely) or switch off the fact that it's greying-out code that it thinks is #ifdef'd out?
(Rearranging the code is not an option - it's a large and complex system whose files are built in various environments, Visual Studio being only one of them. I'm using Visual Studio 2005, but I'd be interested to know whether this is fixed or workaroundable in a later version.)
If someone still interested - to turn off graying out #ifdef:
Go to Tools -> Options
Open Text Editor -> C/C++ -> Formatting
Uncheck Colorize inactive code blocks in a different color
In VS19, it's Tools / Options / Text Editor / C/C++ / View / Inactive Code / Show Inactive Blocks.
Following previous answer of aousov I check my VSCode and found this setting:
C_Cpp: Dim Inactive Regions
Controls whether inactive preprocessor blocks are colored differently than active code. This setting has no effect if IntelliSense is disabled or if using the Default High Contrast theme.
in Extensions / C/C++
This may be related to the version you are using (in my case 1.46.1).
Best,
Geoffroy
The problem you describe is par for the course in VS 2005. It is fixed in Visual Studio 2010 and later due to the completely redesigned Intellisense system. This is not directly applicable to your problem, but here's some info on the underlying architecture: http://blogs.msdn.com/b/vcblog/archive/2009/05/27/rebuilding-intellisense.aspx
There are some things you could try, and some project structure changes that can help minimize the problem's frequency, but whatever you do will be hit or miss, and the problem will eventually resurface again regardless. The only real solution is to use a newer IDE.
You can continue to use the VS 2005 build tools by installing VS 2010 along with Daffodil (http://daffodil.codeplex.com), then build your projects with the v80 platform toolset in VS 2010. This makes the migration fairly straightforward, with no need for any source code changes.
Since #define SOMETHING is defined inside stdafx.hpp, indicating that it's always defined since DEFINE_SOMETHING is defined in project configuration, would it be out of the question to also define SOMETHING explicitly in project configuration?
I used to have similar issues in VS2005 and 2008, and redundant explicit definitions sometimes helped.
I fixed this (in VSCode) by changing C_Cpp.default.intelliSenseMode
"C_Cpp.default.intelliSenseMode": "windows-gcc-x64"
I am building an ARM project on a micro-controller. Its not 64 bit either. But this does parse the directives correctly.
For Science I tried Widows-gcc-ARM and that also correctly lit up the regions that are truly active. I also know for a fact that gcc is setup and configured on my windows machine, and while I have clang and msvc, I dont use them and dont know that they work- so it could be why gcc works better for me.
You can experiment with this setting, but I am fairly certain the resolution resides in this option.
I do not know the equivalent VS option, I am sorry.

Debugging a C executable with a C++-CLI Library compiled with /clr

I am using Visual Studio 2010 to debug an application mostly written in C. Normally, I can attach the debugger just fine, but I am running in to some problems when I link in a library written in C++ / CLI.
If I compile the library with the /clr flag (which I will eventually have to do for this as of yet unwritten library) then I lose all ability to debug the entire C application, even the parts that have nothing to do with the library calls. I get the empty circle with the yellow triangle and exclamation mark where a red break point circle ought to be. Hovering over it gives me only a tool tip that says "The breakpoint will not currently be hit. No executable code is associated with this line. Possible causes include: conditional compilation or compiler optimizations."
Then if I link with the exact same library compiled without the /clr flag, I am again able to debug my application.
I understand that visual studio will not likely be able to debug the library written in C++ / CLI, and that is OK. I just want to keep the ability to debug the rest of the application and at least see the results of my calls to the external library.
Another complicating factor is that this project is not being built by visual studio. It is compiled using an external make system that uses cl, so I can customize any commands that need to be issued to the compiler that way.
Does anyone know why I can't currently debug the libraries the way I want to? Any advice for how I can?
You have to select the kind of debugger when you attach. Note the "Attach to" label in the dialog. Press the Select button and tick "Native" to get support for debugging native code. The DLL also needs to be loaded before any of your breakpoints can hit. If you are not sure whether or not that was done then look in the Debug > Windows > Modules debugger window to see loaded DLLs. The breakpoint indicator turns from hollow to solid red as soon as the debugger saw the DLL load and armed the breakpoint.
Debugging C++/CLI is otherwise supported, you can tick both the "Managed" and "Native" checkboxes. And set breakpoints in either kind of code. The only thing not supported is single-stepping from managed to native code and back. A mode-switch is required to activate the correct debugging engine, that requires code to hit a breakpoint.
And consider the Debug options in your native project, you can specify an EXE to start. So that you can simply press F5 to start debugging and skip the attach hassle.
It might also have to do with the Debugger Type!
(but that depends on your specific building configuration about which I do not know enough)
If any of your projects is complied with Common Language Runtime Support (/clr) you should set the Debugger Type in your startup project to "Mixed", since the default setting "Auto" might fail!
Imagine, you have two projects:
1) A non-CLR C++ project, which is your startup project that generates some .exe file.
2) A C++ project, that generates mylibrary.dll, which is compiled with Common Language Runtime Support, because it uses some managed code. The .exe from the first project calls mylibrary.dll.
If you start the first project with Debugger Type set to its default value "Auto", you'll be able to debug into the first project, but not into the second one. The debugger selector does not realize that you will be calling a CLR-library.
So set Project Properties -> Configuration Properties -> Debugging -> Debugger Type to "Mixed"!

PCL Rigid Transformation using IterativeClosestPoint - Error: vector erase iterator outside range

I am currently trying to calculate rigid transformation between two point sets so I tried to use the code given by the tutorial on pointclouds.org:
http://www.pointclouds.org/documentation/tutorials/iterative_closest_point.php#iterative-closest-point
For my case I only changed the part where the data is randomly generated to something that loads the point data I want to analyze. Everything else is exactly like in the tutorial...
(I also tried testing exactly the tutorial code with the random data, in case I had somethign wrong with reading my input data)
Since I work with Qt I integrated the PCL library, Eigen library and FLANN library to my project. It finds all headers and successfully compiles with MSVC 2008...
Unfortunately I always get a runtime error at
icp.setInputTarget(cloud_out);
saying:
Debug Assertion failed! Program:
...MSVC2008_Qt_SDK_Release\release\Project.exe File: c:\Program
Files\Microsoft VIsual Studio 10.0\CV\include\vector Line: 1200
Expression: vector erase iterator outside range
[..] ... check documentation ... [..]
Does anybody know what that means? The input clouds both have the same size and have filled values.
I would be thankful for any help!
UPDATE 1:
The error message shows some file path for MSVC 2010 (10.0) ... So I tried to uninstall Visual Studio 2010 since I don't really need it. But still, if I compile in Debug mode, it shows me an error message, but with Expression: vector iterators incompatible instead... If I now run it in Release mode, it just crashes at runtime (at the same line), but doesn't show that error message.
This seems so be a problem with the library you use. Assuming you have done a clean build, checked your PATH variable and everything and that Visual Studio 2010 is removed, this might be a problem with the library itself. Are you you using the right one?
The current Qt SDK has MSVC2008 in it, so I guess it takes everything from where it needs. But either the compiler in Qt or one of the libraries you use that might want the 2010 version...
Hope it helps!

How to get around "Binary was not built with debug information"?

I'm trying to debug a C DLL that I'm using with a Delphi program. I built the DLL with Visual C++, with debug information enabled. I built the Delphi program with Delphi 2009, with debug information enabled. But apparently they use different formats, because when I try to attach the VC++ debugger to my program, it says "binary was not built with debug information" and won't even accept as valid the breakpoints I put in the C code, which was built with debug info in the format VC++ understands.
Does anyone know how I can get this to work?
When you say "won't accept as valid" the debug breakpoints in the C code what do you mean exactly? Does it not enable them? If so has the DLL been loaded yet when you set the breakpoints? I find it can simplify matters if I wait to set the breakpoints until after I'm sure the DLL in question has been loaded. If this is not what is happening, please elaborate on what you mean by "valid" breakpoints.
Other options are to set function breakpoints, or the compile the DLL with strategically placed DebugBreak() calls.
Are you sure it's the right DLL that's being loaded (i.e. the debug version)?. Again, even the right DLL is being loaded I'm not sure the error is necessarily applying to the DLL and not just the main executable. Or it could be having problems loading the symbol database as suggested by jdigital, assuming you extract them out for debug builds of your DLL. Even with no debugging symbols, debugging should still be feasible, especially since it's a DLL, you can work from the exported symbols.
This isn't a COM component is it? If it is, I'd double check that the debug version was the one registered before you start up your process.
Again I'd still be interested in hearing exactly what happens when you try to set a break point. If you go to the breakpoints window in VS it should clarify why the breakpoint couldn't be set, if that's what is happening.
Hmm. I don't have much experience with /Z7, do you still have the .obj file for the DLL? The docs seem to imply that's necessary for debugging. Alternatively I'd try building with /Zi instead and getting a .pdb for that sucker.
Not sure about Visual C++ (don't have it installed at all anymore), but maybe this will help...
If you were writing a DLL in Delphi and using it from C++, and you wanted to debug the DLL, you'd open the source for the DLL in Delphi and set a breakpoint. You'd then use Run|Parameters and set the C++ application as the host executable and hit run in the Delphi IDE. The IDE would then launch the C++ application and run it as usual until the breakpoint in the DLL was hit, and then would break as you'd expect.
Is something similar available in VC++? (You didn't say which version of VC++, or which version of Visual Studio or the earlier IDE you were using.)
If not, the only alternative I could think of is to do a quick VC++ app that uses the DLL and debug via that instead.
Debug formats are not standardised - basically you can't use Delphi to debug MS compiled code or vice versa.
You can debug Delphi DLLs in Delphi and you can use those DLLs with other apps not compiled with Delphi, provided you mark the Delphi functions for export. What you can't do is debug those DLLs symbolically in a 3rd party debugger, which would have to understand Object Pascal name mangling at the very least.
Have you pointed the debugger at the symbols for your DLL? If there's any doubt, try running with Filemon to see if the debugger is failing when it tries to load the symbols.
Insure that is opening the DLL in the Debug Folder, not another one in some other folder.
Ten years later and this is happening to me, while debugging a custom DLL used in Team Developer, setting debugging command to start the Team Developer IDE. The objective is to step through the 3rd party code to the point of invocation of an exported function from the DLL.
Starting the debugger launches the IDE without error, but running the TD project within the TD IDE causes an exception in VS on a DLL used by the TD IDE.
How do I ignore the exceptions from outside the project? Has anyone been able to get around this since '09?

Resources