Start debugger in c++ code - visual-studio

I am trying to debug some problems in a native C++ COM DLL I have created and would like to launch the Visual Studio (2008) debugger from this. This dll is called by another dll that I was given, so basically my code plugs into another application through this. Anyway, in the past I had used C# to create this dll and was able to use the System.Diagnostics.Debugger.Break() method. Is there anything similar to this that can be used? I saw a few posts about DebugBreak and __debugbreak but this did not seem to work, it just appeared to stop my code from executing beyond that statement.
EDT: I followed the advice of paulsm4 below and found that the breakpoints did not work, it would tell me: "The breakpoint will not currently be hit. No Symbols have been loaded for this document"

Ideally, you'll have the project source for your COM/ActiveX .dll.
If that's the case, just set a break point in your .dll code, run your program in the debugger ... and that's it.
Please note the distinction between "managed code" (e.g. a C#/.net .exe or .dll) and "native code" (like your COM/ActiveX .dll). To debug "native code" under MSVS2008, you'll need to specify "mixed mode":
http://msdn.microsoft.com/en-us/library/kbaht4dh%28v=vs.90%29.aspx

Related

Debugging a C executable with a C++-CLI Library compiled with /clr

I am using Visual Studio 2010 to debug an application mostly written in C. Normally, I can attach the debugger just fine, but I am running in to some problems when I link in a library written in C++ / CLI.
If I compile the library with the /clr flag (which I will eventually have to do for this as of yet unwritten library) then I lose all ability to debug the entire C application, even the parts that have nothing to do with the library calls. I get the empty circle with the yellow triangle and exclamation mark where a red break point circle ought to be. Hovering over it gives me only a tool tip that says "The breakpoint will not currently be hit. No executable code is associated with this line. Possible causes include: conditional compilation or compiler optimizations."
Then if I link with the exact same library compiled without the /clr flag, I am again able to debug my application.
I understand that visual studio will not likely be able to debug the library written in C++ / CLI, and that is OK. I just want to keep the ability to debug the rest of the application and at least see the results of my calls to the external library.
Another complicating factor is that this project is not being built by visual studio. It is compiled using an external make system that uses cl, so I can customize any commands that need to be issued to the compiler that way.
Does anyone know why I can't currently debug the libraries the way I want to? Any advice for how I can?
You have to select the kind of debugger when you attach. Note the "Attach to" label in the dialog. Press the Select button and tick "Native" to get support for debugging native code. The DLL also needs to be loaded before any of your breakpoints can hit. If you are not sure whether or not that was done then look in the Debug > Windows > Modules debugger window to see loaded DLLs. The breakpoint indicator turns from hollow to solid red as soon as the debugger saw the DLL load and armed the breakpoint.
Debugging C++/CLI is otherwise supported, you can tick both the "Managed" and "Native" checkboxes. And set breakpoints in either kind of code. The only thing not supported is single-stepping from managed to native code and back. A mode-switch is required to activate the correct debugging engine, that requires code to hit a breakpoint.
And consider the Debug options in your native project, you can specify an EXE to start. So that you can simply press F5 to start debugging and skip the attach hassle.
It might also have to do with the Debugger Type!
(but that depends on your specific building configuration about which I do not know enough)
If any of your projects is complied with Common Language Runtime Support (/clr) you should set the Debugger Type in your startup project to "Mixed", since the default setting "Auto" might fail!
Imagine, you have two projects:
1) A non-CLR C++ project, which is your startup project that generates some .exe file.
2) A C++ project, that generates mylibrary.dll, which is compiled with Common Language Runtime Support, because it uses some managed code. The .exe from the first project calls mylibrary.dll.
If you start the first project with Debugger Type set to its default value "Auto", you'll be able to debug into the first project, but not into the second one. The debugger selector does not realize that you will be calling a CLR-library.
So set Project Properties -> Configuration Properties -> Debugging -> Debugger Type to "Mixed"!

createObject in VB6 does fails when running exe

I have a VB6 program which tries to run a DLL written in C#.
This DLL has a COM interface so I can create an object of a class in it with "CreateObject".
The problem is that it runs and works well when I run it from the VB6 IDE, but when I make an EXE and try to run it, it throws the exception:
"Automation error. The system cannot find the file specified (-2147024894)."
Why is it happening and how can i solve it?
Look at Project, References in the IDE and look which dll or ocx file belongs to the object you are referencing with CreateObject (the Object Manager might also help to find out).
This dll file must be available when the exe is compiled, too. Usually, you need to have it registered with regsvr32.exe.
A technique I use to figure issues of this type is to open the add reference dialog in Visual Basic 6. I scroll the list of available COM Libraries and see if the problem DLL is listed. If it is then CreateObject should work, you should be able to assign it do a variant variant and use late binding to access it's members.
In addition try temporally set a reference to the variable and instead of using CreateObject use the = New and see what error messages, if any, it gives you. Generally I found them to be more informative then the ones thrown by CreateObject.
Finally it would help if you post the reason why you are choosing to use CreateObject instead of setting of a reference. If the DLL is a known object that will be continually used by the program then a reference should be set and early binding generally used.
Finally it may be that the error is resulting from a dependency of the C# COM DLL not the DLL itself. If for example I was to take a Com Library and properly register it but it relies on the COM Library Widget2000 and it NOT registered then it will throw the automation error. Especially if you are testing the EXE in it's installed environment and not the environment in which you complied it.
For example suppose I have a CAD program written in VB6 and I have source tree that begins with MyCAD. THe exe is in MyCAD/MainEXE and the shape library is in MyCAD/ShapeLibrary. I run the IDE everything is fine. Then I make my setup and goto my test machine and install it and it error on the creation of shapelibrary.
The first thing I would do it check if MainEXE will run straight out of the MainEXE directory of my source tree. That test will eliminate whether it is a install issue or a quirk of the IDE vs complied version. Then I would look at the setup and see what not being registered. Also look at either the source for the C# library or the setup for the library and see what dependencies it needs. Since it a complied COM DLL you should be able to use a dependency walker tool to see what COM references it needs. Finally make sure the correct version of the .NET framework is installed.
If you are compiling the C# DLL on your test machine - make sure you have ticked the register for COM Interop setting. If you are not compiling on the same machine you need to run RegAsm with the /codebase option.
try compiling it as an installer and include the dll/com that you use in the compilation of the installer package so that the dll/com that you use will be include in the compilation of your exe.., and install it in the windows not just copy past it.

Get the callstack(s) when a kernel32.dll function is called

I have a process that changes its current directory, and I would like to know when and where it happens. How could I do that?
I tried setting a breakpoint in SetCurrentDirectoryA/SetCurrentDirectoryW with Visual Studio, but it does not work.
Are you debugging one of your own programs, or one that you don't have the source code for? The Visual Studio debugger isn't very friendly with regards to debugging no-source applications; in that case, I would recommend WinDbg or OllyDbg - or even skipping the debugger and write an instrumented logger using EasyHook.
EDIT:
Try setting a breakpoint at {,,kernel32.dll}_SetCurrentDirectoryA#4 - peculiar syntax and requires decorated names. Haven't tried it myself, but found it here. Google keywords: "visual studio breakpoint api" :)
You need to attach/debug the process using native code. If you by mistake are debugging using managed code you will not hit those breakpoints.
Your program may be changing directories using the msvcrt functions.
You should try placing breakpoints on these functions as well:
_chdir
_chdrive

What is a Windows command-line EXE's "side-by-side configuration" and how do I correct it?

I have a simple .exe written in C++ (built with Visual Studio 2005) that tests some hardware using a supplied API. It works fine on the Windows 7 machine I built it on, but when I copy it to another (Windows 7) machine and run it (from the command-line) I get:
The application has failed to start
because its side-by-side configuration
is incorrect. Please see the
application event log or use the
command-line sxstrace.exe tool for
more detail.
What is "side-by-side configuration"?
I ran sxstrace.exe and read the usage info. It appears I would need to instrument my exe to generate a log file for sxstrace.exe to be useful?
I imagine the problem is my exe requires DLLs that either don't exist on the other machine, or are the wrong version. How do I find out what DLLs my exe uses, and what versions it links to on my machine (where it works)? Any other advice on copying it to another machine and getting it running? Would more information help?
mfawzymkh's answer to the "application has failed to start because the side by side configauration is incorrect" question (linked to in the question spirulence linked to in his answer to this question) appears to apply to this question also. mfawzymkh writes:
You can resolve this issue by either
one of these 1- Install VC8 Debug CRT
2- Build you app as statically linked
And mfawzymkh's comment on that same answer explains how to build as statically linked:
when you build it in VS, go to
projects->settings->C/C++->Code
Generation and choose Runtime Lib
options to be /MTd instead of /MDd
I did that and the side-by-side configuration message is gone. (And after installing something else for the DLL I was using, my EXE works.)
For what it's worth, I encountered the same issue. In the Event Viewer I had an error message that read:
Activation context generation failed for "C:\\MyExe.exe".Error in manifest or policy file "C:\\MyExe.exe.Config" on line 12. Invalid Xml syntax.
Sure enough, I'd changed a connection string and left out the closing quote. Added that back in and it solved the issue.
"when you build it in VS, go to projects->settings->C/C++->Code Generation and choose Runtime Lib options to be /MTd instead of /MDd" worked for me, although I was interested in the Release version instead of the Debug version.
Microsofto says:
/MT Causes your application to use the multithread, static version of the run-time library. Defines _MT and causes the compiler to place the library name LIBCMT.lib into the .obj file so that the linker will use LIBCMT.lib to resolve external symbols.
/MD
Causes your application to use the multithread- and DLL-specific version of the run-time library. Defines _MT and _DLL and causes the compiler to place the library name MSVCRT.lib into the .obj file.
Applications compiled with this option are statically linked to MSVCRT.lib. This library provides a layer of code that allows the linker to resolve external references. The actual working code is contained in MSVCR100.DLL, which must be available at run time to applications linked with MSVCRT.lib.
Are you suffering from the same issue as this guy? Side-by-side assemblies, Windows 7, and Visual Studio 2005

How to get around "Binary was not built with debug information"?

I'm trying to debug a C DLL that I'm using with a Delphi program. I built the DLL with Visual C++, with debug information enabled. I built the Delphi program with Delphi 2009, with debug information enabled. But apparently they use different formats, because when I try to attach the VC++ debugger to my program, it says "binary was not built with debug information" and won't even accept as valid the breakpoints I put in the C code, which was built with debug info in the format VC++ understands.
Does anyone know how I can get this to work?
When you say "won't accept as valid" the debug breakpoints in the C code what do you mean exactly? Does it not enable them? If so has the DLL been loaded yet when you set the breakpoints? I find it can simplify matters if I wait to set the breakpoints until after I'm sure the DLL in question has been loaded. If this is not what is happening, please elaborate on what you mean by "valid" breakpoints.
Other options are to set function breakpoints, or the compile the DLL with strategically placed DebugBreak() calls.
Are you sure it's the right DLL that's being loaded (i.e. the debug version)?. Again, even the right DLL is being loaded I'm not sure the error is necessarily applying to the DLL and not just the main executable. Or it could be having problems loading the symbol database as suggested by jdigital, assuming you extract them out for debug builds of your DLL. Even with no debugging symbols, debugging should still be feasible, especially since it's a DLL, you can work from the exported symbols.
This isn't a COM component is it? If it is, I'd double check that the debug version was the one registered before you start up your process.
Again I'd still be interested in hearing exactly what happens when you try to set a break point. If you go to the breakpoints window in VS it should clarify why the breakpoint couldn't be set, if that's what is happening.
Hmm. I don't have much experience with /Z7, do you still have the .obj file for the DLL? The docs seem to imply that's necessary for debugging. Alternatively I'd try building with /Zi instead and getting a .pdb for that sucker.
Not sure about Visual C++ (don't have it installed at all anymore), but maybe this will help...
If you were writing a DLL in Delphi and using it from C++, and you wanted to debug the DLL, you'd open the source for the DLL in Delphi and set a breakpoint. You'd then use Run|Parameters and set the C++ application as the host executable and hit run in the Delphi IDE. The IDE would then launch the C++ application and run it as usual until the breakpoint in the DLL was hit, and then would break as you'd expect.
Is something similar available in VC++? (You didn't say which version of VC++, or which version of Visual Studio or the earlier IDE you were using.)
If not, the only alternative I could think of is to do a quick VC++ app that uses the DLL and debug via that instead.
Debug formats are not standardised - basically you can't use Delphi to debug MS compiled code or vice versa.
You can debug Delphi DLLs in Delphi and you can use those DLLs with other apps not compiled with Delphi, provided you mark the Delphi functions for export. What you can't do is debug those DLLs symbolically in a 3rd party debugger, which would have to understand Object Pascal name mangling at the very least.
Have you pointed the debugger at the symbols for your DLL? If there's any doubt, try running with Filemon to see if the debugger is failing when it tries to load the symbols.
Insure that is opening the DLL in the Debug Folder, not another one in some other folder.
Ten years later and this is happening to me, while debugging a custom DLL used in Team Developer, setting debugging command to start the Team Developer IDE. The objective is to step through the 3rd party code to the point of invocation of an exported function from the DLL.
Starting the debugger launches the IDE without error, but running the TD project within the TD IDE causes an exception in VS on a DLL used by the TD IDE.
How do I ignore the exceptions from outside the project? Has anyone been able to get around this since '09?

Resources