QTAgent32.exe keeps a handle to a DLL open after execution - visual-studio-2010

I have run into a bit of a problem using Visual Studio 2010's unit test framework. Currently the QTAgent32 will maintain a reference to a DLL after the execution of a test has finished.
The DLL in question is a c++/cli wrapper around some native c++ code. The object exposed by the wrapper is actually never assigned to by the managed code. The only reference it makes is a final check during disposal to see if it has been set during the class's life time.
If(_obj != null)
{
_obj.Dispose();
_obj = null;
}
I know this for a fact since if i step through the code and watch the debugger console output I can see the symbols for the DLL are not loaded until hitting the if (which makes sense). All managed objects involved implement IDisposable to make sure that all native objects are taken care to avoid memory leaks.
Based on this question: QTAgent32 Holding File Open I made sure that no filestreams were explicitly opened (including Console) and no files are even used yet the problem remains. I am running out of ideas on what to do.
Can anyone help?
TL;DR: QTAgent32.exe keeps an open reference to a c++/cli wrapper that is never instantiated.

I ran into the same problem while using a Fortran DLL. The problem persisted even if the DLL function was completely empty.
I still don't know what the issue is, but an easy workaround is to specify the killing of the QTAgent32 process as a pre-build event in your project.
taskkill /f /im QTAgent32.exe
exit 0

It probably opens the DLL in metadata only mode in order to search for test attributes. This wouldn't trigger symbol loading, but it could lock the file.

Related

Easiest way to remove dependency on certain ActiveX library in MFC project?

I am working on greatly simplifying an old MFC application, and I'd like to use that opportunity to remove a dependency on a certain third-party ActiveX library that has been causing headaches. I first simply tried to remove all references to these controls in the cpp code. It compiles fine, but when installed on a new computer, it crashes if the library .msm file is not included in the setup project.
The problem now is that I don't know where any remaining references to these objects in the project is, and therefore have no easy way of tracking them down and removing them. Simply using the Find-functionality in VS only returns results in cpp files which I've already cleaned up, not any control instances in f.ex. dialog resource files. Is there a way to search for all these objects in the project, or check for them compile-time so that I can use compile errors to see where any library objects remain?
First of all: CoCreateInstance, CoGetClassObject are the normal ways how COM Objects are created. Should be easy to set break points here or to set a hook to monitor what's going on.
Seams that you have a bad error handling in you software. Otherwise you application should show an error message if an object could not be created.
Try to debug.... use remote debugging, even a crash dump will lead you to the code location.
Easiest approach: In your development machine. Try to look into the output window and see if any of the old ActiveX DLLs gets loaded. Trigger down to the location were any of those objects is created.
You can do the same with remote debugging on the test machine.

VB6 Service debugging with WINDBG

First and foremost, I'm an absolute newbie at debugging compiled code (until recently, I didn't even know it was possible!). I've successfully used the VC6 IDE to attach to a process but I can't get WinDbg to work...
The problem is that I have a VB6 service that I need to debug in a production server. I really don't understand much of how the .EXE is compiled because it is generated by an external tool called "NT Service Toolkit". The code I have is compiled into an ActiveX DLL that is used by this .EXE.
I've done everything I found on the internet to make the service debuggable with the VC6 IDE: compile without optimizations and generate symbols. When I use the VC6 IDE to attach to it, it works as expected... I can set breakpoints and everything works fine except for the annoying bug that kills the service when the debugging session stops.
When I try to use WinDbg however, I cannot set breakpoints; it fails with an error:
Unable to insert breakpoint 0 at 00000000`0046f715, Win32 error 0n299
"Only part of a ReadProcessMemory or WriteProcessMemory request was completed."
bp0 at 00000000`0046f715 failed
WaitForEvent failed
I've already downloaded symbols for windows' components from the symbol servers, as instructed in the documentation I could find, and I also included the PDB file for my service in the symbols folder...
One thing I noted is that I can get symbolic information when I check the "Noninvasive" box when selecting the process to attach to because I see things like wow64win!ZwUserGetMessage instead of just the address`offset syntax... but when I do, I also get this warning:
WARNING: Process <PID> is not attached as a debuggee
The process can be examined but debug events will not be received
In this case, I can set breakpoints, but when I try to run the service, it fails with:
0:000> g
^ No runnable debuggees error in 'g'
UPDATE
When I set a breakpoint in "Noninvasive" mode, detach, and reattach with "Noninvasive" un-checked, the breakpoints I set while in "Noninvasive" mode remain, and are hit successfully, but I can't set new breakpoints unless I repeat the whole process (detach, attach in "Noninvasive" mode and reattach). Does this make sense? What could I be doing wrong?
I found the problem. It was a conflict in the module names.
When WinDbg loads the modules for a process, it names them as the file where they live, just WITHOUT extension. Therefore, if, as in my case, two modules that must be loaded share the same name, say MyService.exe and MyService.dll, there will be two modules named MyService; in this case, WinDbg breaks the ambiguity by appending _<start-address> to the name, resulting in, for example:
...
MyService
MyService_11000000
...
(The following is a rationalization of what happened, since like I said, I'm pretty new to "attach to process"-type debugging and I lack knowledge of WinDbg internals...)
My problem was that the PDB file for the DLL (MyService.pdb), which was the module I was interested in debugging, was being matched to the EXE file's module (MyService), so the DLL's symbols were not loaded before trying to "restore" the breakpoints, resulting in the Win32 error 0n299 I was seeing earlier... It appears that the g command also tries to "set" the saved breakpoints, and since the symbols were not yet loaded, it would not let me start the thing after the default breakpoint either...
...
Or something like that... Obviously, if someone has a more accurate explanation, I'd very much like to be enlightened. :)
For the moment, I just changed the service EXE's name, and now everything works fine.

debugging a dynamically loaded executable

I've been handed an application to support, and I'm trying to figure out how to do it. I do have the source, and can make some changes, but I obviously don't want to completely change the architecture of the application.
The app is in a VS2010 solution composed of 9 different projects. The main one is a Windows Form application, but it spins off others in other threads.
Here's the difficulty. Even though the different projects are parts of the same solution, they are separate executables, not DLLs. When the main program starts one of the other projects, it does so by creating a new process, setting the filename of the executeable, the startup arguments and other assorted parameters into the process.StartInfo object, and then calls process.Start().
How can I set breakpoints and debug subordinate executables? I can't attach to them until they are loaded, but they don't get loaded until process.Start() is called, and by then it's too late. Is there a method call I can insert into the main program to get it to load the executable (so I can set breakpoints in it) before it actually begins execution?
Thanks.
Are you able to recompile the other executables? If so, have you tried putting DebugBreak in suitable places? (or _asm int 3).
You can't load the process (usefully), since by definition it will be run in a different addres space from the one you are debugging before it starts.
One simple solution could be adding a call to DebugActiveProcess function to the "main" function of every process which participates in your application.

Debugging across projects in VS2008?

We have a DLL which provides the data layer for several of our projects. Typically when debugging or adding a new feature to this library, I could run one of the projects and Step Into the function call and continue debugging code in the DLL project. For some reason, that is no longer working since we switched to Visual Studio 2008... It just treats the code from the other project as a DLL it has no visibility into, and reports an exception from whatever line it crashes on.
I can work around that by just testing in the DLL's project itself, but I'd really like to be able to step in and see how things are working with the "real" code like I used to be able to do.
Any thoughts on what might have happened?
Is the pdb file for the dll in the same directory as the dll? This should all work -- I do just this on a regular basis. Look in the Modules window which will show you whether it's managed to load symbols for the dll. If it hasn't then you won't be able to step into functions in that dll.
It sounds like you have "Just My Code" enabled and VS is considering the other projects to not be your code. Try the following
Tools -> Options -> Debugger
Uncheck "Just my Code"
Try again
I've gotten around this issue by opening a class that will be called in the project you need, placing a breakpoint, keep the file open, and run the debugger. The debugger will hit the breakpoint and the relative path that VS uses will be updated so that future classes will be opened automagically.

CComPtr CoCreateInstance() fails

I have a COM .dll registered successfully with regsvr32 but somehow CoCreateInstance() fails to create one of its interfaces. Is there a freeware tool which can determine the reason for the failure?
First of all, check the return value of the CoCreateInstance() call. Second, you can use a tool like Regmon or Process Monitor to see what registry lookup fails. This way, you can quickly determine what exactly wasn't registered the way you'd expect it to be.
If your com dll is implemented in C++ & has debug info, you could also try debugging with MSVC to step into CoCreateInstance.
My guess is that you missed associating your class with one of the interfaces properly. I've done that a number of times by mistake. If you're using ATL you need to make sure your implementing class derives from the interface & also you have added COM_INTERFACE_ENTRY(I____) for your interface in the COM_MAP:
BEGIN_COM_MAP(CFileHelper)
COM_INTERFACE_ENTRY(IFileHelper)
COM_INTERFACE_ENTRY(IDispatch)
COM_INTERFACE_ENTRY(IStream)
COM_INTERFACE_ENTRY(ISupportErrorInfo)
END_COM_MAP()
Or maybe the GUID differs between the IDL file and what's in the implementing C++ file. MSVC6 has an annoying bug where if the wizard fails to create a new class because of file permissions (e.g. some of the files it wants to change are read-only) but it has already added a GUID to the IDL file, and you fix the file permissions and go to do it again, there will be an inconsistency in the GUIDs and it's a pain to catch this. In an ideal world, the GUID definitions would reside ONLY in one file and then you wouldn't have to worry about this.
If not that, sometimes there are weird errors regarding marshaling & apartments but that only pops up if you've got multiple threads and are sending interface pointers across thread or interface boundaries.

Resources