Questions about SetWindowsHookEx() and hooking - windows

Here is a little background information. I'm working on replacing a dll that has been used in a dll injection technique via the AppInit_DLLs registry entry. Its purpose was to be present in every process and set hooks into the GDI32.dll to gather information about printing. This is kind of a funky way to get what we want. The .dll itself is over 10 years old (written in Visual Studio 97) and we'd like to replace it with something a little less invasive than an injected dll.
It appears SetWindowsHookEx() maybe what we are looking for. I've been having some trouble with it, but I've also had some discussions with co-workers about whether or not this tree is worth barking up. Here are some questions that we could not determine:
When we hook a routine out of a dll, for example StartDoc() from GDI32.dll, do we really get a notification every time any other process uses that rotuine out of that dll? This is kind of the functionality we were getting with our injected .dll and we need the same functionality going forward.
When the hook is triggered, does the hook handling procedure run in the process space of the process that initiated the actual call, or in the process space of the process that set up the hook? My opinion is that it has to run in the process space of the process that called the routine. For example, if a program calls StartDoc() from GDI32.dll, it will have the hook handling procedure code "injected" into its space and executed. Otherwise, there would have to be some inter-process communication that automatically gets set up between the calling process and the process that set up the hook, and I just don't see that as being the case. Also, its kind of necessary that this hook handling routine run in the process space of the calling process since one of the things it needs to know is the name of that calling process, and I'm not sure on how to get that information if it wasn't actually running in that process.
If the hook handling routine is written using the .NET managed environment, will it break when getting hooked into a process not utilizing the .NET managed environment? We'd really like to move away from C++ here and use C#, but what would happen if we our hook gets called from a process that isn't managed? As stated before, I think that our hook handling procedure will run in the process that originally called the routine that was hooked. But if this is true, than I would think that we'd run into trouble if this process was not using the .NET run time environment, but the incoming hooked handling code is.

Yes.
Generally, it's the former: it executes in the context of the process whose event it is hooking.
After a successful call to SetWindowsHookEx, the operating system automatically injects the hook DLL (the one that contains the callback function) into the address space of all target processes that meet the requirements for the specified hook type. (Of course, the hooking code is not necessarily injected immediately.)
The exception to this general rule are the low-level keyboard and mouse hooks (WH_LL_KEYBOARD and WH_LL_MOUSE). Since those hook types are not injected into the client processes, the callback is called in the same thread that originally called SetWindowsHookEx.
That last point is important to keep in mind to answer your third question. Because the low-level keyboard and mouse hooks are the only two global hooks that do not require DLL injection, they are also the only two types of hooks that can be written in managed .NET code.
For the other hook types, your concerns expressed in the question are precisely correct. You would need to write these hook DLLs in C or C++. Of course, the rest of your application's pieces could still be written in a managed language. The only thing that matters is the hook DLL.
You might consider looking into either Microsoft Detours or EasyHook.

Related

Can Microsoft Detours be used to hook system-wide calls without injecting into every proc?

I want to hook every call to CreateProcess (and a few other related APIs), no matter the process. Can modern Detours be used to do this without enumerating all processes and performing injections?
I've read several articles on both API hooking and doing so specifically with Detours, re:
API Hooking with MS Detours
API Hooking with MS Detours (InfoSec Institute)
API Hooking Revealed
I have two problems with these articles:
They are quite old and things may have changed.
They do not thoroughly address system-wide hooking using Detours specifically.
I am aware that this can be done with a driver, but Detours is said to be a powerful tool, and is still currently supported, so I wanted to know how it could be done with Detours.
The literature I've read thus far is geared toward targeting a specific program using Detours, and although it has presented an idea of how it's possible to enumerate every process and perform a DLL injection and then try to use Detours, this seems like a very unreliable way.
NO
Detours cannot be used globally without per-process injection.
However, you only need to enumerate processes manually once, when you want to set an initial hook after the system is already running. One option (if you are careful with it) is to use the AppInit_DLLs Registry setting to have your DLL loaded into new processes (well, at least processes that use user32.dll, and that don't opt-out of letting AppInit_DLLs run, and providing that AppInit_DLLs is even enabled on the system to begin with).
Otherwise, the alternative is to write a kernel driver that implements a process creation callback that is registered via PsSetCreateProcessNotifyRoutine(). That callback will be called every time a process is created or destroyed.

Can I unload a DLL from another process? (Win32)

I want to unload a DLL from another process. Is it possible?
If yes, how to do that? (I'm using Win32 API)
Thank you very much.
Yes, this is possible. It is called DLL ejection and is featured by some DLL injectors. The way a DLL is usually loaded is via LoadLibrary and it is subsequently unloaded via FreeLibrary. FreeLibrary takes only one parameter which is a handle to the module to be unloaded. If you injected the DLL in the first place, you should be able to find this very easily. Otherwise there are ways of obtaining the handle such as CreateToolHelp32Snapshot with further enumeration with Module32First/Module32Next. Suppose you have obtained the handle through some means, then the steps to eject the DLL are simple:
Get the address of FreeLibrary with GetProcAddress. This address will match the one for the same function in the target because of how Windows works.
Call CreateRemoteThread on the target process, specifying lpStartAddress as the address of FreeLibrary, and with lpParameter as the module's handle
There are several caveats to DLL ejection.
You should only ever eject a DLL which you are certain no code is going to make use of again in the future. If any dynamically linked code attempts to make a call to your code after it has been freed, it will most likely trigger some form of page access violation.
You should ensure that no threads are executing within the code of the DLL whilst ejection is being performed for similar reasons.
DLL ejection should be avoided with general. If the library wants to have the option of being freed, it should supply some interface which users can access it through which eventually calls FreeLibraryAndExitThread.
If you require a code example for this, I have written an ejector as part of an injector I wrote in the past in C. I can search it up and find it but it's from many years ago and the code quality is not likely to be good.
You don't want to do this.
"Loading" a DLL is much more than simply opening (and locking) a file. When the NT loader starts an executable, it processes all the DLLs referenced by the image (recursively) and wires up the function calls (recursively): loading the DLLs, calling the DLL initialization code, etc.
Unloading a DLL would mean that you'd need to stop all processes that loaded your DLL, load the new DLL, and perform all the operations the NT loader would. Of course, unloading and re-loading a DLL would need to restore that old DLL's state (initialized variables etc), an operation which is not specified in Win32.
For a bit of background information, see this article on MSDN and this Under the Hood article in MSJ.
Short answer: No, it is impossible.
Win32 doesn't provide an API to unload a DLL of another process. If a library is freed unexpectedly, the process will crash. This leads to a serious security hole as it breaks process protection mechanism.
If you can modify both of the processes, you can modify the application and add routines to free a library, and let the other application to send the message.
I would instead look to change the function called when the process tries to invoke the functions in that dll. I know this is possible in theory.
It would mean a bit of memory hacking and knowing where the pointers to the functions are stored, but all of that can be found easily enough (ollydbg manages to do it), it would be harder if they use ordinals, even harder if they hard code the pointers, but no one does that nowadays. You could then inject your own code that (ideally) mimics the functions they mask, but does not actually do anything. They will probably have to be injected into the process, and that way you could get it to work without the process ever knowing, and without any crashes.

CreateRemoteThread, WriteProcessMemory, VirtualAllocEx - why use them?

Recently I have been reading up articles about DLL injection and I understand them fairly well.
However, what I don't understand is why APIs such as CreateRemoteThread, WriteProcessMemory(in being able to write to the memory of another process) and VirtualAllocEx(in being able to allocat memory in the context of another process) were implemented in the first place.
What was the original need for such APIs? Just curious.
WriteProcessMemory was made for ring3 debuggers that need to securely write process memory, most commonly for INT 3 breakpoints or user provided memory edits.
along the same line, CreateRemoteThread can also be used for debugging purposes, however, MSDN can enlighten us on CreateRemoteThread a bit more:
A common use of this function is to inject a thread into a process
that is being debugged to issue a break. However, this use is not
recommended, because the extra thread is confusing to the person
debugging the application and there are several side effects to using
this technique:
It converts single-threaded applications into
multithreaded applications.
It changes the timing and memory layout of
the process.
It results in a call to the entry point of each DLL in
the process.
IIRC, CreateRemoteThread is also used by debuggers to hook application native expection handlers, commonly set by SetExceptionHandler, which requires call from the target process as the handler is stored in the PEB.
VirtualAllocEx is just how windows virtual memory system operates, it needs a context to allocate in, be it in the current process, a child process or a remote process. VirtualAlloc in fact is nothing more than a pass through wrapper of the Ex variant, it just passes a special constant that indicates the handle of the caller process is to be used.

Why call CoFreeUnusedLibraries() before OleUninitialize()?

While trying to resolve process hanging on CoUninitialize() I came upon a piece of code shared by many of our projects. When a program is going to quit it first calls CoFreeUnusedLibraries(), then immediately OleUninitialize().
While the effect of OleUninitialize() is quite clear I can't find why one would want to call CoFreeUnusedLibraries() before calling OleUnitialize(). What might be the use of this call at this specific point?
CoFreeUnusedLibraries() will trigger a call to the DllCanUnloadNow for each in-process COM DLL that exports this function. Not sure about threading issues or out-of-process COM components as it relates to this API.
Presumably, someone who wrote the code that calls DllCanUnloadNow before OleUnitialize was attempting to reduce working set and ensure cleanup.
I don't think there's much value in calling CoFreeUnusedLibraries right before application shutdown (the DLLs will get unloaded anyway).
My experience is that calling CoFreeUnusedLibraries results in crashes and hangs in 3rd party COM DLLs that never had their DllCallUnloadNow implementation tested before release. (Because not too many apps call this function).
You didn't provide a call stack or hint as to where the hang was occurring (did you break into a debugger to see what DLL is at the top of the stack?). My guess is that you can likely take this call out if you can't fix the offending DLL.
Docs indicate that
This function is provided for
compatibility with 16-bit Windows.
Hmmm...
Have you seen this problem report? This call seems redundant to me - maybe this leaves one or more DLLs in a state where OleUninitialize does not work properly - waiting for some state change due to the earlier call. However this does allude to the need to wait a while between calls...
CoFreeUnusedLibraries does not
immediately release DLLs that have no
active object. There is a ten minute
delay for multithreaded apartments
(MTAs) and neutral apartments (NAs).
For single-threaded apartments (STAs),
there is no delay. The ten minute
delay for CoFreeUnusedLibraries is to
avoid multithread race conditions
caused by unloading a component DLL.
There are also comments elsewhere re a 6 -minute closedown timeout when using DCOM - is that applicable to you?

How do I dispose of a VB6 COM object i'm using via an interop assembly from IronPython?

I'm using a third party COM component by means of a .NET interop assembly in IronPython, as seen here: How can I use a VB6 COM 'reference' in IronPython?
My experience in this area has been great, I'm very impressed by the amount of stuff that works seamlessly... except one thing.
The 3rd party COM component uses Microsoft's DAO library (not sure what version) to work with a database created by yet another 3rd party application. The problem is that this database file gets recreated during my program's runtime, and I'd like to 'de-initialize' this COM object. In particular, running my code (which accesses the COM component's attributes, calls it's methods, etc) works fine the first time I run my routine. The next time, however, the third party COM component displays a messagebox saying the database (MSAccess 95 .mdb file) is already in use, and offers an option to retry. Pressing retry works reliably, so the main issue is that this dialog comes up.
So, my theory is the COM component is leaking handles to the db, and I can't find a 'cleanup' method to call. I've tried .Dispose(), but that has not worked.
My last resort is making the code that calls to the COM object a separate IronPython process that interacts with my main process via std in/out as the role the COM object serves is more of a 'give me all this information right now' use case instead of a continually required dependency.
I am hoping to avoid that scenario, and since i'm not to familiar with COM (or really, .NET for that matter) I can only hope i'm missing an obvious .Dispose method or the like.
If there's no clean way, can I forcibly unload the assembly from my process, analogous to repeated FreeLibrary calls in native code? (I guarantee that my code won't be using the object anymore, so I shouldn't need to worry about missing refs on my part leading to a memory protection error)
EDIT:
I wasn't able to solve this, so I went the out of process method and let windows clean up when my child batch process shuts down.
Not sure about IronPython, but when working with COM in C#, you need to call Marshal.ReleaseComObject after use to ensure the reference count is properly decremented.
Otherwise you will leak...Dispose() does not do this for you.
http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.marshal.releasecomobject.aspx

Resources