I'm looking to develop a program that detects calls to certain Windows API functions and simply records the calling process, call count, and hopefully their arguments, to later mark them as benign or malicious.
The GUI program API monitor is a good example of the functionality I'm trying to achieve. Ideally I would like to track each desired API function individually and get the caller PID and parameters when or after it is used, without user input. The program should be able to run on any windows 7 machine, but can be limited to 32bit applications.
I understand there are several methods of hooking a function, and from my understanding Microsoft detours implements one of these, but I don't know if its the one best suited to what I want to do. I've seen detours, easyhook, deviare API hook, and others mentioned on very old posts, but I have a hard time getting my head around the differences and features of each.
So my question is, given what I'm trying to do, what do you recommend and why?
For reference I'm an intermediate level programmer, but a beginner at Windows programming.
Thanks for your help
I'm part of Nektra Deviare API Hook team. Our hooking engine is used by a large number of companies all over the world, in different types of end-user products (e.g.: Anti Virus, Data Loss Prevention, AI, handicapped software, Data Classification, App virtualization).
Deviare-InProc is the MS Detours replacement and Deviare2 has built in all the RPC you need to hook another process and get the calls in your own process.
We continuously fix all reported issues. You can verify it in our GitHub:
https://github.com/nektra/Deviare2
https://github.com/nektra/Deviare-InProc
You can see Deviare2 running in Nektra's SpyStudio API Monitor.
Detours is an excellent software but very expensive (USD 10k). In addition to this, it completely lacks of support. It can be compared to Deviare InProc,
EasyHook used to be a good start point because it was the only free option. But, now Deviare2 family is open source and EasyHook has a lot of stability issues for the real world.
Related
I was told by someone with a lot of experience that there is at least one alternate way to monitor Windows OS calls, in the context of for example, creating Process Monitor-like functionality or developing an anti-virus scanner, which is not "hooking" and that this alternate method is more efficient than hooking. However, I am unsure of which alternatives to hooking there are in the context of developing a filter driver to perform this task.
No. There is no equivalent for SSDT hooking but this does't mean you can not monitor events. We have several CBs in KM which can be used for monitoring and altering the data/access rights, etc. If you're talking about AV scanner those CBs are sufficient most of the time. I think it's better to describe for what you need this. Then it's easier to say what to use instead.
side note this has be put as a comment. I think it's crazy that you have to earn more than 50 rep to be able to put comments!!! Comments are useful for better understanding the questions and clarify things. Why someone needs to have rep for that?!
On Windows, all disk I/O ultimately happens via Win32 API calls like CreateFile, SetFilePointer, etc.
Now, is it possible to intercept these disk I/O Win32 calls and hook in your own code, at run time, for all dynamically-linked Windows applications? That is, applications that get their CreateFile functionality via a Windows DLL instead of a static, C library.
Some constraints that I have are:
No source code: I won't have the source code for the processes I'd like to intercept.
Thread safety: My hook code may dynamically allocate its own memory. Further, because this memory is going to be shared with multiple intercepted processes (and their threads), I'd like to be able to serialize access to it.
Conditional delegation and overriding : In my hook code, I would like to be able to decide whether to delegate to the original Win32 API functionality, or to use my own functionality, or both. (Much like the optional invocation of the super class method in the overriding method of the subclass in C++ or Java.)
Regular user-space code: I want to be able to accomplish the above without having to write any device-driver, mainly due to the complexity involved in writing one.
If this is possible, I'd appreciate some pointers. Source code is not necessary, but is always welcome!
You may want to look into mhook if Detours isn't what you want.
Here are a couple of problems you may run into while working with hooks:
ASLR can prevent injected code from intercepting the intended calls.
If your hooks are global (using AppInit_DLLs for example), only Kernel32.dll and User32.dll are available when your DLL is loaded. If you want to target functions outside of those modules, you'll need to manually make sure they're available.
I suggest you start with Microsoft Detours. It's free edition also exists and its rather powerful stable as well. For injections you will have to find which injection method will work for your applications in target. Not sure whether you need to code those on your own or not, but a simple tool like "Extreme Injector" would serve you well for testing your approaches. And you definitely do not need any kernel-land drivers to be developed for such a simple task, in my opinion at least. In order to get the full help of me and others, I'd like to see your approach first or list more constraints to the problem at hand or where have you started so far, but had problems. This narrows down a lot chit-chats and can save your time as well.
Now, if you are not familiar with Detours from Microsoft (MSFT) please go ahead and download it from the following link: http://research.microsoft.com/en-us/projects/detours/ once you download it. You are required to compile it yourself. It's very straightforward and it comes with a compiled HTML help file and samples. So far your profiles falls under IAT (Import Address Table) and EAT (Export Address Table).
I hope this non-snippet answer helps you a little bit in your approach to the solution, and if you get stuck come back again and ask. Best of luck!
I must intercept execution in very big application in many places.
What programs I can use to do this? What techniques exists for this problems?
Manually reverse engineering and adding hooks is maybe not optimal solution for this problem, because application is very big and some part of application can be updated in some time, i think with some tools or good practices for this problem i can do this faster, anyone know how to do?
Anybody help me?
seeing as the tools part has been covered, here is something for the techniques.
Depending what it is you need to hook and whether or not there is protection invloved, there are a few methods:
Relative call/jmp patching in the virtualized binary: this is the simplest, but also a lot of work if you can't automatically find all references to a function, this probably won't work in this cause due to your criteria.
IAT/EAT hooking: this is use for imports(IAT) and exports(EAT), great if your targeting a known importted/exported set of API functions. a good example of this can be found here or here
Hot-Patching: Windows XP SP2 introduced something called "hot-patching" (used for realtime system function updates), where all its (the WinAPI) functions start with a 'mov edi,edi', allowing a relative jump to be patched into the free space created above every hot-patchable function(one can do it too). this is generally used for programs that checksum there IAT's or have other funny forms of protection, more info can be found here and here
Code-Caving: capturing execution flow by placing redirections in arbitrary code space. see here, here or here
VFT/COM Redirection: basically overwriting entries in a objects virtual function table, useful for OOP/COM based applications. see this
There are a lot of 3rd party libraries, most famous would probably be MS Detours, one can also look at APIHijack or a mini-hook engine.
Ofcourse nothing can substitute for the initial poking you'll need to do with a debugger like ollydbg, but knowing the method your gonna use can drastically short them amount time time spent poking around
Some details on what exactly you need to do (e.g. how do you determine where to break) would be nice. Depending on your situation, something like Pin might work.
I suggest using Deviare API Hook. It's the easiest way you can do what you need. It has some COM objects that you can use to hook an application from a different process. In your process you get full parameter information and you can use it in any programming language (I'm using C# and it works like a charm).
If you need to intercept registry API I suggest using Deviare to debug what you need to intercept but then you will have to make your own hooks, otherwise, you'll find performance issues.
You can do API Hooking if you are interested in intercepting method calls.
Or use some disassembler like softice or ollydbg or win32dasm.
I don't mean for this question to be a flame bait but I'll be using Microsoft and their win32 API as a example of a legacy API.
Now what I am wondering here is Microsoft is spending a lots of their money and energy in maintaining their legacy API, including all of the "glitches/bugs/workaround" that are needed to keep the API functioning the same. Now I'm aware that in Windows 7 they are providing a way for the customer to run their application in a "Windows XP" VM which would be one such way for them to start cleaning up their win32 API because they could then push all of the application into the "Windows XP" VM.
So now what I am wondering is, is it possible to virtualization a legacy API in such way that an customer/program can still access and use it, yet at the same time be able to take advantage of the newer version/API? Because as far as I understand it, if the application is ran in the "Windows XP" VM, it won't be able to access any of the newer API/feature of Windows 7.
The thing that puzzles me about this question when it comes up is that Windows has been doing this since NT came out in the mid nineties. This is how NT runs DOS and Win16 programs, and how it always has. The NTVDM virtualization layer runs 16-bit apps under Win32 with very little special support from the core OS. This is just one example - another is WINE, which as I understand it does a pretty reasonabl job of running windows apps on top of an API set which is very different from that of windows. So it is definitely possible.
The more pertinent question would be why Microsoft would consider it. In order for you to think it is necessary you have to think two things. 1) There is something better to replace the win32 API with and 2) Maintaining the Win32 API is a burden.
Both of these are questionable. In the case of kernel duties, such as accessing hardware and synchronizing and doing threads and processes and memory the Win32 API does a pretty good job, and is ultimately quite close to what the kernel really does. If you think there is a better API then that must mean there is also a better kernel. I personally don't think that NT needs replacing right now. For graphics and windowing, admitedly gdi32 is a bit long in the tooth. But Microsoft solved that problem by building WPF right alongside it. This then brings in the burden question. Well, sure there are two APIs to maintain, but if you virtualized GDI on top of WPF you'd still have to maintain both anyway so there is no benefit there. The advantage of running both in parallel is that GDI already exists and is already tested. All you have to do is to fix the occasional bug, whereas a new virtualization layer would have to be written and tested all over again, which takes time away from making WPF better.
In terms of maintaining back compat, that isn't as much of a burden as it sounds. It is mainly a test question - you have to test that the API behaviour doesn't change, but again - those tests have already been written, so it isn't really any extra work.
So, to answer a question with a question, why would they bother?
This is an interesting question, at least to me, here are some of my thoughts.
Your understanding is correct, an application running in the XP VM only has access to the Win32 APIs provided by XP in the VM. One of the many ways that I have seen Microsoft's approach to enhancing specific APIs is to create new functions with the enhanced/fixed functionality and name the new function by append Ex and even ExEx to the original name, for example
GetVersion
GetVersionEx
For functions that accept pointers to structures, the structures are 'versioned' by using the size of the structure to determine the functionality required, so older code would be passing a previous size of the structure while newer code would be passing in the newer larger strucure and the API functions accordingly.
I guess, the problem has become that it is no longer just differences in how an API works, but more integral to the functioning of the operating system and the internal structures which have changes significantly enough that arguably badly written code is effectively broken.
As to your actual question, I guess it would be quite tough. Even if one thought to let the OS adjust how it executes code based on a target OS version in the PE header of the executable, what would happen if a newer DLL was loaded into the process that targeted the latest OS, now how should the OS handle this when the code is executing? IMHO, I think this would be very challenging, one frought with pitfalls that would ultimately fail.
Of course that is just my raw thoughts on the topic so I might be 100% wrong and there is some simple approach that just did not come to mind.
I couldn't find a suitable title for this. I'm going to express my query with examples.
Consider following softwares:
Process explorer from sysinternals (an advanced task manager)
Resource Manager : resmon.exe (lists each and every fine detail about resource usage about each process).
For me these softwares seems like miracles. I wonder how these are even made. C'mon how a user process can know such fine details about other processes? Who tells this software, what processes are running and what all resources are utilized? Which dlls are used? etc..
Does windows operating system give these software that information? I mean though (obviously the most lower level api) WIN32API. Are there some functions,which on calling return these values
abstractly say:
GetAllRunningProcesses()
GetMemoryUsedByProcess(Process* proc)
etc..
Other similar applications are
network Packet Capture software. How does it get information about all those packets? It clearly sits just infront of the NIC card. How is it possible?
Anti-virus: It scans memory for viruses. Intercepts other processes. Acts like a sandbox for the user application space. How? How??
If its WIN32API. I swear, I'm going to master it.
I don't want to create a multi-threaded application. I want to get information about other multithreaded applications.
I don't want to create a program which communicates using sockets. I want to learn how to learn how to capture all communication packets.
I actually want to work at the lower level. But I don't know, what should I learn. Please guide me in proper direction.
This is really a pretty open-ended question. For things like a list of running processes, look up "PSAPI" or "Toolhelp32". For memory information about a particular process, you can use VirtualQuery.
Capturing network packets is normally done by installing a device driver. If you look, you should be able to find a fair amount about how to write device drivers, though don't expect to create wonders overnight, and do expect to crash your machine a few times in the process (device drivers run in kernel mode, so it's easy for a mistake to crash the machine hard).
I can't say as much with any certainty about anti-virus, because I've never tried to write one. My immediate guess would be that their primary technique is API hooking. There's probably more to it than that, but offhand I've never spent enough time looking at them to know what.
Mark Russinovich's classic, Windows Internals, is the go-to book if you want to get deep in this kind of stuff. I notice that the just-released 5th edition includes Vista. Here's a sample chapter to peek at.
If you like Process Explorer, this is the guy who wrote that, and there are lots of examples using it in the book.
Plus, at 1232 hardcover pages, you can use it to press your clothes.