I have an application, which displays me some data. I need to attach to this app's process, find the data I need in memory (one single number, actually), and save it somewhere. This application doesn't seem to use standard windows controls, so things aren't going to be as simple as reading controls data using AutoIt or something similar.
Currently I'm a self-learner database guy and have quite shallow knowledge about windows apps debugging. Not even sure if I asked my question correctly enough.
So, can you give me some starter guidelines about, say, what should I read first, and general directions I should work on?
Thanks.
To read memory of other application you need to open the process with respect of OpenProcess with at least PROCESS_VM_READ access rights and then use ReadProcessMemory to read any memory address from the process. If you are an administrator or have debug privilege you will be able to open any process with maximal access rights, you need only to enable SeDebugPrivilege before (see for example http://support.microsoft.com/kb/131065).
If you don't know a much about the memory of the destination process you can just enumerate the memory blocks with respect of VirtualQueryEx (see How does one use VirtualAllocEx do make room for a code cave? as an example where I examine the program code. The program data you can examine in the same way).
The most practical problem which I see is that you ask your question in too general way. If you explain more what kind of the data you are looking for I could probably suggest you a better way. For example if you could see the data somewhere you could examine the corresponding windows and controls with respect of Spy++ (a part of Visual Studio Tools). The most important are the class of windows (or controls) and the messages which will be send at the moment when the most interesting window are displayed. You can also use Process Monitor to trace all file and registry access at the time when the windows with the interesting information will be displayed. At least at the beginning you should examine the memory of the process with ReadProcessMemory at the moment when the data which you are looking for are displayed on the window.
If you will have no success in your investigations I'd recommend you to insert in your question more information.
My primary advice is: try to find any other method of integration than this. Even if you succeed, you'll be hostage to any kinds of changes in the target process, and possibly in the Windows O/S. What you are describing is behaviour most virus scanners should flag and hinder: if not now, then in the future.
That said, you can take a look at DLL injection. However, it sounds as if you're going to have to debug the heck out of the target process at the disassembly level: otherwise, how are you going to know what memory address to read?
I used to know the windows debugging API but it's long lost memory. How about using ollydbg:
http://www.ollydbg.de/
And controlling that with both ollydbg script and autoit?
Sounds interesting... but very difficult. Since you say this is a 'one-off', what about something like this instead?
Take a screenshot of this application.
Run the screenshot through an OCR program
If you are able to read the text you are looking for in a predictable way, you're halfway there!
So now if you can read a OCR'd screenshot of your application, it is a simple matter of writing a program that does the following:
Scripts the steps to get the data on the screen
Creates a screenshot of the data in question
Runs it through an OCR program like Microsoft Office Document Imaging
Extracts the relevant text and does 'whatever' with it.
I have done something like this before with pretty good results, but I would say it is a fragile solution. If the application changes, it stops working. If the OCR can't read the text, it stops working. If the OCR reads the wrong text, it might do worse things than stop working...
As the other posters have said, reaching into memory and pulling out data is a pretty advanced topic... kudos to you if you can figure out a way to do that!
I know this may not be a popular answer, due to the nature of what this software is used for, but programs like CheatEngine and ArtMoney allow you to search through all the memory reserved by a process for a given value, then refine the results till you find the address of the value you're looking for.
I learned this initially while trying to learn how to better protect my games after coming across a trainer for one of them, but have found the technique occasionally useful when debugging.
Here is an example of the technique described above in use: https://www.youtube.com/watch?v=Nv04gYx2jMw&t=265
Related
I am trying to implement some basic anti debugging functionality in my application. One area that I wanted to focus on in particular, is attempting to prevent people from easily taking a usable memory dump from my application. I read the article at:
http://www.codeproject.com/KB/security/AntiReverseEngineering.aspx
and that gave me a lot of tips for how to detect if a debugger is present, as well as some information on how I might prevent memory dumps. But the author notes that one should be careful about using these techniques, such as removing the executable header in memory. He mentions that there might be times when the OS or other programs may want to use this information, but I cannot see for what purpose.
Has anyone got some other tips as to how I could stop reverse engineers from dumping my program?
I am on Windows.
Kind regards,
Philip Bennefall
There is no reasonable way to prevent someone from capturing a memory dump of your process. For example, I could attach a kernel debugger to the system, break all execution, and extract your process' dump from the debugger. Therefore, I would focus on making analysis more difficult.
Here are some ideas:
Obfuscate and encrypt your executable code. Decrypt in-memory only, and do not keep decrypted code around for longer than you need it.
Do not store sensitive information in memory for longer than necessary. Use RtlZeroMemory or a similar API to clear out buffers that you are no longer using. This also applies to the stack (local variables and parameters).
I must intercept execution in very big application in many places.
What programs I can use to do this? What techniques exists for this problems?
Manually reverse engineering and adding hooks is maybe not optimal solution for this problem, because application is very big and some part of application can be updated in some time, i think with some tools or good practices for this problem i can do this faster, anyone know how to do?
Anybody help me?
seeing as the tools part has been covered, here is something for the techniques.
Depending what it is you need to hook and whether or not there is protection invloved, there are a few methods:
Relative call/jmp patching in the virtualized binary: this is the simplest, but also a lot of work if you can't automatically find all references to a function, this probably won't work in this cause due to your criteria.
IAT/EAT hooking: this is use for imports(IAT) and exports(EAT), great if your targeting a known importted/exported set of API functions. a good example of this can be found here or here
Hot-Patching: Windows XP SP2 introduced something called "hot-patching" (used for realtime system function updates), where all its (the WinAPI) functions start with a 'mov edi,edi', allowing a relative jump to be patched into the free space created above every hot-patchable function(one can do it too). this is generally used for programs that checksum there IAT's or have other funny forms of protection, more info can be found here and here
Code-Caving: capturing execution flow by placing redirections in arbitrary code space. see here, here or here
VFT/COM Redirection: basically overwriting entries in a objects virtual function table, useful for OOP/COM based applications. see this
There are a lot of 3rd party libraries, most famous would probably be MS Detours, one can also look at APIHijack or a mini-hook engine.
Ofcourse nothing can substitute for the initial poking you'll need to do with a debugger like ollydbg, but knowing the method your gonna use can drastically short them amount time time spent poking around
Some details on what exactly you need to do (e.g. how do you determine where to break) would be nice. Depending on your situation, something like Pin might work.
I suggest using Deviare API Hook. It's the easiest way you can do what you need. It has some COM objects that you can use to hook an application from a different process. In your process you get full parameter information and you can use it in any programming language (I'm using C# and it works like a charm).
If you need to intercept registry API I suggest using Deviare to debug what you need to intercept but then you will have to make your own hooks, otherwise, you'll find performance issues.
You can do API Hooking if you are interested in intercepting method calls.
Or use some disassembler like softice or ollydbg or win32dasm.
I couldn't find a suitable title for this. I'm going to express my query with examples.
Consider following softwares:
Process explorer from sysinternals (an advanced task manager)
Resource Manager : resmon.exe (lists each and every fine detail about resource usage about each process).
For me these softwares seems like miracles. I wonder how these are even made. C'mon how a user process can know such fine details about other processes? Who tells this software, what processes are running and what all resources are utilized? Which dlls are used? etc..
Does windows operating system give these software that information? I mean though (obviously the most lower level api) WIN32API. Are there some functions,which on calling return these values
abstractly say:
GetAllRunningProcesses()
GetMemoryUsedByProcess(Process* proc)
etc..
Other similar applications are
network Packet Capture software. How does it get information about all those packets? It clearly sits just infront of the NIC card. How is it possible?
Anti-virus: It scans memory for viruses. Intercepts other processes. Acts like a sandbox for the user application space. How? How??
If its WIN32API. I swear, I'm going to master it.
I don't want to create a multi-threaded application. I want to get information about other multithreaded applications.
I don't want to create a program which communicates using sockets. I want to learn how to learn how to capture all communication packets.
I actually want to work at the lower level. But I don't know, what should I learn. Please guide me in proper direction.
This is really a pretty open-ended question. For things like a list of running processes, look up "PSAPI" or "Toolhelp32". For memory information about a particular process, you can use VirtualQuery.
Capturing network packets is normally done by installing a device driver. If you look, you should be able to find a fair amount about how to write device drivers, though don't expect to create wonders overnight, and do expect to crash your machine a few times in the process (device drivers run in kernel mode, so it's easy for a mistake to crash the machine hard).
I can't say as much with any certainty about anti-virus, because I've never tried to write one. My immediate guess would be that their primary technique is API hooking. There's probably more to it than that, but offhand I've never spent enough time looking at them to know what.
Mark Russinovich's classic, Windows Internals, is the go-to book if you want to get deep in this kind of stuff. I notice that the just-released 5th edition includes Vista. Here's a sample chapter to peek at.
If you like Process Explorer, this is the guy who wrote that, and there are lots of examples using it in the book.
Plus, at 1232 hardcover pages, you can use it to press your clothes.
I am facing a severe problem with my program, which gets reproduced only in the customer place. Putting logs, are not helping as I doubt the failure is happening in a third party dll. For some reasons, I couldn't get help from the library provider. I am thinking of producing a dump at the point of failure, so that to analyze it offline. Is this a recommended practice? Or any alternatives?
Yes, this is something that every program should have and utilize as often as possible.
I suggest that you don't use third party libraries. Create your own dumps instead. It's very simple and straight forward. You basically need to do the following:
Your program needs to access dbghelp.dll. It's a windows dll that allows you to create human readable call stacks etc. The debugger uses this dll to display data in your process. It also handles post mortem debugging, i.e. dumps of some sort. This dll can safely be distributed with your software. I suggest that you download and install Debugging Tools for Windows. This will give you access to all sorts of tools and the best tool WinDbg.exe and the latest dbghelp.dll is also in that distribution.
In dbghelp.dll you call e.g. MiniDumpWriteDump(), which will create the dump file and that's more or less it. You're done. As soon as you have that file in your hands, you can start using it. Either in the Visual Studio Debugger, which probably even might be associated with the .dmp file extension, or in WinDbg.
Now, there are a few things to think of while you're at it. When checking dump files like this, you need to generate .pdb files when you compile and link your executable. Otherwise there's no chance of mapping the dump data to human readable data, e.g. to get good callstacks and values of variables etc. This also means that you have to save these .pdb files. You need to be able to match them exactly against that very release. Since the dump files are date stamped with the date stamp of the executable, the debugger needs the exact pdb files. It doesn't matter if your code hasn't changed a single bit, if the .pdb files belong to another compilation session, you're toast.
I encourage every windows win32 developer to check out Oleg Starodumov's site DebugInfo.com. It contains a lot of samples and tutorials and how you can configure and tune your dump file generation. There are of course a myriad of ways to exclude certain data, create your custom debug message to attach to the dump etc.
Keep in mind that minidumps will contain very limited information about the application state at exception time. The trade off is a small file (around 50-100 kB depending on your settings). But if you want, you can create a full dump, which will contain the state of the whole application, i.e. globals and even kernel objects. These files can be HUGE and should only be used at extreme cases.
If there are legal aspects, just make sure your customers are aware of what you're doing. I bet you already have some contract where you aren't supposed to reveal business secrets or other legal aspects. If customers complain, convince them how important it is to find bugs and that this will improve the quality of the software drastically. More or less higher quality at the cost of nothing. If it doesn't cost them anything, that's also a good argument :)
Finally, here's another great site if you want to read up more on crash dump analysis: dumpanalysis.org
Hope this helps. Please comment if you want me to explain more.
Cheers !
Edit:
Just wanted to add that MiniDumpWriteDump() requires that you have a pointer to a MINIDUMP-EXCEPTION-INFORMATION (with underscores) struct. But the GetExceptionInformation() macro provides this for you at time of exception in your exception handler (structured exception handling or SEH):
__try {
}
__except (YourHandlerFunction(GetExceptionInformation())) {
}
YourHandlerFunction() will be the one taking care of generating the minidump (or some other function down the call chain). Also, if you have custom errors in your program, e.g. something happens that should not happen but technically is not an exception, you can use RaiseException() to create your own.
GetExceptionInformation() can only be used in this context and nowhere else during program execution.
Crash dumps are a pretty common troubleshooting method and can be very effective, especially for problems that only reproduce at the customer's site.
Just make sure the customer/client understands what you're doing and that you have permission. It's possible that a crash dump can have sensitive information that a customer may not want (or be permitted) to let walk out the door or over the wire.
Better than that there are libraries that will upload crash data back you.
BugDump and BugSplat
And there's the Microsoft way:
http://msdn.microsoft.com/en-us/library/aa936273.aspx
Disclaimer: I am not a lawyer, nor do I pretend to be one, this is not legal advice.
The data you can include in logs and crash dumps also depend on what domain you are working in. For example, medical equipment and patient information systems often contain sensitive data about patients that should not be visible to unauthorized persons.
The HIPAA Privacy Rule regulates
the use and disclosure of certain
information held by "covered entities"
(...) It establishes regulations for
the use and disclosure of Protected
Health Information (PHI). PHI is any
information held by a covered entity
which concerns health status,
provision of health care, or payment
for health care that can be linked to
an individual.[10] This is interpreted
rather broadly and includes any part
of an individual's medical record or
payment history. --Wikipedia
It should not be possible to link health information to an individual. The crash dumps and logs should be anonymized and stripped of any sensitive information, or not sent at all.
Maybe this does not apply to your specific case, so this is more of a general note. I think it applies to other domains that handle sensitive information, such as military and financial, and so on.
Basically the easiest way to produce a dumpfile is by using adplus. You don't need to change your code.
Adplus is part of the debugging tools for windows, as mentioned in the article above.
Adplus is basically a huge vbscript automation of windbg.
What you have to do to use adplus:
Download and install Debugging tools for windows to c:\debuggers
start your application
open a commandline and navigate to c:\debuggers
run this line "adplus -crash your_exe.exe"
reproduce the crash
you'll get a minidump with all the information you need.
you can open the crash dump in your favorite debugger.
within windbg, the command "analyze -v" helped me in at least 40% of all the crashes that only happened at customer site and were not reproducible in house.
I am trying to help a client with a problem, but I am running out of ideas. They have a custom, written in house application that runs on a schedule, but it crashes. I don't know how long it has been like this, so I don't think I can trace the crashes back to any particular software updates. The most unfortunate part is there is no longer any source code for the VB6 DLL which contains the meat of the logic.
This VB6 DLL is kicked off by 2-3 function calls from a VB Script. Obviously, I can modify the VB Script to add error logging, but I'm not having much luck getting quality information to pinpoint the source of the crash. I have put logging messages on either side of all of the function calls and determined which of the calls is causing the crash. However, nothing is ever returned in the err object because the call is crashing wscript.exe.
I'm not sure if there is anything else I can do. Any ideas?
Edit: The main reason I care, even though I don't have the source code is that there may be some external factor causing the crash (insufficient credentials, locked file, etc). I have checked the log file that is created in drwtsn32.log as a result of wscript.exe crashing, and the only information I get is an "Access Violation".
I first tend to think this is something to do with security permissions, but couldn't this also be a memory access violation?
You may consider using one of the Sysinternals tools if you truly think this is a problem with the environment such as file permissions. I once used Filemon to figure out all the files my application was touching and discovered a problem that way.
You may also want to do a quick sanity check with Dependency Walker to make sure you are actually loading the DLL files you think you are. I have seen the wrong version of the C runtime being loaded and causing a mysterious crash.
Depending on the scope of the application, your client might want to consider a rewrite. Without source code, they will eventually be forced to do so anyway when something else changes.
It's always possible to use a debugger - either directly on the PC that's running the crashing app or on a memory dump - to determine what's happening to a greater or lesser extent. In this case, where the code is VB6, that may not be very helpful because you'll only get useful information at the Win32 level.
Ultimately, if you don't have the source code then will finding out where the bug is really help? You won't be able to fix it anyway unless you can avoid that code path for ever in the calling script.
You could use the debugging tools for windows. Which might help you pinpoint the error, but without the source to fix it, won't do you much good.
A lazier way would be to call the dll from code (not a script) so you can at least see what is causing the issue and inspect the err object. You still won't be able to fix it, unless the problem is that it is being called incorrectly.
The guy of Coding The Wheel has a pretty interesting series about building an online poker bot which is full of serious technical info, a lot of which is concerned with how to get into existing applications and mess with them, which is, in some way, what you want to do.
Specifically, he has an article on using WinDbg to get at important info, one on how to bend function calls to your own code and one on injecting DLLs in other processes. These techniques might help to find and maybe work around or fix the crash, although I guess it's still a tough call.
There are a couple of tools that may be helpful. First, you can use dependency walker to do a runtime profile of your app:
http://www.dependencywalker.com/
There is a profile menu and you probably want to make sure that the follow child processes option is checked. This will do two things. First, it will allow you to see all of the lib versions that get pulled in. This can be helpful for some problems. Second, the runtime profile uses the debug memory manager when it runs the child processes. So, you will be able to see if buffers are getting overrun and a little bit of information about that.
Another useful tool is process monitor from Mark Russinovich:
http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx
This tool will report all file, registry and thread operations. This will help you determine if any you are bumping into file or registry credential issues.
Process explorer gives you a lot of the same information:
http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx
This is also a Russinovich tool. I find that it is a bit easier to look at some data through this tool.
Finally, using debugging tools for windows or dev studio can give you some insight into where the errors are occurring.
Access violation is almost always a memory error - all the more likely in this case because its random crashing (permissions would likely be more obviously reproducible). In the case of a dll it could be either
There's an error in the code in the dll itself - this could be something like a memory allocation error or even a simple loop boundary condition error.
There's an error when the dll tries to link out to another dll on the system. This will generally be caused by a mismatch between dll versions on the machine.
Your first step should be to try and get a reproducible crash condition. If you don't have a set of circumstances that will crash the system then you cannot know when you have fixed it.
I would then install the system on a clean machine and attempt to reproduce the error on that. Run a monitor and check precisely what other files (dlls etc) are open when the program crashes. I have seen code that crashes on a hyperthreaded Pentium but not on an earlier one - so restoring an old machine as a testbed may be a good option to cover that one. Varying the amount of ram in the machine is also worthwhile.
Hopefully these steps might give you a clue. Hopefully it will be an environment problem and so can be avoided by using the right version of windows, dlls etc. However if you're still stuck with the crash at this point with no good clues then your options are either to rewrite or attempt to hunt down the problem further by debugging the dll at assembler lever or dissassembling it. If you are not familiar with assembly code then both of these are long-shots and it's difficult to see what you will gain - and either option is likely to be a massive time-sink. Myself I have in the past, when faced with a particularly low-level high intensity problem like this advertised on one of the 'coder for hire' websites and looked for someone with specialist knowledge. Again you will need a reproducible error to be able to do this.
In the long run a dll without source code will have to be replaced. Paying a specialist with assembly skills to analyse the functions and provide you with flowcharts may well be worthwhile considering. It is good business practice to do this sooner in a controlled manner than later - like after the machine it is running on has crashed and that version of windows is no longer easily available.
You may want to try using Resource Hacker you may have luck de-compiling the in house application. it may not give you the full source code but at least maybe some more info about what the app is doing, which also may help you determine your culrpit.
Add the maximum possible RAM to the machine
This simple and cheap hack has work for me in the past. Of course YMMV.
Reverse engineering is one possibility, although a tough one.
In theory you can decompile and even debug/trace a compiled VB6 application - this is the easy part, modifying it without source, in all but the most simple cases, is the hard part.
Free compilers/decompilers:
VB decompilers
VB debuggers
Rewrite would be, in most cases, a more successful and faster way to solve the problem.