I'm building a Haskell command-line application in Windows 10, and am trying to debug an issue around the Windows 260-character file path limitation by tracing system calls and seeing which ones fail.
I've used procmon (https://learn.microsoft.com/en-us/sysinternals/downloads/procmon) for this, which seems quite nice, but although it displays many related log entries, I was surprised to find that it doesn't display an entry for the particular CreateFileW call that actually exceeds 260 chars and crashes my application.
I briefly tried Win32 API Monitor (https://www.apimonitor.com) but couldn't make heads or tails of it; it seems better suited to attaching to already-running GUI applications than command-line applications that need to be launched in a particular directory, etc.
Is there a better alternative to these, or a better approach?
#RemyLebeau's comment was the one I needed:
"I was surprised to find that it doesn't display an entry for the
particular CreateFileW call that actually exceeds 260 chars and
crashes my application"
Because there is nothing for it to log. The call likely gets rejected at the higher level API layer, when the input data is first
validated, long before it reaches the file system layer. Procmon logs
lower level activity, not higher level APIs. Tools like API Monitor
are what you are looking for. If you are having trouble with it, ask a
new question about that.
My eventual solution to the problem that inspired this was to upgrade from base-4.11 to base-4.12 which handles windows paths > 260 chars better.
I don't think I'd even need the registry switch anymore.
Related
I am trying to understand how IOFilterScheme KEXTs work in order to eventually develop one myself. I have tried a few sample programs and have gotten basic encryption to work, for example by using this sample.
However, when I add printf() statements and look at the console logs, I am seeing some confusing behavior. Specifically, I almost never see read() calls coming in, except for a few processes when I configure things (like mount_hfs and fsck_hfs).
For example, if I write a new file on a Volume (from a mounted disk image) from some application (ex: vim), when I will see a corresponding write() in the console logs, with the proper PID of 'vim'. I see this when using other applications as well.
However, if I try to read that same file from another application (say, Sublime Text editor), the file opens fine, but I never see any corresponding read() entry in the console log.
While I can tell that the sample encryption is working by looking at the DMG file, I have two problems with the behavior I am seeing:
1) It is hard for me to understand what is going on in terms of reads and writes.
2) Eventually I would like to write a KEXT where the behavior differs depending on the application that is going the reading or writing. To do this, I would need an actual read() from each application that tries to access the file (at least the first time that app does).
After doing some research it seems that block devices on Mac OS have some sort of buffering, but I was not able to find too many details. Experimentally, I tried executing this line in the read() and write() calls, but it had no effect
super::IOStorage::synchronize(client, 0, 0);
If someone can tell me how I can gain more control over the buffering so I can see actual read() calls coming in, that would be great. If that is not possible, then I might have to write my encryption driver at a different level. However, an IOFilterScheme KEXT seems like (except for this point) it really fits my use case, so I am hoping I can make it work.
The caching mostly happens at the file level, in the Unified Buffer Cache (UBC). The file system itself may do any kind of caching, especially for metadata (internal tree structures, etc.). This is many layers above the IOKit, so you have no influence over any of this from an IOStorage driver.
2) Eventually I would like to write a KEXT where the behavior differs depending on the application that is going the reading or writing. To do this, I would need an actual read() from each application that tries to access the file (at least the first time that app does).
Trying to do this in a block I/O kext is probably a fools' errand. Modern file systems are complex beasts, so you can't expect any kind of 1:1 correspondence between file I/O and block I/O. If you want application/file level granularity, you'll need to work at the VFS layer, either through your own file system or depending on what exactly you're trying to do, using kauth. If you're not afraid of using APIs Apple has specifically said are not to be used (e.g. internal use/learning), you can also use the MAC framework kext APIs, which give you more control than kauth.
I am looking for an approach to register to XML-based performance counters without using lodctr executable.
While this question and this question has given some insight to the problem, none gives a proper resolution.
The function LoadPerfCounterTextStrings seems to do the trick, but doesn't seem to take XML as input (i.e. /M argument for lodctr command). The function would anyway call the lodctr command but will save the programmer from calling lodctr.exe programmatically.
Is there an approach to call this function (or any other) that would behave as if:
lodctr.exe /M:xmlfile.xml
?
Just an additional information that LoadPerfCounterTextStrings is available in Windows XP, however, the new API (perflib 2) is available only after Vista. I am using V2.
I don't think it is possible, at least with a documented way. If you'd find and use some undocumented way, it may break with some OS/service pack or with future versions of Windows.
Best bet is you call LODCTR and UNLODCTR for installation/registration of performance counters within your process. Since these are command line tools, they would probably show up with a command (black) console window. You may hide with input/output redirection with pipes, so that console window doesn't appear.
Note that these tools don't take up much of time or resources, so calling these EXEs from your process won't take up much time (user won't notice, and even won't be able to easily figure out with some process monitoring tool).
Note that registering/unregistering PC is a rare activity - hence Microsoft won't bother to provide you a nice callable Windows API.
I am running Windows 7 on which I want to do kernel debugging and I do not want to mess with boot loader. So I've downloaded LiveKd as suggested here and make it run and seems it is working. If I understand correct it is some kind of read only debugging. Here is mentioned that it is very limited and even breakpoint cannot be used. I would like to ask if is possible in this mode to periodically dump all the instructions that are being executed or basically all events which are happening on current OS? I would like to have some system wide strace (Linux users know) and to do some statistical analysis on this. I suppose it depends on more factors like installed debug symbols to begin able resolve addresses etc.
I'm not sure if debugger is the best tool you can use for tracing live system calls. As you've mentioned LiveKd session is quite limited and you are not allowed to place breakpoints in it (otherwise you would hang your own system). However, you still can create memory dumps using the .dump command (check windbg help: .hh .dump). Keep in mind though that getting a full dump (/f) of a running system might take a lot of time.
Moving back to the subject of your question, by using the "dump approach" you will miss many system calls as you will have only snapshots of a system at given points in time. So if you are looking for something similar to Linux strace I would recommend checking those tools:
Process Monitor (procmon) - it's a tool which will show you all I/O requests in the system, as well as operations performed on the registry or process activity events
Windows Performance Toolkit - it contains tools for collecting (WPR) and analysing (WPA) system and application tracing events. It might be a lot of events and it's really important to filter them accordingly to your needs. ETW (Event Tracing for Windows) is a huge subject and you probably will need to read some tutorials or books before you will be able to use it effectively (but it's really worth it!).
API Monitor - it's one of many (I consider it as one of the best) tracing applications - this tool will allow you to trace method calls in any of the running processes. It has a nice interface and even allows you to place breakpoints on methods you'd like to intercept.
There are many other tools which might be used for tracing on Windows, but I would start with the ones I listed above. You may also check a great book on this subject: Inside Windows Debugging. Good luck! :)
Is it possible to have certain code executed whenever a file of a certain type is opened? In my case, I want to "listen" for when video files (".avi, mp4, etc.") are opened (either via the windows file explorer shell, or maybe directly from a video player?), so that I can store a history of played videos.
An hour's worth of googling turned up nothing, so I turn to you stackoverflow. Please point me in the right direction.
Thanks.
The best (and only reasonable way) to capture file system events (open/read/write) from arbitrary processes is by writing a File System MiniFilter
If you're developing a commercial product, please refrain from "hooking" Usermode APIs like CreateFile. Doing so requires numerous, platform-specific hacks, and is a compatibility nightmare.
I wouldn't hook CreateFile for this job. Windows has mechanisms built-in to handle jobs like this much more cleanly.
The easy way to handle this would be with ReadDirectoryChangesW with the FILE_NOTIFY_CHANGE_LAST_ACCESS flag. Any time a file is opened, its last-access time will be updated, so this tells you any time the file was opened.
Although it's pretty rare, that can "miss" changes under rare circumstances1. If you must have 100% accuracy (instead of, say, 99.9%), you can read change journals instead, but it's a fair amount of extra work for an advantage you may not care about.
1. There is one circumstance that isn't (necessarily) rare that you might care about though: ReadDirectoryChangesW will only work when/if your program is running. Change journals will let you know about things that happened even when your code isn't running at all.
I have an application, which displays me some data. I need to attach to this app's process, find the data I need in memory (one single number, actually), and save it somewhere. This application doesn't seem to use standard windows controls, so things aren't going to be as simple as reading controls data using AutoIt or something similar.
Currently I'm a self-learner database guy and have quite shallow knowledge about windows apps debugging. Not even sure if I asked my question correctly enough.
So, can you give me some starter guidelines about, say, what should I read first, and general directions I should work on?
Thanks.
To read memory of other application you need to open the process with respect of OpenProcess with at least PROCESS_VM_READ access rights and then use ReadProcessMemory to read any memory address from the process. If you are an administrator or have debug privilege you will be able to open any process with maximal access rights, you need only to enable SeDebugPrivilege before (see for example http://support.microsoft.com/kb/131065).
If you don't know a much about the memory of the destination process you can just enumerate the memory blocks with respect of VirtualQueryEx (see How does one use VirtualAllocEx do make room for a code cave? as an example where I examine the program code. The program data you can examine in the same way).
The most practical problem which I see is that you ask your question in too general way. If you explain more what kind of the data you are looking for I could probably suggest you a better way. For example if you could see the data somewhere you could examine the corresponding windows and controls with respect of Spy++ (a part of Visual Studio Tools). The most important are the class of windows (or controls) and the messages which will be send at the moment when the most interesting window are displayed. You can also use Process Monitor to trace all file and registry access at the time when the windows with the interesting information will be displayed. At least at the beginning you should examine the memory of the process with ReadProcessMemory at the moment when the data which you are looking for are displayed on the window.
If you will have no success in your investigations I'd recommend you to insert in your question more information.
My primary advice is: try to find any other method of integration than this. Even if you succeed, you'll be hostage to any kinds of changes in the target process, and possibly in the Windows O/S. What you are describing is behaviour most virus scanners should flag and hinder: if not now, then in the future.
That said, you can take a look at DLL injection. However, it sounds as if you're going to have to debug the heck out of the target process at the disassembly level: otherwise, how are you going to know what memory address to read?
I used to know the windows debugging API but it's long lost memory. How about using ollydbg:
http://www.ollydbg.de/
And controlling that with both ollydbg script and autoit?
Sounds interesting... but very difficult. Since you say this is a 'one-off', what about something like this instead?
Take a screenshot of this application.
Run the screenshot through an OCR program
If you are able to read the text you are looking for in a predictable way, you're halfway there!
So now if you can read a OCR'd screenshot of your application, it is a simple matter of writing a program that does the following:
Scripts the steps to get the data on the screen
Creates a screenshot of the data in question
Runs it through an OCR program like Microsoft Office Document Imaging
Extracts the relevant text and does 'whatever' with it.
I have done something like this before with pretty good results, but I would say it is a fragile solution. If the application changes, it stops working. If the OCR can't read the text, it stops working. If the OCR reads the wrong text, it might do worse things than stop working...
As the other posters have said, reaching into memory and pulling out data is a pretty advanced topic... kudos to you if you can figure out a way to do that!
I know this may not be a popular answer, due to the nature of what this software is used for, but programs like CheatEngine and ArtMoney allow you to search through all the memory reserved by a process for a given value, then refine the results till you find the address of the value you're looking for.
I learned this initially while trying to learn how to better protect my games after coming across a trainer for one of them, but have found the technique occasionally useful when debugging.
Here is an example of the technique described above in use: https://www.youtube.com/watch?v=Nv04gYx2jMw&t=265