I want my code to be notified when any file under (either directly or indirectly) a given directory is modified. By "modified", I mean I want my code to be notified whenever a file's contents are altered, it's renamed, or it's deleted; or if a new file is added. For my application, there can be thousands of files.
I looked as FSEvents, but its Technology Overview says, in part:
The important point to take away is that the granularity of notifications is at a directory level. It tells you only that something in the directory has changed, but does not tell you what changed.
It also says:
The file system events API is also not designed for finding out when a particular file changes. For such purposes, the kqueues mechanism is more appropriate.
However, in order to use kqueue on a given file, one has to open the file to obtain a file descriptor. It's impractical to manage thousands of file descriptors (and would probably exceed the maximum allowable number of open file descriptors anyway).
Curiously, under Windows, I can use the ReadDirectoryChangesW() function and it does precisely what I want.
So how can one do what I want under Mac OS X? Or, asked another way: how would one go about writing the equivalent of ReadDirectoryChangesW() for Mac OS X in user-space (and do so very efficiently)?
EDIT: Not verified, but Konstantin indicates below that this code sample is obsolete as of 2012.
I don't believe there's a specific API for what you're looking for. Apple provides sample code for a similar problem called Watcher. It is not what you are looking for, but it is about the best you can do at this point. You have to take snapshots of the directory, and rescan it when you find out something changed. Modification time is the best thing to check of course, if you can trust modification time.
You are probably correct that trying to register for an unbounded number of kqueues would likely be unworkable.
I haven't tried this myself, but it seems like FSEvents is able to provide file-level notifications as of 10.7 (Lion). From the description of FSEventStreamCreateFlags:
kFSEventStreamCreateFlagFileEvents
Request file-level notifications. Your stream will receive events about individual files in the hierarchy you're watching instead of only receiving directory level notifications. Use this flag with care as it will generate significantly more events than without it.
Available in OS X v10.7 and later.
The closest utility (that I know of) that matches your needs on Mac OS X is fslogger.
See the link for a description, dmg and source code:
OSXBook - fslogger
You might want to check out man fs_usage, though it's not specific to a directory and requires root privileges.
Related
I want to be able to set the process name (as it appears in the Activity Monitor) of arbitrary binaries in MacOS. Preferably, this wouldn't require any manual reverse engineering.
To clarify, I'm not asking to change it at runtime. Rather, to set up a script which will change the code of another executable.
(similar to how you can change argv[0] in C code)
You may achieve it for some apps, but with relevant consiquences.
First of all, let's think of what is going to be the name in Activity monitor. It is a name of a executable file for shell scripts, and it is a Bundle or displayed Bundle name for bundles.
So, to change the name you will basically need to change the name of file or the record inside Info.plist.
Both is not doable for Apple provided apps, until you disable SIP (if this could be the way for you - all is doable).
Speaking about non-Apple apps, you could change their names or Info.plist records, but it would result in non-valid code signature for them. You will need to manually allow such apps to run on your mac and get over all Apple popups about that.
There is no API you can use to achieve this goal. You are either going to use hacky ways, or need to change what you want to have.
Is it possible to have certain code executed whenever a file of a certain type is opened? In my case, I want to "listen" for when video files (".avi, mp4, etc.") are opened (either via the windows file explorer shell, or maybe directly from a video player?), so that I can store a history of played videos.
An hour's worth of googling turned up nothing, so I turn to you stackoverflow. Please point me in the right direction.
Thanks.
The best (and only reasonable way) to capture file system events (open/read/write) from arbitrary processes is by writing a File System MiniFilter
If you're developing a commercial product, please refrain from "hooking" Usermode APIs like CreateFile. Doing so requires numerous, platform-specific hacks, and is a compatibility nightmare.
I wouldn't hook CreateFile for this job. Windows has mechanisms built-in to handle jobs like this much more cleanly.
The easy way to handle this would be with ReadDirectoryChangesW with the FILE_NOTIFY_CHANGE_LAST_ACCESS flag. Any time a file is opened, its last-access time will be updated, so this tells you any time the file was opened.
Although it's pretty rare, that can "miss" changes under rare circumstances1. If you must have 100% accuracy (instead of, say, 99.9%), you can read change journals instead, but it's a fair amount of extra work for an advantage you may not care about.
1. There is one circumstance that isn't (necessarily) rare that you might care about though: ReadDirectoryChangesW will only work when/if your program is running. Change journals will let you know about things that happened even when your code isn't running at all.
Ok i wrote and application that use Adobe ActiveX control for displaying PDF files.
Adobe ActiveX control load files only from file system. So i nead to feed a file path to this control.
Problem is that i don't want to store PDF files on file system. Event temporary! I wan't to store my PDF files only in memory, and i want to use Adobe ActiveX control.
So i nead:
1) A way to fake file on a file system. So this control would "think" that there is a file, but would load it from memory
2) A way to create file on file system that would be "visible" to only one application, so my PDF control could load it, and other users won't even see it..
3) Something else
PS: I'm not asking to "finish my home work", i'm just asking - is there a way to do this?
You can almost do it (means: no you can't, but you can do something that comes close).
Creating a file with FILE_ATTRIBUTE_TEMPORARY does in principle create a file, temporarily. However, as long as there is sufficient buffer cache (which is normally always the case unless your file is tens to hundreds of megabytes), the system will not write to disk. This is not just something that happens accidentially, but the actual specified behaviour of this flag.
Further, specifying 0 as share mode and FILE_FLAG_DELETE_ON_CLOSE will prevent any other process from opening your file for as long as you keep it open, even if someone knows it's there, and the file will "disappear" when you close it. Even if your application crashes, the OS will clean up behind you (if DRM is the reason). If you're in super paranoia mode and worried about the system bluescreening while your file exists, you can additionally schedule a pending move too. This will, in case of a system crash, remove the file during boot.
Lastly, given NTFS, you can create an alternate stream with a random, preferrably unique name (e.g. SHA1 of the document or a UUID) on any file or even directory. Alternate streams on directories are ... a kind of nasty hack, but entirely legal and they work just fine, and don't appear in Explorer. This will not really make your file invisible, but nearly so (in almost every practical aspect, anyway). If you're a good citizen, you will want to use the system temp folder for such a thing, not the program folder or some other place that you shouldn't write to.
Creating an alternate stream is dead easy too, just use the normal file or directory name and append a colon (:) and the name of the stream you want. No extra API needed.
Other than that, it gets kind of hard. You can of course always create something like a ramdisk (would be tough to hide it, though), or try to use one of the stream-from-memory functions to fool an application into reading from a memory buffer on the allegation of a file... but that's not trivial stuff.
If something needs to be on a file system to pass to another application, you can not hide it/limit it to certain processes. Anything your app can see, anything else at the same privilige level can also see/access. You may be able to lock it but how depends on why you want to protect against.
Remember that the user's PC is theirs, not yours so they have full access to everything on it.
You can create a virtual disk and limit access to it to only specific application. Do to this you would have to write a file system driver or a filesystem filter driver. Both work in kernel mode and are tricky to write and maintain. Our company offers components that let you avoid writing drivers yourself and write business logic in user-mode (we provide drivers in those products).
Your most obvious option is to get rid of Adobe Reader control and use some third-party component that displays PDFs and can load them from memory.
But in general a smart hacker would be able to capture your data unless you have (a) non-standard data format, and/or (b) stream the data from the server dynamically, not keeping the complete data on the computer. These are not bulletproof solutions either, but they make hacker's work much harder.
I have a Mac application in the App Store and am looking to adopt sandboxing before it becomes a mandatory requirement. I've run into two issues and was hoping to post here for some insight into best practice in the following situations:
Within my application I use an NSOpenPanel to prompt the user to load a proprietary file format. After loading the file my application parses it and gathers a list of NSURLs to local files. These local files are then passed to NSImage's initWithContentsOfURL: method. Unfortunately, the act of loading the image files causes the sandbox to cancel the action. I understand that this happens because the user has given my application explicit permission to open the file selected by the NSOpenPanel, but not for the files referenced within my proprietary format. How can I handle this (supposedly fairly common) situation?
I have a unix executable file contained within my applications bundle that I would like to execute using an NSTask. Is this legal under sandboxing, given that the script is contained within my bundle?
If anyone could clarify the above points, that would be appreciated.
1) From my understanding the NSURL object contains the permissions necessary to re-access the files later, so if you are using hardcoded paths, you could replace them with archived NSURL objects. This is also assuming the user selected those filies within an NSOpenPanel at an earlier point.
2) You can run a NSTask but it inherits the permissions of your main app.
Hopefully others can chime in with more information. I've found the Mac Developer Boards, specifically the "Application Sandboxing" forums to be helpful, as Apple employees often drop in. So far, I've found sandboxing to be an unusable mess.
Easy one first: you can run your helper with NSTask and it will inherit the sandbox of your app.
Those URLs: not easily/reliably/at all. There is a way to save NSURLs to files you have access to in such a way that a subsequent run of your application and re-load them and regain access, however it is deemed fragile and not to be recommended. Read the Apple developer forums and this is an Apple acknowledged problem they are "working on"; given this using the fragile solution is probably not worth the effort - search the developer forums for the fragile solution if you really want to hack something that sort of works now.
What would be the best way of inserting functionality into a binary application (3d party, closed source).
The target application is on OSX and seems to have been compiled using gcc 3+. I can see the listing of functions implemented in the binary and have debugged and isolated one particular function which I would like to remotely call.
Specifically, I would like to call this function - let's call it void zoomByFactor(x,y) - when I receive certain data from a complex HIDevice.
I can easily modify or inject instructions into the binary file itself (ie. the patching does not need to occur only in RAM).
What would you recommend as a way of "nicely" doing this?
Edit:
I do indeed need to entire application. So I can't ditch it and use a library. (For those who need an ethical explanation: this is a proprietary piece of CAD software whose company website hasn't been updated since 2006. I have paid for this product (quite a lot of money for what it is, really) and have project data which I can not easily migrate away from it. The product suits me just fine as it is, but I want to use a new HID which I recently got. I've examined the internals of the application, and I'm fairly confident that I can call the correct function with the relevant data and get it to work properly).
Here's what I've done so far, and it is quite gheto.
I've already modified parts of the application through this process:
xxd -g 0 binary > binary.hex
cat binary.hex | awk 'substitute work' > modified.hex
xxd -r modified.hex > newbinary
chmod 777 newbinary
I'm doing this kind of jumping through hoops because the binary is almost 100 megs large.
The jist of what I'm thinking is that I'd jmp somewhere in the main application loop, launch a thread, and return to the main function.
Now, the questions are: where can I insert the new code? do I need to modify symbol tables? alternatively, how could I make a dylib load automatically so that the only "hacking" I need to do is inserting a call to a normally loaded dylib into the main function?
For those interested in what I've ended up doing, here's a summary:
I've looked at several possibilities. They fall into runtime patching, and static binary file patching.
As far as file patching is concerned, I essentially tried two approaches:
modifying the assembly in the code
segments (__TEXT) of the binary.
modifying the load commands in the
mach header.
The first method requires there to be free space, or methods you can overwrite. It also suffers from extremely poor maintainability. Any new binaries will require hand patching them once again, especially if their source code has even slightly changed.
The second method was to try and add a LC_ LOAD_ DYLIB entry into the mach header. There aren't many mach-o editors out there, so it's hairy, but I actually modified the structures so that my entry was visible by otool -l. However, this didn't actually work as there was a dyld: bad external relocation length at runtime. I'm assuming I need to muck around with import tables etc. And this is way too much effort to get right without an editor.
Second path was to inject code at runtime. There isn't much out there to do this. Even for apps you have control over (ie. a child application you launch). Maybe there's a way to fork() and get the initialization process launched, but I never go that.
There is SIMBL, but this requires your app to be Cocoa because SIMBL will pose as a system wide InputManager and selectively load bundles. I dismissed this because my app was not Cocoa, and besides, I dislike system wide stuff.
Next up was mach_ inject and the mach_star project. There is also a newer project called
PlugSuit hosted at google which seems to be nothing more than a thin wrapper around mach_inject.
Mach_inject provides an API to do what the name implies. I did find a problem in the code though. On 10.5.4, the mmap method in the mach_inject.c file requires there to be a MAP_ SHARED or'd with the MAP_READ or else the mmap will fail.
Aside from that, the whole thing actually works as advertised. I ended up using mach_ inject_ bundle to do what I had intended to do with the static addition of a DYLIB to the mach header: namely launching a new thread on module init that does its dirty business.
Anyways, I've made this a wiki. Feel free to add, correct or update information. There's practically no information available on this kind of work on OSX. The more info, the better.
In MacOS X releases prior to 10.5 you'd do this using an Input Manager extension. Input Manager was intended to handle things like input for non-roman languages, where the extension could popup a window to input the appropriate glyphs and then pass the completed text to the app. The application only needed to make sure it was Unicode-clean, and didn't have to worry about the exact details of every language and region.
Input Manager was wildly abused to patch all sorts of unrelated functionality into applications, and often destabilized the app. It was also becoming an attack vector for trojans, such as "Oompa-Loompa". MacOS 10.5 tightens restrictions on Input Managers: it won't run them in a process owned by root or wheel, nor in a process which has modified its uid. Most significantly, 10.5 won't load an Input Manager into a 64 bit process and has indicated that even 32 bit use is unsupported and will be removed in a future release.
So if you can live with the restrictions, an Input Manager can do what you want. Future MacOS releases will almost certainly introduce another (safer, more limited) way to do this, as the functionality really is needed for language input support.
I believe you could also use the DYLD_INSERT_LIBRARIES method.
This post is also related to what you were trying to do;
I recently took a stab at injection/overriding using the mach_star sources. I ended up writing a tutorial for it since documentation for this stuff is always so sketchy and often out of date.
http://soundly.me/osx-injection-override-tutorial-hello-world/
Interesting problem. If I understand you correctly, you'd like to add the ability to remotely call functions in a running executable.
If you don't really need the whole application, you might be able to strip out the main function and turn it into a library file that you can link against. It'll be up to you to figure out how to make sure all the required initialization occurs.
Another approach could be to act like a virus. Inject a function that handles the remote calls, probably in another thread. You'll need to launch this thread by injecting some code into the main function, or wherever else is appropriate. Most likely you'll run into major issues with initialization, thread safety, and/or maintaining proper program state.
The best option, if its available, is to get the vendor of your application to expose a plugin API that lets you do this cleanly and reliably in a supported manner.
If you go with either hack-the-binary route, it'll be time consuming and brittle, but you'll learn a lot in the process.
On Windows, this is simple to do, is actually very widely done and is known as DLL/code injection.
There is a commercial SDK for OSX which allows doing this: Application Enhancer (free for non-commercial use).