Are there any patterns or best practices for keeping a hierarchy of objects consistent with the file system?
Each object corresponds with a file on the file system. Files can be added, changed, deleted etc and the object hierarchy should be adapted accordingly.
Please don't answer along the lines of "discard files, use a database".
A good starting point is to check how the target OS lets you watch the filesystem. On Linux there's the inotify API. On Windows you wait on the notification handle created by the FindFirstChangeNotification() API call. Both let you avoid active polling of the directory for changes.
Related
If I understand right, through Windows' Filter Manager we can write a filter to make an operation to read one file instead actually read another. So for example, if you've added hooks to monitor certain DLLs and an attacker attempts to load a fresh DLL from source, I'm trying to figure out if I can use a mini-filter to have that attempt actually result in loading a copy of that DLL in another folder.
Is Filter Manager capable of facilitating that outcome? If so, is there any limitation on, for example, affecting read operations against a protected directory such as the System32 folder?
I built a very simple minifilter driver as part of a lesson on minifilters. I've also read the minifilter documentation that Microsoft provides which is in the form of a PDF doc, as well as this reference. These guides explain how to set up a context and an instance. However, they do not explain why one would use a context and/or instance and what they are for. My very small filter driver used NULL for both context and instance and still operates, so I am wondering the use-case for these constructs.
There are many reasons why you would want to use contexts for files, volumes etc.. Certainly filters and even file-systems could operate without them, but the performance would be really bad.
Imagine this scenario: you are an AV (AntiVirus) and want to scan some files to check if they contain malicious code or not.
You register your minifilter and callbacks and now you are being called and you need to make a decision on a file as it is opened.
There are a few steps involved:
You query the file name and security context
You read the file contents
Alternatively hash the file with a SHA256 to see if it matches in your AV database for example
You check if the file is digitally signed, also part of your check
You parse the file's PE header if it has one to see what kind of file or executable it is to help you in your decision
You apply your policy on the file based on all the information above
Now let's assume the file is clean and goes away. If you cannot hold on to the information that you just learnt about the file, the next time the file is opened you will have to re-do it all over again. Your performance will suck, and your OS will crash and burn slowly to the ground.
This is where contexts come in handy.
Now that you have all this information about the file, you store all of it in your context that is then associated with this file. Next time you see the file you simply query its context and have all the information you need.
Of course some things will need to be updated, for example if you notice the file has been changed then you mark it as dirty and update as needed on the next Create or Cleanup callback.
Alternatively you could use a cache, where after the file is closed for good and the minifilter wants to free the context you have associated with the file you can save it yourself.
Now, the next time the file is opened you look for the context of the file ( NTFS support unique file ids for files ) and just associated it with your file and know immediately everything you need to know about that file
This is only one usage, but now you can think for yourself of many more scenarios where they are useful.
Is it possible to append/remove a ressource file to a binary at execution time?
I have an application written with go, which saves/searches data from a database file, and i would like this database file to be embedded to the binary, and updated by the application itself.
This way the application would be self contained with its database.
Modifying the executable, this is generally a very bad idea.
Several issues pop right into my head, such as:
Does the current user have sufficient permissions?
Is the file locked during execution?
What about multiple running instances of the application?
Even if you manage to do just that, think of what anti-virus and firewall applications will say to it: most when they detect the change will flag the executable and/or contain it, or deny running it, or some may even delete it. Rightfully, as this is what many viruses do: modify existing executables.
Also virus scanner databases maintain reports where files (their contents) are identified based on the hash of their content. Modifying the executable will naturally change the file content hash, thus render the file unknown / suspicious to these databases.
As mentioned, just write / cache data in separate file(s), preferably in user's home folder or in the application folder (next to the executable, optionally in sub-folders). Or make the cache file / folder a changeable option (command line flags).
Technically, this is possible, but this is a bad idea. Your application could be run by users not having write permissions to your binary.
If you're talking about a portable app, your best option might be using a file in the same directory the binary is located, otherwise - use the user's home directory according to the conventions of the OS you're running on. You can use the os/user package to find the home directory.
hi guys i have a test system at work which is used to test final builds. I am wondering if there is a way to track if someone dumps a file on the system... I don't care if it tracks it for entire system or a specific directory any ideas?
Use FileSystemWatcher Class. It looks for different changes in folder.
I want to monitor a directory (of thousands of files, with about 5 levels of sub directories) for when files are changed. I know I can use the FSEvents API to monitor a directory for when files change inside that directory, but I can't seem to figure out how to determine which file(s) changed. This reference suggests I build a binary tree and traverse the tree each time an event is triggered, is that the best way to determine which files were changed? If not, what are some better alternatives?
Is it better to recursively scan the directory and attach kqueue to every file? I'm not sure how well that would work on thousands of files?
I've used UKKQueue before with mixed results.
I've recently become aware of a better solution, but I haven't tried this. If you only need to target Lion, I think that the new best-practice way to do this is to use an NSFileCoordinator. You implement the methods of NSFilePresenter to indicate which directory you're interested in (the presentedItemURL property) and the system will notify you when a sub item moves/changes/is deleted (via methods like presentedSubitemDidChangeAtURL:)
I'd love to hear how that works out if you do go that route.
If you create your stream using kFSEventStreamCreateFlagFileEvents then you will get events for the changes to each file rather than just a notification of the change to the watched directory. Unfortunately this is only available in OSX 10.7 and later.