I want to monitor a directory (of thousands of files, with about 5 levels of sub directories) for when files are changed. I know I can use the FSEvents API to monitor a directory for when files change inside that directory, but I can't seem to figure out how to determine which file(s) changed. This reference suggests I build a binary tree and traverse the tree each time an event is triggered, is that the best way to determine which files were changed? If not, what are some better alternatives?
Is it better to recursively scan the directory and attach kqueue to every file? I'm not sure how well that would work on thousands of files?
I've used UKKQueue before with mixed results.
I've recently become aware of a better solution, but I haven't tried this. If you only need to target Lion, I think that the new best-practice way to do this is to use an NSFileCoordinator. You implement the methods of NSFilePresenter to indicate which directory you're interested in (the presentedItemURL property) and the system will notify you when a sub item moves/changes/is deleted (via methods like presentedSubitemDidChangeAtURL:)
I'd love to hear how that works out if you do go that route.
If you create your stream using kFSEventStreamCreateFlagFileEvents then you will get events for the changes to each file rather than just a notification of the change to the watched directory. Unfortunately this is only available in OSX 10.7 and later.
Related
This is likely not a simple topic - I have researched this to the best of my abilities and realize that it is not supported in any typical fashion.
My goal is to enable something similar to .app files from OSX, where the application, as well as its user data, can exist in the same file. I imagine it would require writing a tool to manage this behaviour, but this question is more about how to achieve this in the Windows OS. I am quite flexible regarding the implementation details, but the more straightforward the behaviour, the better (i.e. avoiding copying or compressing/decompressing entire directories/archives at runtime would be ideal).
Approaches I have considered:
Find a way to get explorer to treat a directory as a file, so that it can be associated. I have found a way to get explorer to treat a directory as a control panel item, I have thus far been unable to find a way to use this to associate a custom program. See the infamous "godmode hack" for Windows (name a directory something to the effect of "GodMode.{ED7BA470-8E54-465E-825C-99712043E01C}"). This one seems the most hopeful, but I'm at wits end trying to find information about creating a new association of this type.
Come up with some kind of archive format which can extract executable information to a temporary directory, launch this executable passing the archive as a commandline parameter. This seems like the ugliest solution, from a performance perspective. I would prefer a different solution if at all possible, one which doesn't involve making duplicates of the program or its data to run.
Find a way to associate a directory directly, though I have found no trace of this being supported in Windows, and I assume this is a dead-end.
Find a way to get an executable to include writeable embedded files. I have been unable to make any headway with this- I even tried a resource hacker approach, but obviously you cannot modify the assembly while its in use.
Tried to make a self-modifying JAR file with Java, but the route I took would add the JDK as a runtime requirement, which seems a bit overkill. Even then, it would be limited to Java, and I'm pretty sure it's not actually supposed to allow that in the first place.
Modify Windows Explorer. I shudder at the amount of work this would take, not to mention the at-best gray area it falls under legally. Perhaps there's a way to extend explorer to achieve this, I'm not sure.
A custom archive file. This seems like the most straightforward way to do it. But it would ideally need to be an archive format that has very little overhead for file I/O. Could even be some kind of virtual disk that gets mounted, but I am imagining that would be pretty heavy.
I would appreciate any insight that anyone has on this topic. I won't go into reasons as they are irrelevant to the question itself- I'm aware it is likely not the most practical solution to anything in particular. Consider it a novel pursuit.
It can be done by application virtualization,
Read this wikipedia page theory:
https://en.wikipedia.org/wiki/Portable_application
https://en.wikipedia.org/wiki/Application_virtualization
And two pages about software:
https://en.wikipedia.org/wiki/VMware_ThinApp
https://en.wikipedia.org/wiki/Turbo_(software)
Windows 7 added the ability for a Desktop.ini file to add/change the folder verbs on a per-folder basis. Using that trick it is possible to create a "folders as applications" style setup.
Are there any patterns or best practices for keeping a hierarchy of objects consistent with the file system?
Each object corresponds with a file on the file system. Files can be added, changed, deleted etc and the object hierarchy should be adapted accordingly.
Please don't answer along the lines of "discard files, use a database".
A good starting point is to check how the target OS lets you watch the filesystem. On Linux there's the inotify API. On Windows you wait on the notification handle created by the FindFirstChangeNotification() API call. Both let you avoid active polling of the directory for changes.
As I create more applications, my /code/shared/* increases.
this creates a problem: zipping and sending a project is no longer trivial. it looks like my options are:
in Xcode set shared files to use absolute path. Then every time I zip and send, I must also zip and send /code/shared/* and give instructions, and hope the recipient doesn't have anything already at that location.
this is really not practical; it makes the zip file too big
maintain a separate copy of my library files for each project
this is not really acceptable as a modification/improvements would have to be implemented everywhere separately. this makes maintenance unreasonably cumbersome.
some utility to go through every file in the Xcode project, figure out the lowest common folder, and create a zipped file structure that only contains the necessary files, but in their correct relative folder locations, so that the code will still build
(3) is what I'm looking for, but I have a feeling it doesn't as yet exist.
Anyone?
You should rethink your current process. The workflow you're describing in (3) is not normal. This all sounds very complicated and all basically handled with relative ease if you were using source control. (3) just doesn't exist and likely never will.
A properly configured SCM will allow you to manage multiple versions of multiple libraries (packages) and allow you to share projects (in branches) without ever requiring zipping up anything.
I would like to generate some temporary files in the course of my application. Specifically, I'm using AVAudioRecorder to record a file that I, upon stopping the recording, would like to load and edit/process. My question is:
What is the appropriate standard place to create temporary files. Is there some generally accepted approach to this for Mac or for iPad programming in general? I don't want to simply create a directory and write files into it if there is a proper protocol to this.
The answer to this question is actually a lot more complicated then one might assume. One cannot necessarily just use NSTemporaryDirectory and be done. I cocoadev.com has some good pages on this topic and I would suggest that you study them yourself and determine what will work best for your circumstance.
http://www.cocoadev.com/index.pl?NSTemporaryDirectory
http://www.cocoadev.com/index.pl?GettingTemporaryFolderOnSpecificVolume
The usual place for applications to store temporary data is /var/tmp. You could also use /tmp but this directory is for system-generated temporary files and anything in /tmp is deleted when the machine reboots.
What I found was that according to the iOS Application Programming Guide, I am supposed to query for the appropriate temporary folder for my application via NSTemporaryDirectory(). I tried this and it returned a folder within the /var directory, in my case '/var/folders/pQ/pQ+ZqZCSHWSIHftcbIo57U+++TI/-Tmp-/'.
/tmp or /usr/tmp are the usual places to store temporary files in Unix (which Mac OS X and iOS are).
I would like to upload documents to GoogleDocs every time the OS hears that a file was added/dragged/saved in a designated folder, just the way DropBox uploads a file when you save it in the DropBox folder.
What would this take in Ruby, what are the parts?
How do you listen for when a File is Saved?
How do you listen for when a File is added to a Folder?
I understand how to use the GoogleDocs API and upload things once I get these events, but I'm not sure how this would work.
Update
While I still don't know how to check if a file is added to a directory, listening for when a file is saved is now dirt simple, thanks to Guard for ruby.
If I were faced with this, I would use something like git or bzr to handle the version checking and just call add then commit from your script and monitor which files have changed (and therefore need to be uploaded).
This adds the benefit of full version control over files and it's mostly cross platform (if you include binaries for each platform).
Note this doesn't handle your listening problem, just what you do when you know something has changed. You could schedule the task (via various routes) but I still like the idea of a proper VCS under the hood.
I just found this: http://www.codeforpeople.com/lib/ruby/dirwatch/
You'd need to read over it as I can't vouch for its efficiency or reliability. It appears to use SQLite, so it might be better just to manually check once every 10 seconds (or something along those lines).
Ruby doesn't include a built-in way to "listen" for updates to files. If you want to stick to pure Ruby, your best bet would be to perform the upload on a fixed schedule (say every 5 minutes) regardless of when the file is saved.
If this isn't an acceptable alternative, you could try writing the app (or at least certain parts of it) in Java, which does support this type of thing. Take a look at JRuby for integrating the Ruby and Java portions of your app.
Here is a pure ruby gem:
http://github.com/TwP/directory_watcher
I don't know the correct way of doing this, but a simple hack would be to have a script running in the background which checks the contents of a bunch of folders every n minutes and uses the associated timestamps to determine if the file was modified in that span of time
You would definitely need some native OS code here, to write the monitoring service/client. I'd select C++ if you want it to be cross platform. If you decide to go with .Net, for example, you can use the FileSystemWatcher class to achieve what you need (documentation and here's a related article).
Kind of an old thread, but I am faced with doing something similar and wanted to throw in my thoughts. The route I'm going is to have a ruby script that watches a given directory and checks the timestamps. Once all files have been uploaded, the script saves the latest timestamp and then polls the directory again, checking if any files/folders have been added. If files are found, then the script uploads them and updates the global timestamp, etc...
The downside is that setting up a ruby script to run continually (or as a service) is somewhat painful. But it's not an overwhelming task, just needs to be thought out properly.
Also depends on if your users are competent enough to have ruby installed or if you have to package everything up into a one-click installer as well. That, to me, is the hardest part to figure out.