I have attached a script with a folder to run whenever something is added to the folder. The problem is if multiple files are added one after another the process gets queued. How to make the script run even if its already running for another file.
You can't stop the process queuing. The "on adding folder items to xxfolder after receiving xxfiles", gets all files dropped at same time (xxfiles is a list). When system takes too much time to add all files (copy via slow network, ...), the systems split the list of added files in sub-lists, each one calling the script. That's the way it has been built ! but what is your troubles having several calls instead of only 1 ? ..as long as all files added are processed, it should be OK.
fyi, the system does periodic calls to check folder actions, using launchd process, but it is not documented where the period is set.
Related
In my backup app I am monitoring changes to a user-selected directory A and mirroring all file changes to another directory B. The only problem seems to be: how do I know if a file has been created and is still being written to, such as when an app creates a very big file or the Finder copies a file?
I would like to ignore this event because another event will follow corresponding to the moment in which the write operation has finished and it is almost certainly safe to copy the file from A to B (starting a copy while the original file is still being written usually results in an error). I tried to analyze the flags passed to the FSEvents callback, but it seems like different applications cause different flags to be passed (for instance the cp and touch commands, a copy initiated in the Finder, creating a file from TextEdit and Word all result in different flags).
I have a GUI (lxn/walk) app patcher that downloads a file via ftp, streams it to a temporary file and extracts the contents to update the local files. The remove file command is deferred.
This works unless the user exits the program while the file is downloading, then the file isn’t deleted.
I tried to fix this by doing a graceful exit by catching the signal and removing the file there. But unfortunately it throws an error that the file can’t be deleted because it is being used by another program. Which makes sense because the another program is actually itself still writing to the temporary file.
Now I’m stuck and don’t know what to do to make sure that the temporary file is automatically gone once the patcher is not running. How do I do that correctly?
The file could also be created as a normal file, not just a temp file. I would just like to ask too, where in windows is best to write a temporary file?
Now I’m stuck and don’t know what to do to make sure that the temporary file is automatically gone once the patcher is not running. How do I do that correctly?
There are no guaranteed ways to accomplish this as many things beyond the control of the application can cause it to exit. A power failure or kernel panic due to some hardware issue can crash the machine or force it to be restarted.
A strategy that is in common use is to implement a check on program startup for the status of the previous run. Some applications create a lock file at start and remove it on graceful exit. If this lock file exists when the program is restarted, this means the previous run did not result in a clean exit, and the application can take any corrective action. The exact action to be taken depends on the nature of the application, some refuse to start, others give warnings to users.
I would just like to ask too, where in windows is best to write a temporary file?
Each OS has its own location for temporary files. If you eliminate the dir argument to TempFile, it will create it in the appropriate location, as mentioned in the documentation:
TempFile creates a new temporary file in the directory dir, opens the
file for reading and writing, and returns the resulting *os.File. The
filename is generated by taking pattern and adding a random string to
the end. If pattern includes a "*", the random string replaces the
last "*". If dir is the empty string, TempFile uses the default
directory for temporary files (see os.TempDir). Multiple programs
calling TempFile simultaneously will not choose the same file. The
caller can use f.Name() to find the pathname of the file. It is the
caller's responsibility to remove the file when no longer needed.
From os.TempDir we see the following:
On Unix systems, it returns $TMPDIR if non-empty, else /tmp. On
Windows, it uses GetTempPath, returning the first non-empty value
from %TMP%, %TEMP%, %USERPROFILE%, or the Windows directory. On
Plan 9, it returns /tmp.
The directory is neither guaranteed to exist nor have accessible
permissions.
I have 2 applications running in parallel, both doing the following:
check for file not containing "processed"
process the file and then rename it to filename+processed
for every file, only one application shall use it (on a first come first served basis)
I get the files and I also lock them so the other application cannot process it. But when it comes to renaming the file I get a problem. To rename the file, wanted to use the File.renameTo function. However, for that to work, I have to release the lock on the file. But when I release the lock another process may try to use the file. Exactly that should not happen.
Is there any way to prevent the application B from using the file between application A releasing the lock and finishing renaming the file?
EDIT
Some more information:
File creation if the file doesn't exist has to be prevented.
The file will be processed RandomAccessFile (with read and write permission; this creates a new file if it doesn't exist).
Note: On linux, one can rename a file that is locked, so this problem doesn't occur there. However, on Windows a locked file cannot be renamed; I have to release the lock, then rename it. But the time, during which the lock is released creates enables other applications to see that the file is available and then they will try to use it.
Windows applications can do this using the SetFileInformationByHandle function, which allows you to rename the file using the handle you already have open. You probably can't do this natively from Java.
However, a more straightforward solution would be to rename the file (to filename+processing, for example) before you start processing it. Whichever process successfully renames the file in this way is the one responsible for processing it and eventually renaming it to filename+processed.
I am trying to keep two directories synchronized with the same files in them.
Files are dropped into Directory A throughout the day. I would like to create a file watcher script that will copy files from Directory A to Directory B as soon as they are dropped.
My thought was to run the job every minute and simply copy everything that dropped in the last minute, but I am wondering if there is a better solution out there.
I'm running MKS toolkit under Windows. Different servers, same operating system.
Thanks for your help!
If you use Linux, you can hook into the kernel using the inotify API to get notified if something in a folder changes. There are command line versions like inotifywatch(1) as well.
To copy the files, I suggest to use rsync(1): it is clever, knows how to clean up after itself and it will create new files hidden while they are copied so users and programs are less likely to pick them up before they are complete.
I'm currently in working on a script to create a custom backup script, the only piece I'm missing is a file monitor. I need some form of a script that will monitor a folder for file changes, and then run a command with the file that's changed.
So, for example, if the file changes, it'll execute "c:/syncbatch.bat %Location_Of_File%"
In VBScript, you can monitor a folder for file changes by subscribing to the WMI __InstanceModificationEvent event. These articles contain sample code that you can learn from and adapt to your specific needs:
WMI and File System Monitoring
How Can I Monitor for Different Types of Events With Just One Script?
Calling WMI is fairly cryptic and it causes the WMI service to start running which can contribute to bloat since its fairly large and you really can't cancel the file change notifications you've requested from it without rebooting. Some people experimenting with remote printing from a Dropbox folder found that a simple VBScript program that ran an endless loop with a 10 second WScript.Sleep call in the loop used far less resource. Of course to stop it you have to task kill that script or program into it some exit trigger it can find like a specifically named empty file in the watch folder, but that is still easier to do than messing with WMI.
The Folder Spy http://venussoftcorporation.blogspot.com/2010/05/thefolderspy.html
is a free lightweight DOT.NET based file/folder watching GUI application I'ved used before to run scripts based on file changes. It looks like the new version can pass the event filename to the launched command. The old version I had didn't yet support file event info so when launched, my script had to instance a File System Object and scan the watched folder to locate the new files based on criteria like datestamps and sizes.
This newer version appears to let you pass the file name to the script if you say myscript.vbs "*f" on the optional script call entry. Quotes can be important when passing file paths that have spaces in folder names. Just remember if you are watching change events you will get a lot of them as a file grows or is edited, usually you just want notification of file adds or deletes.
Another trick your script can do is put the file size in a variable, sleep for a few seconds, and check the file again to see if its changed. if it hasn't changed in a few seconds you can usually assume whatever created it is done writing it to disk. if it keeps changing just loop until its stable.