Can I make the function call SHFileOperation a transactional one?
I mean, I want to move/copy/delete few files using this function and want to ensure that entire file operation either fully succeeds or fully fails (leaves no change).
Related
The Windows API function CopyFileEx allows a callback function to be specified which is then being called when a part of the file has been copied to e.g. provide the user with feedback about the progress.
This works fine in my program, but I have now an additional requirement: I need to temporarily pause the copying process in order to keep system load low for a time (may be up to several hours) and then resume it. This may happen at any time and usually there is a large file (several 100 Gigabytes) that is only partly copied yet, so aborting the process and restarting it is not a good option, as that would mean that possibly 99% of a file that have already been copied, would need to be copied again.
One option to do that is simply let the thread that executes the copy sleep when the callback is called, e.g.
function ProgressCallback(
_TotalFileSize, _TotalBytesTransferred, _StreamSize,
_StreamBytesTransferred: LARGE_INTEGER;
_StreamNumber, _CallbackReason: LongWord;
_SourceFile, _DestinationFile: THandle; _Data: Pointer): LongWord;
far; stdcall;
begin
// [...]
while gblPauseFlag do
sleep(100);
end;
(This is of course simplified code. The original is too complex to post here.) The gblPauseFlag would be set and reset by a different thread in the application that checks for the condition to pause.
I am aware that there are better ways to pause the thread (events, critical sections or whatever), but the question here is just: Is is safe to pause the copying thread in the callback function? Or would it block some Windows functionality or resources?
As you can see from the sample code, this is a Delphi program, but I don't think that matters much, so I have not tagged the question as [delphi].
EDIT: I could pass the flag COPY_FILE_RESTARTABLE, but:
Progress of the copy is tracked in the target file in case the copy fails.
The failed copy can be restarted at a later time by specifying the same
values for lpExistingFileName and lpNewFileName as those used in the call
that failed. This can significantly slow down the copy operation as the
new file may be flushed multiple times during the copy operation.
Source: CopyFileEx documentation
So I would rather try to avoid this.
Ideally when you have such a large to copy, you should be using utility like robocopy for large files. If you still prefer to write your own utility, make it such that you can restart the aborted copy. You can use the file stream to store metadata about your progress of your copy in the destination file.
From the doc (ReadDirectoryChangesW):
"Retrieves information that describes the changes within the specified directory. The function does not report changes to the specified directory itself."
My question is: What do I use to report changes to the specified directory itself?
I want to be able to capture changes not only to things and sub-things in the folder itself but also to detect for example, when the folder itself has been deleted.
One strategy would be to actually monitor for changes on the parent of the folder I'm really interested in and then use that to generate an event when the folder I'm interested in is deleted. This works but has the potential to generate thousands of 'uninteresting' events.
A second strategy is to have a recursive monitor for stuff under the folder I'm actually interested in and then a non-recursive monitor on it'a parent. The non-recursive monitor would then be able to tell me when the real folder of interest is deleted.
The latter, second strategy, generates fewer events and is the strategy I would like to use. BUT: It doesn't work 'in process'. That is, if I start to monitor the folder of interest recursively (HANDLE A), and it's parent non-recursively (HANDLE B) and then in the same process, I try and delete the folder of interest, no removal event is generated for it (even though I verify from a console that the thing no longer exists). My suspicion is that this is due to HANDLE A on the folder still being open, and even though I have included the "FILE_SHARE_DELETE" flag in the call to CreateFileW that gave me HANDLE A, it simply can't work.
Note that 'Out of process', i.e. when I delete the folder from within a completely separate process, the above strategy does work.
So, what are my options?
Many thanks,
Ben.
I have some code which writes some PDFDocument objects to a user-chosen destination. This works fine.
Perhaps some of these files (chosen by the user) may be pretty large (maybe hundreds of megabytes) and now I wonder whether there is a possibility to cancel the current writeToFile:withOptions: call (e.g. the user changed his mind and wants to stop it).
I doubt you can do it with that method, since it provides no canceling functionality.
I suggest you use the dataRepresentation method of PDFDocument to first get the PDF data. You can then split up the data using NSData’s subdataWithRange:. And then you can successively write out the data to a file using NSFileHandle’s fileHandleForWritingToURL:error:, writeData:, and closeFile methods.
Writing it out in chunks like this from a non-main thread, in a for-loop say, you can cancel it any time you wish.
Observed that FWRITE or KAUTH_FILEOP_CLOSE_MODIFIED is not consistenly set in action KAUTH_FILEOP_CLOSE during file modification or file copy.
My usecase is - I am trying to figure out whether the file being closed is modified file or newly created file. I want to ignore files that are not modified.
As per documentation, I am checking for KAUTH_FILEOP_CLOSE_MODIFIED flag when the file action is KAUTH_FILEOP_CLOSE. Most of the time, I have observed KAUTH_FILEOP_CLOSE_MODIFIED is not set when file is copied from one location to other or file is modified.
I also observed that FWRITE flag is set, but not consistently for modified or copied files. I am just wondering why the behavior is so inconsistent.
Another way I was thinking was to rely on vnode actions KAUTH_VNODE_WRITE_DATA, But I have observed that there KAUTH_VNODE_WRITE_DATA multiple calls comes after KAUTH_FILEOP_CLOSE and even when file is not modified.
Any idea why such behavior exist?
Thanks in advance.
Regards,
Rupesh
KAuth and especially KAUTH_FILEOP_CLOSE_MODIFIED is buggy, and I already reported some problems related to it to Apple (a long time ago):
Events happening on file descriptor inherited from parent process seem to not trigger call to the KAuth callback at all. (See http://www.openradar.me/8898118)
The flag KAUTH_FILEOP_CLOSE_MODIFIED is not specified when the given file has transparent zlib compression enabled. (See http://www.openradar.me/23029109)
That said, I am quite confident that (as of 10.5.x, 10.6.x, 10.7.x) the callbacks are always called directly from the kernel thread doing the syscall. For example when open(2) is called, it calls the kauth callbacks for the vnode context and then (if allowed by return value) calls the VFS driver to realize the operation. The fileop (KAUTH_FILEOP_CLOSE) works also from the same thread but is just called after the closing itself.
Hence I don't think KAUTH_VNODE_WRITE_DATA can come after KAUTH_FILEOP_CLOSE for the same event.
Either you have a bug in your code, or it is another event (e.g. next open of the same file after it was closed in the same or other process.)
Still there are some traps you must be aware of:
Any I/O which is performed by kernel itself (including other kexts) does not trigger the kauth callbacks at all.
If there are multiple callbacks for the vnode context (e.g. from multiple Kexts), kernel then calls them one by one for every event. However as soon as some of them returns KAUTH_RESULT_ALLOW or KAUTH_RESULT_DENY, it finally decides and the rest of the callbacks is not called. I.e. all callbacks are called only if all of them but the last return KAUTH_RESULT_DEFER. (AFAIK, for fileop this is not true, because in this case the return value is completely ignored.)
I have a Simulink model that uses an embedded MATLAB function for a block, and I haven't been able to figure out how to move data between the embedded MATLAB block and a GUI in real-time (i.e. while the model is running). I tried to implement a "to workspace" block in my model but I don't know how to correctly use it.
Does anyone know how to move data from a Simulink block into a GUI in real-time?
Non-real-time solution:
If you want to set parameters in a GUI, simulate a model with those parameters, and then display the simulation output in the GUI, there is a good tutorial on blinkdagger.com. One solution they describe is using the SIMSET function to define which workspace the Simulink model interacts with. You should be able to supersede the base workspace so that data is instead sent to and from the workspace of the GUI functions that are calling the Simulink model.
Real-time solution
As suggested by MikeT, you can use a RuntimeObject. You first have to use the get_param function to get the RuntimeObject from the block:
rto = get_param(obj,'RuntimeObject');
Where obj is either a block pathname or a block-object handle. You can get the pathname of the most recently selected block using the GCB function (in which case you can replace obj with gcb). You can then get the block's output with the following:
blockData = rto.OutputPort(1).Data
One additional caveat from the documentation:
To ensure the Data field contains the
correct block output, turn off the
Signal storage reuse option (see
Signal storage reuse) on the
Optimization pane in the Configuration Parameters dialog box.
You would likely end up with a loop or a timer routine running in your GUI that would continuously get the output data from the RuntimeObject for as long as the simulation is running. The documentation also states:
A run-time object exists only while
the model containing the block is
running or paused. If the model is
stopped, get_param returns an empty
handle. When you stop or pause a
model, all existing handles for
run-time objects become empty.
Your loop or timer routine would thus have to keep checking first that the RuntimeObject exists, and either stop (if it doesn't) or get the data from it (if it does). I'm unsure of exactly how to check for existence of a RuntimeObject, but I believe you would either check if the object is empty or if the BlockHandle property of the object is empty:
isempty(rto) % Check if the RuntimeObject is empty
%OR
isempty(rto.BlockHandle) % Check if the BlockHandle property is empty
From your responses, I'm guessing you want to see the results while the simulation is running, is that correct? The blinkdagger.com tutorial lets you view the results of a simulation after it is done, but not while it is running. Do you basically want to embed something like a scope block into your GUI?
There's a few ways to do this, the best is probably using the EML block's runtime object. If you use this, you should be able to look at the output of the EML block while it is running.