Why does ReadDirectoryChangesW omit events? - windows

Question

I use ReadDirectoryChangesW to watch a specified directory and update indexing structures whenever a change is detected. I use the following code (roughly)
var
InfoPointer : PFileNotifyInformation;
NextOffset : DWORD;
...
while (not Terminated) do begin
if ReadDirectoryChangesW (FDirHandle, FBuffer, FBufferLength, True,
FFilter, #BytesRead, #FOverlap, nil) then
begin
WaitResult := WaitForMultipleObjects (2, #FEventArray, False, INFINITE);
if (WaitResult = waitFileChange) then
begin
InfoPointer := FBuffer;
repeat
NextOffset := InfoPointer.NextEntryOffset;
...
PByte (InfoPointer) := PByte (InfoPointer) + NextOffset;
until NextOffset = 0;
end;
end;
end;
Filter is
FFilter := FILE_NOTIFY_CHANGE_FILE_NAME or
FILE_NOTIFY_CHANGE_DIR_NAME or
FILE_NOTIFY_CHANGE_SIZE or
FILE_NOTIFY_CHANGE_LAST_WRITE;
and the directory handle is obtained like this:
FDirHandle := CreateFile (PChar (FDirectoryWatch.WatchedDirectory),
FILE_LIST_DIRECTORY or GENERIC_READ,
FILE_SHARE_READ or FILE_SHARE_WRITE or FILE_SHARE_DELETE,
nil, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS or
FILE_FLAG_OVERLAPPED, 0);
When I delete multiple files I get only one event and NextOffset is 0! And when I delete a directory I get only one event for the directory. What if I want one event for each file in the directory?
Any help would be appreciated.

Answer 1

It seems to me that you are mixing the various ways to use ReadDirectoryChangesW(), you do both specify the FILE_FLAG_OVERLAPPED flag when opening the directory and provide a pointer to the lpOverlapped parameter, meaning you want to wait on the event in the structure and handle the asynchronous I/O; and at the same time you call ReadDirectoryChangesW() in a loop in a worker thread. I would first try again with lpOverlapped set to nil, as you have a dedicated thread and can use the synchronous mode.
In the documentation of the ReadDirectoryChangesW() API function the different ways to use it are described. Note that it is also possible that the buffer overflows, so change events can be lost anyway. Maybe you should rethink your strategy of relying solely on this function, comparing snapshots of directory contents could work as well.
Edit:
Your edited code looks better. In my tests however ReadDirectoryChangesW() did work as advertised, there were either several data entries in the returned buffer, or there were more than one buffer to process. This depends on timing, after hitting a breakpoint in Delphi I get several entries in one buffer.
For completeness I attach the test code, implemented using Delphi 5:
type
TWatcherThread = class(TThread)
private
fChangeHandle: THandle;
fDirHandle: THandle;
fShutdownHandle: THandle;
protected
procedure Execute; override;
public
constructor Create(ADirectoryToWatch: string);
destructor Destroy; override;
procedure Shutdown;
end;
constructor TWatcherThread.Create(ADirectoryToWatch: string);
const
FILE_LIST_DIRECTORY = 1;
begin
inherited Create(TRUE);
fChangeHandle := CreateEvent(nil, FALSE, FALSE, nil);
fDirHandle := CreateFile(PChar(ADirectoryToWatch),
FILE_LIST_DIRECTORY or GENERIC_READ,
FILE_SHARE_READ or FILE_SHARE_WRITE or FILE_SHARE_DELETE,
nil, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS or FILE_FLAG_OVERLAPPED, 0);
fShutdownHandle := CreateEvent(nil, FALSE, FALSE, nil);
Resume;
end;
destructor TWatcherThread.Destroy;
begin
if fDirHandle <> INVALID_HANDLE_VALUE then
CloseHandle(fDirHandle);
if fChangeHandle <> 0 then
CloseHandle(fChangeHandle);
if fShutdownHandle <> 0 then
CloseHandle(fShutdownHandle);
inherited Destroy;
end;
procedure TWatcherThread.Execute;
type
PFileNotifyInformation = ^TFileNotifyInformation;
TFileNotifyInformation = record
NextEntryOffset: DWORD;
Action: DWORD;
FileNameLength: DWORD;
FileName: WideChar;
end;
const
BufferLength = 65536;
var
Filter, BytesRead: DWORD;
InfoPointer: PFileNotifyInformation;
Offset, NextOffset: DWORD;
Buffer: array[0..BufferLength - 1] of byte;
Overlap: TOverlapped;
Events: array[0..1] of THandle;
WaitResult: DWORD;
FileName, s: string;
begin
if fDirHandle <> INVALID_HANDLE_VALUE then begin
Filter := FILE_NOTIFY_CHANGE_FILE_NAME or FILE_NOTIFY_CHANGE_DIR_NAME
or FILE_NOTIFY_CHANGE_SIZE or FILE_NOTIFY_CHANGE_LAST_WRITE;
FillChar(Overlap, SizeOf(TOverlapped), 0);
Overlap.hEvent := fChangeHandle;
Events[0] := fChangeHandle;
Events[1] := fShutdownHandle;
while not Terminated do begin
if ReadDirectoryChangesW (fDirHandle, #Buffer[0], BufferLength, TRUE,
Filter, #BytesRead, #Overlap, nil)
then begin
WaitResult := WaitForMultipleObjects(2, #Events[0], FALSE, INFINITE);
if WaitResult = WAIT_OBJECT_0 then begin
InfoPointer := #Buffer[0];
Offset := 0;
repeat
NextOffset := InfoPointer.NextEntryOffset;
FileName := WideCharLenToString(#InfoPointer.FileName,
InfoPointer.FileNameLength);
SetLength(FileName, StrLen(PChar(FileName)));
s := Format('[%d] Action: %.8xh, File: "%s"',
[Offset, InfoPointer.Action, FileName]);
OutputDebugString(PChar(s));
PByte(InfoPointer) := PByte(DWORD(InfoPointer) + NextOffset);
Offset := Offset + NextOffset;
until NextOffset = 0;
end;
end;
end;
end;
end;
procedure TWatcherThread.Shutdown;
begin
Terminate;
if fShutdownHandle <> 0 then
SetEvent(fShutdownHandle);
end;
////////////////////////////////////////////////////////////////////////////////
procedure TForm1.FormCreate(Sender: TObject);
begin
fThread := TWatcherThread.Create('D:\Temp');
end;
procedure TForm1.FormDestroy(Sender: TObject);
begin
if fThread <> nil then begin
TWatcherThread(fThread).Shutdown;
fThread.Free;
end;
end;
Deleting a directory does indeed only return one change for it, nothing for the files contained in it. But it does make sense, as you are watching the handle of the parent directory only. If you need notifications for subdirectories you probably need to watch them as well.

Answer 2

We've had the same problem with losing events, especially if a lot of changes happens at the same time, ie. 500 files are copied to the monitored directory.
In the end we found Cromis and use the Directory watch. We have never looked back again.