I have noticed that notepad can take a very long time to load a large where it appears to be completely unresponsive but the file handle only seems to be active for a very short period of time during the beginning. Once the loading has begun another process can open a file for exclusive share mode, ie using the value 0 for dwShareMode
How does notepad continue to do it's loading with a closed handle or whatever magic it uses?
Nevermind I have figured out what notepad does. Here's the sollution if anyone else needs it:
hFile = CreateFile("file", 0x80000000, 3, NULL, 3, 0x80, NULL);
hMap = CreateFileMapping(hFile, NULL, 2, sizeHigh, sizeLow, NULL);
ptr = MapViewOfFile(hMap, 4, 0, 0, size);
CloseHandle(hMap);
CloseHandle(hFile);
/* At this point the handles are closed so programs that try
to get share exclusive on the file succeed but I still have
a pointer for reading the memory myself */
UnmapViewOfFile(ptr);
Related
When working with filesystem files on Windows, and specifically with the CreateFile API:
With regard to access sharing, that is having multiple, independent, CreateFile calls to open the same file, possibly with different flags and sharing modes, does it matter in any way whether file access is performed from within the same process or from a different process?
That is, once someone has opened a file with CreateFile(..., FILE_SHARE_READ, ...), noone should be able to open the same file with GENERIC_WRITE access. Does it matter whether different calls originate from within the same process, or from different processes?
My impression so far is that process boundaries do not matter for independent CreateFile calls to the same file. (They do matter for handle inheritance, etc.)
But that docs contain such gems as:
To enable a process to share a file or device while another process
has the file or device open, use a compatible combination of one or
more of the following values. For more information about valid
combinations of this parameter with the dwDesiredAccess parameter, see
Creating and Opening Files.
which doesn't exactly inspire confidence.
No. In general such access restrictions to files don't care if they are done from the same or different process.
The file sharing flags work always the same way.
As #xMRi answered, it does not matter if the Valid second call to CreateFile is in the same process or not.
Creating and Opening Files
illustrates the valid combinations of two calls to CreateFile using
various access modes and sharing modes (dwDesiredAccess, dwShareMode
respectively). It does not matter in which order the CreateFile calls
are made.
An example to illustrate.
#include <Windows.h>
#include <Tchar.h>
int main()
{
//...
HANDLE hFile1 = CreateFile(TEXT("tempfile"),
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ,
NULL,
CREATE_ALWAYS,
0,
NULL);
HANDLE hFile2 = CreateFile(TEXT("tempfile"),
GENERIC_READ,
FILE_SHARE_READ| FILE_SHARE_WRITE,
NULL,
OPEN_ALWAYS,
0,
NULL);
if (hFile1 != INVALID_HANDLE_VALUE && hFile2 != INVALID_HANDLE_VALUE)
{
HANDLE hFile = hFile1;
//HANDLE hFile = hFile2;
FILE_BASIC_INFO fdi{};
fdi.FileAttributes = FILE_ATTRIBUTE_TEMPORARY | FILE_ATTRIBUTE_NORMAL;
BOOL fResult = SetFileInformationByHandle(hFile,
FileBasicInfo,
&fdi,
sizeof(FILE_BASIC_INFO));
if (fResult)
{
// File will be deleted upon CloseHandle.
_tprintf(TEXT("SetFileInformationByHandle marked tempfile for deletion\n"));
// ...
// Now use the file for whatever temp data storage you need,
// it will automatically be deleted upon CloseHandle or
// application termination.
// ...
}
else
{
_tprintf(TEXT("error %lu: SetFileInformationByHandle could not mark tempfile for deletion\n"),
GetLastError());
}
CloseHandle(hFile);
// At this point, the file is closed and deleted by the system.
}
else
{
_tprintf(TEXT("error %lu: could not create tempfile\n"),
GetLastError());
}
//...
}
Editor FooEdit (let's call it) uses ReplaceFile() when saving to ensure that the save operation is effectively atomic, and that if anything goes wrong then the original file on disc is preserved. (The other important benefit of ReplaceFile() is continuity of file identity - creation date and other metadata.)
FooEdit also keeps open a handle to the file with a sharing mode of just FILE_SHARE_READ, so that other processes can open the file but can't write to it while it while FooEdit has it open for writing.
"Obviously", this handle has to be closed briefly while the ReplaceFile operation takes place, and this allows a race in which another process can potentially open the file with write access before FooEdit re-establishes it's FILE_SHARE_READ lock handle.
(If FooEdit doesn't close its FILE_SHARE_READ handle before calling ReplaceFile(), then ReplaceFile() fails with a sharing violation.)
I'd like to know what is the simplest way to resolve this race. The options seem to be either to find another way to lock the file that is compatible with ReplaceFile() (I don't see how this is possible) or to replicate all the behaviour of ReplaceFile(), but using an existing file handle to access the destination file rather than a path. I'm a bit stuck on how all of the operations of ReplaceFile() could be carried out atomically from user code (and reimplementing ReplaceFile() seems a bad idea anyway).
This must be a common problem, so probably there's an obvious solution that I've missed.
(This question seems related but has no answer: Transactionally write a file change on Windows.)
Here's a minimal verifiable example showing what I am trying to achieve (updated 13:18 30/9/2015 UTC). You must supply three file names as command line arguments, all on the same volume. The first must already exist.
I always get a sharing violation from ReplaceFile().
#include <Windows.h>
#include <stdio.h>
#include <assert.h>
int main(int argc, char *argv[])
{
HANDLE lock;
HANDLE temp;
DWORD bytes;
if (argc != 4)
{
puts("First argument is the project file. Second argument is the temporary file.");
puts("The third argument is the backup file.");
}
/* Open and lock the project file to make sure no one else can modify it */
lock = CreateFile(argv[1], GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_DELETE, NULL, OPEN_EXISTING, 0, 0);
assert(lock != INVALID_HANDLE_VALUE);
/* Save to the temporary file. */
temp = CreateFile(argv[2], GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_DELETE, NULL, CREATE_ALWAYS, 0, 0);
assert(temp != INVALID_HANDLE_VALUE);
WriteFile(temp, "test", 4, &bytes, NULL);
/* Keep temp open so that another process can't modify the file. */
if (!ReplaceFile(argv[1], argv[2], argv[3], 0, NULL, NULL))
{
if (GetLastError() == ERROR_SHARING_VIOLATION)
puts("Sharing violation as I expected");
else
puts("Something went wrong");
}
else
puts("ReplaceFile worked - not what I expected");
/* If it worked the file referenced by temp would now be called argv[1]. */
CloseHandle(lock);
lock = temp;
return EXIT_SUCCESS;
}
Thanks to Hans Passant, who provided some valuable clarifying thoughts in an answer now deleted. Here's what I discovered while following up his suggestions:
It seems ReplaceFile() allows lpReplacedFileName to be open FILE_SHARE_READ | FILE_SHARE_DELETE, but lpReplacementFileName can't be. (And this behaviour doesn't seem to depend on whether lpBackupFileName is supplied.) So it's perfectly possible to replace a file that another process has open even if that other process doesn't allow FILE_SHARE_WRITE, which was Hans' point.
But FooEdit is trying to ensure no other process can open the file with GENERIC_WRITE in the first place. To ensure in FooEdit that there's no race where another process can open the replacement file with GENERIC_WRITE, it seems that FooEdit has to keep hold continuously of a FILE_SHARE_READ | FILE_SHARE_DELETE handle to lpReplacementFileName, which then precludes use of ReplaceFile().
Actually I think there might be a solution that doesn't involve transactions (although transactions are still available as far as I know). I haven't tried it myself, but I think on NTFS it should be possible to create a new file stream (use a long random name to ensure there are no collisions), write your data, and then rename that stream to the stream you actually wanted to write to.
FILE_RENAME_INFORMATION suggests this should be possible, since it talks about renaming data streams.
However, this would only work on NTFS. For other file systems I don't think you have a choice.
I'd like to know what is the simplest way to resolve this race.
There is no simple way to resolve this race. It's an inherent part of the file system which is not transactional. MS introduced a transactional file API with Vista but now strongly advise developers not to use it as it may be removed in a future release.
I have had some experience with ReplaceFile but I think it caused more trouble than it was worth. My recollection was that whilst meta data was preserved, a new file was created. A consequence of this was very annoying behaviour for files saved on the desktop. Because such files have their position preserved, creating a new file resulted in the default position being used. So you'd save a file, you'd drag it to the place on the desktop where you wanted to keep it, and then when you saved the file again, it moved back to the default position.
Starting with the ndisprot sample from Microsoft I try to write a NDIS protocol driver. From User space I try to read and write to the device simultaneous (out of two threads). Since I don't receive any packets, the ReadFile system call blocks. I'm not able to complete a WriteFile system call in this state.
CHAR NdisProtDevice[] = "\\\\.\\\\NDISprot";
CHAR * pNdisProtDevice = &NdisProtDevice[0];
this.iHandle = CreateFile(pNdisProtDevice,
GENERIC_WRITE | GENERIC_READ, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
// Blocks, because no frames arrive
bSuccess = (BOOLEAN)ReadFile(Handle,
(LPVOID)pReadBuf,
PacketLength,
&BytesRead,
NULL);
...
// Called some seconds later from another thread, while ReadFile still blocking...
bSuccess = (BOOLEAN)WriteFile(Handle,
pWriteBuf,
PacketLength,
&BytesWritten,
NULL);
I added some debug messages and discovered that the driver function associated with IRP_MJ_WRITE (NdisprotWrite) gets not even called! Something between the user space application and the driver blocks concurrent access to the device \Device\NDISprot.
How can I concurrent Read and Write to the file?
By default, you can only have one outstanding I/O request per usermode handle. Either open multiple handles, or open your one handle with FILE_FLAG_OVERLAPPED. (Once you use FILE_FLAG_OVERLAPPED, you also generally need to use OVERLAPPED structures - make sure you've got the gist of it by skimming this and this.)
Let us say that i am writing data to a file handle:
hFile = CreateFile(filename, GENERICREAD | GENERICWRITE, 0, null, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
//[snip error check]
try
{
if (!WriteFile(hFile, buffer, count, ref bytesWritten, null))
throw new Win32Exception(GetLastError());
}
finally
{
CloseHandle();
}
If my write of data failed, i want the file to be deleted when i close the handle. I.e.: i want the file to be "un-CreatFile'd".
i've tried the obvious, delete the file if there was a problem:
hFile = CreateFile(filename, GENERICREAD | GENERICWRITE, 0, null, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
//[snip error check]
try
{
try
{
if (!WriteFile(hFile, buffer, count, ref bytesWritten, null))
throw new Win32Exception(GetLastError());
}
finally
{
CloseHandle();
}
}
catch
{
DeleteFile(filename);
throw;
}
There are two problems with that approach:
There's a race condition, where someone could open my file after i close it, but before i delete it
i might have permission to Create a file, but not the Delete it.
What i would like is a way to retroactively specify:
FILE_FLAG_DELETE_ON_CLOSE: The file is to be deleted immediately after all of its handles are closed, which includes the specified handle and any other open or duplicated handles.
to the file that i have open.
i created the file! Surely i can uncreate it! All because i forgot to specify a flag before-hand?
In all versions of Windows, you can
Specify FILE_SHARE_DELETE when opening the file originally.
Open it a second time, specifying FILE_FLAG_DELETE_ON_CLOSE
Close the second handle.
The file will be deleted when all handles are closed.
This is explained in the FILE_FLAG_DELETE_ON_CLOSE description in the documentation for CreateFile:
The file is to be deleted immediately after all of its handles are closed, which includes the specified handle and any other open or duplicated handles.
If there are existing open handles to a file, the call fails unless they were all opened with the FILE_SHARE_DELETE share mode.
If the automatic deletion isn't good enough, you could look into Transactional NTFS, putting file creation and all writes into a transaction, and then rolling it back if necessary.
In Windows Vista and newer, this could be done using SetFileInformationByHandle and FILE_DISPOSITION_INFO.
If you're using Windows 2003 server or Vista or higher, ReOpenFile() is what you're looking for. It allows you to specify a new 'flags' parameter for the reopened handle. Shame it's apparently not available on XP.
I'm trying to call IOCTL_BTH_GET_LOCAL_INFO using DeviceIoControl, which I believe it can be done (accordingly to Bluetooth Profile Driver IOCTLs).
I'm on a Windows 7 x64 using Visual Studio 2012 (probably with default configuration).
The handle have a valid value (I removed the validation code) but DeviceIoControl always returns ERROR_INVALID_USER_BUFFER (error 1784).
Here's the code:
int main() {
BTH_LOCAL_RADIO_INFO buffer;
BOOL fStatus;
HANDLE h;
DWORD returned = 0;
h = CreateFile(
TEXT("\\\\.\\BthPan"),
GENERIC_READ | GENERIC_WRITE ,
FILE_SHARE_READ | FILE_SHARE_WRITE,
NULL,
OPEN_EXISTING,
0,
NULL);
fStatus = DeviceIoControl(
h,
IOCTL_BTH_GET_LOCAL_INFO,
NULL, 0,
(LPVOID)&buffer, sizeof(BTH_LOCAL_RADIO_INFO),
&returned,
(LPOVERLAPPED) NULL
);
(...)
After some research I tried the following solutions:
Changing the structure pack alignment to 1/4/8 byte (with VS options);
Using values which are 8-byte aligned (later I've found out that
this was already happening, even with data types smaller than 8 bytes). After a while I've read somewhere that DeviceIoControl deals with misaligment for you, so probably no need to worry about that.
All of the solutions above have failed. What do you think it is? VS have a bunch of configurations for Win32, but that never gave me a problem before (first time with IOCTL though).
I've seen some of that code on 32feet.NET, so probably it's just an error of mine (I can't see any difference).
You're sending IOCTL_BTH_GET_LOCAL_INFO to the wrong device (Bluetooth Personal Area Network instead of Bluetooth Radio).
So I suggest you to use BluetoothFindFirstRadio, BluetoothFindNextRadio and BluetoothFindRadioClose to simply iterate through local Bluetooth radios, rather than to guess the correct DOS Device Names for them.