I'm trying to use a memory-mapped file under windows (using CreateFile/CreateFileMapping/MapViewOfFile functions), and I'm currently specifying FILE_SHARE_READ and FILE_SHARE_WRITE when calling CREATE_FILE. However, this is locking the file from being using by other processes.
What I want is to memory-map a snapshot of the file at the time I call CreateFileMapping or MapViewOfFile, so that I don't see any changes (writes or deletes) made to the file by other processes. Sort of like copy-on-write, but other processes are doing the write. Can I do this using memory-mapped files on windows?
That's just not how memory mapped files work. Windows puts a hard lock on the file so that nobody can change its content and make it different from the pages mapped into RAM. Those pages in RAM are shared between all processes that created a view on the file. There is no 'tear off' option.
You could simply map the file and make a copy of the bytes in the view. Some synchronization with the other processes would typically be required.
Related
Is there an API or straight forward way to simulate any data as a windows explorer drive/file-hierarchy? I don't want to create actual files I want to have a view on my data as though it were.
E.g. if my real data was in a database but I want it to look like a drive/folder/files?
You can create shell namespace extension or a virtual file system.
Shell namespace extension has many shortcomings but if you need files to be only exposed and copied, it can work.
Virtual file system or disk can be created only with help of third-party solutions such as our Callback File System or CallbackDisk. This approach fully emulates the local disk and the filesystem up to possibility to manipulate individual sectors (but this depends on the way used, i.e. whether you emulate the disk or the filesystem).
I have two questions, both are them may be related so I am asking at once.
Linux has /proc directory which is runtime data structure and gives information about running process. Does windows have any such directory where I can get runtime info about process, like its layout and open handles. Please do no suggest tools like Process Explorer, its good but they are not part of core windows os.
Secondly, it is said for Windows that not everything is file, like socket is not a file. Does it mean that it is not a sort of file you can see in your hard disk but a runtime it creates file and in proc like data structure it has some entry.
Thanks.
While Windows has the ability to create virtual files (device drivers use this), there are no such files for process information.
Information about processes is available either through the process functions, the undocumented functions used by Process Explorer, or not at all.
Not every file is stored on some disk.
Virtual files are essentially just some value in memory, or some callback function that generates the file contents dynamically when you're trying to read it.
I am currently on a mission loading files into pagecache, and I want to load locked files, too. The goal is nothing more than pro-actively keeping a dataset in RAM, reducing loading times within third party applications.
Shadow copies were my first thought on this, but unfortunately seem to have separated pagecaches.
So is there any way cheating around the exclusive lock mechanism? Like fetching file fragment location on disk, accessing whole disk and reading directly (which I fear is another separated pagecache, anyways)?
Or is there a very different approach to directing the pagecache, e.g. some Windows API that can be told to load a specific file into pagecache?
You can access locked files in Windows from kernel-mode driver, or using our RawDisk product. But for your task (speed up DB file access) this won't work right as Windows' filesystem cache size is limited (it won't accommodate GBs of data).
In general, if I were to develop a large software project (for small application the amount of work needed is just enormous) I'd do the following: create a virtual drive backed by in-memory storage, present the DB file to the application via that virtual disk and flush drive contents to the disk on change asynchronously. All of this should be done in kernel mode (this is where development time grows to 12-15 man-months of work).
In theory the same can be done using one of our Virtual Storage products, but going back into user mode for callback handling would eliminate all that you gain from moving the data into RAM.
This isn't necessarily a programming question, but I've hit a performance bottleneck with disk IO and I'd like to try writing and reading from RAM instead of the hard drive. I want to create my file in RAM and then run my application against it.
There are lots of tools for creating RAM drives. None of them seem to work for windows 2008 R2. Does anyone know if this is possible and if so how. Does anyone know of a tool that works?
Use Memory-Mapped Files to map the file into RAM (including memory backed to pagefile, if it's large. so be careful).
File mapping is the association of a
file's contents with a portion of the
virtual address space of a process.
The system creates a file mapping
object (also known as a section
object) to maintain this association.
A file view is the portion of virtual
address space that a process uses to
access the file's contents. File
mapping allows the process to use both
random input and output (I/O) and
sequential I/O. It also allows the
process to work efficiently with a
large data file, such as a database,
without having to map the whole file
into memory.
whatever ram disk you choose to purchase or write, remember check if the ram disk driver responds the SetDispositionInformationFile call.
I ended up finding and using Vsuit Ramdisk (Server Edition). It works great but its not free.
I want to call the Windows API OpenProcess function on another process running on the machine. Will this always cause the file whose process I am opening to be write locked? Or does it depend on the access rights I request?
Yes, it is a fundamental property of Windows. When an executable file gets loaded (EXE or DLL), Windows creates a memory mapped view of the file. Chunks of code or data from the executable file get page-faulted into RAM, as needed to keep the program running. It works the other way around too, when Windows needs to make RAM available for another program then it throws chunks of mapped pages away, the ones that weren't used in a while. Those pages don't take up space in the paging file if they are code, they can be reloaded from the executable file.
Very efficient, code that was written when 16 megabytes of RAM was a luxury. The memory mapped section keeps a write lock on the file. Still useful in this day and age, it prevents some kind of malware with fecking with the code of a running process.
The process file is locked while the process is running; it doesn't have anything to do with OpenProcess. The file is unlocked when the process terminates.