I want to create memory mapped file on windows (the method I know is WINAPI's CreateFileMapping if there are other methods please mention them), then use fread to read from it, is it possible to read from it using fread? if it is possible,is it good programming? is it efficient?
Thanks.
fread() requires a FILE* to read from.
Once you have mapped the file into memory using MapViewOfFile(), you can wrap the memory pointer into a FILE* using fmemopen(), if your compiler/rtl provides an implementation for it, or if you install a third-party implementation for your compiler.
Related
I understand ChronicleMap creates a mmap file under the hood but I'm looking through the code and getting a bit lost.
Could someone kindly show me where in the code it creates the memory mapped file? I was expecting to see something using a MappedByteBuffer or something like that but I can't see it.
We don't use MappedByteBuffer as this is limited to just under 2 GB at a time. Instead, we call map directly so we can map in 100 TB if we want (I have done so on Linux)
The call is made in net.openhft.chronicle-core.OS.map(FileChannel, FileMode, long, long) in the Chronicle-Core library.
I am a beginner to kernel module programming. And using resources online (especially this tutorial) I have managed to write most of my driver.
The driver basically is a character device which maps different areas of SRAM into separate files. You can see the actual code here.
Right now I am able to successfully create 4 files under /dev and reading/writing the first file works too but the other 3 files do not work. I am using minor number to distinguish between the files and assign the starting address accordingly.
Questions:
Why are the other files not working?
Is there a better way to implement the module?
Thanks.
Line 141,
"if (cdev_add(&c_dev, first, 1) == -1)",
only applies the file_operations struct to the first device.
You should use MAXDEVICES instead of 1 here
On another note, the init code is messy (better use goto and not duplicate the cleanup for every function that can fail) and in some cases plain wrong (device_destroy() before any devices were created, resource leak in case you fail the create a device that isn't the first).
The entire file does not stand up to kernel coding conventions.
I want to make a tool similar to zerofree for linux. I want to do it by allocating a big file without zeroing it, look for nonzero blocks and rewrite them.
With admin privileges it is possible, uTorrent can do this: http://www.netcheif.com/Articles/uTorrent/html/AppendixA_02_12.html#diskio.no_zero , but it's closed source.
I am not sure this answers your question (need), but such a tool already exists. You might have a look at fsutil.exe Fsutil command line tool. This tool has a huge potential to discover the internal structures of NTFS files and can also create file of any size (without the need to zeroing it manually). Hope that helps.
Wrote a tool https://github.com/basinilya/winzerofree . It uses SetFileValidData() as #RaymondChen suggested
You should try SetFilePointerEx
Note that it is not an error to set the file pointer to a position
beyond the end of the file.
So after you create file, call SetFilePointerEx and then SetEndOfFile or WriteFile or WriteFileEx and close the file, size should be increased.
EDIT
Raymonds suggested SetValidData is also good solution, but this requares privileges, so shouldn't be used often by anyone.
My solution is the best on NTFS, because it supports feature known as initialized size it means that after using SetFilePointerEx data won't be initialized to zeros, but after attempt to read uninitialized data you will receive zeros.
To sum up, if NTFS use SetFilePointerEx, if FAT (not very likely) - use SetValidData
For linux and windows, in the same process, how to make two page entries reference to the same physical page?
For windows, by reading at the MSDN, looks like I can call CreateFileMapping by passing INVALID_HANDLE_VALUE to create a file mapping without backed by a file. Then I can call MapViewOfFileEx twice with different lpBaseAddress, which essentially makes two different addresses reference to the same physical address.
My question is, how to do it under linux? I read manual for mmap, and didn't see a way to do it, unless the region is backed by a file (with flag MAP_SHARED) but modifications to this region will be written to the file, which is not what I want. Does anyone aware of someway to do this? I am not against backing by a file, as long as the writing to the region doesn't actually goes to the disk. Using tmpfs is not an option because I can't guarantee user has a tmpfs mounted.
By the way, the code should be user mode code, not kernel mode.
Use shm_open() to create a file for mmap().
"I want to write some emulator" is the same purpose as mine when I used this trick.
I did use ipc/shm, but I forgot the detail. It was very very very very probably: shmget()+shmat()
Based on a file name, or a file handle, is there a Win-API method of determining what physical sector the file starts on?
You can get file cluster allocation by sending FSCTL_GET_RETRIEVAL_POINTERS using DeviceIoControl.
You'd have to read the allocation table directly.
I suspect that there is no such function.
Even if you know where the file starts, what good would it do? The rest of the file could be anywhere as soon as the file is larger than a single sector due to fragmentation.
You would probably need to develop deeper understanding of the file system involved and read the necessary information from the file allocation table or such mechanism.
No. Why? Because a file system is an abstraction of physical hardware. You don't need to know if you're on a RAM disk, hard drive, CD, or network drive, or if your data is compressed or encrypted -- Windows takes care of these little details for you.
You can always open the physical disk, but you'd need knowledge of the file system used.
What are you trying to accomplish with this?