On UNIX, I can, for example, tell the OS that the mapping will be needed in the future with posix_fadvise(POSIX_FADV_WILLNEED). It will then read-ahead the data if it feels so.
How to tell the access intend to Windows ?
Beginning with Windows 8, there is the PrefetchVirtualMemory function for this purpose.
Actually, as Anders mostly suggested, there is no such method in the memory management functions available in Windows 7 and earlier.
2 different ways exists to do something similar :
Read the data asynchronously with ReadFileEx. The data might then still be in the file cache when needed later.
Open the file with a streaming hint with the FILE_FLAG_SEQUENTIAL_SCAN attribute of CreateFile. Readahead would then perhaps be automatically done.
You can pass FILE_FLAG_RANDOM_ACCESS or FILE_FLAG_SEQUENTIAL_SCAN to CreateFile()
Related
I want to make a tool similar to zerofree for linux. I want to do it by allocating a big file without zeroing it, look for nonzero blocks and rewrite them.
With admin privileges it is possible, uTorrent can do this: http://www.netcheif.com/Articles/uTorrent/html/AppendixA_02_12.html#diskio.no_zero , but it's closed source.
I am not sure this answers your question (need), but such a tool already exists. You might have a look at fsutil.exe Fsutil command line tool. This tool has a huge potential to discover the internal structures of NTFS files and can also create file of any size (without the need to zeroing it manually). Hope that helps.
Wrote a tool https://github.com/basinilya/winzerofree . It uses SetFileValidData() as #RaymondChen suggested
You should try SetFilePointerEx
Note that it is not an error to set the file pointer to a position
beyond the end of the file.
So after you create file, call SetFilePointerEx and then SetEndOfFile or WriteFile or WriteFileEx and close the file, size should be increased.
EDIT
Raymonds suggested SetValidData is also good solution, but this requares privileges, so shouldn't be used often by anyone.
My solution is the best on NTFS, because it supports feature known as initialized size it means that after using SetFilePointerEx data won't be initialized to zeros, but after attempt to read uninitialized data you will receive zeros.
To sum up, if NTFS use SetFilePointerEx, if FAT (not very likely) - use SetValidData
For linux and windows, in the same process, how to make two page entries reference to the same physical page?
For windows, by reading at the MSDN, looks like I can call CreateFileMapping by passing INVALID_HANDLE_VALUE to create a file mapping without backed by a file. Then I can call MapViewOfFileEx twice with different lpBaseAddress, which essentially makes two different addresses reference to the same physical address.
My question is, how to do it under linux? I read manual for mmap, and didn't see a way to do it, unless the region is backed by a file (with flag MAP_SHARED) but modifications to this region will be written to the file, which is not what I want. Does anyone aware of someway to do this? I am not against backing by a file, as long as the writing to the region doesn't actually goes to the disk. Using tmpfs is not an option because I can't guarantee user has a tmpfs mounted.
By the way, the code should be user mode code, not kernel mode.
Use shm_open() to create a file for mmap().
"I want to write some emulator" is the same purpose as mine when I used this trick.
I did use ipc/shm, but I forgot the detail. It was very very very very probably: shmget()+shmat()
Say I have app TestApp.exe
While TestApp.exe is running I want a separate program to be able to read the executable code that is resident in memory. I'd like to ignore stack and heap and anything else that is tangential.
Put another way, I guess I'm asking how to determine where the memory-side equivalent of the .exe binary data on disk resides. I realize it's not a 1:1 stuffing into memory.
Edit: I think what I'm asking for is shown as Image in the following screenshot of vmmap.exe
Edit: I am able to get from memory all memory that is tagged with any protect flag of Execute* (PAGE_EXECUTE, etc) using VirtualQueryEx and ReadProcessMemory. There are a couple issues with that. First, I'm grabbing about 2 megabytes of data for notepad.exe which is a 189 kilobyte file on disk. Everything I'm grabbing has a protect flag of PAGE_EXECUTE. Second, If I run it on a different Win7 64bit machine I get the same data, only split in half and in a different order. I could use some expert guidance. :)
Edit: Also, not sure why I'm at -1 for this question. If I need to clear anything up please let me know.
Inject a DLL to the target process and call GetModuleHandle with the name of the executable. That will point to its PE header that has been loaded in the memory. Once you have this information, you can parse the PE header manually and find where .text section is located relative to the base address of the image in the memory.
no need to inject a dll
use native api hooking apis
I learned a ton doing this project. I ended up parsing the PE header and using that information to route me all over. In the end I accomplished what I set out to and I am more knowledgeable as a result.
I'm using the GoAsm assembler on a Windows 7 - 64 bit OS and I'll be asking you a few (not so dumb) questions.
First question :
How can I find the physical address of a file ?
Let's suppose file "Text.txt" is at the root of my C:\ partition.
Is there a way to get the exact memory address where this file is ?
Second question :
Is it possible to call a routine which will just do like if I invoked a C function ?
(i.e. : Consider a C function "WriteToScreen", is it possible to have the same function, but in assembler format, that means without having the need to use high-level invokes to do that work ?
Third question :
Are there somewhere on the net some include files for GoAsm containing useful routines like (move, copy, edit, erase) commands ? I've first thought of ms-dos interrupts but I can't manage to get them to work without crashing the program. I guess it just not compatible with Windows OS even though the command prompt acts like ms-dos... ?
Fourth question :
I've heard from different sources and myself that NASM works pretty bad on Win7 x64, is it just true, or am I doing it the wrong way ?
1
An hard drive, from a logical point of view, can be seen as a sequence of "blocks" (the more common name is sectors). How these blocks are organized physically on the disks can be disregarded, but the driver must know someway how to get data of course, though you send to modern hd driver "high level" commands that, as far as you know, are not strongly related to where data physically are (you can say "read the block 123", but there's no extern evidence of where that block lives).
However this way you can "name" a block with a number, and say e.g. that block 0 is the MBR. Each block contains several bytes (512, 1024...). Not all used blocks contain actual data of a file, in fact there are metainformations of any sort, depending on the filesystem but even related to the "structure" of the hd (I mean, partitions).
A file located on an hd is not automatically loaded into memory, so it has no memory address. Once you read it, piece of it if not all are of course copied into the memory you give, which is not an intrinsic property of the file. (Filesystems retrieve the blocks belonging to the file and "show" them as we are used to see them, as a single "unit", the file)
Summarizing: files have no memory address. The physical address could be the set of blocks holding data (and metadata, like inodes ) of the file, or just the first block (but if a block of data is N, N+1 could not belong to the same file - the blocks need no to be one next to the other). To know them, you have to analyse the structure of the filesystem you use. I don't know if there's an API to retrieve them easily, but in the worst case you can analyse the source code of the filesystem... good luck!
2
C functions are translated into assembly. If you respect the C calling convention, you can write a "C function" directly in assembly. Try reading this and this for x86.
3
You can call windows API from asm. Forget MS-DOS, MS-DOS is dead, MS-DOS is not Windows, the cmd is a sort of "emulation"... indeed no, not an emulation but just a command line interface that resemble the one MS-DOS users was used to. But it is not exaclty the same, i.e. there are no MS-DOS system interrupt you can use. Iczelion's assembly tutorials, though old, could be an interesting resource. (If links expire, try with the wayback machine)
4
I do not own Win7 and never installed nasm on windows, so I can't say anything about.
For the first question just drag the file into the address bar in the browser
To restrict the scope, let assume we are in Windows world only.
Also assume we don't want to play with permission policy.
Is it possible for us to create a file that cannot be copied?
Thank you in advance.
"Trying to make digital files uncopyable is like trying to make water not wet." ~ Bruce Schneier
No. You can't create a file that a SYSADMIN can't copy. You could encrypt it, though.
Well, how about creating a file that uses up more than 50% of the total space on that machine and that is not compressible?
For instance, let us assume that you want to save a boolean (true or false) in such a fashion.
Depending on its value, you could then write a bit stream of ones or zeroes and encrypt said stream using some kind of encryption algorith, such as AES in CBC mode. This gives you the added advantage of error correction. Even in case of massive data corruption, you should be able to recover your boolean by checking whether ones or zeroes are prevalent in the decrypted stream.
In that case you cannot copy it around (completely) on the machine...
Of course, any type of external memory that can be added to the system would pose a problem in this scenario. But the file would be already encrypted, so don't worry about it too much...
Any file that can be read can have its contents written to another location (such as another file, i.e. copied).
The only thing you can do is limit who/what can read the file.
What is the motivation behind? If it is a read-only file, you can have it as embedded resources within your assembly.
Nice try, RIAA.
But seriously, no you can not. It is always possible to copy, you can just make it it more difficult for people to make sense of the file or try to hide it using like encryption. Spotify does it.
If you really try hard thou, you cold make a root-kit for windows and use it to prevent windows from even knowing about the file and also prevent copies. The file will still be there and copy-able by other tools, or Linux accessing the ntfs.
If in a running process you open a file and hold an exclusive lock, then other processes cannot read the file until you close the handle or your process terminates. However, as admin you could forcibly remove the lock handle.
Short answer: No.
You can, of course, use security settings to limit who can read the file. But if someone can read it, then they can copy it. Even if you found some operating system trick to disable "ordinary" copying, if someone can read the file, they can extract the contents, store it in memory, and then write it somewhere else.
You can encrypt the contents so it's only useful to your own program, that knows how to decrypt it.
That's about it.
When using Windows 7 to copy some files from a hard drive, certain files popped up a message saying they could not be copied in their entirety; certain data would be omitted from the copy. I suspect that had something to do with slack space at the end of the files, though I thought the message was curious. I would have expected the copy operation to just ignore the slack space.
If you are running old (OLD) versions of windows, there are certain characters you can put in the filename that make it invalid, not listed in folders, etc. They were used a lot in the old pub ftp days of filesharing ;)
In the old DOS days, you used to be able to flag disk sectors as bad and still read from them. This meant the OS ignored the sector in question but your application would know where to look and be able to get the data. Not sure this would work these days.
Another old MS-DOS trick was to put a space character in the middle of the filename (yes, spaces were valid characters for filenames). Since there was no method on the command line to escape a space, the file couldn't be copied using the DOS commands.
This answer is outside Windows so yeah
Dont know if its already been said but what about a file that is an inseperable part of the firmware so that it is always on AND running, perhaps it has firmware that generates a sequence that is required for the other . AN incedental effect of its running is to prevent any 80% or more of its code from being replicated. Lets say its on an entirely different board, protected by surge protectors, heavy em proof shielding and anything else required to make it completely unerasable.
If its possible to make a program that is ALWAYS on and running as long as the copying software is running then yes.
I have another way and this IS with windows. I will come to your house and give you a disk, i will then proceed to destroy every single computer you put the disk into. This doesnt work on XP
Well technically you could create and write to a write-only network share.