Here's another question I met when reading < Windows via C/C++ 5th Edition >. First, let's see some quotation.
LPVOID WINAPI VirtualAlloc(
__in_opt LPVOID lpAddress,
__in SIZE_T dwSize,
__in DWORD fdwAllocationType,
__in DWORD fdwProtect
);
The last parameter, fdwProtect,
indicates the protection attribute
that should be assigned to the region.
The protection attribute associated with the region has no effect on the
committed storage mapped to the
region.
When reserving a region, assign the protection attribute that will be used
most often with the storage committed
to the region. For example, if you
intend to commit physical storage with
a protection attribute of
PAGE_READWRITE, you should reserve the
region with PAGE_READWRITE. The
system's internal record keeping
behaves more efficiently when the
region's protection attribute matches
the committed storage's protection
attribute.
(When commiting storage)...you usually
pass the same page protection
attribute that was used when
VirtualAlloc was called to reserve the
region, although you can specify a
different protection attribute.
The above quotation totally puzzled me.
If the protection attribute associated with the region has no effect on the committed storage, why do we need it?
Since it is recommended to use the same protection attribute for both reserving and committing, why does Windows still offer us the option to use different attribute? Isn't it mis-leading and kind of a paradox?
Where exactly is the protection attribute stored for reserved region and committed storage, repectively?
Many thanks for your insights.
It's important to read it in context.
The protection attribute associated
with the region has no effect on the
committed storage mapped to the
region.
was referring to reserving, not committing regions.
A reserved page has no backing store, so it's protection is always conceptually PAGE_NOACCESS, regardless of what you pass to VirtualAlloc. I.e. if a thread attempts to read/write to an address in a reserved region, an access violation is raised.
From linked article:
Reserved addresses are always
PAGE_NOACCESS, a default enforced by
the system no matter what value is
passed to the function. Committed
pages can be either read-only,
read-write, or no-access.
Re:
Where exactly is the protection
attribute stored for reserved region
and committed storage, repectively?
The protection attributes for virtual address regions are stored in the VAD tree, per process. (VAD == Virtual Address Descriptor, see Windows Internals, or linked article)
Since it is recommended to use the same protection attribute for both reserving and committing, why does Windows still offer us the option to use different attribute? Isn't it mis-leading and kind of a paradox?
Because the function always accepts a protection parameter, but its behaviour depends on fdwAllocationType. Protection only makes sense for committed storage.
The reason Richter suggests using the same protection setting is presumably because fewer changes in the protection flags in a region mean fewer "blocks" (see your book for definition), and hence a smaller AVL tree for the VADs. I.e. if all pages in a region are committed with the same flags, there'll only be 1 block. Otherwise there could be as many blocks as pages in the region. And you need a VAD for each block (not page).
Block == set of consecutive pages with identical protection/state.
If the protection attribute associated
with the region has no effect on the
committed storage, why do we need it?
As above.
Well... One reason could be so you could use guard pages so you can commit memory as you use it.
Think of the thread stack in Windows; the page immediately below the stack is set as a guard page, typically with read and write ability. Once the guard page is touched, an exception handler runs and commits the guard page and makes the next page a guard.
See here for a better description. Also, that link is part of a series on how windows handles low level resources and is pretty good reading.
Another reason for allowing you to respecify the protection attributes could be for copy on write techniques. Pages are set to read only until they're changed, which can raise an exception you can handle etc etc etc.
On the 386 family of Intel chips, the commit, read/write/reserve flags are stored in the page tables. Take a look at a 386 chip reference for more details.
Edit: I poked around for a bit and could not find where MS stores the PAGE_GUARD bit. Now I'm curious where I saw it. :) Too bad I threw out about 500 pounds of old reference material last spring...
Hope this helps :)
Related
I'm investigating a set of Windows API system calls made by a piece of malware running in a sandbox so that I can understand its malicious intent. Unfortunately, I'm struggling to understand the ZwMapViewOfSection function described in documentation: https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/content/wdm/nf-wdm-zwmapviewofsection
Now, I do understand that this function is related to the mapping of physical memory to virtual memory in a page table. Apart from that, I find the documentation arcane and not friendly to beginners. I am also confused why they are calling blocks of physical memory "sections" rather than "frames" (if that is what they are indeed referring to -- its not clear to me). Can anyone provide a more intuitive explanation about this system call and what it does in general? Is this a common system call for programs or is it limited to malware? Thank You.
It is extremely common for normal programs to make this call (not directly of course), every program is going to invoke it multiple times during initialization at the very least (ZwMapViewOfSection is used when performing the memory backing used to implement the executable sections of code itself). Not so common during normal program code but not uncommon either. Particularly common if the program performs dynamic DLL loads but legitimate programs can also do memory-mapped IO for their own reasons.
It operates on memory section objects (I've never really understood that name either) which are one part of the link between disc files and memory-mapped regions, the section is created via ZwCreateSection or opened with ZwOpenSection and then the other part comes into play with ZwMapViewOfSection.
What part of this, exactly, is confusing you? Knowing that would make it far easier to provide an informative response.
As far as I understand it, you have to open the file and acquire a file handle which you then map with CreateFileMapping, which will call NtCreateSection, which calls MmCreateSection. If the file is mapped for the first time a new segment object and control area are created first then depending on whether the section is created for a data, image or page-file backed MiCreateDataFileMap, MiCreateImageFileMap or MiCreatePagingFileMap is called.
MiCreateDataFileMap sets up the subsection object and section object. In the normal case only one subsection is created, but under some special conditions multiple subsections are used, e.g. if the file is very large. For data files, the subsection object field SubsectionBase is left blank. Instead the SegmentPteTemplate field of the segment object is setup properly which can be used to create the PPTEs when necessary. This defers the creation of PPTEs until a view is mapped for the first time which avoids wasting memory when very large data files are mapped. Note a PPTE is a PTE that is serving as a prototype PTE, but an _MMPTE_PROTOTYPE is a PTE that is pointing to a prototype.
MiCreateImageFileMap creates the section object and loads the PE header of the specified file and verifies it then one subsection is created for the PE header and one for each PE section. If a very small image file is mapped then only one subsection is created for the complete file. Besides the subsections also the related PPTEs for each of them are created and their page protection flags are set according to the protection settings of the related PE section. These PPTEs will be used as a template for building the real PTEs when a view is mapped and accessed.
After a section is created it can be mapped into the address space by creating a view from it. The flProtect passed to CreateFileMapping specifies the protection of the section object. All mapped views of the object must be compatible with this protection. You specify dwMaximumSizeLow and dwMaximumSizeHigh to be 0 in order for dwMaximumSizeHigh to be set to the length of the file automatically.
You then pass the returned section object handle to MapViewOfFile, which will calls NtMapViewOfSection on it, which calls MmMapViewOfSegment, which calls MmCreateMemoryArea, which is where the view is mapped into the VAD of the process with the protection dwDesiredAccess supplied to MapViewOfFile, which serves as the protection type for all PTEs that the VAD entry covers. dwNumberOfBytesToMap = 0 and dwFileOffsetLow = 0 in MapViewOfFile maps the whole file.
When a view is mapped, I believe that all of the PTEs are made to point to the prototype PTEs and are given the protection of the PPTE. For an image file, the PPTEs have already been initialised to subsection PTEs. For a data file, the PPTEs for the view need to be initialised to subsection PTEs. The VAD entry for the view is now created. The VAD entry protection isn't always reflective of the protection of the PTEs it covers, because it can cover multiple subsections and multiple blocks within those subsections.
The first time an address in the mapping is actually accessed, the subsection prototype PTE is filled in on demand with the allocated physical page filled with the I/O write for that range and the process PTE is filled in with that same address. For an image, the PPTE was already filled in when the subsections were created along with protection information derived from the section header characteristics in the image, and it just fills in the PTE with that address and the protection information in it.
When the PTE is trimmed from the process working set, the working set manager accesses the PFN to locate the PPTE address, decreases the share count, and it inserts the PPTE address into the PTE.
I'm not sure when a VAD PTE (which have a prototype bit and prototype address of 0xFFFFFFFF0000 and is not valid) occurs. I would have thought the PPTEs are always there at their virtual address and can be pointed to as soon as the VAD entry is created.
I have a process with some sensitive memory which must never be written to disk.
I also have a requirement that I need core dumps to satisfy first-time data capture requirements of my client.
Does locking a page using mlock() prevent the page from appearing in a core dump?
Note, this is an embedded system and we don't actually have any swap space.
Taken from man 2 madvise:
The madvise() system call advises the kernel about how to handle
paging input/output in the address range beginning at address addr and
with size length bytes. It allows an application to tell the kernel
how it expects to use some mapped or shared memory areas, so that the
kernel can choose appropriate read-ahead and caching techniques.
This call does not influence the semantics of the application (except
in the case of MADV_DONTNEED), but may influence its performance. The
kernel is free to ignore the advice.
Particularly check the option MADV_DONTDUMP :
Exclude from a core dump those pages in the range specified by addr
and length. This is useful in applications that have large areas of
memory that are known not to be useful in a core dump. The effect of
MADV_DONTDUMP takes precedence over the bit mask that is set via the
/proc/PID/coredump_filter file (see core(5)).
I am writing a kernel module that has access to a particular process's memory. I have done an anonymous mapping on some of the user space memory with do_mmap():
#define MAP_FLAGS (MAP_PRIVATE | MAP_FIXED | MAP_ANONYMOUS)
prot = PROT_WRITE;
retval = do_mmap(NULL, vaddr, vsize, prot, MAP_FLAGS, 0);
vaddr and vsize are set earlier, and the call succeeds. After I write to that memory block from the kernel module (via copy_to_user), I want to remove the PROT_WRITE permission on it (like I would with mprotect in normal user space). I can't seem to find a function that will allow this.
I attempted unmapping the region and remapping it with the correct protections, but that zeroes out the memory block, erasing all the data I just wrote; setting MAP_UNINITIALIZED might fix that, but, from the man pages:
MAP_UNINITIALIZED (since Linux 2.6.33)
Don't clear anonymous pages. This flag is intended to improve performance on embedded
devices. This flag is only honored if the kernel was configured with the
CONFIG_MMAP_ALLOW_UNINITIALIZED option. Because of the security implications, that option
is normally enabled only on embedded devices (i.e., devices where one has complete
control of the contents of user memory).
so, while that might do what I want, it wouldn't be very portable. Is there a standard way to accomplish what I've suggested?
After some more research, I found a function called get_user_pages() (best documentation I've found is here) that returns a list of pages from userspace at a given address that can be mapped to kernel space with kmap() and written to that way (in my case, using kernel_read()). This can be used as a replacement for copy_to_user() because it allows forcing write permissions on the pages retrieved. The only drawback is that you have to write page by page, instead of all in one go, but it does solve the problem I described in my question.
In userspace there is a system call mprotect that can modify the protection flags on existing mapping. You probably need to follow from the implementation of that system call, or maybe simply call it directly from your code. See mm/protect.c.
I am trying to learn how to access other process's memory. I'm using ReadProcessMemory but it fails, supposedly because i'm reading from an unaccessible memory. I think I should use VirtualQueryEx first to find out if the memory I'm trying to read is readable, but I don't know how to parse the results of VirtualQueryEx.
VirtualQueryEx returns a structure described here http://msdn.microsoft.com/en-us/library/windows/desktop/aa366775%28v=vs.85%29.aspx
Is there any good simple way to know whether the values in the MEMORY_BASIC_INFORMATION structure tell that the page is readable by ReadProcessMemory or not?
This structure has several parameters that, from what I understand, define whether or not memory is accessible:
DWORD AllocationProtect;
DWORD State;
DWORD Protect;
DWORD Type;
Its not even clear on what combination of parameters actually allows reading the memory (for example, should I read Protection flag from AllocationProtect or Protect, does State other than Commit mean that memory is unreadable, etc) and making a huge if (this is that and that is that or this is that....) doesn't seem like a clean solution.
There's got to be a better way, but I don't seem to be able to find it...
Actually the clue is in the documentation you linked:
AllocationProtect
The memory protection option when the region was initially allocated. This member can be one
of the memory protection constants or 0 if the caller does not have access.
VirtualProtect or VirtualProtectEx could have been used meanwhile to change the protection (per-page, while the initial protection was for the whole allocated range). So you'd have to check the Protect member of the structure against the Memory Protection Constants in the MSDN documentation that state that read is allowed. Checking for a single bit doesn't seem to be enough.
I have some code (which I cannot change) that I need to get working in a native Win32 environment. This code calls mmap() and munmap(), so I have created those functions using CreateFileMapping(), MapViewOfFile(), etc., to accomplish the same thing. Initially this works fine, and the code is able to access files as expected. Unfortunately the code goes on to munmap() selected parts of the file that it no longer needs.
x = mmap(0, size, PROT_READ, MAP_SHARED, fd, 0);
...
munmap(x, hdr_size);
munmap(x + foo, bar);
...
Unfortunately, when you pass a pointer into the middle of the mapped range to UnmapViewOfFile() it destroys the entire mapping. Even worse, I can't see how I would be able to detect that this is a partial un-map request and just ignore it.
I have tried calling VirtualFree() on the range but, unsurprisingly, this produces ERROR_INVALID_PARAMETER.
I'm beginning to think that I will have to use static/global variables to track all the open memory mappings so that I can detect and ignore partial unmappings, but I hope you have a better idea...
edit:
Since I wasn't explicit enough above: the docs for UnMapViewOfFile do not accurately reflect the behavior of that function.
Un-mapping the whole view and remapping pieces is not a good solution because you can only suggest a base address for a new mapping, you can't really control it. The semantics of munmap() don't allow for a change to the base address of the still-mapped portion.
What I really need is a way to find the base address and size of a already-mapped memory area.
edit2: Now that I restate the problem that way, it looks like the VirtualQuery() function will suffice.
It is quite explicit in the MSDN Library docs for UnmapViewOfFile:
lpBaseAddress A pointer to the
base address of the mapped view of a
file that is to be unmapped. This
value must be identical to the value
returned by a previous call to the
MapViewOfFile or MapViewOfFileEx
function.
You changing the mapping by unmapping the old one and creating a new one. Unmapping bits and pieces isn't well supported, nor would it have any useful side-effects from a memory management point of view. You don't want to risk getting the address space fragmented.
You'll have to do this differently.
You could keep track each mapping and how many pages of it are still allocated by the client and only free the mapping when that counter reaches zero. The middle sections would still be mapped, but it wouldn't matter since the client wouldn't be accessing that memory anyway.
Create a global dictionary of memory mappings through this interface. When a mapping request comes through, record the address, size and number of pages that are in the range. When a unmap request is made, find out which mapping owns that address and decrease the page count by the number of pages that are being freed. When that count reaches zero, really unmap the view.