I have the following scenario:
qemu-kvm(guest) ---has virtual memory, get physical--->
virtio (send physical address to host) ---map physical to host virtual memory--->
host
The physical memory is preallocated. Is there a method to preallocate non mapped virtual memory on the host so that it won't have to search for free virtual address spaces?
Would this be a justifiable design concern if the buffers are pretty big?
The end result that I want is a pool of virtual address spaces to map received buffers.
After every job I want to unmap them and send them back the virtual address space to the pool.
To reserve a virtual memory range without actually committing any physical pages to it, pass PROT_NONE as protection parameter to mmap(). Later on, you can use mprotect() on that range to make it readable/writable when necessary - the kernel will commit the physical pages on first access. When you're done, you can use mprotect() again to reset the protection status of the address range back to PROT_NONE.
Related
For example, if a process on the guest has data allocated on 0x8000(as in Guest Virtual Memory), and that has been mapped into 0x4000 in Guest Physical Memory Address Space, is the data located on 0x4000 in the virtual memory address space of host-side QEMU-KVM's (sub)process(per VM)? In other words, if I write new code within QEMU's source(so I can use the QEMU-KVM's page table), compile, and run it, then can I access to the data of guest process directly just with the guest's physical memory address matching to the guest virtual memory address?
No, the page table mapping used for the user-space QEMU process is unrelated to either the page tables used by the guest itself for guest virtual -> guest physical address mapping, or to the page tables used by the KVM kernel code for guest physical address -> host physical address mapping for RAM.
When you're writing code in QEMU that needs to access guest memory, you should do so using the APIs that QEMU provides for that, which deal with converting the guest address to a host virtual address for RAM and also with handling the case when that guest address has an emulated device rather than RAM. The QEMU developer internals docs have a section on APIs for loads and stores, and the functions are also documented in doc comments in the header files.
Usually the best advice is "find some existing code in QEMU which is doing something basically the same as what you're trying to do, and follow that as an example".
I'm assuming your question is about x86 platform.
QEMU+KVM use extended page table (EPT) to map host virtual addresses (HVA) to guest physical addresses (GPA). To find HVA corresponding to GPA, you should traverse EPT of your guest.
When you need to read a virtual address from your guest, you can use GDB functions in QEMU source. This snippet will read 4 bytes at virtualAddress in the current executing process in your guest:
uint8_t instr[4];
if( cpu_memory_rw_debug( cpu, virtualAddress, (uint8_t *)instr, sizeof( instr ), 0 ) )
{
printf( "Failed to read instr!\n" );
return;
}
cpu is an CPUState* of a cpu of your guest.
When you want to read an address of a concrete process in your guest, you should set env->cr[3] to CR3 value of this process (env is an CPUX86State*). Dont forget to restore the original value after you finish reading. And of course read memory only when your guest is not executing, otherwise there can be races.
as I read here the virtual address space of a 32 bit Windows application has 2GB of storage (from 0x00000000-0x7FFFFFFF). The other 2GB are reserved for the system address space.
However, I found a pointer in a 32bit program (using Cheat Engine) which is pointing to an address which isn't in range of the virutal address space. The addresses in my last exploration were 0x301DDC3C -> 0x87F56190 like you can see in the picture:
(The expansion in the first line means a dereference of the pointer 0x301DDC3C, in the next line you can see what's in the dereference location 0x87F56190 in RAM)
After dereferencing the pointer there are pointers back into the process virtual address space.
How is it possible that a user mode application has a valid pointer into system address space?
Does this mean the pointer in location 0x301DDC3C is pointing to an location in the system address space? And so the process I'm examining is using kernel mode stuff?
from Memory and Address Space Limits
Limits on memory and address space vary by platform, operating system, and by whether the IMAGE_FILE_LARGE_ADDRESS_AWARE flag in the IMAGE_FILE_HEADER.Characteristics. IMAGE_FILE_LARGE_ADDRESS_AWARE (The application can handle addresses larger than 2 GB) is set or cleared by using the /LARGEADDRESSAWARE linker option.
by default IMAGE_FILE_LARGE_ADDRESS_AWARE cleared for 32-bit PE and set for 64-bit PE, but we can overwrite default:
so 32-bit process with set IMAGE_FILE_LARGE_ADDRESS_AWARE flag - up to 4Gb memory is avaible.
really of course [0, 0x800000000000) (win8.1 +) or [0, 0x80000000000) (before win 8.1) memory space is avaible for user mode in x64 windows. but system artificially restrict this by reserve big range of memory (this allocation is protected and can not be free)
for 32-bit process this reservation begin from 7FFF0000 or FFFE0000 and up to 64-bit ntdll.dll. very interesting that in 64-bit process, where IMAGE_FILE_LARGE_ADDRESS_AWARE cleared - also was such reserved memory space begin from 0x80000000. also interesting that in this case kernel32.dll is loaded at another address compare usual 64-bit process. so base of kernel32.dll not the same in general in all 64-bit processes. but ntdll.dll loaded at the same address in all processes anyway.
usual memory allocations on x64 windows:
32 bit process, IMAGE_FILE_LARGE_ADDRESS_AWARE cleared (default)
32 bit process, IMAGE_FILE_LARGE_ADDRESS_AWARE set
64 bit process, IMAGE_FILE_LARGE_ADDRESS_AWARE cleared
64 bit process, IMAGE_FILE_LARGE_ADDRESS_AWARE set (default)
ALL of the addresses you see are virtual addresses, of the process (not "physical" addresses). A user-space process may use pointers that happen to come from "system space", but that does NOT mean a process can freely access kernel resources , nor does it mean that these pointers necessarily map to physical addresses.
Here is another Microsoft link, that might help clarify:
Virtual Address Space
When a processor reads or writes to a memory location, it uses a
virtual address. As part of the read or write operation, the processor
translates the virtual address to a physical address.
...
The range of
virtual addresses that is available to a process is called the virtual
address space for the process. Each user-mode process has its own
private virtual address space. For a 32-bit process, the virtual
address space is usually the 2-gigabyte range 0x00000000 through
0x7FFFFFFF.
...
Processes like Notepad.exe and MyApp.exe run in user
mode. Core operating system components and many drivers run in the
more privileged kernel mode. For more information about processor
modes, see User mode and kernel mode. Each user-mode process has its
own private virtual address space, but all code that runs in kernel
mode shares a single virtual address space called system space. The
virtual address space for a user-mode process is called user space.
...
In 32-bit Windows, the total available virtual address space is
2^32 bytes (4 gigabytes). Usually the lower 2 gigabytes are used for
user space, and the upper 2 gigabytes are used for system space.
...
Code running in user mode has access to user space but does not have
access to system space. This restriction prevents user-mode code from
reading or altering protected operating system data structures. Code
running in kernel mode has access to both user space and system space.
That is, code running in kernel mode has access to system space and
the virtual address space of the current user-mode process.
...
It's also worthwhile to note the difference betwee kernel mode and user mode:
User mode and kernel mode
When you start a user-mode application, Windows creates a process for
the application. The process provides the application with a private
virtual address space and a private handle table. Because an
application's virtual address space is private, one application cannot
alter data that belongs to another application. Each application runs
in isolation, and if an application crashes, the crash is limited to
that one application. Other applications and the operating system are
not affected by the crash.
...
In addition to being private, the virtual address space of a user-mode application is limited. A processor running in user mode
cannot access virtual addresses that are reserved for the operating
system. Limiting the virtual address space of a user-mode application
prevents the application from altering, and possibly damaging,
critical operating system data.
...
I know how vmalloc() does。 When a process(in kernel space) want to access the memory that belongs to vmalloc()，a page fault happens and does the synchronization。
But when it invokes the vfree()， how the process update its page table to sync with the master kernel page table？ Or I have some understandings with it.
Thanks.
your understanding of memory allocation is seems to be incorrect. No memory belongs to vmalloc. A fixed virtual address (of kernel space) is assigned to vmalloc at boot up time. Later when vmalloc is called then virtual addresses are picked from fix allocated range and physical memory pages are allocated from buddy system.
Virtual addresses and physical pages are mapped one to one.
when vfree() is called, the virtual address range is freed again , so does the physical pages are return to buddy system.
Hope this correct your understanding.
I suggest you go through some online tutorial about kernel memory as also reading them now.
Is device address is virtual address? what is functionality of mmap in this case?or device address mapped to physical address
Usally, device address are allocated by specific system/host bus. It can identify devices on the bus.
Virtual address and physical address are used in memory system.
For mmap, the system allocate an I/O address for specific device in physical address space, application can access device in the way of memory access.
Usually devices comes with resources like registers, internal memory, etc that can be accessed from the CPU.
In order to be able to access a specific device register for example from the CPU you need to know the physical address of this device registers and then map this physical address to either kernel or user depending on your use case.
mmap maps resources to be accessed from user space. The result of mmap is a user space cpu address that is mapped to this resource.
This resource can be anything. It can be:
a file
anonymous memory
some external device resource ( memory, registers, etc )
mmap can't directly map device registers for example simply because it doesn't know how to do that. In this case you will probably need add some kernel space support for your mmap operation.
If there is a 32 bit system (assume Windows), the virtual address space is 4GB. So CPu can generate any address between this range. Then shoudn't a process also be able to address anywhere in this range?
It is said that each process has its own private virtual address space.Then How does the system facilitate this?
In other words the CPU generates a 32 bit address, and that gets translated into physical address. Now how does CPU know that a specific process has to address only a specific part of the virtual address space(its private virtual address space).
Suppose a process addresses an address out of its private virtual address space, what happens?
A program has to call VirtualAlloc() on Windows to tell the operating system that it wants to use a chunk of virtual memory. Often called indirectly as a result of allocating memory from a heap or loading a DLL.
The operating system, in turn, sets up the page mapping tables that the CPU uses to translate a virtual address as used in the program to a physical RAM address as output on its address bus pins. One of three unusual things can happen whenever the CPU reads or writes data or executes code at a virtual memory address:
if there is no entry in the page mapping tables then the CPU raises a general protection fault trap. The operating system verifies that the address is invalid and terminates the program
if the page is not mapped to RAM yet then the CPU raises a page fault trap. The operating system finds a page of RAM that's unused, swapping out a used page if necessary. And ensures the content is valid, loading it from a file or the paging file if necessary. And updates the table entry so it now has the physical address of the RAM page. Execution resumes as normal
the CPU verifies that access to the page is allowed. A write to a page that is marked as read-only or an execute of a instruction in the page that's marked as no-execute generates a general protection fault trap. The operating system terminates the program.
Every process has its own set of page mapping tables, ensuring that one process cannot access the RAM pages that are used by another. Unless sharing is specifically requested, common for pages of code loaded from an executable file and memory mapped files. A context switch loads the CR2 register, the CPU register that contains the address of the page mapping table.
So there is no scenario where a process can ever address memory outside of its private virtual address space, the lack of a matching paging table entry ensures that this terminates the program.
The whole 4 GB address space is available to the process (although typically the upper half is reserved for kernel data), and the MMU maps parts of it to physical memory. The process cannot go "out" of its address space (all the 4 GB of it are allowed to be used), but if some part of it hasn't been mapped to physical memory a hardware exception is raised.
The address space is said to be private since the operating system changes the settings of the MMU at task switch, so every process sees a different independent memory layout (although parts of the address space can be shared with other processes).