I'm trying to map a virtual memory page using VirtualAlloc, but it's an specific address outside 8TB Windows 7 64 bits allowed range, i've read at Windows Internals book this address range is used
for "page table self mapping structures", anyway,is there any way i could map a memory page with the 0x7ffffffff000 address?
Related
How can I get the physical address of page table itself?
I tried to use __pa(), virt_to_phys() but it seems to translate to another virtual address.
For example, the address of the page table of the first page is:
0xffffffbf00000000 --> 0x0000003f78800000.
The system has only 5G physical memory.
So, the address must be another virtual address.
How can I get its actual physical address?
I'm writing a memory manager for my kernel (32 bit x86)
In the course of this... I'm facing a bit of a dilemma....
Description of virtual memory map:
Identity map of first 4 Mb
Virtual address 0xC0000000 mapped to physical address 0x100000 (Also a 4 Mb map)
My page directory is at physical address 0x9c000.
My page table 1 is at physical address 0x9d000.
My page table 2 is at physical addres 0x9e000.
(I need only two page tables here :) ... These correspond to the identity map and higher memory map respectively)
Bless the identity mapping.... I can safely access my page directory and page tables as if paging wasn't even enabled. This makes it really easy for me to modify page tables,etc.
Now comes the issue: I may remove this identity mapping... If so, I can already imagine problems creeping up..
Eg. I have physical addresses that I want to access... But I can only access virtual ones. In order to map the virtual address to the required physical address, I need to access the page directory. But I have the physical address of the page directory... *I realize that I'm back where I started.
So, I'm guessing there's a need for some permanent mapping (or some sort of identity mapping for tables and the directory) so that I can forget about all this and get on with my life.
But if I map something permanently, I feel that I'm reducing the flexibilty of the program(kernel) in some sort of way.
What's the way one deals with this issue?
What happens when you lose the virtual address of the page directory? You can always get the physical address from cr3, but you have no idea where it's mapped, how to access it, and whatnot. In this case, I don't think one can even change the page directory location using cr3 because you'd be loading a physical address into it, but all that you can view are virtual addresses... It seems like a really scary situation here
Am I missing something?
Don't lose your virtual Page directory address! You need at least a small portion of the page directory at a know physical and a known virtual address (they do not need identity mapped).
Next, the loader creates a basic page table. This page table maps the 64 MB at the base of virtual memory (starting at virtual address
0) directly to the identical physical addresses. It also maps the same
physical memory starting at virtual address LOADER_PHYS_BASE, which
defaults to 0xc0000000 (3 GB). The Pintos kernel only wants the latter
mapping, but there's a chicken-and-egg problem if we don't include the
former: our current virtual address is roughly 0x20000, the location
where the loader put us, and we can't jump to 0xc0020000 until we turn
on the page table, but if we turn on the page table without jumping
there, then we've just pulled the rug out from under ourselves.
A citation comes from: https://web.stanford.edu/~ouster/cgi-bin/cs140-winter16/pintos/pintos_6.html
I think that I don't understand exactly what does it is written above.
My issues:
Let's assume that kernel was loaded at 0x20000 physical address by bootloader. From what I understand two different virtual address are mapped to the same place- to the kernel. The first, direct mapping: ~0x20000 -> ~0x20000. And the second ~0xc0000000 -> ~0x20000. But, why there is need to use two mappings?
I cannot see why the page table ( and page directory) must be mapped by identity.
Please explain.
My understanding in address translation process in MMU(memory management unit)
-> logical address : generated by cpu.programmer concern with this address.
-> virtual address : reside in the hard disk , as a pages.
-> physical address : reside in the RAM. It is the actual address.
1: cpu generate the logical address and send it to the MMU.
2: MMU translate the logical address into the virtual address then translate it to the physical address and send the physical address to RAM.
3: when ever the RAM is full , the page which is not used rapidly is returned to the hard disk , to allocate memory to the other pages(processes).
my questions are :
1) where the value of Relocation register is added?
2) who decide the value of Relocation Register?
3) what to do with the Base register and Limit register , how to use it?
4) where the logical address goes off?
If any body can answer it , It would be grateful to me.
It is requested that , let me know it any misunderstanding in this topic.
-thanks
I can tell you how this works on x86.
All programs in non-64-bit modes operate with addresses combined of two items: segment selector (for brevity "selector" is often omitted in text and that may be confusing) and offset. This selector:offset pair is called the logical address.
The selector portion isn't always explicitly specified or manipulated with in code since the CPU has "default" associations of segment registers containing selectors with specific instructions or specific instruction encodings. It's also uncommon to manipulate selectors in 32-bit mode, but is very often necessary in 16-bit code.
The virtual address is formed from the logical address either "directly" (in real or 8086 virtual mode) or "indirectly" (in protected mode).
"Direct" virtual address = selector * 16 + offset.
"Indirect" virtual address = SegmentDescriptorTable[selector].Base + offset.
SegmentDescriptorTable is either the Global Descriptor Table (AKA GDT) or the Local Descriptor Table (AKA LDT). It's set up by the OS and describes the location and size of various segments of memory. selector is used to select a segment in the table. The Base entry of the table tells the segment's beginning (virtual address). The Limit entry tells the segment size (generally; the details are a little more complex).
When a program tries to access memory with an offset resulting access beyond the end of the segment (the CPU compares offset and Limit), the CPU generates an exception and the OS handles it, by usually terminating the program.
Btw, in real/v86 mode, even though the virtual address is formed directly from selector:offset, there's still a 16-bit Limit imposed on offsets, which is why you need to use a different selector to access more than 64KB of memory.
The Base entry in a segment descriptor can be used to either isolate the segment from the rest of the memory (Limit helps here) or to place or move the entire segment to an arbitrary virtual address without having to modify anything (or much) in the program it belongs to (if we're moving a segment, the data has to be moved in the memory, obviously). Basically, it can be used for relocation purposes. In real/v86 mode for relocation purposes the selector is changed.
The virtual address can be further translated to the physical address if the CPU is running in protected mode and has set up page tables. If there're no page tables, the physical address is the same as the virtual address. The translation is done in blocks of physical memory and address ranges that are called pages (often 4KB).
There's no dedicated relocation register on x86 CPUs. Relocation can be achieved by adjusting:
segment selectors in CPU registers or program's code
segment base addresses in GDT/LDT
offsets in program's code
physical addresses in page tables
As for virtual address : reside in the hard disk , as a pages, I'm not sure what exactly you want to say with this, but just because there's virtual to physical address translation, it doesn't mean there's also virtual on-disk memory. There are other uses for the translation besides virtual on-disk memory. And the addresses reside in the CPU and wherever your (and OS's) code writes them to, not necessarily on the disk.
Your description has a number of mistakes, much of which may be the result of imprecise documentation and common usage.
First of all, there really is no such a thing as a virtual address. There are physical and logical addresses. Sadly, the term virtual address is frequently (even in hardware documentation) used when logical address is what is meant..
The CPU instruction stream always operates on logical addresses (values may refer to physical addresses).
When the CPU needs to access a logical address, the MMU attempts to translate it to a physical addresses. It does that by looking up the address in a page table.
Several things can happen at that point:
There may not be a page table entry for the address => Access violation.
The page table entry is marked invalid => Access violation.
The page table entry indicates that no physical memory is mapped to it => Page fault.
(I omit mode access checks).
It is this last step that last step where virtual memory comes into play. At that point the page fault handler of the operating system needs to find where the corresponding page has been stored to disk, load it, update the page table, and restart the instruction.
The operating system manages the available physical memory by paging writeable memory (that has changed) to disk (read only data does not have to be written back) when there is high demand for physical memory.
I have never heard of a "relocation register" before. But doing a GOOGLE search I can see that some academic material uses it as a confusing pedagogical concept (i.e., with no relation to reality).
Some systems define the page table using base and limit registers. The base registers indicate where the page table starts in memory (this can be either a physical or logical addresses) and the limit register indicates the side of the table.
The registers are usually not loaded directly. Their values are usually written to the hardware Process Context Block (PCB). When the process context is loaded, the page table base and limit are loaded automatically.
On some systems there are multiple page tables. If there are system and user page tables, the user page tables can refer to logical addresses in the system space and the system page tables refer to physical addresses.
I'm just asking this question because I'm curious how the Linux kernel works. According to http://i-web.i.u-tokyo.ac.jp/edu/training/ss/lecture/new-documents/Lectures/02-VirtualMemory/VirtualMemory.ppt Windows uses special entries in its page directory and page tables named self-map in order to be able to manipulate page directory/tables content from kernel virtual address space. If anyone is familiar with Linux memory management, please tell me if Linux kernel handle this problem in a similar or different way. Thanks.
Yes, in Linux also page tables are mapped to address space. But paging data structures in some of the architectures may use physical addresses. So it not fixed in Linux. But you can access the table easily.
Here is the kernel code to access the page table
struct mm_struct *mm = current->mm;
pgd = pgd_offset(mm, address);
pmd = pmd_offset(pgd, address);
pte = *pte_offset_map(pmd, address);
To understand more about Linux memory management see this
Cr3 register on IA32 stores the page table base pointer (pgd pointer), which stores physical address. This is true even for Windows (as it is a feature of the x86 processor, not of the OS).
Read this article to understand IA32 paging.
Edit2:
Task struct contains a mm_struct instance related to Memory management of that task (so a process), this mm_struct has a pgd_t * pgd. load_cr3 loads a physical address of page directory table in cr3 register but it takes the virtual address of pgt. So mm_struct contains the virtual address of pgt.
Since page tables are in kernel space and kernel virtual memory is mapped directly to ram it's just easy macro.