I was learning Linux memory management recently, now I am stopped by the paging mechanism.
As with Regular Paging for 32-bit processors, why page directory entries (32 bits in total) need 20 bits to indicate 2^10 Page Tables? I think 10-bits is just enough and no waste.
What is wrong with my understanding?
Thank you.
A page has a size of 4096 bytes, i.e., 2^12 bytes.
This means that pages are aligned to a multiple of 4 KB, and that the address of a page is xxxxxxxxxxxxxxxxxxxx000000000000.
In other words, a page address needs 12 bits less than the address bus size.
For 32-bit addresses, this ends up being 20 bits.
A page directory entry has 32 bits, so 2^10 of them fit into a 4 KB page.
Regular x86 uses 2 level pagetable, but i think the case is that they talk here about one-level page table ... So you have one huge structure with 2^20 entries, each entry associate virtual page address (mentioned 20 bits) with physical page address. Can you provide a link where have you found this picture?
For a 32-bit processor, it will generate 32-bit address.
If an address generated is 32-bit then, addressable memory is 4GB.(As 2^32 = 4GB)
Now, both page table and page directory reside on memory within a single page.
Also, page size is 4kB.
And in page directory and page table, entries always point to border or edge of page table and page directory respectively.
If you divide 4G(1G=2^30) by 4k(1k = 2^10), you'll get 2^20
That is we need 20bits to access all of 4kb chunks within 4GB memory or maximum addressable memory.
That is why, entries for page table and page directory are always 20-bits
Related
Im having trouble understanding the process of finding the virtual page offset, the value of the physical page offset, and the virtual page number
Ive tried looking at various places but I cannot find any examples or formulas for finding the following parameters. Below is an example of a problem, but I have left out the virtual address from the actual problem.
Given a virtual memory system with the following specification:
The memory is byte addressable.
Each memory access is to 1-byte word.
Each virtual address is 16-bit wide.
Each physical address is 12-bit wide.
The page size is 32 bytes (25)
The TLB is 4-way set associative with 16 total entries.
The L1 data-cache is physically addressed and direct mapped, with a 4 byte line size and 16 total sets.
If the CPU issues the following virtual address: (some hex number)
What is the virtual page offset?
What is the value of the physical page offset?
What is the value of the TLB tag?
What is the value of the TLB index?
What is the value of the virtual page number?
(Answers in binary)
After turning the hex number to binary, I am lost. How can I find the virtual page offset? Whats the difference between the virtual page offset and the physical page, and how do I find the value of the physical page?
I think the TLB tag is the remaining digits after the VPO and PPO, but I would need to know how to get the VPO and the PPO first. I am unsure what the TLB index is. The VPN is a combination of some of these answers but I am not sure which.
What happens if the set associative is changed to a 2 way TLB? does that change the way I find these values?
Finally, If I were given a 2-way set associative TLB and a page table with the same information above, how can I determine if there is a page fault or not? From what I understand, if there is a TLB hit there is no page fault, but if not, how can I determine this? He also talks about a physical address, what is the process of finding this?
What is the virtual page offset?
What is the value of the physical page offset?
VPO are addresses of a byte within a page. Concerning physical pages, they are generally called frames, but the offset definition is identical.
VPO = VAdr % PageSize
What is the value of the TLB tag?
What is the value of the TLB index?
TLB index maps a VAdr to a TLB set.
Number-of-sets = number-of-TLB-entries / number-of-ways
and ìndex-size =log_2(number-of-sets)
TLB tag is the remaining MS bits in virtual address
What is the value of the virtual page number?
Virtual-page-number = virtual-adresse / page-size
If address is in binary abdefghijklmnopq its interpretation as (TLBTag,TLBidx,VPO) is (abdefghij,kl,mnopq) and virtual page address is abdefghijkl.
What happens if the set associative is changed to a 2 way TLB? does that change the way I find these values?
If the number of entries is unchanged, this divides the number of sets by 2 and adds one bit to TLBIdx (and removes one from TLBTag).
Finally, If I were given a 2-way set associative TLB and a page table with the same information above, how can I determine if there is a page fault or not? From what I understand, if there is a TLB hit there is no page fault, but if not, how can I determine this?
If there is a TLB hit, there is no page fault. If there is a TLB miss, page address has to be generated by looking at process page tables. These tables indicates correspondence between virtual and logical page addresses and if a page is in memory or swapped to disk. If the page is swapped to disk, this is a page fault. This generates an exception and the OS has to write the page back to memory.
He also talks about a physical address, what is the process of finding this?
For simple situations like yours, it can just be a table lookup. A page address is 11 bits wide and a 2048 entry table indicates the address of the corresponding physical frame. In real systems, this is more complex and generally involve several cascading tables.
What is the virtual page offset?
Address MOD Page Size
What is the value of the physical page offset?
Address / Page Size
What is the value of the TLB tag?
Unknown from what you have given.
What is the value of the TLB index?
Unknown from what you have given.
What is the value of the virtual page number? (Answers in binary)
Address / Page Size
From my understanding of how virtual address translation happens (assuming a 32 bit virtual address space, as in the x86 architecture):
bits 31:22 of the virtual address indicate the proper Page Directory Entry (i.e. aligned physical address of the Page Table) to access
similarly, bits 21:12 of the virtual address indicate the proper Page Table Entry (i.e. aligned physical address of the Page itself) to access
lastly, bits 11:0 indicate the proper offset into the Page, and are merely appended onto the end of the final physical address to access
As for the physical address
bits 31:12 are the physical base address (aligned) of the Page (found from the PTE)
bits 11:0 are the same as they are in the virtual address
Visually, this is what happens
I know that a Page is a chunk of virtual memory. But conceptually, I have a hard time visualizing what is actually "inside" of a Page. Ie, if I were to index into a "Page entry" (if such a thing even makes sense), what would I get in return?
This post
seems to refer this value as a "desired byte." What exactly is the "desired byte"? Am I overthinking the functionality of Pages?
From reading your question, it sounds like your misunderstanding is because you do not understand the format of a page table entry.
The memory management unit (MMU) of the CPU divides the physical memory into PAGE FRAMES of some fixed size (typically 512K to 1MB).
The operating system manages PAGES of memory.A page must have the same size as the page frame. User mode processes only see pages; not page frames.
The operating system maintains sets of PAGE TABLES that provide the mapping between the pages and page frames.
In a logical memory system, the bits within an address consists of two bit fields. One bit field identifies the page and the other specifies the byte offset into the page.
When a process accesses an address, the MMU divides it into the two bit field. It then uses the page identifier to look up the what page frame the page is mapped to in the page tables.
. Ie, if I were to index into a "Page entry" (if such a thing even makes sense), what would I get in return?
The page entry (or entry in the page table) specifies the number of the physical page frame.
[This is the part it sounds like you are missing.]
In your example, you discuss a multi-level page table, but for simplicity, let's assume there is no page directory, and just a page table.
In a 32-bit system, the page entry will typically be 32-bit and 64-bits on a 64-bit system. The format of the page entry varies among system but it will likely have bit fields that define:
The index of the page frame mapped to.
Bits indicating if the entry is valid.
A bit the indicates if the corresponding page has been written to.
Bits the specify the protection for the page.
In your example you have omitted the format of the page table entry.
So once you have the entry, the next step is to get the page frame from it.
In your example, this is 4096 bytes of data.
The MMU could either just use the page frame index to identify the page. Or it could multiply that value by the page size to get the byte that starts the page.
To get the specific byte within the 4096, the MMU uses the offset (bits 0:11 in your example)
The MMU does all this behind the scenes so the process never sees it. One of the chief jobs of the operating system is to maintain the page tables and the entries within them.
You can think of each page as a 4096 byte "array". (On x86 with 4k pages). It's not an "array" in the high-level-language sense of the word (unless you happen to have a page-aligned array of bytes in your program), but it is a linear collection of bytes that you can index with an offset.
A one-byte load from any specific address has an offset within a specific page, which determines which of those 4096 bytes should be loaded. The low 12 bits of the address determine this offset. (i.e. the "page offset").
Note that 2^12 = 4096, that's why it's the low 12 bits that represent the offset within the page in an address.
Further reading: What Every Programmer Should Know About Memory?
A computer system has a 36-bit virtual address space with a page size of 4K (small modification for hex representation), and 4 bytes per page table entry. (example found here, 2nd problem)
PTE->0x11223344 (32 bits)
FullAddress(PTE<<12+PageOffset)->0x11223344AAA (44 bits)
But the offset in the page table cannot be bigger than 2^24 (36-PAGE_SIZE which is 12=24)
So, let's say there is a function f that generates the PTE address, f: {0,1}^24->{0,1}^32, which effectively allows access to 2^24 pages per process.
Bottom line, i would say that one process cannot address the full 2^44 bytes but only 2^36 and it could be potentially beneficial when there are multiple processes.
e.g. The system could allocate to 2^8 processes different chunks of 2^36 memory.
This is the potential benefit?
(This is for single level of page table, for multilevel it will grow even bigger)
I guess the question is similar with: Does Physical Address Extension (PAE) allows a process to utilize more than 4GB or does it just allows a number of processes to utilize more than 4GB?
I was reading the dinosaur book on Operating System about memory management. I assume this is one of the best books but there's something about paging written in the book which I don't get.
The book says, "A 32-bit CPU uses 32-bit addresses, meaning that a given process space can only be 2^32 bytes (4 TB ). Therefore, paging lets us use physical memory that is larger than what can be addressed by the CPU’s address pointer length."
I don't quite get this part because if the CPU can only refer to 2^32 different physical addresses, if there were 2^32+1 physical addresses, the last address won't be able to be reached by the CPU. So how can paging help with this?
Also, earlier the book says "Frequently, on a 32-bit CPU , each page-table entry is 4 bytes long, but that size can vary as well. A 32-bit entry can point to one of 2^32 physical page frames. If frame size is 4 KB (2^12 ), then a system with 4-byte entries can address 2^44 bytes (or 16 TB ) of physical memory."
I don't see how that is even possible in ideal/theoretical situations, cuz as I understand it, part of the virtual address will refer to an entry of the page table while the other part of the virtual address will refer to the off-set of that particular type in that page. So in the above-mentioned situation put forward by the book, even if the CPU could point to 2^32 different page entries, it won't be able to read any particular byte within that page cuz it doesn't specify the office.
Maybe I've misunderstood the book or there is some part that I missed out. I much appreciate your help! Thanks a lot!
It sounds like you need to burn your book. It's useless.
"[P]aging lets us use physical memory that is larger than what can be addressed by the CPU’s address pointer length" is complete nonsense (unless the book is assigning two different meanings to the term "paging," in which it is still useless).
Let's start with logical addressing. A logical address is composed of a page selector and and offset into the page. Some number (P) of bits will be assigned to the page selector and the remained will be assigned to the offset. If pages are 2^9 bits, there are 23 bits in the page selector and 9 bits for the byte offset within the page.
Note that the 9/23 pick are arbitrary on my part. Most systems these days use larger pages but these are values have been used in the past.
The 23 bits in the page selector are indices into the process page table.
The size of entries in the page table are going to be a power of 2 (and I have never seen one less than 4). For our purposes let's say that each entry is 8-bytes long.
The bits in the page table entry are divided between those that index physical page frames and control bits. let's make the arbitrary choice that 32 bits index page frames and 32 bits are used for control.
That means the system can theoretically MANAGE 2^32 pages that are 2^9 bytes large or a total of 2^41 bytes. If we were to increase the page size from 2^9 to 2^20, the system could theoretically MANAGE 2^52 (32+20) bytes of memory.
Note that each process can still only ACCESS 2^32 bytes. But in my 9-bit page system, 2^9 processes could each access 2^32 pages simultaneously on a system with 2^41 physical bytes of memory (ignoring the need for a shared system address space in this gross oversimplification).
Note that if I change my page table to 32-bits and assign 9 of those bits to control and and 23 to page frame selection, the system can only MANAGE 2^32 bytes of memory (and that was more common than managing greater than 2^32 bytes).
You quote: "Frequently, on a 32-bit CPU , each page-table entry is 4 bytes long, but that size can vary as well. A 32-bit entry can point to one of 2^32 physical page frames. If frame size is 4 KB (2^12 ), then a system with 4-byte entries can address 2^44 bytes (or 16 TB ) of physical memory."
This is theoretical BS. A system that used all 32 bites of the page table entry as an index to page frames could not function. There would have to be some control bits in the page table.
The quotes you are taking from this book are highly misleading. Few (any?) 32-bit processors could even access 2^32 bytes of memory due to address line limitations.
While it is possible that the use of logical pages could allow a processor to manage more memory that the logical address size suggests, that was not the purpose of managing memory in pages.
The purpose of paging—which in its normal and customary usage refers to the movement of virtual memory pages between physical page frames and secondary storage—is to allow processes to access more virtual memory than there was physical memory on the system.
There is an additional system of memory management that is (thankfully) dying out: segments. Segments also provided a means for systems to manage more physical memory than the logical address space would allow.
Lets consider that size of page is equal to 1 KB. One entry in table
takes 2B. Table of pages takes not more than one page (so <= 1KB).
Can we conclude that size of operational memory is <= 512 KB ?
A correct answer is No, however I can't understand it. For me, answer is yes - look at my reasoning at show me where I am wrong, please.
Table contains <=1024B/2B=512=2^9 entries in table of pages. Size of page is 1024B=2^10B, so offset takes not more than <=10 bits. Number of page takes <=9 bits - because we have 512=2^9 entries. Hence, 9+10=19. Therefore <=2^19 bits make it possible to address <= 2^19 B=2^9KB=512KB.
Where am I wrong?
A page table entry is 2B, so a table entry can point to one of 2^16 physical pages. That's 2*16 kiB (because pages are 1kB). So a kernel can make use of up to 65536 kiB (64MiB) of physical memory.
This assumes that the entire page table entry (PTE) is a physical page number. That can work if the position of a PTE within the page table associates it with a virtual address. i.e. 9 bits of a virtual address are used as an index into the PT, to select a PTE.
We can't assume that the machine does work this way, but similarly we can't assume that it doesn't, until we have more information.
Given just the info in the question, we can conclude:
Usable physical memory can be anything up to 65536kiB.
Up to 512 virtual pages can be mapped at once (512kiB).
Not much else! Many possibilities are open.
A more likely design would have a valid/invalid bit in each PTE.
If a PTE's contents (rather than its location) indicate which virtual page it maps, it would be possible to map a large virtual address space onto very few physical pages. This address space would necessarily be sparsely mapped, but the kernel could give a process the illusion of having a lot of active mappings by keeping a separate table of what the mappings should be.
So on a page fault, the kernel checks to see if the page should have been mapped. If so, it puts the mapping in the PTE and resumes the process. (This kind of thing happens in real OSes, and it's called a minor page fault.)
In this design, the kernel's use of the page table to hold a subset of the real mappings would be analogous to a software-managed TLB. (Of course, this architecture could have a non-coherent TLB that requires invalidation after every PTE modification, so this would probably perform horribly).