How is the Virtual address space greater than Physical address space ?
suppose a Virtual 0x7000 maps to physical address 0x8000, can another virtual address lets say
0x7500 map to the same physical location as 0x8000, if not then how can there be more virtual
address and limited physical memory since mapping has to convert to physical address?
Please help me understand this concept.
http://en.wikipedia.org/wiki/Virtual_memory.
Virtual Memory uses both physical ram and hard disk space to represent more memory than may physically exist, and provides an interface whereby each program can request memory resources without having to be concerned with the other programs existent on the machine and which memory addresses they may request.
The whole virtual address space does not have to be mapped to physical memory at the same time. That's what makes it "virtual". The contents of that virtual memory which is allocated but not currently mapped to physical memory reside on some form of external storage, typically disk.
It is the memory management system's job to move virtual memory pages into and out of physical memory as needed, and the requirement to do so is why virtual-memory computers can slow down overall when enough memory is allocated that it no longer all fits in physical memory at the same time.
Related
Let me put my understanding.
Suppose we have a 32-bit memory address space for a system. So a process can access any memory in the 4GB range
If the RAM in the system we have of 4GB, kernel divides it into 1:3 . 1GB for kernel , and rest 3GB for the user space process.
A user space process will get the system memory access within that 3GB memory only and which address it gets is determined by the page table.
Kernel logical address is that 1GB ( approx ~896MB) memory which is being reserved only for the kernel. Is this correct?
kernel virtual address is the memory left i.e. 104MB + 3GB that also can be assigned to userspace. Is this correct?
user virtual address is the address generated by the user space process and its corresponding memory would be assigned from the 3GB reserved for the user space process by the kernel.
Let me know if my above understanding is correct? If not can you please explain in detail the difference between kernel logical address space , kernel virtual address space and user virtual address space.
your understanding is a mixture of right and wrong, I'll try to point to some of them:
in 32 bit machines, we're not always limited by 4GB addressable RAM, check this question for more detail: link
the memory is an abstraction for the user space programs, they see it a a continuous big chunk of memory, but the kernel manages this abstraction with some hardware support named MMU, to map the used virtual space in the user space program into an actual physical address or even some bloc in hard drive if swapping is activated.
the kernel can actually access to the physical memory, in order to manage the abstraction mentioned above, it can also use this abstraction, this depends on the designer of the kernel.
as for the difference between the virtual and logical addressing, check this answer: link
In my research project, I have to reserve a large virtual memory address space inside a kernel module and handle memory accesses to that area (in a 64-bit system). I have modified do_page_fault function in arch/x86/mm/fault.c so that I can handle page faults when there is no page mapping record in TLB.
In my project, the reserved virtual address space is much larger than physically available memory. For example, let's say we have 4GB available memory space. Inside my kernel module, I what to have 16GB address space which is accessible by load/store instructions. I want to compress a 4KB page into 1KB and maintain 16GB compressible data into 4GB memory; once a page fault occurs, it'll decompress the corresponding page and copy it into the page cache.
I've tried memory allocate functions such as kmalloc and vmalloc, but they literally allocate memory while I only need some non-one-to-one virtual address mapping via calling a [decompression] function. How can I do that?
I have come across a term - holes in the memory in Linux. I believe this is the memory that is I/O remapped. Is my understanding correct?
Holes in memory can mean different things:
1) It can refer to physical memory addressing: For historical and boot-strapping reasons, in the "standard PC" (x86) architecture, all of system RAM is not contiguous. There are "holes" in the address space where memory-mapped I/O resides. For example, from the earliest days, there has been an area reserved for boot ROM (BIOS) and video memory. Also, there is a large area of the address space which is reserved for dynamic assignment to PCI (and PCI-X or PCI-Express) peripherals. These areas are often mapped as needed into kernel virtual address space by device drivers (which may be referred to as "I/O remapping").
Memory controllers built-in to the motherboard allow the physical address of the RAM to be configured (this is typically handled by the BIOS in the standard PC architecture). Other [non-x86] architectures often have similar holes in the physical address space.
2.) The term can also refer to unassigned regions in the virtual address space. Both kernel virtual address space and user processes' virtual address space typically have "holes" in them. For example, linux doesn't map any physical memory corresponding to virtual address 0 (i.e. the first page of the address space never has valid memory) -- this allows null pointer references to be trapped.
In some kinds of memory allocations, the linux kernel maintains unmapped areas between properly allocated virtual memory regions in order to trap faulty memory references (i.e. that stray beyond the end of the allocated space).
Does virtual address space resides in virtual memory ? I have a confusion like , Each process
has its own virtual memory and page table and conversion to physical address from virtual address takes place while loading it into physical memory , but where does the virtual address space comes into picture ? I have gone through many operating systems books but everywhere it gives just explaination about particular word not where it resides and what is the relationship between them and how it operates.
please just explain me theoretically , example not needed.
Thanks in advance.
(Virtual) Address space is the set of allowable addresses for a given address width (that is 2^32 bytes on x86, 2^64 on x64). Virtual memory usually means almost the same. It is the set of allowable addresses for a certain process or application or also for the whole system. The virtual memory for a single application can be at most as big as the virtual address space of the system. Each application can "see" only the virtual address space that is allocated to it by the OS (and due to some trickery, it is possible that each application can have a virtual address space of the same size and the sum might be larger than the address space of the system).
Physical memory (more correct: physical RAM), is the amount of effectively installed RAM modules. It is usually smaller than the virtual address space. The OS does swapping to bring the requires memory pages from the hard disk into the physical memory if needed. A memory page in physical memory has a physical address and a virtual one. Normal applications only see the virtual address, and they don't (and must not) care about where the memory page is physically loaded. Therefore an address seen in an application or a debugger is really a virtual address in the virtual address space of that application. The physical address is only ever needed when directly interfacing to hardware. It can even constantly change if the OS decides to do so.
Hope this makes it a bit clearer.
I'm not a specialist but i think virtual addressing and paging is a part of the cpu protected mode introduced after 80386 and it is not part of the operating system. The operating system controls the page tables. For the virtual addresses they are just numbers in your executable file for example
objdump -d will display them
I am not able to exactly difference between kernel logical address and virtual address. In Linux device driver book it says that all logical address are kernel virtual address, and virtual address doesn't have any linear mapping. But logically wise when we say it is logical and when we say virtual and in which situation we use these two ?
The Linux kernel maps most of the virtual address space that belongs to the kernel to perform 1:1 mapping with an offset of the first part of physical memory. (slightly less then for 1Gb for 32bit x86, can be different for other processors or configurations). For example, for kernel code on x86 address 0xc00000001 is mapped to physical address 0x1.
This is called logical mapping - a 1:1 mapping (with an offset) that allows the kernel to access most of the physical memory of the machine.
But this is not enough - sometime we have more then 1Gb physical memory on a 32bit machine, sometime we want to reference non contiguous physical memory blocks as contiguous to make thing simple, sometime we want to map memory mapped IO regions which are not RAM.
For this, the kernel keeps a region at the top of its virtual address space where it does a "random" page to page mapping. The mapping there do not follow the 1:1 pattern of the logical mapping area. This is what we call the virtual mapping.
It is important to add that on many platforms (x86 is an example), both the logical and virtual mapping are done using the same hardware mechanism (TLB controlling virtual memory). In many cases, the "logical mapping" is actually done using virtual memory facility of the processor, so this can be a little confusing. The difference therefore is the pattern according to which the mapping is done: 1:1 for logical, something random for virtual.
Basically there are 3 kinds of addressing, namely
Logical Addressing : Address is formed by base and offset. This is nothing but segmented addressing, where the address (or offset) in the program is always used with the base value in the segment descriptor
Linear Addressing : Also called virtual address. Here adresses are contigous, but the physical address are not. Paging is used to implement this.
Physical addressing : The actual address on the Main Memory!
Now, in linux, Kernel memory (in address space) is beyond 3 GB ( 3GB to 4GB), i.e. 0xc000000..The addresses used by Kernel are not Physical addresses. To map the virtual address it uses PAGE_OFFSET. Care must be taken that no page translation is involved. i.e. these addresses are contiguous in nature. However there is a limit to this, i.e. 896 MB on x86. Beyond which paging is used for translation. When you use vmalloc, these addresses are returned to access the allocated memory.
In short, when someone refers to Virtual Memory in context of User Space, then it is through Paging. If Kernel Virtual Memory is mentioned then it is either PAGE_OFFSETed or vmalloced address.
(Reference - Understanding Linux Kernel - 2.6 based )
Shash
Kernel logical addresses are mappings accessible to kernel code through normal CPU memory access functions. On 32-bit systems, only 4GB of kernel logical address space exists, even if more physical memory than that is in use. Logical address space backed by physical memory can be allocated with kmalloc.
Virtual addresses do not necessarily have corresponding logical addresses. You can allocate physical memory with vmalloc and get back a virtual address that has no corresponding logical address (on 32-bit systems with PAE, for example). You can then use kmap to assign a logical address to that virtual address.
Simply speaking, virtual address would include "high memory", which doesn't do the 1:1 mapping for the physical address,if your RAM size is more than the address range of kernel(typically,For 1G/3G in X86,your RAM is 3G but your kernel addressing range is 1G) and also the address return from kmap() and vmalloc(), which requires the kernel to establish page table for the memory mapping. since logic address is always memory mapped by the kernel(1:1 mapping), you don't need to explicitly call kernel API,like set_pte to set up the page table entry for the particular page.
so virtual address can't be logic address all the time.