A comment in this blog states:
We know how to make chunked heaps, but there would be some overhead to
using them. We have more requests for faster storage management than
we do for larger heaps in the 32-bit JVM. If you really want large
heaps, switch to the 64-bit JVM. We still need contiguous memory,
but it's much easier to get in a 64-bit address space.
This implication of the above statement is that it is easier to get contiguous memory in a 64-bit address space. Is this true? If so why?
That's very true. A process must allocate memory from the virtual memory address space. Which stores both code and data and whose size is restricted by the addressing capability of the architecture. You can never address more than 2^32 bytes in a 32-bit process, not counting bank-switching tricks. That's 4 gigabytes. The operating system typically takes a big chunk out of that as well, on 32-bit Windows for example that cuts down the addressable VM size to 2 gigabytes.
Ideally, allocations are made so that they fit snugly together. That very rarely works out in practice. Shared libraries or DLLs in particular need to pick a preferred load address and that has to be guessed up front when the library is built.
So in practice, the allocations are made from the holes in between existing ones and the largest possible contiguous allocation you can get is restricted by the size of the largest hole. Usually much smaller than the addressable VM size, on Windows it is typically around 650 megabytes. That tends to go down-hill from there as the available address space is getting fragmented by allocations. Particularly by native code that can't afford to have allocations moved by a compacting garbage collector. If you use Windows then you can get insight in the VM allocations with the SysInternals' VMMap utility.
This problem completely disappears in a 64-bit process. The theoretical addressable virtual memory size is 2^64, an enormous number. So large that current processors don't implement it, they can go up to 2^48. Further restricted by the operating system version you have and its willingness to keep page mapping tables for that much VM. Eight terabytes is a typical limit. By implication, the holes between allocations are huge. Your program will keel over on paging file thrashing before it dies from OOM.
I can't speak for how the JVM is implemented obviously, but from a purely theoretical viewpoint, if you have a significantly larger virtual address space (eg 64-bit as compared with 32-bit) it should be significantly easier to find a large block of contiguous memory which is available for allocation (going to extremes - you've got no chance of finding a contiguous 4GB of free memory in a 32-bit address space, but a significant chance of finding this space in a full 64-bit address space).
It should be noted that whatever the virtual address space size, this is still going to be implemented by allocation of (probably) non-contiguous physical memory pages, particularly if the requested allocation is large - the larger virtual address space just means there are a likely to be a lot more contiguous virtual addresses available for use.
Related
In CS, there are different memory types, such as the Stack and the Heap. However, on a physical level, are they the same, and these concepts are just for the software writers? Or are there different types of memory in a given computer. I know that there are differences in a HDD and SSD storage, but what about a given disc? On a given HDD, would every block of memory be the same as others?
The answer to the title question is, "Not necessarily." Software determines how memory is allocated. It's possible that the software will use memory starting at physical address and moving up, or in reverse order, or randomly, etc. So if you're using 50% of the total memory, it could be scattered all over the place, or it could be the bottom half of physical memory.
Same is true of a disk. It's up to the software (probably the driver software, but possibly the disk controller) where things are placed physically on the disk.
There are, of course, reserved areas, but again those are by convention. For example, boot loaders are placed in fixed areas on disk drives, BIOS entry points are placed at specific memory addresses, etc.
In general, a memory location is a memory location. Your program asks for a block of memory of a certain size, the operating system's memory allocator will return you a pointer to memory. You might be using that block of physical memory for a program's heap, whereas in the past the same block might have been used for the executable code of some other program.
Memory is memory. With the exception of a few reserved areas used by the operating system or the machine BIOS, it's all the same. The same goes for disk space.
In several books and on websites a reason given for virtual memory management is that it allows only part of a program to be loaded in to RAM and therefore more efficient use of RAM is made.
1) Why do we need virtual memory management to only load part of a program? Why could we not load part of a program using physical addresses?
2) Beyond the security reasons for separating the different parts (stack, heap etc) of a process' memory in to various physical locations, I really don't see what other benefits there are to virtual memory?
3) Why is it important the process thinks the addresses are continuous (courtesy of virtual addresses) when in reality they are discontinuous?
EDIT: I know the obvious reason that virtual memory allows more memory to be treated as if it were RAM.
There are a number of advantages to using virtual memory over strictly physical memory, some of which you've already listed. Basically it allows your programs to just use memory without having to worry about where it comes from or what else might be competing for it. It makes memory appear to be flat and contiguous, even if it's spread out across various sections of physical memory and to disk.
1) Why do we need virtual memory management to only load part of a
program? Why could we not load part of a program using physical
addresses?
You can try that with purely physical addresses, but what if there's not a large enough single block available? With virtual addresses you can bridge sections of physical RAM and make them appear as one large block. You can also move things around in memory without interrupting processes that would be surprised to have that happen.
2) Beyond the security reasons for separating the different parts
(stack, heap etc) of a process' memory in to various physical
locations, I really don't see what other benefits there are to virtual
memory?
It also helps keep memory from getting overly fragmented. Makes it easier to segregate memory in use by one process from memory in use by another.
3) Why is it important the process thinks the addresses are continuous
(courtesy of virtual addresses) when in reality they are
discontinuous?
Try iterating over an array that's split between two discontinuous sections of memory and then come ask that again. Or allocating a buffer for some serial communications, or any number of times that your software expects a single chunk of memory.
1) Why do we need virtual memory management to only load part of a program? Why could we not load part of a program using physical addresses?
Some of us are old enough to remember 32-bit systems with 8MB of memory. Even compressing a small image file would exceed the physical memory of the system.
It's likely that the paging aspect of virtual memory will vanish in the future as system memory and storage merge.
2) Beyond the security reasons for separating the different parts (stack, heap etc) of a process' memory in to various physical locations, I really don't see what other benefits there are to virtual memory?
See #1. The amount of memory required by a program may exceed the physical memory available.
That said, the main security reasons are to separate the various processes and the system address space. Any separation of stack, heap, code, are usually for convenience and error detection.
Advantages then include:
Process memory in excess of physical memory
Separation of processes
Manages access to the kernel (and in some systems other modes as well)
Prevents executing non-executable pages.
Prevents writing to read only pages (code, data).
Easy of programming
3) Why is it important the process thinks the addresses are continuous (courtesy of virtual addresses) when in reality they are discontinuous?
I presume you are referring to virtual addresses. This is simply a matter of convenience. It would make no sense to make them non-contiguous.
1) Why do we need virtual memory management to only load part of a
program? Why could we not load part of a program using physical
addresses?
Well, you obviously are aware that programs' size can range from some KB's to several GBs or even more than that. But, as we have kind of a limitation on our main memory aka RAM (because of cost issues), so the whole program bigger than the size of RAM can't be loaded as whole at once. So, to achieve the desired result scientists (computer scientists) developed a method virtual memory. It'd help to achieve
a) first space equal to size of some portion of hard-disk(not total),but the major part that would easily accomodate parts of running program. Say, if the running program's size exceeds the size of RAM, then the program is kinda cut into segments (not really), only the relevant part which could easily fit into memory is called, and the subsequent codes are called as per address by address in sequence or as per instruction call!
b) less burden on physical memory and thereby enabling other programs in the main memory to keep running. Well there are several more reasons!
2) Beyond the security reasons for separating the different parts
(stack, heap etc) of a process' memory in to various physical
locations, I really don't see what other benefits there are to virtual
memory?
Separation of heap,stack,etc. is for storing several kinds of operations running at times. They all are different data-structures and hence they will be storing possibly different program's values or, even if similar program's values, then also distinct instruction sequence's address! Say, stack would be storing a recursive call's return address (the calling address) whereas the heap would be pointing at current code of execution of the program!
Also, it is not the virtual memory which has this storage scheme but it's actually fit in the main memory. Also,there are several portions of heap also, which performs different functions entirely! Also, I already mentioned benefits of virtual memory -- helps in running several programs simultaneously,optimises caching, addressing using paging, segmentation, etc.
3) Why is it important the process thinks the addresses are continuous
(courtesy of virtual addresses) when in reality they are
discontinuous?
Would it be better in the world if there had been counting like 1,2,3,4,5,etc. which we are familiar or had it started like 1,5,2,4,3, etc. even though knowing the true pattern rejecting the choice to render it discontinuous? Well, at least I'd have chosen the pattern option to perform any task. Similar is the case with physical (main) memory! The physical memory renders the exact address and it clearly fetches the addresses in a dis-continuous manner -- kinda mingled.
But wait, WOW, we have a mechanism like virtual memory which has led to the formation of the actual discontinuous memory locations to a fixed regular/continuous memory location! Virtual memory using paging, segmentation has made the work same, but to make us understand easier. Also,due to relative indexing in paging and due to segmentation -- the address/memory location appears continuous though the actual address is always determined the paging scheme or segment's starting address! Hence, the virtual-memory renders as if we are working with a continuous memory location. Isn't it good/better!
Hi people I was wondering about something called addressing schemes that the operating systems use for expandable RAM's. Let us consider an example to throw some light on this.
"If we have a 32 bit architecture for the computer then this means we have a computer address that is 32-bit long which amounts to 2^32 addressable memory location approximately 4GB of data."
But if we add another 4GB of main memory that is now 8GB of RAM in effect, how does the computer address the extra main memory locations because that additional amount exceeds the range of a 32 bit address which is 2^32.
Can anyone throw some light on this question.
Basically, you can't address 8GB with just 32bits. At any given point in time using 32bits you can only choose from 4G memory locations .
A popular workaround is to use physical addresses which are larger than 32 bits in the page tables. This allows the operating system to define which subset of the 8GB a program is able to access. However, this subset can never be bigger than 4GB. x86 PAE is one example, but there are others which do just the same.
With this workaround the operating system itself can access the whole 8GB only by changing it's own page table. E.g. to access a memory location, it first has to map the memory location into its own address space by changing the page table, and only then can start accessing the memory location. Of course this is very cumbersome (to say the least). It can also lead to problems if parts of the operating system were written without considering this type of memory extension, device drivers are a typical example.
The problem is not new. 8bit Computers like Commodores C64 used bank switching to access more than 64KB with 16bit addresses. Early PCs used expanded memory to work around the 640KB limit. The Right Thing (TM) is of course to switch to bigger addresses before you have to resort to ugly solutions.
sorry for my rather general question, but I could not find a definite answer to it:
Given that I have free swap memory left and I allocate memory in reasonable chunks (~1MB) -> can memory allocation still fail for any reason?
The smartass answer would be "yes, memory allocation can fail for any reason". That may not be what you are looking for.
Generally, whether your system has free memory left is not related to whether allocations succeed. Rather, the question is whether your process address space has free virtual address space.
The allocator (malloc, operator new, ...) first looks if there is free address space in the current process that is already mapped, that is, the kernel is aware that the addresses should be usable. If there is, that address space is reserved in the allocator and returned.
Otherwise, the kernel is asked to map new address space to the process. This may fail, but generally doesn't, as mapping does not imply using physical memory yet -- it is just a promise that, should someone try to access this address, the kernel will try to find physical memory and set up the MMU tables so the virtual->physical translation finds it.
When the system is out of memory, there is no physical memory left, the process is suspended and the kernel attempts to free physical memory by moving other processes' memory to disk. The application does not notice this, except that executing a single assembler instruction apparently took a long time.
Memory allocations in the process fail if there is no mapped free region large enough and the kernel refuses to establish a mapping. For example, not all virtual addresses are useable, as most operating systems map the kernel at some address (typically, 0x80000000, 0xc0000000, 0xe0000000 or something such on 32 bit architectures), so there is a per-process limit that may be lower than the system limit (for example, a 32 bit process on Windows can only allocate 2 GB, even if the system is 64 bit). File mappings (such as the program itself and DLLs) further reduce the available space.
A very general and theoretical answer would be no, it can not. One of the reasons it could possibly under very peculiar circumstances fail is that there would be some weird fragmentation of your available / allocatable memory. I wonder whether you're trying get (probably very minor) performance boost (skipping if pointer == NULL - kind of thing) or you're just wondering and want to discuss it, in which case you should probably use chat.
Yes, memory allocation often fails when you run out of memory space in a 32-bit application (can be 2, 3 or 4 GB depending on OS version and settings). This would be due to a memory leak. It can also fail if your OS runs out of space in your swap file.
I'm trying to get a better understanding of how Windows, 32-bit, calculates the virtual bytes for a program. I am under the impression that Virtual Bytes (VB) are the measure of how much of the user address space is being used, while the Private Bytes (PB) are the measure of actual committed and reserved memory on the system.
In particular, I have a server program I am monitoring which, when under heavy usage, will climb up to the 3GB limit for VBs. Around the same time the PB climb as well, but then quickly drop down to around 1 GB as the usage drops. The PB tend to then stay low, around the 1 GB mark, but the VB stay up around the 3 GB mark. I do not have access to the source code, so I am just using the basic Windows performance counters to monitor all of this. From a programming point of view, what memory concept do I not understand that makes this all possible? Is there a good reference to learn more about this?
What your reporting is most likely being caused by the process heap. There are two pieces to a memory allocation in Windows. The first piece is the continuous address space in your application for the memory to accessed through. On a 32 bit system not running the /3GB switch all your allocations must come out of the lower 2 GB of user address space. The second piece of the memory allocation is the actually memory for the allocation. This can be either RAM or part of the page file system on the hard disk. The OS handles moving allocations between RAM and the page file system in the background.
Most likely your application is using a Windows heap to handle all memory allocations. When a heap is created is reserves 1 MB of address space for the memory it will allocate. Until it actually needs memory associated with this address space no physical memory is actually used. If the heap needs more memory than 1 MB it uses a doubling algorithm to reserve more address space, and then commits physical memory when it needs it. The important thing to note is that once a heap reserves address space it never releases it.
Personally I found the following books and chapters useful when trying to understand memory management.
Advanced Windows Debugging - Chapter 6 This book has the most detailed look into the heap I have seen.
Windows Internals - Chapter 7 This book adds a bit of information not found in Advanced Windows Debugging; however, it does not give as good an overview.
It sounds to me like you have a garbage collector that's only kicking in once the memory pressure hits 1/3 (1 GB out of 3 GB).
As for the VB - don't worry! It's virtual! Honestly, nothing's been allocated, nothing's been committed. Focus on your private bytes - your real allocations.
There is such a thing as "Virtual Memory". It's a rather non-OS-specific concept in computer science. Microsoft has also written about Windows implementation of the thing.
A long story short, in Windows you can ask to reserve some memory without actually allocating any physical memory. It's like making some memory addresses reserved for future use. When you really need the memory, you allocate it physically (aka "commit" it).
I haven't needed to use this feature myself, so I don't know how it's used in real life programs, but I know it's there. I think the idea might be something like preserving pointers to some memory address and allocating the memory when needed, without having to change what the pointers actually point to.
Windows is notorious for having a variety of types of memory allocations, some of which are supersets of others. You've mentioned Private Bytes and Virtual Bytes. Private bytes, as you know, refers to memory allocated specifically to your process. Virtual bytes includes private bytes, as well as any shared memory, reserved memory, etc.
Even though in real life you only need to be concerned with private bytes and perhaps shared memory (Windows handles the rest, anyways), the Virtual Bytes count is often what users (and evaluators of your software) see and interpret as the memory usage of your process.
A good and up-to-date reference on the subject is the book titled Windows Via C/C++ by Jeffrey Richter, you should look for Chapter 13: "Windows Memory Architecture".