Image demonstrating the memory window while debugging
When using the memory window in visual studio, do we see the virtual address of that process or the physical address of RAM?
User-level code always sees virtual addresses. It has no way to know what physical address (if any!) presently corresponds to one of these addresses. "Virtual" is the only world that it lives in, and the only one that it ever knows. For all of a program's intents and purposes, "virtual is reality."
To clarify: all operating systems (except MS-DOS and the like ...) run user programs in a virtual memory space. Each program has its own perception of what "location $12345678 contains," and each program's perception is, for it, correct. Each program can have a different number there, and can change it at its own pleasure.
They can do this because none of them actually know:
Where, in physical memory, "their 'location $12345678'" actually resides. (If it does ... and quite-possibly it doesn't!)
What is in physical address $12345678.
The operating system maintains virtual memory for each process, using a combination of physical-RAM and (if necessary) page-file and/or swap-file space. Information that is actively being used by a process is made available to it "on demand," at a physical RAM-location that is both unknown to it and unknowable by it. Information that has not recently been used is eventually "stolen" from physical RAM and moved to external storage ... until it is referenced again, triggering what is known as a "page fault."
The memory view that is given to you by a debugger is the memory view that is perceived by and that is correct for the program being debugged: virtual addresses, in the process's virtual memory.
Related
Say two processes are using Kernel32.dll, does Windows map the DLLs to the same virtual address space in both processes? If not, how does paging mechanism end up using the same physical address where the DLL is in fact loaded for both processes? I tried finding this info in the windows internals book but didn't find anything
TL;DR: No, it might be loaded somewhere else in another process.
Ntdll and Kernel32 are special and always load at the same address so it is better to focus on something else, Shell32 for example.
A dll has what is known as a preferred base address and this is stored in the PE header (ImageBase). The loader will first attempt to load the dll at this address. If that address range is free then loading will succeed with no extra work required.
If the address is not free then the loader has to load it somewhere else. Loading at a different address usually requires relocation information and if this was removed during linking (/FIXED) then loading will fail! If there was space somewhere else to load the dll, the loader will use the relocation information to patch the given locations in the dll with the new base address. Because dlls are loaded as copy-on-write, this will cause extra memory usage compared to loading at the preferred address since each memory page that needed a patch is now a private copy in the process. This means that the answer to your question is no, a dll might not load at the same address in a different process if that process already has something else loaded there.
So far I have only talked about the loader. The loader is implemented in Ntdll as normal usermode code and is not involved with how a file mapped into memory actually works. Memory mapped files (known as Sections internally in NT) is a co-operation between the operating system kernel and the CPU hardware. This is a whole topic in of itself but the important thing to know is that physical memory and the page/swap file mechanism is completely disconnected from how a usermode process accesses its virtual memory pages. The kernel can map a physical memory page to zero, one, or multiple places in a processes virtual memory and the CPU will automatically translate when a virtual page is accessed by the process.
As a final note, ASLR does complicate things a little bit but the "offset" only changes on reboot and should not have an impact on this specific question in current implementations. In theory Windows could change this in the future and always load things at different addresses in different processes but this is unlikely to happen because of the copy-on-write downsides.
How can I make a program use virtual memory in Windows?
I have a long perl script which is using 6GB+ of memory and increasing. My machine only has 8GB or RAM. It is probably caused by a memory leak in a module, but there is nothing I can do about that now.
Is it possible to make it use virtual memory, or is this something controlled by Windows only?
The OS will provide virtual memory automatically if needed and if it's configured to have swap space. You cannot control that from a Perl program.
If your Perl program has a memory leak eventually it will start being swapped to the page file. When its memory consumption causes total memory to exceed the sum of your physical RAM plus page file, things will slow to a crawl and processes may become unresponsive and/or crash.
In any case, the size of the page file cannot be change dynamically, a reboot is required. The only long-term fix is to find and fix the leak.
Create a shortcut of program that u want to run in virtual ram.
Right click on shortcut and click properties.
In properties, locate for target.
Copy and this at the end of target( --profile-directory="Profile 1"--disk-cache-dir=C:\ ).
Restart your pc.
I want to change the protection for the memory pages of a hole function inside a Dll to be no more executable.
Does this affect other processes which mapped the same dll into virtual address space?
TL;DR: No, it doesn't.
The NX bit works at the virtual/page level. Each process has their own virtual address space, and cannot directly affect another process' address space (barring shared memory).
Different processes can share mapped pages of a DLL. If one process alters a page, copy-on-write kicks in and that process gets a unique copy of the page, the other processes' view(s) of the DLL are unaffected.
For more information on NX, see Intel® 64 and IA-32 Architectures
Software Developer’s Manual (Volume 3A linked, 7 volumes total).
Also, here is an older but still relevant Microsoft article on copy-on-write protection.
I watched a ASLRed dll images's based address for 32bit Process.
It's not a fully randomization. It just randomizated 1/2 probability.
For example, once I load a dll then the image is loaded on 0x12345678.
And I load the image again, the image is loaded on 0x23456789.(Base address is changed!)
But I load the image again
0x12345678
0x23456789
0x12345678
0x23456789
...
Why they did implement like this?
Is it for a crash report's frequency?(For getting same crash addresses of re-deployed dlls)
This is by design. Normally, Windows selects a preferred base address for an ASLR DLL when the DLL is first loaded, and then it keeps using that address until the system is rebooted. That way the DLL will be mapped at the same address in every process that loads it, allowing code pages to be shared.
However, if a DLL has been unloaded from every process, the system may sometimes select a different base address the next time the DLL is loaded. It does this to reduce virtual address space fragmentation, not for security reasons. This is what seems to be happening in your case.
It's documented as being at one of 1 of 256 possible starting addresses.
But i didn't think it even applied to a process, but to shared DLL's.
ASLR: is not on by default for process images. It's an opt-in thing, for compatiblity.(3)
Address Space Layout Randomization
(ASLR)
ASLR moves executable images into
random locations when a system boots,
making it harder for exploit code to
operate predictably. For a component
to support ASLR, all components that
it loads must also support ASLR. For
example, if A.exe consumes B.dll and
C.dll, all three must support ASLR. By
default, Windows Vista and later will
randomize system DLLs and EXEs, but
DLLs and EXEs created by ISVs must opt
in to support ASLR using the
/DYNAMICBASE linker option.
ASLR also randomizes heap and stack
memory:
When an application creates a heap in
Windows Vista and later, the heap
manager will create that heap at a
random location to help reduce the
chance that an attempt to exploit a
heap-based buffer overrun succeeds.
Heap randomization is enabled by
default for all applications running
on Windows Vista and later.
When a
thread starts in a process linked with
/DYNAMICBASE, Windows Vista and later
moves the thread's stack to a random
location to help reduce the chance
that a stack-based buffer overrun
exploit will succeed.
Had installed new Win8 RC x64 yesterday.
Watch out!
Kernel32.dll (64-bit version) have different base address in different processes (in single session, of course). Only ntdll.dll base address remains constant. I had to change the code, you can no longer rely on the permanent address Loadlibrary.
Two applications share memory by MMF.
A create MMF (about 1GB), B open that MMF file by name.
When I see Windows Task Manager, A has 1GB memory.
But, after several closing and launching B app again,
(or after 1 days later? I'm not sure how to reproduce)
A's memory in Windows Task Manager is below 1K bytes.
My guess is,
maybe because A app doesn't do anything after create MMF,
so, Windows thinks MMF is belong to B app. (Just guess).
My OS is Windows 2003 Enterprise x64, SP2.
Is there somebody who knows the reason?
Thanks in advance.
Memory mapped file is still part of your Virtual Address Space, use perfmon to get reliable counters instead of Task Manager, which changes with each release of Windows. The Perfmon counter of Process | Virtual Bytes (total VAS) is the most interesting.
My understanding is that 1GB is reserved in the virtual address space, but memory is only actually allocated for pages that are touched. Memory mapped files are implemented parallel to the Virtual Memory API, and both build upon the NT Virtual Memory Manager. See this article and diagram for an explanation.
Did you fill your entire file with data, or did you just allocate 1GB?
UPDATE:
Which column are you viewing in Task Manager?
The default Memory (Private Working) represents physically allocated memory.
You can add the column Commit Size to see the total amount of virtual address space allocated to the process.
Here is a summary of the various memory statistics you can see in Task Manager and what they mean.
It's because of memory working set minimize.
Thanks for everyone. :)