I am building a Memory Scanner to find malware strings in a process.
Btw, when I was searching about the VirtualQueryEx dll, I saw that people starts its variable lpAdress ( which is supposed to be the Base Address of the process) with a NULL/0 value
LPVOID lpAdress = 0
and in each loop they increase the adress value by the size of the page they just read, so that way they go to the next page and can map all process virtual memory
lpAdress += mbi.RegionSize # mbi is a variable with MEMORY_BASIC_INFORMATION structure
So, is lpAdress the value of memory considering 0 as a start of the own process virtual memory and you dont need to get the actual base adress of the process in memory ? Sorry if my question looks dumb, but the MSDN documentation is confusing me.
Each process has it's own virtual address space that starts at 0. The various executable files (.exe / .dll / whatever) are loaded either at addresses specified in the file or more recently at random addresses for security purposes.
A process can easily have mapped memory regions at addresses lower than where the process executable is loaded. For this reason, if you want to examine a process' entire memory space you need to start at 0.
Related
I'm currently studying memory management of OS by the video lecture. The instructor says,
In fact, you may have, and it is quite often the case that there may
be several parts of the process memory, which are not even accessed at
all. That is, they are neither executed, loaded or stored from memory.
I don't understand the saying since even if in a simple C program, we access whole address space of it. Don't we?
#include <stdio.h>
int main()
{
printf("Hello, World!");
return 0;
}
Could you elucidate the saying? If possible could you provide an example program wherein "several parts of the process memory, which are not even accessed at all" when it is run.
Imagine you have a large and complicated utility (e.g. a compiler), and the user asks it for help (e.g. they type gcc --help instead of asking it to compile anything). In this case, how much of the utility's code and data is used?
Most programs have various optional parts that aren't used (e.g. maybe something that works with graphics will have some code for 16 bits per pixel and other code for 32 bits per pixel, and will determine which code to use and not use the other code). Most heap allocators are "eager" (e.g. they'll ask the OS for 20 MiB of space and then might only "malloc() 2 MiB of it). Sometimes a program will memory map a huge file but then only access a small part of it.
Even for your trivial "hello world" example code; the virtual address space probably contains a huge (several MiB) shared library to support lots of C standard library functions (e.g. puts(), fprintf(), sprintf(), ...) and your program only uses a small part of that shared library; and your program probably reserves a conservative amount of space for its stack (e.g. maybe 20 KiB of space for its stack) and then probably only uses a few hundred bytes of stack.
In a virtual memory system, the address space of the process is created in secondary store at start up. Little or nothing gets placed in memory. For example, the operating system may use the executable file as the page file for the code and static data. It just sets up an internal structure that says some range of memory is mapped to these blocks in the executable file. The same goes for shared libraries. The other data gets mapped to the page file.
As your program runs it starts page faulting rapidly because nothing is in memory and the operating system has to load it from secondary storage.
If there is something that your program does not reference, it never gets loaded into memory.
If you had global variable declared like
char somedata [1045] ;
and your program never references that variable, it will never get loaded into memory. The same goes for code. If you have pages of code that done get execute (e.g. error handling code) it does not get loaded. If you link to shared libraries, you will likely bece including a lot of functions that you never use. Likewise, they will not get loaded if you do not execute them.
To begin with, not all of the address space is backed by physical memory at all times, especially if your address space covers 248+ bytes, which your computer doesn't have (which is not to say you can't map most of the address space to a single physical page of memory, which would be of very little utility for anything).
And then some portions of the address space may be purposefully permanently inaccessible, like a few pages near virtual address 0 (to catch NULL pointer dereferences).
And as it's been pointed out in the other answers, with on-demand loading of programs, you may have some portions of the address space reserved for your program but if the program doesn't happen to need any of its code or data there, nothing needs to be loader there either.
This [wiki article] about Virtual memory says:
The process then starts executing bytes in the exe file. However, the
only way the process can use or set '-' values in its VAS is to ask
the OS to map them to bytes from a file. A common way to use VAS
memory in this way is to map it to the page file.
A diagram follows :
0 4GB
VAS |---vvvvvvv----vvvvvv---vvvv----vv---v----vvv--|
mapping ||||||| |||||| |||| || | |||
file bytes app.exe kernel user system_page_file
I didn't understand the part values in its VAS is to ask the OS to map them to bytes from a file.
What is the system page file here?
First off, I can't imagine such a badly written article to exist in Wikipedia. One has to be an expert already familiar with the topic before being able to understand what was described.
Assuming you understand the rest of the article, the '-' part represents unallocated virtual address within the 4GB address space available to a process. So the sentence "the only way the process can use or set '-' values in its VAS is to ask the OS to map them to bytes from a file" means to allocate virtual memory address e.g. in a Windows native program calling VirtualAlloc(), or a C program calling malloc() to allocate some memory to store program data while those memory were not already existing in the current process's virtual address space.
When Windows allocates memory to a process address space, it normally associate those memory with the paging file in the hard disk. The c:\pagefile.sys is this paging file which is the system_page_file mentioned in the article. Memory page is swapped out to that file when there is not enough physical page to accommodate the demand.
Hope that clarifies
I have some shared libraries mapped into virtual address space of my task. What happens when I change some data for example in .bss section? I do it using kmap with physical page address as argument. I can suggest 2 ways. Data is changed and it influences at all tasks which use the library or the certain page is copied due to COW.
I think it's neither. The .bss area is set up when the executable is loaded. Virtual memory space is allocated for it at that time, and that space won't be shared with any other task. Pages won't be allocated initially (by default, mlock* can change that); they will be faulted in (i.e. demand-zeroed) as referenced.
I think that even if the process forks before touching the memory, the new process would then just get the equivalent (same virtual memory space marked as demand-zero).
So if you already have a physical address for it, I would think that's already happened and you won't be changing anything except the one page belonging to the current process.
a few questions about windows memory segmentation.
every process in windows got his own virtual memory. does it mean that each each process has it own task
(I mean own Task descriptor or Task gate) ?
I opened a simple exe with ollydbg and I saw that for each CALL intruction to a dll function is taking me to the jumping table. the jumping table had jumping instructions to the DLLs like this one :
JMP DWORD PTR DS:[402058]
my question is why its uses the data segment and not the CS selector for the base address?
if I open the memory map and find what stored at 402058 I find that it containes resorces.
if I understand correctly the addresses of the DLL function stored in the DS ?
I noticed that the memory map is organized by owner. shouldn't it be organized with segments like all the code be in CS data in DS etc ?
thank you
1.
A Process has it's own virtual address space.
I do not understand what you're referring to as "Task descriptor or Task gate", but the Windows operating system holds a descriptor for each process, called the Process Control Block, that contains information about the process (such as identification, access tokens, execution state, virtual memory mapping, etc).
A Task is a logical unit that can be used to manage a single process, or multiple processes.
Job -> Tasks
Task -> Processes
Process -> Threads
2.
In the case you mentioned, which is common for compilers, the program uses the .DATA section to store the jump table after loading the function addresses.
The reason why this happens in the first place is because the compiler cannot know the DLL base address at compile-time, therefore the address has to be fixed at load-time to point to the function. This is known as Relocation.
In order to maintain the jump table seperately from the code, compilers store it in the .DATA section. This way, we can also give it write permissions (usually the .DATA segment has write permissions) and modify it as necessary without sacrificing stability and security.
3.
Each module loaded in the process' virtual address space contains it's own sections - that's why you see a different set of .text, .data, .reloc etc for each module. The "Owner" column is the module name.
P.S. Please ask one question per post - that way it will be easily accesible by other users after you get answered, and each question will likely get more accurate answers.
How can I locate which areas of memory of a Win32 process contain the global data and the stack data for each thread?
There is no API (that I know of) to do this. But If you have a DLL in the process, then you will get DLL_PROCESS_ATTACH/DLL_THREAD_ATTACH notifications in DllMain when each thread is created. You can record the thread ID and the address of a stack object for that thread when you get these notifications, because you will get called on the new thread. So store the thread id and stack address in some table that you create at that time. Don't try to do a lot of work in DllMain, just record the stack location and return.
You can then use VirtualQuery to get turn the address of a variable on each thread stack into a virtual allocation range, that should give you the base address of the stack (remember that stacks grow from high addresses to low addresses). The default allocation size for a stack is 1Mb, but that can be overridden by a linker switch or by the thread creator, but a stack must be contiguous. So what you get back from VirtualQuery will be the full stack at that point in time
As for the heap location - there can be multiple locations for the heap, but in general if you want to assume a single contigous heap location then use HeapAlloc to get the address of a heap object and then VirtualQuery to determine the range of pages for that section of the heap.
Alternatively You can use VirtualQuery on the hModule for the EXE and for each DLL. and then you can assume that anything that is read-write and isn't a stack or a module is part of the heap. Note that this will be true in most processes, but may not be true in some because an application can call VirtualAlloc or CreateFileMapping directly, resulting in valid data pointers that are not from either stack or heap.
Use EnumProcessModules to get the the list of modules loaded into a process.
VirtualQuery basically takes a random address, and returns the base address of the collection of pages that that address belongs to, as well as the page protections. So it's good for going from a specific pointer which 'type' of allocation.
Take the address of variables allocated in the memory regions you are interested. What you do with the addresses when you have them is another question entirely.
You can also objdump -h (I think it's -h, might be -x) to list the section addresses, including data sections.
Global Data
By "Global" I'm going to assume you mean all the data that is not dynamically allocated using new, malloc, HeapAlloc, VirtualAlloc etc - the data that you may declare in your source code that is outside of functions and outside of class definitions.
You can locate these by loading each DLL as a PE File in a PE file reader and determining the locations of the .data and .bss sections (these may have different names for different compilers). You need to do this for each DLL. That gives you the general locations for this data for each DLL. Then, if you have debugging information, or failing that, a MAP file, you can map the DLL addresses against the debug info/mapfile info to get names and exact locations for each variable.
You may find the PE Format DLL helps you perform this task much easier than writing the code to query the PE file yourself.
Thread Stacks
Enumerate the threads in the application using ToolHelp32 (or PSAPI library if on Windows NT 4). For each thread, get the thread context and read the ESP register (RSP for x64). Now do a VirtualQuery on the address in the ESP/RSP register read from each context. The 1MB (default value) region around that address (start at mbi.AllocationBase and work upwards 1MB) is the stack location. Note that the stack size may not be 1MB, you can query this from the PE header of the DLL/EXE that started the thread if you wish.
EDIT, Fix typo where I swapped some register names, thanks #interjay