I have gone through with memory managment concepts of Operating system concept of Galvin , I have read a statment :
If you know at compile time where the process will reside in memory, then absolute code can be generated.
How at compile time processor got to know at which memory location in main memory process is going to store.
Can someone explain , what is the exactly does it means , if we know at compile time where process will reside in memory ,
As memory be allocated when program is moving from ready to running state .
Generally, machine code isn't position-independent. In order to be able to load it at an arbitrary starting address and run there one needs some extra information about the machine code (e.g. where it has addresses to the various parts of itself), so it can be adjusted to the arbitrary position.
OTOH, if the code is always going to be loaded at the same fixed address, you don't need any of that extra information and processing.
By absolute he means fixed + final, already adjusted to the appropriate address.
The processor does not "know" anything. You "tell" it.
I don't know exactly what he means with "absolute code", depending which operating system you use, the program with it's code and data will be loaded to a virtual address and executed from there.
Beside of this not the compiler but the linker sets the address where the program will be loaded to.
Modern operating systems like Linux are using Address Space Layout Randomization to avoid having one static address where every program is loaded and to avoid the possibility of exploiting software flaws.
If you're writing your own operating system maybe the osdev.org wiki could be a good ressource for you. If you can read/speak german I recommened lowlevel.eu either.
Related
Wikipedia says this about relocations:
PE files normally do not contain position-independent code. Instead
they are compiled to a preferred base address, and all addresses
emitted by the compiler/linker are fixed ahead of time. If a PE file
cannot be loaded at its preferred address (because it's already taken
by something else), the operating system will rebase it. This involves
recalculating every absolute address and modifying the code to use the
new values. The loader does this by comparing the preferred and actual
load addresses, and calculating a delta value. This is then added to
the preferred address to come up with the new address of the memory
location. Base relocations are stored in a list and added, as needed,
to an existing memory location.
I am confused as to why there would be anything else at 0x00400000 (default preferred base address) besides the base address for the process. It is my understanding that in virtual memory, the process has the view of an empty memory space in which it is the only thing that exists. With this in mind, how would anything be there before the process itself initially?
As a matter of fact, in most cases, when a process starts there is no issue regarding the preferred base address! In some situations like "Process Hollowing" (a technique where an application replaces another one in memory), the preferred base address is an important issue that must be handled. See following link for more (low-level) technical details about this issue related to the preferred address.
Introduction to Process Hollowing
I'm trying to count hits of a specific set of pages, by hacking the reference bits in the page table. Is there any system call or any other way to unset reference bits (in UNIX-like systems)?
A page table is the data structure used by a virtual memory system in a computer operating system to store the mapping between virtual addresses and physical addresses. (https://en.wikipedia.org/wiki/Page_table)
In unix-like systems there is a bit associated with each page table entry, called "reference" bit, which indicates if a page was accessed since the bit was unset.
The linux kernel unsets these reference bits periodically and checks a while after that to know what pages have been accessed, in order to detect "hot" pages. But this information is very coarse grain and low-precision since it doesn't say anything about the number of accesses and their time.
I want to count accesses to specific pages during shorter epochs by unsetting reference bits then check if the pages have been accessed after a short time.
Therefore, I was wondering if there is any system call or CPU interrupt which provides means to unset "reference bits". Otherwise, I need to dive deep into kernel to see what goes on down there.
There is no API for resetting the page reference bits. Page management is a very twitchy aspect of kernel tuning and no one wants to upset it. Of course you could modify the kernel to your needs.
Instead, you might look into Valgrind which is a debugging and profiling tool for running a single program. Ordinarily it detects subtle memory errors such as detecting use of a dynamic memory block after it has been freed.
If you need page management information for the system as a whole, I think the most expedient solution is hacking the kernel.
Is it possible to build a position independant code for windows ?
I am not talking about a dll, i am talking about an executable (PE).
What i want to do is that functions (main for example) which are in the program should be mapped at a different memory address between 2 executions.
Thanks
You should read up on the image base relocation directory in a PE file. I was wondering the same thing you are asking now a while back. From the research I had done I couldn't find any compiler that generates completely base independent code. I ended up writing my program entirely in x86 asm with some tricks to load strings etc independent form the address. I can tell you its not worth the trouble. Just remap all the addresses to your target location.
I suggest you start reading this
https://msdn.microsoft.com/en-us/library/ms809762.aspx
You should also check out Icezilon's (not sure how you spell his name) tutorials. They're very helpful
xmalloc can be used in the process environment only when I write a AIX kernel extension.
what's the memory allocation functions can be called from the interrupt environment in AIX?
thanks.
The network memory allocation routines. Look in /usr/include/net/net_malloc.h. The lowest level is net_malloc and net_free.
I don't see much documentation in IBM's pubs nor the internet. There are a few examples in various header files.
There is public no prototype that I can find for these.
If you look in net_malloc.h, you will see MALLOC and NET_MALLOC macros defined that call it. Then if you grep in all the files under /usr/include, you will see uses of these macros. From these uses, you can deduce the arguments to the macros and thus deduce the arguments to net_malloc itself. I would make one routine that is a pass through to net_malloc that you controlled the interface to.
On your target system, do "netstat -m". The last bucket size you see will be the largest size you can call net_malloc with the M_NOWAIT flag. M_WAIT can be used only at process time and waits for netm to allocate more memory if necessary. M_NOWAIT returns with a 0 if there is not enough memory pinned. At interrupt time, you must use M_NOWAIT.
There is no real checking for the "type" but it is good to pick an appropriate type for debugging purposes later on. The netm output from kdb shows the type.
In a similar fashion, you can figure out how to call net_free.
Its sad IBM has chosen not to document this. An alternative to get this information officially is to pay for an "ISV" question. If you are doing serious AIX development, you want to become an ISV / Partner. It will save you lots of heart break. I don't know the cost but it is within reach of small companies and even individuals.
This book is nice to have too.
i've googled a lot about what is "page share factor per proc" responsible for and found nothing. It's just interesting for me, i have no current problem with it for now, just curious (wnat to know more). In sysctl it is:
vm.pmap.shpgperproc
Thanks in advance
The first thing to note is that shpgperproc is a loader tunable, so it can only be set at boot time with an appropriate directive in loader.conf, and it's read-only after that.
The second thing to note is that it's defined in <arch>/<arch>/pmap.c, which handles the architecture-dependent portions of the vm subsystem. In particular, it's actually not present in the amd64 pmap.c - it was removed fairly recently, and I'll discuss this a bit below. However, it's present for the other architectures (i386, arm, ...), and it's used identically on each architecture; namely, it appears as follows:
void
pmap_init(void)
{
...
TUNABLE_INT_FETCH("vm.pmap.shpgperproc", &shpgperproc);
pv_entry_max = shpgperproc * maxproc + cnt.v_page_count;
and it's not used anywhere else. pmap_init() is called only once: at boot time as part of the vm subsystem initialization. maxproc, is just the maximum number of processes that can exist (i.e. kern.maxproc), and cnt.v_page_count is just the number of physical pages of memory available (i.e. vm.stats.v_page_count).
A pv_entry is basically just a virtual mapping of a physical page (or more precisely a struct vm_page, so if two processes share a page and both have them mapped, there will be a separate pv_entry structure for each mapping. Thus given a page (struct vm_page) that needs to be dirtied or paged out or something requiring a hw page table update, the list of corresponding mapped virtual pages can be easily found by looking at the corresponding list of pv_entrys (as an example, take a look at i386/i386/pmap.c:pmap_remove_all()).
The use of pv_entrys makes certain VM operations more efficient, but the current implementation (for i386 at least) seems to allocate a static amount of space (see pv_maxchunks, which is set based on pv_entry_max) for pv_chunks, which are used to manage pv_entrys. If the kernel can't allocate a pv_entry after deallocating inactive ones, it panics.
Thus we want to set pv_entry_max based on how many pv_entrys we want space for; clearly we'll want at least as many as there are pages of RAM (which is where cnt.v_page_count comes from). Then we'll want to allow for the fact that many pages will be multiply-virtually-mapped by different processes, since a pv_entry will need to be allocated for each such mapping. Thus shpgperproc - which has a default value of 200 on all arches - is just a way to scale this. On a system where many pages will be shared among processes (say on a heavily-loaded web server running apache), it's apparently possible to run out of pv_entrys, so one will want to bump it up.
I don't have a FreeBSD machine close to me right now but it seems this parameter is defined and used in pmap.c, http://fxr.watson.org/fxr/ident?v=FREEBSD7&im=bigexcerpts&i=shpgperproc