I was wondering what differences and relations are between
segmentation fault and page fault?
Does segmentation fault only belong to segmented memory model?
Does page fault only belong to paged memory model?
If both are yes, since most computer systems such as x86 and Linux use paged memory model instead of segmented memory model, why does GCC C compiler sometimes report segmentation fault error?
Thanks and regards!
These two things are very dissimilar, actually. A segmentation fault means a program tried to access an invalid or illegal memory address: for example, 0, or a value larger than any valid pointer. A page fault is when a pointer tries to access a page of address space that's currently not mapped onto physical memory, so that the MMU needs to grab it off of disk before it can be used. The former is an illegal condition and the program will generally be aborted; the latter is perfectly normal and the program won't even know about it.
"Segmentation" isn't at all related to the old "segmented memory model" used by early x86 processors; it's an earlier use which just refers to a portion or segment of memory.
Segmentation faults occur when the memory is not allowed to be accessed (does not exist, or is forbidden). Most frequently they occur when you dereference a null variable or run off the end of an array. Page faults occur when memory that is mapped but not loaded is accessed. They are not errors, and signal to the operating system that it should load the appropriate page into memory.
Related
I was curious about how the kernel prevents the stack from growing too big, and I found this Q/A:
Q: how does the linux kernel enforce stack size limits?
A: The kernel can control this due to the virtual memory. The virtual
memory (also known as memory mapping), is basically a list of virtual
memory areas (base + size) and a target physically memory area that
the kernel can manipulate that is unique to each program. When a
program tries to access an address that is not on this list, an
exception happens. This exception will cause a context switch into
kernel mode. The kernel can look up the fault. If the memory is to
become valid, it will be put into place before the program can
continue (swap and mmap not read from disk yet for instance) or a
SEGFAULT can be generated.
In order to decide the stack size limit, the kernel simply manipulates
the virtual memory map. - Stian Skjelstad
But I didn't quite find this answer satisfactory. "When a program tries to access an address that is not on this list, an exception happens." - But wouldn't the text section (instructions) of the program be part of the virtual memory map?
I'm asking about how the kernel enforces the stack size of user programs.
There's a growth limit, set with ulimit -s for the main stack, that will stop the stack from getting anywhere near .text. (And the guard pages below that make sure there's a segfault if the stack does overflow past the growth limit.) See How is Stack memory allocated when using 'push' or 'sub' x86 instructions?. (Or for thread stacks (not the main thread), stack memory is just a normal mmap allocation with no growth; the only lazy allocation is physical pages to back the virtual ones.)
Also, .text is a read+exec mapping of the executable, so there's no way to modify it without calling mprotect first. (It's a private mapping, so doing so would only affect the pages in memory, not the actual file. This is how text relocations work: runtime fixups for absolute addresses, to be fixed up by the dynamic linker.)
The actual mechanism for limiting growth is by simply not extending the mapping and allocating a new page when the process triggers a hardware page fault with the stack pointer below the existing stack area. Thus the page fault is an invalid one, instead of a soft aka minor for the normal stack-growth case, so a SIGSEGV is delivered.
If a program used alloca or a C99 VLA with an unchecked size, malicious input could make it jump over any guard pages and into some other read/write mapping such as .data or stuff that's dynamically allocated.
To harden buggy code against that so it segfaults instead of actually allowing a stack clash attack, there are compiler options that make it touch every intervening page as the stack grows, so it's certain to set off the "tripwire" in the form of an unmapped guard page below the stack-growth limit. See Linux process stack overrun by local variables (stack guarding)
If you set ulimit -s unlimited could you maybe grow the stack into some other mapping, if Linux truly does allow unlimited growth in that case without reserving a guard page as you approach another mapping.
What happens if a process calls out, for code execution, to a far pointer that belongs to another process?
In 32-bit operating mode, the GDT is non-trivially used and segmentation can be fully used.
So, what happens if the requested CS pointer points to another process's segment?
Sometimes this is allowed, I think.
But, generally it's not.
In 64-bit mode, segmentation is halfway turned off.
All segments run from 0 to the top of the processor's linear space.
Also, the GDT seems to be trivially used; Linux, for example, throws all user code from all processes into one segment.
Same for system code.
So, in 64-bit mode, you wouldn't even need a far pointer to call out to another process's code.
Maybe my question can be boiled down to this:
What happens when a process tries to call another process's code?
What OS are you running?
In general, your OS runs your process in protected memory, meaning there is no access to memory not owned by your process. Attempting to access memory outside your virtual memory results in a segmentation fault. To access such memory you would need to be part of the kernel.
I'm creating a block of shared memory using CreateFileMapping and MapViewOfFile thus obtaining a pointer. I then apply QueryWorkingSetEx to to it, the problem is that i keep getting invalid pages in the PSAPI_WORKING_SET_EX_INFORMATION return structure. I'm on a NUMA architecture however the same thing happens on other non-NUMA machines.
If i try the exact same procedure on memory allocated with malloc and get valid results, is it possible that QueryWorkingSetEx does not support shared memory pointers?
after talking with Microsoft's support i was given the solution for this, since QueryWorkingSetEx is being called immediately after MapViewOfFile the memory address hasn't yet been touched so the pages are not yet backed by any physical memory.
The solution is to simply do a read loop over the memory address before QueryWorkingSetEx is invoked, this forces the memory manager to back up the pages with physical memory.
In the case of osx, gcc, modern x86:
How is the x86 segmentation h/w and paging h/w used?
For the most part1, the segmentation hardware isn't used. Most current OSes set CS, DS, SS, and ES to all point to all memory (base address of 0, limit of 4Gig). Each is set to allow full access to all that memory (CS->execute, DS, ES, SS->read/write).
That means nearly all real access control is done with the paging unit. The basic idea is that pages accessible by a particular process are mapped to that process. Pages that are in virtual memory are mapped, but marked not present, so attempting to read/write them will cause an exception; the OS reads the data from the paging file into RAM, marks the data as present, and re-starts the instruction.
As far as how pages are marked, most executable code will be marked read-only, and will be shared between processes. Most data and stack will be marked read/write and will not be shared. Depending on the exact system, stack space will usually have the NX bit set to prevent it from being executed.
There are a few other bits and pieces that are a bit different. For example, most OSes (including OS/X, if memory serves) set up a stack guard page -- a page at the top of the stack that allows no access. When/if you try to access it, the OS catches an exception, allocates another page of stack space, and re-starts the instruction. This means you can allocate (say) 4 megabytes of address space for the stack, but only allocate actual RAM for roughly the space that's been used (obviously in page-sized increments).
The hardware also supports "large" (4 megabyte) pages. These are used primarily for mapping large chunks of contiguous memory like the part of the memory on the graphics card that's directly visible to the CPU.
That's only a very high-level view, but it's hard to provide more detail without knowing what you care about. Trying to cover all the use of paging by an entire OS could occupy an entire (large) book.
1 Windows (unlike most other systems) does make a minimal use of segmentation -- it sets up FS as a pointer to a Thread Information Block (TIB), which gives access to some basic information about the current thread. This is useful (and used) particularly by Windows' Structured Exception Handling (and Vectored Exception Handling).
The more I read about low level languages like C and pointers and memory management, it makes me wonder about the current state of the art with modern operating systems and memory protection. For example what kind of checks are in place that prevent some rogue program from randomly trying to read as much address space as it can and disregard the rules set in place by the operating system?
In general terms how do these memory protection schemes work? What are their strength and weaknesses? To put it another way, are there things that simply cannot be done anymore when running a compiled program in a modern OS even if you have C and you own compiler with whatever tweaks you want?
The protection is enforced by the hardware (i.e., by the CPU). Applications can only express addresses as virtual addresses and the CPU resolves the mapping of virtual address to physical address using lookaside buffers. Whenever the CPU needs to resolve an unknown address it generates a 'page fault' which interrupts the current running application and switches control to the operating system. The operating system is responsible for looking up its internal structures (page tables) and find a mapping between the virtual address touched by the application and the actual physical address. Once the mapping is found the CPU can resume the application.
The CPU instructions needed to load a mapping between a physical address and a virtual one are protected and as such can only be executed by a protected component (ie. the OS kernel).
Overall the scheme works because:
applications cannot address physical memory
resolving mapping from virtual to physical requires protected operations
only the OS kernel is allowed to execute protected operations
The scheme fails though if a rogue module is loaded in the kernel, because at that protection level it can read and write into any physical address.
Application can read and write other processes memory, but only by asking the kernel to do this operation for them (eg. in Win32 ReadProcessMemory), and such APIs are protected by access control (certain privileges are required on the caller).
Memory protection is enforced in hardware, typically with a minimum granularity on the order of KBs.
From the Wikipedia article about memory protection:
In paging, the memory address space is
divided into equal, small pieces,
called pages. Using a virtual memory
mechanism, each page can be made to
reside in any location of the physical
memory, or be flagged as being
protected. Virtual memory makes it
possible to have a linear virtual
memory address space and to use it to
access blocks fragmented over physical
memory address space.
Most computer architectures based on
pages, most notably x86 architecture,
also use pages for memory protection.
A page table is used for mapping
virtual memory to physical memory. The
page table is usually invisible to the
process. Page tables make it easier to
allocate new memory, as each new page
can be allocated from anywhere in
physical memory.
By such design, it is impossible for
an application to access a page that
has not been explicitly allocated to
it, simply because any memory address,
even a completely random one, that
application may decide to use, either
points to an allocated page, or
generates a page fault (PF) error.
Unallocated pages simply do not have
any addresses from the application
point of view.
You should ask Google for Segmentation fault, Memory Violation Error and General Protection Failure. These are errors returned by various OSes in response for a program trying to access memory address it shouldn't access.
And Windows Vista (or 7) has routines for randomized dll attaching, which means that buffer overflow can take you to different addresses each time it occurs. This also makes buffer overflow attack a little bit less repeatable.
So, to link together the answers posted with your question. A program that attempts to read any memory address that is not mapped in its address space, will cause the processor to issue a page fault exception transferring execution control to the operating system code (trusted code), the kernel will then check which is the faulty address, if there is no mapping in the current process address space, it will send the SIGSEV (segmentation fault) signal to the process which typically kills the process (talking about Linux/Unix here), on Windows you get something along the same lines.
Note: you can take a look at mprotect() in Linux and POSIX operating systems, it allows you to protect pages of memory explicitly, functions like malloc() return memory on pages with default protection, which you can then modify, this way you can protect areas of memory as read only (but just in page size chunks, typically around 4KB).