I have been playing a while with ptrace. I followed some tutorials like this one or this one. So far, when I have a ptrace-d child process, I am able to:
Detect system calls and browse the registers.
Fetch the strings contained in addresses pointed by the registers, thanks to the PTRACE_PEEKDATA option of ptrace.
Change the values of those registers and change memory values in the user space of the child process thanks to the PTRACE_POKEDATA option of ptrace.
My problem is the following: let's say that for example I have just detected an open system call. I can modify the filename of the file to be opened thanks to the address stored in the ebx register. However, I wonder if I can just change the filename to anything I want, any size. If the name I am changing to is really large (let's say 50 times the original filename length), wouldn't I be messing with some memory I should not be writing on? Should I 'allocate' some memory in the child's memory space? If so, how would this be done?
Note that the child process is some program executed with execve, I cannot access its source code.
The pathname passed to open could be dynamically allocated by the program (so its on the heap or stack somewhere), or it could be in the read-only section if it was a compile-time constant. In either case, you don't know what other parts of the program might be using it, so its probably not a good idea to change its contents. You would definitely overwrite adjacent memory if you wrote past the current length (which would probably lead to subtle problems like corrupting heap meta-data or corrupting other random allocation objects).
Here are some random ideas (totally untested) on how to allocate memory in a child process:
invoke an mmap syscall on its behalf (this would probably be pretty tricky) but would get you a page (or more) of memory to play with
allocate some space in the current stack (don't change the child's registers, but use your knowledge of which part of the stack the child is using to put temporary objects in the unused section). Technically its legal for the child process to do this same thing (so you could end up corrupting that data), but its very unlikely.
hide stuff at the far end of the stack, (again assuming the child isn't also playing this trick).
I didn't think invoking malloc would be easy, but googling for 'ptrace child allocate memory' I found: http://www.hick.org/code/skape/papers/needle.txt (which finds the malloc routine used by the ELF dynamic linker and constructs a call out to there to allocate memory).
Related
Is it possible in Win32 to get a writeable (or write-only) range of "garbage" virtual address space (i.e., via VirtualAlloc, VirtualAlloc2, VirtualAllocEx, or other) that never needs to be persisted, and thus ideally is never backed by physical memory or pagefile?
This would be a "hole" in memory.
The scenario is for simulating a dry-run of a sequential memory writing operation just in order to obtain the size that it actually consumes. You would be able to use the exact same code used for actual writing, but instead pass in an un-backed "garbage" address range that essentially ignores or discards anything that's written to it. In this example, the size of the "void" address range could be 2⁶⁴ = 18.4ᴇʙ (why not? it's nothing, after all), and all you're interested in is the final value of an advancing pointer.
[edit:] see comments section for the most clever answer. Namely: map a single 4K page multiple times in sequence, tiling the entire "empty" range
This isn't possible. If you have code that attempts to write to memory then the virtual memory needs to be backed with something.
However, if you modified your code to use the stream pattern then you could provide a stream implementation that ignored the write and just tracked the size.
I am learning about destructors right now because I am making this assignment about matrices (we're supposed to make a Matrix class and overload operators to do Matrix operations and me and the person I am going to mention in the next bit were planning to also make it perform Gauss-Jordan elimination, if this is relevant) which are represented in this assignment through dynamic 2D arrays.
I heard someone talk about using a destructor for the deletion process of the arrays. I started reading about destructors and one of the events that calls a destructors that seemed like the only time a destructor would be used in an application like this was the termination of the program, so I am left kind of confused as to why he'd need a destructor? What's the significance of a destructor in an application like this?
The answer to the question in the title is:
Yes. And No.
No:
If the process creates an object with new and terminates without calling delete on that same object, the object is never destructed. Any action that would be done by the destructor is simply not done.
This action can be stuff that is required for consistency of external data. Like pushing something to a database. Or like flushing a cache to disk. What action is missed depends entirely on the destructor.
Yes:
The memory that was occupied by the process is not lost to the system. Your process requested some chunks of memory from the system's kernel, so that it was able to construct its objects within that memory. The kernel keeps track of which memory pages it has allocated to which process, and it does not care a bit what that process does with it. The kernel is entirely oblivious to which objects were constructed within the memory.
When a process exits, the kernel will simply reclaim any memory that's still allocated to the process. As such, you don't permanently loose memory by forgetting to delete objects at shutdown.
However, this reclaiming affects memory use, only. The contents of any cache that wasn't flushed remains unflushed. And the external files that were in an inconsistent state when the process terminated will remain in that inconsistent state forever.
So, bottom line: Memory will be reclaimed by the kernel anyway. But it's generally not a good idea to forget cleanup. It's better not to get into the habit of being lazy, because that habit will bite you severely down the road.
When a fork is called, the stack and heap are both copied from the parent process to the child process. Before using the fork system call, I malloc() some memory; let's say its address was A. After using the fork system call, I print the address of this memory in both parent and child processes. I see both are printing the same address: A. The child and parent processes are capable of writing any value to this address independently, and modification by one process is not reflected in the other process. To my knowledge, addresses are globally unique within a machine.
My question is: Why is it that the same address location A stores different values at the same time, even though the heap is copied?
There is a difference between the "real" memory address, and the memory address you usually work with, i.e. the "virtual" memory address. Virtual memory is basically just an abstraction from the Operating System in order to manage different pages, which allows the OS to switch pages from RAM into HDD (page file) and vice versa.
This allows the OS to continue operating even when RAM capacity has been reached, and to put the relevant page file into a random location inside RAM without changing your program's logic (otherwise, a pointer pointing to 0x1234 would suddenly point to 0x4321 after a page switch has occured).
What happens if you fork your process is basically just a copy of the page file, which - I assume - allows for smarter algorithms to take place, such as copying only if one process actually modifies the page file.
One important aspect to mention is that forking should not change any memory addresses, since (e.g. in C) there can be quite a bit of pointer logic in your application, relying on the consistency of the memory you allocated. If the addresses were to suddenly change after forking, it would break most, if not all, of this pointer logic.
You can read more on this here: http://en.wikipedia.org/wiki/Virtual_memory or, if you're truly interested, I recommend reading "Operating Systems - Internals and Design Principles" by William Stallings, which should cover most things including why and how virtual memory is used. There is also an excellent answer to this in this StackOverflow thread. Lastly, you might want to also read answers from this, this and this question.
I directly started with managed languages and have barely any experience with C++, hence this question might be too basic.
In a managed language like .net, GC frees the memory. From what I read, in C++ this is done by calling delete. But what does it do to free memory? Does it it set all the bits at a memory location to zero? Or does it in some other way tell the operating system that the memory is available for reuse?
Update:
I have been thru this before and I know what GC does. But thats not my question. I am not trying to ask how GC works. What I am trying to understand is, how do you tell some memory is free?
delete does three different things:
Runs the destructor of the object (or of all objects in the array in the case of delete[]).
Marks the chunk of memory previously used by the object as free.
If possible, informs the operating system that a chunk of memory is free for other programs to use.
Your question is about #2 and #3 together, but they are very different things. To understand how #2 works, remember that the (typically) single "heap" provided by the operating system is segmented into smaller chunks of different sizes. When you allocate a chunk of memory with new, you get a pointer to a previously free part of the heap, and the runtime performs the necessary bookkeeping that marks that region as unavailable for further allocations. delete does the reverse: it performs the bookkeeping that marks the region as available again, optionally coalescing it with adjacent free regions to reduce fragmentation. Subsequent calls to new will consider that region when looking for free memory to return.
In other words, it is wrong to ask what happens with the memory when it is deallocated. The real magic happens in the bookkeeping region! To learn about implementation of generic allocators, google for implementation of malloc.
As for #3, it is an optional step and in many cases impossible to perform. It is only possible to "give back" freed memory that happens to reside at the very end of the allocated heap. A single allocation situated after a large region will remove the possibility of giving back.
In C++, if you allocate memory using "new", that portion of memory will be allocated by the OS to that particular process, until you release that memory or until that process exits.
If some portion of memory allocated for a process means that OS does not allow other process to use that portion of memory until that process release that memory. In C++, you have to use "delete" to release memory.
By releasing memory portion, process just inform the OS that it does not use this portion any more so that OS can allocate that memory portion whenever other processes request memory. In that case content of the memory portion will not be changed.
Garbage collection is just automatic memory management (so you never need to delete anything, the system will take care of it for you). I'm not 100% on whether it sets memory locations to 0, but I would assume it doesnt, since when you delete in c++, thats not what happens, it just allows the space to be used for storage. Writing zeros over everything is much more inefficient and not necessary. Here's some links that might be able to help explain this more thoroughly:
How does garbage collection and scoping work in C#?
http://en.wikipedia.org/wiki/Garbage_collection_(computer_science)
Inside each application, dynamic memory are managed by "heaps". When your code asks for a block of memory, it asks the heap manager to allocate a block of memory, when your application frees that block of memory, it returns it back to the heap manager. In a traditional application, you must explicitly return each memory you allocated. Otherwise you will eventually run out of memory.
In languages like C# or Java, the runtime offers a garbage collector. A garbage collector automatically identify "unreachable" memory block and free them. An unreable memory block is a block that is no longer referenced by any variables. For example, if you have a global variable p1 that points to a block of memory, because p1 is global, so it is visible to anywhere in your code, then it is always reachable. Thus it will never be released by garbage collector. On the other hand, if you have local variable p2 in one of your function Foo, vairable p2 is no longer reachable after Foo has returned. The garbage collector is able to identify such variables and free any memory block pointed by them.
As application/garbage collector interact with the heap, the heap may decide to ask for more memory from the OS or return it to the OS. The OS manages all these memory request from different process and it then decide how to allocate the actual physical memory to different process.
No, it does not set the bits to zero. In a very simplified explanation,
First the garbage collector must determine, not what objects are no longer accessible ("not reachable"), but which ones are still accessible or reachable. It does this by simply listing all object roots. A root is a memory location containing a pointer to a reference object (an object on the heap). Then, recursively, it flags as "reachable" every object referenced by a root, or referenced by a field or property of a object already flagged as reachable.
There are four types of roots.
static variables containing reference objects
reference objects on the stack for any currently active thread.
reference types in method parameters
reference objects pointed to by CPU registers.
After determining what reference objects are still accessible (reachable) by any code in the App Domain, it takes all those objects that are still reachable, and if there are any gaps in physical memory between them, it "defragments" them by moving some of them so they are all contiguous, then it sets the pointer which represents the "end" od "used" memory to the end of this new compressed defragmented list. All new memory allocations, for newly instantiated objects, are then allocated from the memory immediately after this pointer location.
If there are no gaps in the memory used by the reachable objects, it just resets the pointer to the end of the last reachable object in the list.
No, deleting a pointer does not set the bytes to zero.
It's not in the standard of course, but it would be a performance overhead and serious implementations don't bother doing it, and it does not even make sense, when the memory is used for complex objects (floats, objects, strings, etc)
You can always try it out.
Declare a pointer to an int, write an integer, delete the pointer.
Then read the content of the deleted pointer again.
Does it have the same content?
int *ptr = new int;
*ptr = 13;
cout << "Before delete: " << *ptr << endl;
delete ptr;
cout << "After delete: " << *ptr << endl;
Yes probably it will, BUT ptr is just a dangling pointer you have there, the memory has been returned to the system and it will be available when you allocate memory again, it's likely that when you allocate another int* it will be pointing where ptr was pointing.
Let's say I have an allocation in memory containing a string, "ABCDEFG", but I only have a pointer to the 'E'. Is it possible, on win32, to free that block, given a pointer that is within the block, but not at the start? Any allocation method would work, but a Heap* function would be the path of least resistance.
If not a native solution, have there been any custom memory managers written which offer this feature?
EDIT: This isn't an excuse to be sloppy. I'm developing an automatic memory management system using 100% compile-time metadata. This odd requirement seems to be the only thing standing in the way of getting it working, and even then it's needed only for data types based on arrays (which are slicable).
It would be possible for the memory allocation routines in the runtime library to check a given memory address against the beginning and end of every allocated block. That search accomplished, it would be easy to release the block from the beginning.
Even with clever algorithms behind it, this would incur some kind of search with each memory deallocation. And why? Just to support erroneous programs too stupid to keep track of the beginning of the blocks of memory they allocated?
The standard C idiom thrives on treating blocks of allocated memory like arrays. The pointer returned from *alloc is a pointer to the beginning of an array, and the pointer can be used with subscripts to access any element of that array, subscripts starting at 0. This has worked well enough for 40 years that I can't think of a sensible reason to introduce a change here.
I suppose if you know what the malloc() guard blocks look like, you could write a function that backs up from the pointer you pass it until it finds a 'best guess' of the original memory address and then calls free(). Why not just keep a copy of the base pointer around?
If you use VirtualAlloc to allocate memory, you can use VirtualQuery to figure out which block a pointer belongs to. Once you have the base address, you can pass this to VirtualFree to free the entire block.