Self modifying algorithms with Virtualprotect problems - windows

Question

I'm having problems with the Virtualprotect() api by windows.
I got an assignment from school, my teacher told us that in the past when memory was scarce and costly. Programmers had to create advanced algorithms that would modify itself on the fly to save memory. So there you have it, we must now write such an algorithm, it doesn't have to be effective but it must modify itself.
So I set out to do just that, and I think that I made it pretty far before asking for any help.
My program works like this:
I have a function and a loop with a built-in stack overflow. The stack gets overflown with the address of a memory location where code resides that is constructed during the loop. Control is passed to the code in memory. The code loads a dll and then exits, but before it exits it has to repair the loop. It is one of the conditions of our assignment, everything changed in the original loop must be restored.
The problem is that I don't have write access to the loop, only READ_EXECUTE, so to change my access I thought, I use virtualprotect. But that function returned an error:
ERROR_NOACCESS, the documentation on this error is very slim, windows only says: Invailid access to memory address. Which figures since I wanted to change the access in the first place. So what's wrong? Here's the code constructed in memory:
The names of all the data in my code is a little vague, so I provided a few comments
Size1:
TrapData proc
jmp pLocals
LocalDllName db 100 dup(?) ; name of the dll to be called ebx-82h
RestoreBuffer db 5 dup(?) ; previous bytes at the overflow location
LoadAddress dd 0h ; ebx - 19h ; address to kernel32.loadlibrary
RestoreAddress dd 0h ; ebx - 15h ; address to restore (with the restore buffer)
AddressToRestoreBuffer dd 0h ; ebx - 11h ; obsolete, I don't use this one
AddressToLea dd 0h ; ebx - 0Dh Changed, address to kernel32.virutalprotect
AddressToReturnTo dd 0h ; ebx - 9h address to return execution to(the same as RestoreAddress
pLocals:
call Refpnt
Refpnt: pop ebx ; get current address in ebx
push ebx
mov eax, ebx
sub ebx, 82h
push ebx ; dll name
sub eax, 19h ; load lib address
mov eax, [eax]
call eax
pop ebx ; Current address
push ebx
;BOOL WINAPI VirtualProtect(
; __in LPVOID lpAddress,
; __in SIZE_T dwSize,
; __in DWORD flNewProtect,
; __out PDWORD lpflOldProtect
;);
mov eax, ebx
mov esi, ebx
sub eax, 82h
push eax ; overwrite the buffer containing the dll name, we don't need it anymore
push PAGE_EXECUTE_READWRITE
push 5h
sub esi, 15h
mov esi, [esi]
push esi
sub ebx, 0Dh
mov ebx, [ebx]
call ebx ; Returns error 998 ERROR_NOACCESS (to what?)
pop ebx
push ebx
sub ebx, 1Eh
mov eax, ebx ; restore address buffer pointer
pop ebx
push ebx
sub ebx, 15h ; Restore Address
mov ebx, [ebx]
xor esi, esi ; counter to 0
#0:
push eax
mov al, byte ptr[eax+esi]
mov byte ptr[ebx+esi], al
pop eax
inc esi
cmp esi, 5
jne #0
pop ebx
sub ebx, 9h
mov ebx, [ebx]
push ebx ; address to return to
ret
Size2:
So what's wrong?
Can you guys help me?
EDIT, Working code:
Size1:
jmp pLocals
LocalDllName db 100 dup(?)
RestoreBuffer db 5 dup(?)
LoadAddress dd 0h ; ebx - 19h
RestoreAddress dd 0h ; ebx - 15h
AddressToRestoreBuffer dd 0h ; ebx - 11h
AddressToLea dd 0h ; ebx - 0Dh
AddressToReturnTo dd 0h ; ebx - 9h
pLocals:
call Refpnt
Refpnt: pop ebx ; get current address in ebx
push ebx
mov eax, ebx
sub ebx, 82h
push ebx ; dll name
sub eax, 19h ; load lib address
mov eax, [eax]
call eax
pop ebx ; Current address
push ebx
;BOOL WINAPI VirtualProtect(
; __in LPVOID lpAddress,
; __in SIZE_T dwSize,
; __in DWORD flNewProtect,
; __out PDWORD lpflOldProtect
;);
mov esi, ebx
push 0
push esp
push PAGE_EXECUTE_READWRITE
push 5h
sub esi, 15h
mov esi, [esi]
push esi
sub ebx, 0Dh
mov ebx, [ebx]
call ebx
pop ebx
pop ebx
push ebx
sub ebx, 1Eh
mov eax, ebx ; restore address buffer pointer
pop ebx
push ebx
sub ebx, 15h ; Restore Address
mov ebx, [ebx]
xor esi, esi ; counter to 0
#0:
push eax
mov al, byte ptr[eax+esi]
mov byte ptr[ebx+esi], al
pop eax
inc esi
cmp esi, 5
jne #0
pop ebx
sub ebx, 9h
mov ebx, [ebx]
push ebx ; address to return to
ret
Size2:
Maybe a little sloppy, but I that doesn't mater ;)

Answer 1

You are trying to make VirtualProtect write lpflOldProtect to a read-only memory location, i.e. your current code section which is what you're trying to unprotect in the first place! My guess is this is what gives you the ERROR_NO_ACCESS. Since you're using the stack anyway, have it write lpflOldProtect to a stack location.

Answer 2

This isn't nearly as easy as it was in the old days; read access used to imply execute access, and a lot of memory mappings were mapped writable.
These days, I'd be surprised if there are many (any?) memory mappings that are both writable and executable. (And modern CPUs with PAE support are sufficient for even 32-bit kernels to provide non-executable-yet-readable mappings.)
I'd say, first things first, find an older Windows system, Win2k or earlier, then start trying to tackle this problem. :)
EDIT: Oh! I thought loading the DLL failed. Good work. :)
What do you mean by 'restore the loop'? Since you smashed the stack to jump to your code, you didn't really destroy the loop's text segment, you only scribbled on the stack. You could insert another function before your loop, then return from your dll to the function that called your loop. (You 'returned' into your injected code from the loop, so you can't return into the loop without building a fake stack frame for it; returning to the previous function seems easier than building a fake stack frame.)