masm call procedure access violation - visual-studio

So I am working on an assignment in assembly to generate a fibonacci sequence. I've written the code successfully in the main procedure but when I try to wrap it in it's own procedure and call that procedure I run into an access violation error. Here's my code:
INCLUDE Irvine32.inc
.data
array DWORD 47 DUP(?)
.code
main proc
mov esi, OFFSET array
call generate_fibonacci
invoke ExitProcess,0
main endp
generate_fibonacci PROC
mov DWORD PTR [esi], 1h
add esi, 4
mov DWORD PTR [esi], 1h
push [esi]
push [esi - 4]
add esi, 4
mov ecx, 45
L1:
pop eax
pop ebx
add eax, ebx
mov DWORD PTR [esi], eax
add esi, 4
push [esi - 4]
push [esi - 8]
loop L1
ret
generate_fibonacci ENDP
end main
The error looks like this: "Exception thrown at some memory location in Project...: Access violation executing location same memory location.
I noticed that the memory location listed in the error message was being loaded onto the EIP register when the call generate_fibonacci instruction is executed. I'm not sure how to fix this.

The pushes and pops in your PROC are not balanced.
Before loop L1: you make 2 pushes. Within the loop L1: you make 2 pops and 2 pushes. When loop L1: ends, that leaves 2 items still on the stack when ret attempts to pull off the return address. So the code tries to resume execution somewhere that causes an access violation.
Please add two lines of code before the ret instruction to clean up the stack
pop eax
pop eax
ret
If the same code worked when it was in main, it worked because main does not end with ret.
EDIT. You could simplify it considerably by keeping the recent terms in registers. The last three terms will be in eax, ebx, edx.
generate_fibonacci PROC
mov eax, 1 ;init first two terms
mov DWORD PTR [esi], eax ;store first two terms
add esi, 4
mov DWORD PTR [esi], eax
add esi, 4
mov ebx, eax
mov ecx, 45 ;init loop count
L1:
mov edx, ebx ;move terms along
mov ebx, eax
add eax, edx ;sum last two terms
mov DWORD PTR [esi], eax
add esi, 4
loop L1
ret
generate_fibonacci ENDP

Related

segmentation fault in x86 trying to do bubble sort

I am trying to implement bubble sort in assembly. Here is my code. I keep getting segmentation fault. I have a function down below. I have been trying to figure this out but I couldn't find a compiler for x86 and I have been checking with my code to check what is wrong but to no avail.
here is my function code:
bubble:
push ebp
mov ebp, esp
push ecx
push edx
push edi
push esi
push ebx
push eax
mov eax, 0
mov ecx, [ebp+12] ; number of elements in the array
mov edx, [ebp+8]; address of array
mov edi, 0
mov esi, 0
dec ecx; n - 1
mov esi, 1
sortOuter:
cmp esi, 0
jg sort
sort:
mov esi, 0 ;count
check:
cmp edi, ecx ; i < n - 1
jl sortInner
sortInner:
mov ebx, [edx+edi*4] ; mov array[i+1] to ebx
cmp [edx], ebx ; cmp array[i] to array[i+1]
jle noswap
swap:
mov eax, ebx ; mov array[i+1] to eax
mov ebx, [edx] ; mov array[i] to array[i+1]
mov [edx], eax ; mov array[i+1] to array[i]
inc esi ; count++
noswap:
inc edi ; i++
jmp check
jmp sortOuter
done:
call print_nl
pop ebx
pop esi
pop edi
pop edx
pop ecx
mov esp, ebp
pop ebp
ret
The segmentation error comes from the infinite loop that you have created with
check:
cmp edi, ecx ; i < n - 1
jl sortInner
sortInner:
If EDI is less than ECX you jump to sortInner, but if it isn't you fall-through into sortInner. No matter, you always end up running the code at sortInner and because the memory addresses that the code uses keep growing, at some point you will be trying to read memory that you don't have access to, hence the segmentation error.
Now if you were to correct this, then there's a second infinite loop waiting.
sortOuter:
cmp esi, 0
jg sort
sort:
Other errors include:
Resetting ESI instead of EDI at the start of the inner loop
Not swapping elements at all but always writing the smallest value in the first array element
Forgetting to restore the register EAX
This is a working BubbleSort. Don't just copy it, but find out how it functions!
In an array with N elements we have to do N-1 comparisons at most (to have the greatest value bubble towards the rear).
Because the InnerLoop uses a counter that gets initialized from the ever decreasing OuterLoop counter, with each iteration of the OuterLoop the portion of the array that is processed in the InnerLoop gets smaller. The portion of the array that we then no longer process contains the greatest elements that have bubbled towards the end of the array, hence the name BubbleSort.
Provisions have been made for an empty array or an array that has but 1 element. Always include some code for the special cases!
bubble:
push ebp
mov ebp, esp
push ...
mov ecx, [ebp + 12] ; Number of elements in the array
sub ecx, 1 ; First time we do n = N-1
jbe Done ; Array is empty or has but 1 element
OuterLoop:
mov edx, ecx ; Current value of the OuterLoop counter (n)
mov esi, [ebp + 8] ; Address of array
InnerLoop:
mov eax, [esi]
mov ebx, [esi + 4]
cmp eax, ebx
jng NoSwap
mov [esi], ebx
mov [esi + 4], eax
NoSwap:
add esi, 4
dec edx
jnz InnerLoop
dec ecx ; Next times we do n = n-1
jnz OuterLoop
Done:
pop ...
pop ebp
ret

Sorting a list through a procedure in flat assembly

I'm pretty new to fasm and I just recently started learning about procedures. My problem is that I have a proc and I want it to sort my list in a certain way. But when I run my code it just seems to sort some random numbers from memory. I don't quite know why it happens and I would appreciate any help.
Here's the code:
format PE gui 5.0
include 'D:\Flat Assembler\INCLUDE\win32a.inc'
entry start
section '.data' data readable writable
mas1 dw 2, -3, 1, -1, 3, -2, 5, -5, -4, 4
N = ($ - mas1) / 2
numStr db N dup('%d '), 0
strStr db '%s', 0
undefStr db 'undefined', 0
buff db 50 dup(?)
Caption db 'Result', 0
section '.code' code readable executable
start:
stdcall bubble, mas1
cinvoke wsprintf, buff, numStr
invoke MessageBox, 0, buff, Caption, MB_OK + MB_ICONINFORMATION
invoke ExitProcess, 0
proc bubble, mas:word
mov ecx, 0
mov ebx, 0
outerLoop:
cmp ecx, 10
je done
mov ebx, 2
innerLoop:
mov eax, 0
mov edx, 0
cmp [mas+ebx], 0 ;if(mas[j] > 0)
jge continue ;continue
mov ax, [mas+ebx-2]
cmp ax, [mas+ebx]
jle continue
mov dx, [mas+ebx]
mov [mas+ebx-2], dx
mov [mas+ebx], ax
continue:
cmp ebx, 18 ;10
je innerDone
add ebx, 2 ;inc ebx
jmp innerLoop
innerDone:
inc ecx
jmp outerLoop
done:
mov ecx, 0
mov ebx, 0
mov ebx, 18
mov ecx, N
print:
mov eax, 0
mov ax, [mas+ebx]
cwde
push eax
sub ebx, 2
loop print
ret
endp
section '.idata' import data readable writeable
library kernel32,'KERNEL32.DLL',\
user32,'USER32.DLL'
include 'D:\Flat Assembler\INCLUDE\API\kernel32.inc'
include 'D:\Flat Assembler\INCLUDE\API\user32.inc'
Error 1
stdcall bubble, mas1
...
proc bubble, mas:word
The parameter mas1 is an address and is pushed to the stack as a dword. Therefore you should not limit the argument mas to a word.
What your bubble procedure needs is the full address of the array. You get this via mov esi, [mas] that FASM will encode as if you would have written mov esi, [ebp+8]. EBP+8 is where the first argument (and in your program the only argument) resides, when the standard prologue push ebp mov ebp, esp is used.
Error 2
In your bubble procedure you push the resulting array to the stack hoping to have wsprintf use it from there, but once the bubble procedure executes its ret instruction, the epilogue code as well as the ret instruction itself will start eating your array and even return to the wrong address in memory!
If you're going to return an array via the stack, then store it above the return address and the argument(s). That's why I wrote in my program below:
sub esp, N*4 ; Space for N dwords on the stack
stdcall bubble, mas1
Error 3
cmp [mas+ebx], 0 ;if(mas[j] > 0)
jge continue ;continue
Your BubbleSort is wrong because you don't allow positive numbers to get compared!
Furthermore you make too many iterations that also continu for too long.
I tested below program on FASM 1.71.22 Don't forget to change the paths!
format PE gui 5.0
include 'C:\FASM\INCLUDE\win32a.inc'
entry start
section '.data' data readable writable
mas1 dw 2, -3, 1, -1, 3, -2, 5, -5, -4, 4
N = ($ - mas1) / 2
numStr db N-1 dup('%d, '), '%d', 0
;strStr db '%s', 0
;undefStr db 'undefined', 0
buff db 50 dup(?)
Caption db 'Result', 0
section '.code' code readable executable
start:
sub esp, N*4 ; Space for N dwords on the stack
stdcall bubble, mas1
cinvoke wsprintf, buff, numStr
invoke MessageBox, 0, buff, Caption, MB_OK + MB_ICONINFORMATION
invoke ExitProcess, 0
proc bubble uses ebx esi, mas
mov esi, [mas] ; Address of the array
mov ecx, (N-1)*2 ; Offset to the last item; Max (N-1) compares
outerLoop:
xor ebx, ebx
innerLoop:
mov ax, [esi+ebx]
mov dx, [esi+ebx+2]
cmp ax, dx
jle continue
mov [esi+ebx+2], ax
mov [esi+ebx], dx
continue:
add ebx, 2
cmp ebx, ecx
jb innerLoop
sub ecx, 2
jnz outerLoop
mov ebx, (N-1)*2
toStack:
movsx eax, word [esi+ebx]
mov [ebp+12+ebx*2], eax
sub ebx, 2
jnb toStack
ret
endp
section '.idata' import data readable writeable
library kernel32,'KERNEL32.DLL',\
user32,'USER32.DLL'
include 'C:\FASM\INCLUDE\API\kernel32.inc'
include 'C:\FASM\INCLUDE\API\user32.inc'
Error 2 revisited
IMO returning the resulting array through the stack would make better sense if your bubble procedure didn't modify the original array.
But in your present code you do, so...
Once you strike the toStack snippet from the bubble procedure, you can simply (after returning from the bubble procedure) push the word-sized elements of the array to the stack as dwords followed by using wsprintf.
...
start:
stdcall bubble, mas1
mov ebx, (N-1)*2
toStack:
movsx eax, word [mas1+ebx]
push eax
sub ebx, 2
jnb toStack
cinvoke wsprintf, buff, numStr
...
sub ecx, 2
jnz outerLoop
; See no more toStack here!
ret
endp
...

Finding Smallest Number in List

My goal in this code is to find the smallest number in the list. I used bubble sort method in this case; unfortunately, the code is not giving me the smallest/minimum number. Please take a look, Thanks:
include irvine32.inc
.data
input byte 100 dup(0)
stringinput byte "Enter any string: ",0
totallength byte "The total length is: ",0
minimum byte "The minimum value is: ",0
.code
stringLength proc
push ebp
mov ebp, esp
push ebx
push ecx
mov eax, 0
mov ebx, [ebp+8]
L1:
mov ecx, [ebx] ;you can use ecx, cx, ch, cl
cmp ecx, 0 ;you can use ecx, cx, ch, cl
JE L2
add ebx, 1
add eax, 1
jmp L1
L2:
pop ecx
pop ebx
mov ebp, esp
pop ebp
ret 4
stringLength endp
BubbleSort PROC uses ECX
push edx
xor ecx,ecx
mov ecx, 50
OUTER_LOOP:
push ecx
xor ecx,ecx
mov ecx,14
mov esi, OFFSET input
COMPARE:
xor ebx,ebx
xor edx,edx
mov bl, byte ptr ds:[esi]
mov dl, byte ptr ds:[esi+1]
cmp bl,dl
jg SWAP1
CONTINUE:
add esi,2
loop COMPARE
mov esi, OFFSET input
pop ecx
loop OUTER_LOOP
jmp FINISHED
SWAP1:
xchg bl,dl
mov byte ptr ds:[esi+1],dl
mov byte ptr ds:[esi],bl
jmp CONTINUE
FINISHED:
pop edx
ret 4
BubbleSort ENDP
main proc
call clrscr
mov edx, offset stringinput
call writeString
mov edx, offset input
call writeString
call stringLength
mov edx, offset input
mov ecx, sizeof input
call readstring
call crlf
mov edx,offset totallength
call writestring
call writedec
call crlf
mov edx, offset minimum
call crlf
call writeString
push offset input
call BubbleSort
mov edx, offset input
call writeString
call crlf
exit
main endp
end main
I haven't looked over your code, because sorting is an over complicated method for what you want to do. Not only that, but most of us don't pay too much attention to uncommented code. Just takes to long to figure out what you're trying to do.
Simply iterate through the entire list and start with 255 (FFH) in AL let's say. Each time you come across a number that is smaller than the one in AL, then replace it with that value and then when loop is finished, AL will have the lowest value.
If you need to know where it is in the list, you could maybe use AH which would be the difference between start address and current address. Knowledge of the instruction set is essential as finding the length of the string can be simplified by;
mov di, input ; Point to beginning of buffer
mov cx, -1 ; for a maximum of 65535 characters
xor al, al ; Looking for NULL
rep scasb
neg cx
dec cx ; CX = length of string.
Remember, ES needs to point to #DATA

Unhandled exception at 0x00000005 in : 0xC0000005: Access violation reading location 0x00000005. when making a ret call

My program is supposed to read an integer n from the user and then calculate all the divisors and if they are prime or not. I am using the Irvine 32 library. Now this is the weird part, when I enter in an even number my program executes as it is supposed to. When I enter in and odd number my program gets the error which is the title of this post.
My main Proc:
main PROC
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; This block displays greeting and newline;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
mov edx, OFFSET greeting
call Writestring
call Crlf
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; This block gets the integer from the user;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
call GetInt
call Crlf
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; This block gets calculates the divsiors and prime divisors.;
; It then puts them in to an array to get ready to display. ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
call CalcDivisors
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; This block displays the results to the screen. ;
; in an n-1 by 3 table. ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
call Display_Results
exit
main ENDP
Now the Proc that has produces the error:
CalcDivisors PROC uses eax ebx ecx edx esi edi
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Function calculates divisors then pushes them on to an array;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
mov eax,0
mov ecx,0
mov ebx,0
mov edx,0
mov esi,0
mov edi,0
mov ecx,n
sub ecx,1
mov eax,n
mov ebx,divisors
mov esi,OFFSET prime_arr
mov edi,OFFSET div_arr
push eax
Calc_Div:
call dumpregs
div ebx
call dumpregs
cmp edx,0
jz Calc_Prime_Div
inc ebx
mov edx,0
mov eax,n
loop Calc_Div
Calc_Prime_Div:
cmp ebx,2
jz Push_2_array
push ebx
push ecx
mov eax,0
mov eax,ebx
mov ecx,ebx
mov divisor_counter,ebx
sub ecx,2
mov ebx,0
mov ebx,prime_divisors
P1:
call dumpregs
div ebx
call dumpregs
cmp edx,0
jz Push_div_array
inc ebx
mov eax,divisor_counter
mov edx,0
loop P1
jmp Push_prime_array
Jump_above:
call dumpregs
loop Calc_div
call dumpregs
jmp foo
Push_prime_array:
pop ecx
pop ebx
mov [esi],ebx
mov eax,[esi]
call writedec
add esi,4
mov eax,0
mov eax,n
call dumpregs
inc ebx
call dumpregs
jmp jump_above
call dumpregs
Push_div_array:
pop ecx
pop ebx
mov [edi],ebx
mov eax,[edi]
call writedec
add edi,4
mov eax,0
mov eax,n
call dumpregs
inc ebx
jmp Jump_above
Push_2_array:
mov [esi],ebx
add esi,4
inc ebx
pop eax
jmp Jump_above
foo:
ret
CalcDivisors ENDP
Now the line that is giving me the exact error is the following:
foo:
ret
It is boggling my mind as to why it is crashing when I enter in an odd number for n and not crashing when n is even. Any suggestions?
It looks like you forget to pop some values from the stack. Check the number of push and pop instructions executed.

Self modifying algorithms with Virtualprotect problems

I'm having problems with the Virtualprotect() api by windows.
I got an assignment from school, my teacher told us that in the past when memory was scarce and costly. Programmers had to create advanced algorithms that would modify itself on the fly to save memory. So there you have it, we must now write such an algorithm, it doesn't have to be effective but it must modify itself.
So I set out to do just that, and I think that I made it pretty far before asking for any help.
My program works like this:
I have a function and a loop with a built-in stack overflow. The stack gets overflown with the address of a memory location where code resides that is constructed during the loop. Control is passed to the code in memory. The code loads a dll and then exits, but before it exits it has to repair the loop. It is one of the conditions of our assignment, everything changed in the original loop must be restored.
The problem is that I don't have write access to the loop, only READ_EXECUTE, so to change my access I thought, I use virtualprotect. But that function returned an error:
ERROR_NOACCESS, the documentation on this error is very slim, windows only says: Invailid access to memory address. Which figures since I wanted to change the access in the first place. So what's wrong? Here's the code constructed in memory:
The names of all the data in my code is a little vague, so I provided a few comments
Size1:
TrapData proc
jmp pLocals
LocalDllName db 100 dup(?) ; name of the dll to be called ebx-82h
RestoreBuffer db 5 dup(?) ; previous bytes at the overflow location
LoadAddress dd 0h ; ebx - 19h ; address to kernel32.loadlibrary
RestoreAddress dd 0h ; ebx - 15h ; address to restore (with the restore buffer)
AddressToRestoreBuffer dd 0h ; ebx - 11h ; obsolete, I don't use this one
AddressToLea dd 0h ; ebx - 0Dh Changed, address to kernel32.virutalprotect
AddressToReturnTo dd 0h ; ebx - 9h address to return execution to(the same as RestoreAddress
pLocals:
call Refpnt
Refpnt: pop ebx ; get current address in ebx
push ebx
mov eax, ebx
sub ebx, 82h
push ebx ; dll name
sub eax, 19h ; load lib address
mov eax, [eax]
call eax
pop ebx ; Current address
push ebx
;BOOL WINAPI VirtualProtect(
; __in LPVOID lpAddress,
; __in SIZE_T dwSize,
; __in DWORD flNewProtect,
; __out PDWORD lpflOldProtect
;);
mov eax, ebx
mov esi, ebx
sub eax, 82h
push eax ; overwrite the buffer containing the dll name, we don't need it anymore
push PAGE_EXECUTE_READWRITE
push 5h
sub esi, 15h
mov esi, [esi]
push esi
sub ebx, 0Dh
mov ebx, [ebx]
call ebx ; Returns error 998 ERROR_NOACCESS (to what?)
pop ebx
push ebx
sub ebx, 1Eh
mov eax, ebx ; restore address buffer pointer
pop ebx
push ebx
sub ebx, 15h ; Restore Address
mov ebx, [ebx]
xor esi, esi ; counter to 0
#0:
push eax
mov al, byte ptr[eax+esi]
mov byte ptr[ebx+esi], al
pop eax
inc esi
cmp esi, 5
jne #0
pop ebx
sub ebx, 9h
mov ebx, [ebx]
push ebx ; address to return to
ret
Size2:
So what's wrong?
Can you guys help me?
EDIT, Working code:
Size1:
jmp pLocals
LocalDllName db 100 dup(?)
RestoreBuffer db 5 dup(?)
LoadAddress dd 0h ; ebx - 19h
RestoreAddress dd 0h ; ebx - 15h
AddressToRestoreBuffer dd 0h ; ebx - 11h
AddressToLea dd 0h ; ebx - 0Dh
AddressToReturnTo dd 0h ; ebx - 9h
pLocals:
call Refpnt
Refpnt: pop ebx ; get current address in ebx
push ebx
mov eax, ebx
sub ebx, 82h
push ebx ; dll name
sub eax, 19h ; load lib address
mov eax, [eax]
call eax
pop ebx ; Current address
push ebx
;BOOL WINAPI VirtualProtect(
; __in LPVOID lpAddress,
; __in SIZE_T dwSize,
; __in DWORD flNewProtect,
; __out PDWORD lpflOldProtect
;);
mov esi, ebx
push 0
push esp
push PAGE_EXECUTE_READWRITE
push 5h
sub esi, 15h
mov esi, [esi]
push esi
sub ebx, 0Dh
mov ebx, [ebx]
call ebx
pop ebx
pop ebx
push ebx
sub ebx, 1Eh
mov eax, ebx ; restore address buffer pointer
pop ebx
push ebx
sub ebx, 15h ; Restore Address
mov ebx, [ebx]
xor esi, esi ; counter to 0
#0:
push eax
mov al, byte ptr[eax+esi]
mov byte ptr[ebx+esi], al
pop eax
inc esi
cmp esi, 5
jne #0
pop ebx
sub ebx, 9h
mov ebx, [ebx]
push ebx ; address to return to
ret
Size2:
Maybe a little sloppy, but I that doesn't mater ;)
You are trying to make VirtualProtect write lpflOldProtect to a read-only memory location, i.e. your current code section which is what you're trying to unprotect in the first place! My guess is this is what gives you the ERROR_NO_ACCESS. Since you're using the stack anyway, have it write lpflOldProtect to a stack location.
This isn't nearly as easy as it was in the old days; read access used to imply execute access, and a lot of memory mappings were mapped writable.
These days, I'd be surprised if there are many (any?) memory mappings that are both writable and executable. (And modern CPUs with PAE support are sufficient for even 32-bit kernels to provide non-executable-yet-readable mappings.)
I'd say, first things first, find an older Windows system, Win2k or earlier, then start trying to tackle this problem. :)
EDIT: Oh! I thought loading the DLL failed. Good work. :)
What do you mean by 'restore the loop'? Since you smashed the stack to jump to your code, you didn't really destroy the loop's text segment, you only scribbled on the stack. You could insert another function before your loop, then return from your dll to the function that called your loop. (You 'returned' into your injected code from the loop, so you can't return into the loop without building a fake stack frame for it; returning to the previous function seems easier than building a fake stack frame.)

Resources