CreateFileA in Windows API in NASM 64: incorrect parameter, but which one? - windows

I am creating a file using CreateFileA from the Windows API in NASM 64-bit (see https://learn.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfilea). With the following parameters, no file is created and it returns an error 87 ("the parameter is incorrect") from GetLastError (see https://learn.microsoft.com/en-us/windows/desktop/debug/system-error-codes--0-499-)
Here are the parameters:
rcx - lpFileName
;dwDesiredAccess
mov rdx,2
I chose FILE_WRITE_DATA from https://learn.microsoft.com/en-us/windows/desktop/FileIO/file-access-rights-constants
; dwShareMode
mov r8,0
According to https://learn.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfilea: If this parameter is zero and CreateFile succeeds, the file or device cannot be shared. According to https://learn.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfilea, the value should be zero for no sharing.
; lpSecurityAttributes
mov r9,const_inf ; (Pointer to null value dq 0xFFFFFFFF)
OR mov r9,const_0
According to https://learn.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfilea: "If this parameter is NULL, the handle returned by CreateFile cannot be inherited by any child processes the application may create and the file or device associated with the returned handle gets a default security descriptor."
sub rsp,24 ; stack space
; dwCreationDisposition
mov rax,2 (CREATE_ALWAYS)
mov [rsp+16],rax
; dwFlagsAndAttribute
mov rax,128
mov [rsp+8],rax
The value 128 is from https://learn.microsoft.com/en-us/windows/desktop/FileIO/file-attribute-constants
; hTemplateFile
mov rax,[const_inf]
mov [rsp+0],rax
Here is the full file creation code:
CreateAuditFile:
push r10
mov r10,rax ; Core #
mov rdi,FileHandles
mov rbx,[rdi+r10]
cmp rbx,0 ; has file been created
jne file_next
mov rcx,FileName_1
mov rdx,2 ;dwDesiredAccess ;0x40000000
push r8
push r9
mov r8,0 ; dwShareMode
mov r9,const_0 ; lpSecurityAttributes
;OR: mov r9,const_inf ; lpSecurityAttributes
; CREATE STACK SPACE FOR REMAINING PARAMETERS:
sub rsp,24
mov rax,2 ; dwCreationDisposition (CREATE_ALWAYS)
mov [rsp+16],rax
mov rax,128
mov [rsp+8],rax ; dwFlagsAndAttributes
mov rax,[const_inf]
mov [rsp+0],rax ; hTemplateFile
push r10
call CreateFileA
pop r10
mov rdi,FileHandles
call GetLastError
mov [rdi],rax
add rsp,24
pop r9
pop r8
pop r10
file_next:
ret
I have looked carefully at the parameter options, but the error message only says "invalid parameter." It doesn't say which parameter.
My question is: which parameter or parameters above is incorrect? Are the parameters on the stack passed correctly?
Thanks for any help.

I solved this problem, and here is the solution. The stack handling on my original question was incorrect. The right way to handle the stack is shown below.
The values for each of the parameters (such as DesiredAccess, ShareMode and Security Attributes) may be different depending on the specific needs of the project, but the parameters are passed as in the code below:
CreateAuditFile:
mov rcx,FileName_1
sub rsp,56 ; 38h
xor eax,eax
mov qword [rsp+48],rax ; 30h
mov eax,80
mov dword [rsp+40],eax ; 28h
mov eax,2
mov dword [rsp+32],eax ; 20h
xor r9,r9
xor r8d,r8d
mov edx,40000000
call CreateFileA
mov rdi,OutputFileHandle
mov [rdi+r15],rax
xor eax,eax
add rsp,56 ;38h
ret
Thanks very much to everyone who responded.

Related

FASM write Hello World to console with NO includes or dependencies at all

I've seen
How to write hello world in assembler under Windows?
and
Writing hello,world to console in Fasm with DOS
How to write to the console in fasm?
I've tried / seen code like this MASM example from this answer
;---ASM Hello World Win64 MessageBox
extrn MessageBoxA: PROC
extrn ExitProcess: PROC
.data
title db 'Win64', 0
msg db 'Hello World!', 0
.code
main proc
sub rsp, 28h
mov rcx, 0 ; hWnd = HWND_DESKTOP
lea rdx, msg ; LPCSTR lpText
lea r8, title ; LPCSTR lpCaption
mov r9d, 0 ; uType = MB_OK
call MessageBoxA
add rsp, 28h
mov ecx, eax ; uExitCode = MessageBox(...)
call ExitProcess
main endp
End
(to which I get an error "Illegal instruction" on windows 64 bit extrn MessageBoxA:PROC because FASM doesn't understand that MASM directive.)
also this FASM example from this question
; Example of 64-bit PE program
format PE64 GUI
entry start
section '.text' code readable executable
start:
sub rsp,8*5 ; reserve stack for API use and make stack dqword aligned
mov r9d,0
lea r8,[_caption]
lea rdx,[_message]
mov rcx,0
call [MessageBoxA]
mov ecx,eax
call [ExitProcess]
section '.data' data readable writeable
_caption db 'Win64 assembly program',0
_message db 'Hello World!',0
section '.idata' import data readable writeable
dd 0,0,0,RVA kernel_name,RVA kernel_table
dd 0,0,0,RVA user_name,RVA user_table
dd 0,0,0,0,0
kernel_table:
ExitProcess dq RVA _ExitProcess
dq 0
user_table:
MessageBoxA dq RVA _MessageBoxA
dq 0
kernel_name db 'KERNEL32.DLL',0
user_name db 'USER32.DLL',0
_ExitProcess dw 0
db 'ExitProcess',0
_MessageBoxA dw 0
db 'MessageBoxA',0
but it displays a message box and also has external dependencies "kernel32.dll" and "user32.dll"
also tried this example from the FASM forum
format pe console
include 'win32ax.inc'
entry main
section '.data!!!' data readable writeable
strHello db 'Hello World !',13,10,0
strPause db 'pause',0
section '.txt' code executable readable
main:
; you can use crt functions or windows API.
cinvoke printf,strHello
cinvoke system,strPause; or import getc()
; or
; invoke printf,srtHello
; add esp, 4
; or use WriteFile and GetStdHandle APIs
push 0
call [ExitProcess]
section '.blah' import data readable
library kernel32,'kernel32.dll',\
msvcrt,'msvcrt.dll' ;; C-Run time from MS. This is always on every windows machine
import kernel32,\
ExitProcess,'ExitProcess'
import msvcrt,\
printf,'printf',\
system,'system'
but it depends on win32ax.inc and other imports
also
format PE console
include 'win32ax.inc'
.code
start:
invoke WriteConsole,<invoke GetStdHandle,STD_OUTPUT_HANDLE>,"Hello World !",13,0
invoke Sleep,-1
.end start
but requires "win32ax.inc" import
closest I could find without the win32ax from the FASM forum:
format pe64 console
entry start
STD_OUTPUT_HANDLE = -11
section '.text' code readable executable
start:
sub rsp,8*7 ; reserve stack for API use and make stack dqword aligned
mov rcx,STD_OUTPUT_HANDLE
call [GetStdHandle]
mov rcx,rax
lea rdx,[message]
mov r8d,message_length
lea r9,[rsp+4*8]
mov qword[rsp+4*8],0
call [WriteFile]
mov ecx,eax
call [ExitProcess]
section '.data' data readable writeable
message db 'Hello World!',0
message_length = $ - message
section '.idata' import data readable writeable
dd 0,0,0,RVA kernel_name,RVA kernel_table
dd 0,0,0,0,0
kernel_table:
ExitProcess dq RVA _ExitProcess
GetStdHandle dq RVA _GetStdHandle
WriteFile dq RVA _WriteFile
dq 0
kernel_name db 'KERNEL32.DLL',0
user_name db 'USER32.DLL',0
_ExitProcess db 0,0,'ExitProcess',0
_GetStdHandle db 0,0,'GetStdHandle',0
_WriteFile db 0,0,'WriteFile',0
but still requires the kernel32.dll and user32.dll
Any way to do this without any external DLLs at all? I know just the program fasm itself does it, and prints to the console, doesn't it?
Any way to do this without any external DLLs at all?
Under Windows: Definitely no!
Windows uses some methods (probably syscall) to enter the operating system, however, there are no official entry points.
This means that it is (unlikely but) possible that exactly the same program that shows the "Hello world" message box in the current Windows version will do something completely different after the next Windows update!
Because Microsoft is assuming that every Windows program is only calling the OS by using the .dll files that match the kernel version, they can do this.
I don't know about Windows 10, but an older Windows version (I don't remember if it was XP, Vista or 7) even simply assumed that an .exe file returns at once if it does not use any .dll file: The program was not even started in this case!
I know just the program fasm itself does it, and prints to the console
That is not the case, fasm is also using the kernel32 APIs.
FWIW kernel32 is loaded into the memory space of every process in Windows, so there is no penalty or overhead in using the kernel32 APIs.
You may like this Windows example in €ASM, which doesn't explicitly mention any DLL and doesn't require other external libraries.
Just save the source as "bluej.asm", assemble and link with euroasm bluej.asm and run as bluej.exe.
Nevertheless, you won't get away without using API functions imported from the default Windows system library "kernel32.dll".
bluej PROGRAM Format=PE, Entry=Start:
IMPORT GetStdHandle,WriteFile,ExitProcess
Start: PUSH -11 ; Param 1: standard output handle identificator.
CALL GetStdHandle; Return StdOutput handle in EAX.
PUSH 0 ; Param 5: no overlap.
PUSH Written ; Param 4: Address of a variable to store number of written bytes.
PUSH MsgSize ; Param 3: Number of bytes to write.
PUSH Msg ; Param 2: Address of text.
PUSH EAX ; Param 1: Output file handle.
CALL WriteFile ; System call.
PUSH 0 ; Errorlevel.
CALL ExitProcess ; System call.
Written DD 0
Msg DB "Hello, world!"
MsgSize EQU $ - Msg
ENDPROGRAM
What constitures as "dependency" to you? If you want to avoid even operating system DLL's, then you're probably out of luck. You can't rely on syscall numbers alone.
"no dependencies" can also mean "just using existing OS DLL's", such as ntdll, kernel32, etc., but without using 3rd party DLL's that may not be present, such as a specific version of the C runtime.
One method I would like to show is retrieving function pointers from the PEB. This is code that I've written and that I personally use, if I want to have shellcode that has no import section.
PebGetProcAddress works similarly to GetProcAddress, except that the DLL name and function name must be a hash, and the DLL must be loaded by using LoadLibrary.
This may not answer your question exactly, but I hope it gets you somewhat closer to your goal or help others who read it.
PebApi.asm
proc PebGetProcAddress ModuleHash:DWORD, FunctionHash:DWORD
local FirstEntry:DWORD
local CurrentEntry:DWORD
local ModuleBase:DWORD
local ExportDirectory:DWORD
local NameDirectory:DWORD
local NameOrdinalDirectory:DWORD
local FunctionCounter:DWORD
; Get InMemoryOrderModuleList from PEB
mov eax, 3
shl eax, 4
mov eax, [fs:eax] ; fs:0x30
mov eax, [eax + PEB.Ldr]
mov eax, [eax + PEB_LDR_DATA.InMemoryOrderModuleList.Flink]
mov [FirstEntry], eax
mov [CurrentEntry], eax
; Find module by hash
.L_module:
; Compute hash of case insensitive module name
xor edx, edx
mov eax, [CurrentEntry]
movzx ecx, word[eax + LDR_DATA_TABLE_ENTRY.BaseDllName.Length]
test ecx, ecx
jz .C_module
mov esi, [eax + LDR_DATA_TABLE_ENTRY.BaseDllName.Buffer]
xor eax, eax
cld
.L_module_hash:
lodsb
ror edx, 13
add edx, eax
cmp al, 'a'
jl #f
sub edx, 0x20 ; Convert lower case letters to upper case
##: dec ecx
test ecx, ecx
jnz .L_module_hash
; Check, if module is found by hash
cmp edx, [ModuleHash]
jne .C_module
; Get module base
mov eax, [CurrentEntry]
mov eax, [eax + LDR_DATA_TABLE_ENTRY.DllBase]
mov [ModuleBase], eax
; Get export directory
mov eax, [ModuleBase]
add eax, [eax + IMAGE_DOS_HEADER.e_lfanew]
mov eax, [eax + IMAGE_NT_HEADERS32.OptionalHeader.DataDirectoryExport.VirtualAddress]
add eax, [ModuleBase]
mov [ExportDirectory], eax
; Get name table
mov eax, [ExportDirectory]
mov eax, [eax + IMAGE_EXPORT_DIRECTORY.AddressOfNames]
add eax, [ModuleBase]
mov [NameDirectory], eax
; Get name ordinal table
mov eax, [ExportDirectory]
mov eax, [eax + IMAGE_EXPORT_DIRECTORY.AddressOfNameOrdinals]
add eax, [ModuleBase]
mov [NameOrdinalDirectory], eax
; Find function in export directory by hash
mov [FunctionCounter], 0
.L_functions:
mov eax, [ExportDirectory]
mov eax, [eax + IMAGE_EXPORT_DIRECTORY.NumberOfNames]
cmp eax, [FunctionCounter]
je .E_functions
; Compute hash of function name
xor edx, edx
mov esi, [NameDirectory]
mov esi, [esi]
add esi, [ModuleBase]
xor eax, eax
cld
.L_function_hash:
lodsb
test al, al
jz .E_function_hash
ror edx, 13
add edx, eax
jmp .L_function_hash
.E_function_hash:
; Check, if function is found by hash
cmp edx, [FunctionHash]
jne .C_functions
; Return function address
mov eax, [ExportDirectory]
mov eax, [eax + IMAGE_EXPORT_DIRECTORY.AddressOfFunctions]
add eax, [ModuleBase]
mov ebx, [NameOrdinalDirectory]
movzx ebx, word[ebx]
lea eax, [eax + ebx * 4]
mov eax, [eax]
add eax, [ModuleBase]
ret
.C_functions:
add [NameDirectory], 4
add [NameOrdinalDirectory], 2
inc [FunctionCounter]
jmp .L_functions
.E_functions:
; Function not found in module's export table
xor eax, eax
ret
.C_module:
; Move to next module, exit loop if CurrentEntry == FirstEntry
mov eax, [CurrentEntry]
mov eax, [eax + LIST_ENTRY.Flink]
mov [CurrentEntry], eax
cmp eax, [FirstEntry]
jne .L_module
; Module not found
xor eax, eax
ret
endp
PebApi.inc
macro pebcall modulehash, functionhash, [arg]
{
common
if ~ arg eq
reverse
pushd arg
common
end if
stdcall PebGetProcAddress, modulehash, functionhash
call eax
}
Example
PEB_User32Dll = 0x63c84283
PEB_MessageBoxW = 0xbc4da2be
; pebcall translates to a call to PebGetProcAddress and the call to the returned function pointer
pebcall PEB_User32Dll, PEB_MessageBoxW, NULL, 'Hello, World!', NULL, MB_OK
How to generate hashes for module names and function names
#define ROTR(value, bits) ((DWORD)(value) >> (bits) | (DWORD)(value) << (32 - (bits)))
DWORD ComputeFunctionHash(LPCSTR str)
{
DWORD hash = 0;
while (*str)
{
hash = ROTR(hash, 13) + *str++;
}
return hash;
}
DWORD ComputeModuleNameHash(LPCSTR str, USHORT length)
{
DWORD hash = 0;
for (USHORT i = 0; i < length; i++)
{
hash = ROTR(hash, 13) + (str[i] >= 'a' ? str[i] - 0x20 : str[i]);
}
return hash;
}

Can anyone explain me how does this Flat Assembler code works?

I am trying to learn me making GUI programs in assembly. I downloaded Flat Assembler and started to read the example programs. There I found this code.
This is 64 bit code in assembly (fasm) for Windows. It makes empty window. But there was very few comments in It, so It's hard for me to understand what's going on. I commented here which parts I don't understand.
format PE64 GUI 5.0
entry start
include 'win64a.inc'
section '.idata' import data readable writeable
library kernel32,'KERNEL32.DLL',\
user32,'USER32.DLL'
include 'api\kernel32.inc'
include 'api\user32.inc'
section '.data' data readable writeable
_title TCHAR 'Win64 program template',0
_class TCHAR 'FASMWIN64',0
_error TCHAR 'Startup failed.',0
wc WNDCLASSEX sizeof.WNDCLASSEX,0,WindowProc,0,0,NULL,NULL,NULL,COLOR_BTNFACE+1,NULL,_class,NULL
msg MSG
section '.text' code readable executable
start:
sub rsp,8 ; Make stack dqword aligned
invoke GetModuleHandle,0 ;GetModuleHandle,0?
mov [wc.hInstance],rax ;wc.hInstance?
invoke LoadIcon,0,IDI_APPLICATION ;LoadIcon,0,IDI_APPLICATION?
mov [wc.hIcon],rax ;wc.hIcon?
mov [wc.hIconSm],rax ;ec.hIconSm?
invoke LoadCursor,0,IDC_ARROW ;LoadCursor,0,IDC_ARROW?
mov [wc.hCursor],rax ;wc.hCursor?
invoke RegisterClassEx,wc ;RegisterClassEx,wc?
test rax,rax ;test?
jz error
invoke CreateWindowEx,0,_class,_title,WS_VISIBLE+WS_DLGFRAME+WS_SYSMENU,128,128,256,192,NULL,NULL,[wc.hInstance],NULL
test rax,rax
jz error
msg_loop: ;What does this function?
invoke GetMessage,msg,NULL,0,0
cmp eax,1
jb end_loop
jne msg_loop
invoke TranslateMessage,msg
invoke DispatchMessage,msg
jmp msg_loop
error:
invoke MessageBox,NULL,_error,NULL,MB_ICONERROR+MB_OK
end_loop:
invoke ExitProcess,[msg.wParam]
proc WindowProc uses rbx rsi rdi, hwnd,wmsg,wparam,lparam ;?
; Note that first four parameters are passed in registers,
; while names given in the declaration of procedure refer to the stack
; space reserved for them - you may store them there to be later accessible
; if the contents of registers gets destroyed. This may look like:
; mov [hwnd],rcx
; mov [wmsg],edx
; mov [wparam],r8
; mov [lparam],r9
cmp edx,WM_DESTROY
je .wmdestroy
.defwndproc: ;What does this?
invoke DefWindowProc,rcx,rdx,r8,r9
jmp .finish
.wmdestroy:
invoke PostQuitMessage,0
xor eax,eax
.finish:
ret
endp

VariantClear releases VT_DISPATCH -vs- MSDN documentation

VS2015 C++ / Windows7 SP1
Considering the following code:
CComPtr<IFontDisp> m_pFont;
::OleCreateFontIndirect(&fdesc,IID_IFontDisp,(void**)&m_pFont);
VARIANT var = m_pFont; // PSEUDO CODE
after this,
var.vt = 9; //VT_DISPATCH
var.DISPATCH = "oleaut32.dll/IFontDisp"
So all looks fine.
Now I call
::VariantClear(var);
And I debug into (ASM), I found this:
74CB2EA6 nop
CFont::Release:
--> 74CB2EA7 sub dword ptr [esp+4],4
74CB2EAC jmp CFont::Release (74CB2E79h)
74CB2EAE nop
74CB2EAF nop
74CB2EB0 nop
Following the code:
CFont::Release:
--> 74CB2E79 mov edi,edi
74CB2E7B push ebp
74CB2E7C mov ebp,esp
74CB2E7E push esi
74CB2E7F mov esi,dword ptr [ebp+8]
74CB2E82 push edi
74CB2E83 lea eax,[esi+0A8h]
74CB2E89 push eax
74CB2E8A call dword ptr [__imp__InterlockedDecrement#4 (74C91298h)]
74CB2E90 mov edi,eax
74CB2E92 test edi,edi
74CB2E94 je CFont::Release+261h (74CB30DAh)
74CB2E9A mov eax,edi
74CB2E9C pop edi
74CB2E9D pop esi
74CB2E9E pop ebp
74CB2E9F ret 4
So as I see, it releases the COM interface.
But if I see MSDN doc about VariantClear:
if the variant to be cleared is a COM object that is passed by
reference, the vtfield of the pvargparameter is VT_DISPATCH | VT_BYREF
or VT_UNKNOWN | VT_BYREF. In this case, VariantClear does not release
the object. Because the variant being cleared is a pointer to a
reference to an object, VariantClear has no way to determine if it is
necessary to release the object. It is therefore the responsibility of
the caller to release the object or not, as appropriate.
According to this, it should not call release on the IFontDisp.
Can anybody explain what is going on here?
Thanks.

Creating variables inside main's frame (Linux)

[SOLVED]
I'm trying to do my own assembly code to do what similar C code will do:
main()
{
scanf("%d",&integer_var); // here must be the address of the integer_var
printf("Your Value is:%d",integer_var);
}
Well this is in C, so I'm doing with NASM under linux with extern functions. scanf and printf and compile first with nasm and then with gcc.
Here's my code (is not right :D)
SECTION .text
argstr: db "%d",10,0
str: db "Your value is:%d",10,0
extern printf
extern scanf
SECTION .data
global main
main:
push ebp
mov esp,ebp
sub esp, 0x10 ;ok integer right?
mov [ebp-0x4],0x0 ;just put 0 number on our integer variable
mov eax,(ebp-0x4) ;here i don't know how to push the address of ebp-0x4
push ecx ;first push is last argument so here's our address to scanf
push argstr ;just the string format
call scanf ;call that to input something
;I have no idea how to do this
;but if i don't do this i get an error
;because the scanf won't clear the arguments on stack
;and what scanf can't return
pop edx ;maybe help here? but it works fine
pop edx
push [-0x4(ebp)] ;i want the value of our var :D
push str
call printf
pop edx ;clear the stack to avoid "segment fault" or something similar
pop edx
mov esp,ebp
pop ebp
ret ;the end :(
Compiler error:
a.asm:18: error: invalid operand type
a.asm:28: error: parser: expecting ]
Another thing: Do I need to align the stack on this case, by the way?
thanks guys ! :)
EDIT solved whole program!
well at least, I can print the variable with printf. scanf i will do later and then I will share here the last result:
SECTION .text
str: db "Value is:%d",10,0
extern printf
SECTION .data
global main
main:
push ebp ;the main function starts here.
mov ebp,esp
;
sub esp,4 ;we need 4bytes of space for the integer
and esp,0xfffffff0 ;align the stack
mov [esp-4], dword 0xff ;move the value 0xff to our var
mov eax,[esp-4] ;move our variable value to the eax
push eax ;second argument of printf
push str ;first argument of printf
call printf ;printf
;
add esp,16 ;this add to the stack pointer what we pushed basicly
mov ebp,esp ;if we don't do add 16 to esp it shows us
pop ebp ;a segment fault cuz ret doesnt pop saved ebp
ret ;of who whatever called this program :)
To load the address EBP-4 into EAX, use lea eax, [ebp-4]. (this is NOT the same as pushing the address.)
In order to push the value at memory location EBP-4, push dword [ebp-4] should work.
Then you need to specify operand size for one of your movs, too: mov [ebp-4], dword 0x0.
These will fix your current assembler errors, and make your program compile, but there are a few other errors in there that will probably prevent it from running.
Here's a working attempt that is close to yours:
;note the sections, the string literals are better in .rodata
;all code goes in .text
SECTION .rodata
;no newline after scanf string
argstr: db "%d",0
str: db "Your value is: %d",10,0
SECTION .text
extern printf
extern scanf
global main
main:
push ebp
mov ebp,esp ;move esp to ebp, NOT other way round!
sub esp, 4 ;4 bytes are enough for the local variable
;there are NO alignment requirements for this program
lea eax,[ebp-4]
push eax
push dword argstr
call scanf
add esp, 8 ;since we don't actually need the popped values
;we can increment esp instead of two pop edx
push dword [ebp-4]
push dword str
call printf
add esp, 8
mov esp,ebp
pop ebp
ret

Push an argument into stack?

I know that the first four arguments are in the register (RCX, RDX, R8, R9), and that additional arguments are pushed on the stack.
Question:
How to push an argument onto the stack? I tried with (push 0) but it does not work?
Code (MASM64)
extrn ExitProcess: PROC
extrn MessageBoxExA: PROC
.data
caption db '64-bit hello!', 0
message db 'Hello World!', 0
.code
Start PROC
sub rsp, 38h
mov rcx, 0 ; hWnd = HWND_DESKTOP
lea rdx, message ; LPCSTR lpText
lea r8, caption ; LPCSTR lpCaption
mov r9d, 0 ; uType = MB_OK
push 0 ; wLanguageId
call MessageBoxExA
mov ecx, eax
add rsp, 38h
call ExitProcess
Start ENDP
End
I'm know that MessageBox and MessageBoxEx work the same way, but im trying to use MessageBoxEx because its need one parameter to be passed (for learning purpose).
I know I've asked similar question, but it is more related to vb.net while this is not.
My assembly is a little rusty, but I was under the impression that all arguments went onto the stack (in reverse order) - I'd have thought you want to be pushing r8 and rdx in as well as the other arguments. Frankly though you might as well just keep doing lea rax, param and push rax for each of the arguments that are pointers.
The order in which the arguments are passed and whether they are passed in registers or on the stack (along with whether caller or callee is responsible for cleanup) is defined by the 'Calling Convention'.
What you are probably thinking of is STDCALL or CDECL, both are calling conventions used in 32-bit Windows that pass arguments on the stack in reverse order (right to left). x64 has moved to a FastCall calling convention where the arguments are passed in forward order (from left to right) and the first 4 arguments are passed in the registers RCX, RDX, R8 & R9. Any arguments beyond 4 are passed on the stack in the same left-to-right order. The original poster had the correct calling convention setup for x64 assembly with MASM. Also, the above responder who said the shadowspace valued subtracted from RSP should be 20h (32d) is correct. The shadow space is allowing space on the stack for the 4 arguments that are passed in by the registers in FastCall.
Changing the code above to:
extrn ExitProcess: PROC
extrn MessageBoxExA: PROC
.data
caption db '64-bit hello!', 0
message db 'Hello World!', 0
.code
Start PROC
sub rsp, 20h
mov rcx, 0 ; hWnd = HWND_DESKTOP
lea rdx, message ; LPCSTR lpText
lea r8, caption ; LPCSTR lpCaption
mov r9d, 0 ; uType = MB_OK
push 0 ; wLanguageId
call MessageBoxExA
mov ecx, eax
add rsp, 20h
call ExitProcess
Start ENDP
End
Works just fine in Visual Studio on a 64-bit machine

Resources