_InterlockedIncrement intrinsic implementation - visual-studio-2010

Visual Studio produces the following machine code when _InterlockedIncrement is used:
; 40 : _InterlockedIncrement(&framecounter);
00078 b8 00 00 00 00 mov eax, OFFSET ?framecounter##3JA ; framecounter
0007d b9 01 00 00 00 mov ecx, 1
00082 f0 0f c1 08 lock xadd DWORD PTR [eax], ecx
If I would be writing this i would use just lock inc DWORD PTR [eax] instead of mov and xadd
Is there a valid reason why Microsoft preferred xadd and using 2 instructions instead of 1?

Because _InterlockedIncrement also returns the new value.
You can't do that with lock inc DWORD PTR [eax], because now neither the old nor the new value are anywhere to be found. Except in memory, but if you do an other read, clearly it won't be atomic (the increment itself would be, but you could get a value back that has nothing to do with what happened at the time of the increment).
Returning the value makes _InterlockedIncrement more useful.

Related

Why does the assembly encoding of objdump vary?

I was reading this article about Position Independent Code and I encountered this assembly listing of a function.
0000043c <ml_func>:
43c: 55 push ebp
43d: 89 e5 mov ebp,esp
43f: e8 16 00 00 00 call 45a <__i686.get_pc_thunk.cx>
444: 81 c1 b0 1b 00 00 add ecx,0x1bb0
44a: 8b 81 f0 ff ff ff mov eax,DWORD PTR [ecx-0x10]
450: 8b 00 mov eax,DWORD PTR [eax]
452: 03 45 08 add eax,DWORD PTR [ebp+0x8]
455: 03 45 0c add eax,DWORD PTR [ebp+0xc]
458: 5d pop ebp
459: c3 ret
0000045a <__i686.get_pc_thunk.cx>:
45a: 8b 0c 24 mov ecx,DWORD PTR [esp]
45d: c3 ret
However, on my machine (gcc-7.3.0, Ubuntu 18.04 x86_64), I got slightly different result below:
0000044d <ml_func>:
44d: 55 push %ebp
44e: 89 e5 mov %esp,%ebp
450: e8 29 00 00 00 call 47e <__x86.get_pc_thunk.ax>
455: 05 ab 1b 00 00 add $0x1bab,%eax
45a: 8b 90 f0 ff ff ff mov -0x10(%eax),%edx
460: 8b 0a mov (%edx),%ecx
462: 8b 55 08 mov 0x8(%ebp),%edx
465: 01 d1 add %edx,%ecx
467: 8b 90 f0 ff ff ff mov -0x10(%eax),%edx
46d: 89 0a mov %ecx,(%edx)
46f: 8b 80 f0 ff ff ff mov -0x10(%eax),%eax
475: 8b 10 mov (%eax),%edx
477: 8b 45 0c mov 0xc(%ebp),%eax
47a: 01 d0 add %edx,%eax
47c: 5d pop %ebp
47d: c3 ret
The main difference I found was that the semantic of mov instruction. In the upper listing, mov ebp,esp actually moves esp to ebp, while in the lower listing, mov %esp,%ebp does the same thing, but the order of operands are different.
This is quite confusing, even when I have to code hand-written assembly. To summarize, my questions are (1) why I got different assembly representations for the same instructions and (2) which one I should use, when writing assembly code (e.g. with __asm(:::);)
obdjump defaults to -Matt AT&T syntax (like your 2nd code block). See att vs. intel-syntax. The tag wikis have some info about the syntax differences: https://stackoverflow.com/tags/att/info vs. https://stackoverflow.com/tags/intel-syntax/info
Either syntax has the same limitations, imposed by what the machine itself can do, and what's encodeable in machine code. They're just different ways of expressing that in text.
Use objdump -d -Mintel for Intel syntax. I use alias disas='objdump -drwC -Mintel' in my .bashrc, so I can disas foo.o and get the format I want, with relocations printed (important for making sense of a non-linked .o), without line-wrapping for long instructions, and with C++ symbol names demangled.
In inline asm, you can use either syntax, as long as it matches what the compiler is expecting. The default is AT&T, and that's what I'd recommend using for compatibility with clang. Maybe there's a way, but clang doesn't work the same way as GCC with -masm=intel.
Also, AT&T is basically standard for GNU C inline asm on x86, and it means you don't need special build options for your code to work.
But you can use gcc -masm=intel to compile source files that use Intel syntax in their asm statements. This is fine for your own use if you don't care about clang.
If you're writing code for a header, you can make it portable between AT&T and Intel syntax using dialect alternatives, at least for GCC:
static inline
void atomic_inc(volatile int *p) {
// use __asm__ instead of asm in headers, so it works even with -std=c11 instead of gnu11
__asm__("lock {addl $1, %0 | add %0, 1}": "+m"(*p));
// TODO: flag output for return value?
// maybe doesn't need to be asm volatile; compilers know that modifying pointed-to memory is a visible side-effect unless it's a local that fully optimizes away.
// If you want this to work as a memory barrier, use a `"memory"` clobber to stop compile-time memory reordering. The lock prefix provides a runtime full barrier
}
source+asm outputs for gcc/clang on the Godbolt compiler explorer.
With g++ -O3 (default or -masm=att), we get
atomic_inc(int volatile*):
lock addl $1, (%rdi) # operand-size is from my explicit addl suffix
ret
With g++ -O3 -masm=intel, we get
atomic_inc(int volatile*):
lock add DWORD PTR [rdi], 1 # operand-size came from the %0 expansion
ret
clang works with the AT&T version, but fails with -masm=intel (or the -mllvm --x86-asm-syntax=intel which that implies), because that apparently only applies to code emitted by LLVM, not for how the front-end fills in the asm template.
The clang error message is:
<source>:4:13: error: unknown use of instruction mnemonic without a size suffix
__asm__("lock {addl $1, %0 | add %0, 1}": "+m"(*p));
^
<inline asm>:1:2: note: instantiated into assembly here
lock add (%rdi), 1
^
1 error generated.
It picked the "Intel" syntax alternative, but still filled in the template with an AT&T memory operand.

Reversed Mach-O 64-bit x86 Assembly analysis

This question is for Intel x86 assembly experts to answer. Thanks for your effort in advance!
Problem Specification
I am analysing a binary file, which match Mach-O 64-bit x86 assembly. I am currently using MacOS 64 OS. The assembly comes from objdump.
The problem is that when I am learning assembly, I can see variable name "$xxx", I can see string value in ascii and I can also see the callee name like "call _printf"
But in this assembly, I can get nothing above:
no main function:
Disassembly of section __TEXT,__text:
__text:
100000c90: 55 pushq %rbp
100000c91: 48 89 e5 movq %rsp, %rbp
100000c94: 48 83 ec 10 subq $16, %rsp
100000c98: 48 8d 3d bf 02 00 00 leaq 703(%rip), %rdi
100000c9f: b0 00 movb $0, %al
100000ca1: e8 68 02 00 00 callq 616
100000ca6: 89 45 fc movl %eax, -4(%rbp)
100000ca9: 48 83 c4 10 addq $16, %rsp
100000cad: 5d popq %rbp
100000cae: c3 retq
100000caf: 90 nop
100000cb0: 55 pushq %rbp
...
The above is codes frame will be executed, but I have no idea where it is executed.
Also, I newbie of AT&T assemble. Hence, could you tell me what is the meaning of instruction:
0000000100000c90 pushq %rbp
0000000100000c98 leaq 0x2bf(%rip), %rdi ## literal pool for: "xxxx\n"
...
0000000100000cd0 callq 0x100000c90
Is it a loop? I am not sure but it seems to be. And why we they use %rip and %rdi register. In intel x86 I know that EIP represents current caller address, but I don't understand the meaning here.
call integer:
No matter what call convention they used, I had never seen code pattern like "call 616":
"100000cd0: e8 bb ff ff ff callq -69 <__mh_execute_header+C90>"
After ret:
Ret in intel x86, means delete stack frame and return control flow to caller. It should be an independent function. However, after this, we can see codes like
100000cae: c3 retq
100000caf: 90 nop
/* new function call */
100000cb0: 55 pushq %rbp
...
It is ridiculous!
ASCII string lost:
I have already viewed the binary in Hexadecimal format, and recognise some ascii string before reverse it to asm file.
However, in this file no ascii string occurrences!
Total architecture review:
Disassembly of section __TEXT,__text:
__text:
from address 10000c90 to 100000ef6 of 145 lines
Disassembly of section __TEXT,__stubs:
__stubs:
from address 100000efc to 100000f14 of 5 lines asm codes:
100000efc: ff 25 16 01 00 00 jmp qword ptr [rip + 278]
100000f02: ff 25 18 01 00 00 jmp qword ptr [rip + 280]
100000f08: ff 25 1a 01 00 00 jmp qword ptr [rip + 282]
100000f0e: ff 25 1c 01 00 00 jmp qword ptr [rip + 284]
100000f14: ff 25 1e 01 00 00 jmp qword ptr [rip + 286]
Disassembly of section __TEXT,__stub_helper:
__stub_helper:
...
Disassembly of section __TEXT,__cstring:
__cstring:
...
Disassembly of section __TEXT,__unwind_info:
__unwind_info:
...
Disassembly of section __DATA,__nl_symbol_ptr:
__nl_symbol_ptr:
...
Disassembly of section __DATA,__got:
__got:
...
Disassembly of section __DATA,__la_symbol_ptr:
__la_symbol_ptr:
...
Disassembly of section __DATA,__data:
__data:
...
Since it might be a virus, I cannot execute it. How should I analyse it ?
Update on May 21
I have already identified where is the output, and if I totally understand the data flow pipeline represented in this programme, I might be able to figure out the possible solutions.
I am appreciated if someone can give me the detailed explanation. Thank you !
Update on May 22
I installed a MacOS in VirtualBox and after chmod privileges , I executed the programme but nothing special except for two lines of output happened. And the result hiding in the binary file.
You don't need a main if you are not using C. The binary header contains the entry point address.
Nothing special about call 616, it's just that you don't have (all) symbols. It's somewhat strange that objdump didn't calculate the address for you, but it should be 0x100000ca6+616.
Not sure what you find ridiculous there. One function ends, another starts.
That's not a question. Yes, you can create strings at runtime so you won't have them in the image. Possibly they are encrypted.

How can I insert repeated NOP statements using Visual C++'s inline assembler?

Visual C++, using Microsoft's compiler, allows us to define inline assembly code using:
__asm {
nop
}
What I need is a macro that makes possible to multiply such instruction n times like:
ASM_EMIT_MULT(op, times)
for example:
ASM_EMIT_MULT(0x90, 160)
Is that possible? How could I do this?
With MASM, this is very simple to do. Part of the installation is a file named listing.inc (since everyone gets MASM as part of Visual Studio now, this will be located in your Visual Studio root directory/VC/include). This file defines a series of npad macros that take a single size argument and expand to an appropriate sequence of non-destructive "padding" opcodes. If you only need one byte of padding, you use the obvious nop instruction. But rather than using a long series of nops until you reach the desired length, Intel actually recommends other non-destructive opcodes of the appropriate length, as do other vendors. These pre-defined npad macros free you from having to memorize that table, not to mention making the code much more readable.
Unfortunately, inline assembly is not a full-featured assembler. There are a lot of things missing that you would expect to find in real assemblers like MASM. Macros (MACRO) and repeats (REPEAT/REPT) are among the things that are missing.
However, ALIGN directives are available in inline assembly. These will generate the required number of nops or other non-destructive opcodes to enforce alignment of the next instruction. Using this is drop-dead simple. Here is a very stupid example, where I've taken working code and peppered it with random aligns:
unsigned long CountDigits(unsigned long value)
{
__asm
{
mov edx, DWORD PTR [value]
bsr eax, edx
align 4
xor eax, 1073741792
mov eax, DWORD PTR [4 * eax + kMaxDigits+132]
align 16
cmp edx, DWORD PTR [4 * eax + kPowers-4]
sbb eax, 0
align 8
}
}
This generates the following output (MSVC's assembly listings use npad x, where x is the number of bytes, just as you'd write it in MASM):
PUBLIC CountDigits
_TEXT SEGMENT
_value$ = 8
CountDigits PROC
00000 8b 54 24 04 mov edx, DWORD PTR _value$[esp-4]
00004 0f bd c2 bsr eax, edx
00007 90 npad 1 ;// enforcing the "align 4"
00008 35 e0 ff ff 3f xor eax, 1073741792
0000d 8b 04 85 84 00
00 00 mov eax, DWORD PTR _kMaxDigits[eax*4+132]
00014 eb 0a 8d a4 24
00 00 00 00 8d
49 00 npad 12 ;// enforcing the "align 16"
00020 3b 14 85 fc ff
ff ff cmp edx, DWORD PTR _kPowers[eax*4-4]
00027 83 d8 00 sbb eax, 0
0002a 8d 9b 00 00 00
00 npad 6 ;// enforcing the "align 8"
00030 c2 04 00 ret 4
CountDigits ENDP
_TEXT ENDS
If you aren't actually wanting to enforce alignment, but just want to insert an arbitrary number of nops (perhaps as filler for later hot-patching?), then you can use C macros to simulate the effect:
#define NOP1 __asm { nop }
#define NOP2 NOP1 NOP1
#define NOP4 NOP2 NOP2
#define NOP8 NOP4 NOP4
#define NOP16 NOP8 NOP8
// ...
#define NOP64 NOP16 NOP16 NOP16 NOP16
// ...etc.
And then pepper your code as desired:
unsigned long CountDigits(unsigned long value)
{
__asm
{
mov edx, DWORD PTR [value]
bsr eax, edx
NOP8
xor eax, 1073741792
mov eax, DWORD PTR [4 * eax + kMaxDigits+132]
NOP4
cmp edx, DWORD PTR [4 * eax + kPowers-4]
sbb eax, 0
}
}
to produce the following output:
PUBLIC CountDigits
_TEXT SEGMENT
_value$ = 8
CountDigits PROC
00000 8b 54 24 04 mov edx, DWORD PTR _value$[esp-4]
00004 0f bd c2 bsr eax, edx
00007 90 npad 1 ;// these are, of course, just good old NOPs
00008 90 npad 1
00009 90 npad 1
0000a 90 npad 1
0000b 90 npad 1
0000c 90 npad 1
0000d 90 npad 1
0000e 90 npad 1
0000f 35 e0 ff ff 3f xor eax, 1073741792
00014 8b 04 85 84 00
00 00 mov eax, DWORD PTR _kMaxDigits[eax*4+132]
0001b 90 npad 1
0001c 90 npad 1
0001d 90 npad 1
0001e 90 npad 1
0001f 3b 14 85 fc ff
ff ff cmp edx, DWORD PTR _kPowers[eax*4-4]
00026 83 d8 00 sbb eax, 0
00029 c2 04 00 ret 4
CountDigits ENDP
_TEXT ENDS
Or, even cooler, we can use a bit of template meta-programming magic to get the same effect in style. Just define the following template function and its specialization (important to prevent infinite recursion):
template <size_t N> __forceinline void npad()
{
npad<N-1>();
__asm { nop }
}
template <> __forceinline void npad<0>() { }
And use it like this:
unsigned long CountDigits(unsigned long value)
{
__asm
{
mov edx, DWORD PTR [value]
bsr eax, edx
}
npad<8>();
__asm
{
xor eax, 1073741792
mov eax, DWORD PTR [4 * eax + kMaxDigits+132]
}
npad<4>();
__asm
{
cmp edx, DWORD PTR [4 * eax + kPowers-4]
sbb eax, 0
}
}
That'll produce the desired output (exactly the same as the one just above) in all optimized builds—whether you optimize for size (/O1) or speed (/O2)—…but not in debugging builds. If you need it in debug builds, you'll have to resort to the C macros. :-(
Base on Cody Gray Answer and code example for metaprogramming using template recursion and inline or forceinline as stated on the code before
template <size_t N> __forceinline void npad()
{
npad<N-1>();
__asm { nop }
}
template <> __forceinline void npad<0>() { }
It won't work on visual studio, without setting some options and is not a guarantee it will work
Although __forceinline is a stronger indication to the compiler than
__inline, inlining is still performed at the compiler's discretion, but no heuristics are used to determine the benefits from inlining this function.
You can read more about this here https://learn.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4714?view=vs-2019

WinExec Return 0x21, But what exactly it means?

When call WinExec to run a .exe, I get return value 0x21.
According to MSDN, a return value greater than 31 (0x1F) means function succeeds.
But what does it mean of 0x21, Why it didn't return other value to me?
It is not useful for you to know what it means. That is an implementation detail. Even if you knew what it meant for this version, the meaning might change in the next version. As a programmer, you are concerned only with programming against the interface, not the underlying implementation.
However, if you are really interested, I will take you through the approach I would take to reverse engineer the function. On my system, WinExec is disassembled to this:
764F2C21 > 8BFF MOV EDI,EDI
764F2C23 55 PUSH EBP
764F2C24 8BEC MOV EBP,ESP
764F2C26 81EC 80000000 SUB ESP,80
764F2C2C 53 PUSH EBX
764F2C2D 8B5D 0C MOV EBX,DWORD PTR SS:[EBP+C]
764F2C30 56 PUSH ESI
764F2C31 57 PUSH EDI
764F2C32 33FF XOR EDI,EDI
764F2C34 47 INC EDI
764F2C35 33F6 XOR ESI,ESI
764F2C37 85DB TEST EBX,EBX
764F2C39 79 4F JNS SHORT kernel32.764F2C8A
764F2C3B 8D45 FC LEA EAX,DWORD PTR SS:[EBP-4]
764F2C3E 50 PUSH EAX
764F2C3F 56 PUSH ESI
764F2C40 57 PUSH EDI
764F2C41 8D45 C8 LEA EAX,DWORD PTR SS:[EBP-38]
764F2C44 50 PUSH EAX
764F2C45 C745 FC 20000000 MOV DWORD PTR SS:[EBP-4],20
764F2C4C E8 90BE0200 CALL <JMP.&API-MS-Win-Core-ProcessThread>
764F2C51 85C0 TEST EAX,EAX
764F2C53 0F84 D2000000 JE kernel32.764F2D2B
764F2C59 56 PUSH ESI
764F2C5A 56 PUSH ESI
764F2C5B 6A 04 PUSH 4
764F2C5D 8D45 F8 LEA EAX,DWORD PTR SS:[EBP-8]
764F2C60 50 PUSH EAX
764F2C61 68 01000600 PUSH 60001
764F2C66 56 PUSH ESI
764F2C67 8D45 C8 LEA EAX,DWORD PTR SS:[EBP-38]
764F2C6A 50 PUSH EAX
764F2C6B C745 0C 00000800 MOV DWORD PTR SS:[EBP+C],80000
764F2C72 897D F8 MOV DWORD PTR SS:[EBP-8],EDI
764F2C75 E8 5CBE0200 CALL <JMP.&API-MS-Win-Core-ProcessThread>
764F2C7A 85C0 TEST EAX,EAX
764F2C7C 0F84 95000000 JE kernel32.764F2D17
764F2C82 8D45 C8 LEA EAX,DWORD PTR SS:[EBP-38]
764F2C85 8945 C4 MOV DWORD PTR SS:[EBP-3C],EAX
764F2C88 EB 03 JMP SHORT kernel32.764F2C8D
764F2C8A 8975 0C MOV DWORD PTR SS:[EBP+C],ESI
764F2C8D 6A 44 PUSH 44
764F2C8F 8D45 80 LEA EAX,DWORD PTR SS:[EBP-80]
764F2C92 56 PUSH ESI
764F2C93 50 PUSH EAX
764F2C94 E8 B5E9F7FF CALL <JMP.&ntdll.memset>
764F2C99 83C4 0C ADD ESP,0C
764F2C9C 33C0 XOR EAX,EAX
764F2C9E 3975 0C CMP DWORD PTR SS:[EBP+C],ESI
764F2CA1 897D AC MOV DWORD PTR SS:[EBP-54],EDI
764F2CA4 0F95C0 SETNE AL
764F2CA7 66:895D B0 MOV WORD PTR SS:[EBP-50],BX
764F2CAB 8D0485 44000000 LEA EAX,DWORD PTR DS:[EAX*4+44]
764F2CB2 8945 80 MOV DWORD PTR SS:[EBP-80],EAX
764F2CB5 8D45 E8 LEA EAX,DWORD PTR SS:[EBP-18]
764F2CB8 50 PUSH EAX
764F2CB9 8D45 80 LEA EAX,DWORD PTR SS:[EBP-80]
764F2CBC 50 PUSH EAX
764F2CBD 56 PUSH ESI
764F2CBE 56 PUSH ESI
764F2CBF FF75 0C PUSH DWORD PTR SS:[EBP+C]
764F2CC2 56 PUSH ESI
764F2CC3 56 PUSH ESI
764F2CC4 56 PUSH ESI
764F2CC5 FF75 08 PUSH DWORD PTR SS:[EBP+8]
764F2CC8 56 PUSH ESI
764F2CC9 E8 A4E3F7FF CALL kernel32.CreateProcessA
764F2CCE 85C0 TEST EAX,EAX
764F2CD0 74 27 JE SHORT kernel32.764F2CF9
764F2CD2 A1 3C005476 MOV EAX,DWORD PTR DS:[7654003C]
764F2CD7 3BC6 CMP EAX,ESI
764F2CD9 74 0A JE SHORT kernel32.764F2CE5
764F2CDB 68 30750000 PUSH 7530
764F2CE0 FF75 E8 PUSH DWORD PTR SS:[EBP-18]
764F2CE3 FFD0 CALL EAX
764F2CE5 FF75 E8 PUSH DWORD PTR SS:[EBP-18]
764F2CE8 8B35 A0054776 MOV ESI,DWORD PTR DS:[<&ntdll.NtClose>] ; ntdll.ZwClose
764F2CEE FFD6 CALL ESI
764F2CF0 FF75 EC PUSH DWORD PTR SS:[EBP-14]
764F2CF3 FFD6 CALL ESI
764F2CF5 6A 21 PUSH 21
764F2CF7 EB 1D JMP SHORT kernel32.764F2D16
764F2CF9 E8 C9E4F7FF CALL <JMP.&API-MS-Win-Core-ErrorHandling>
764F2CFE 48 DEC EAX
764F2CFF 48 DEC EAX
764F2D00 74 12 JE SHORT kernel32.764F2D14
764F2D02 48 DEC EAX
764F2D03 74 0B JE SHORT kernel32.764F2D10
764F2D05 2D BE000000 SUB EAX,0BE
764F2D0A 75 0B JNZ SHORT kernel32.764F2D17
764F2D0C 6A 0B PUSH 0B
764F2D0E EB 06 JMP SHORT kernel32.764F2D16
764F2D10 6A 03 PUSH 3
764F2D12 EB 02 JMP SHORT kernel32.764F2D16
764F2D14 6A 02 PUSH 2
764F2D16 5E POP ESI
764F2D17 F745 0C 00000800 TEST DWORD PTR SS:[EBP+C],80000
764F2D1E 74 09 JE SHORT kernel32.764F2D29
764F2D20 8D45 C8 LEA EAX,DWORD PTR SS:[EBP-38]
764F2D23 50 PUSH EAX
764F2D24 E8 A2BD0200 CALL <JMP.&API-MS-Win-Core-ProcessThread>
764F2D29 8BC6 MOV EAX,ESI
764F2D2B 5F POP EDI
764F2D2C 5E POP ESI
764F2D2D 5B POP EBX
764F2D2E C9 LEAVE
764F2D2F C2 0800 RETN 8
The call convention used under Win32 is stdcall which mandates return values be held in EAX. In the case of WinExec, there is only one exit from the function (0x764F2D2F). Tracing back from there, EAX is set by (at least when the return is 0x21):
764F2D29 8BC6 MOV EAX,ESI
Tracing back further, ESI itself is set from POP ESI which pops the top of the stack into ESI. The value of this is dependent on what was previously pushed on the stack. In the case of 0x21, this happens at:
764F2CF5 6A 21 PUSH 21
Immediately afterwards, a JMP is made to the POP ESI. How we got to the PUSH 21 is interesting only from after the CreateProcess call.
764F2CC9 E8 A4E3F7FF CALL kernel32.CreateProcessA
764F2CCE 85C0 TEST EAX,EAX
764F2CD0 74 27 JE SHORT kernel32.764F2CF9
764F2CD2 A1 3C005476 MOV EAX,DWORD PTR DS:[7654003C]
764F2CD7 3BC6 CMP EAX,ESI
764F2CD9 74 0A JE SHORT kernel32.764F2CE5
764F2CDB 68 30750000 PUSH 7530
764F2CE0 FF75 E8 PUSH DWORD PTR SS:[EBP-18]
764F2CE3 FFD0 CALL EAX
764F2CE5 FF75 E8 PUSH DWORD PTR SS:[EBP-18]
764F2CE8 8B35 A0054776 MOV ESI,DWORD PTR DS:[<&ntdll.NtClose>] ; ntdll.ZwClose
764F2CEE FFD6 CALL ESI
764F2CF0 FF75 EC PUSH DWORD PTR SS:[EBP-14]
764F2CF3 FFD6 CALL ESI
764F2CF5 6A 21 PUSH 21
To see how the path will take you to the PUSH 21, observe different branches. The first occurs as:
764F2CD0 74 27 JE SHORT kernel32.764F2CF9
This is saying if CreateProcess returned 0, call Win-Core-ErrorHandling. The return values are then set differently (0x2, 0x3 and 0xB are all possible return values if CreateProcess failed).
The next branch is a lot less obvious to reverse engineer:
764F2CD9 74 0A JE SHORT kernel32.764F2CE5
What it does is read a memory address which probably contains a function pointer (we know this because the result of the read is called later on). This JE simply indicates whether or not to make this call at all. Regardless of whether the call is made, the next step is to call ZwClose (twice). Finally 0x21 is returned.
So one simple way of looking at it is that when CreateProcess succeeds, 0x21 is returned, otherwise 0x2, 0x3 or 0xB are returned. This is not to say these are the only return values. For example, 0x0 can also be returned from the branch at 0x764F2C53 (in this case, ESI is not used in the same way at all). There are a few more possible return values but I will leave those for you to look into yourself.
What I've shown you is how to do a very shallow analysis of WinExec specifically for the 0x21 return. If you want to find out more, you need to poke around more in-depth and try to understand from a higher level what is going on. You'll be able to find out a lot more just by breakpointing the function and stepping through it (this way you can observe data values).
One other way is to look at the Wine source, where someone has already done all the hard work for you:
UINT WINAPI WinExec( LPCSTR lpCmdLine, UINT nCmdShow )
{
PROCESS_INFORMATION info;
STARTUPINFOA startup;
char *cmdline;
UINT ret;
memset( &startup, 0, sizeof(startup) );
startup.cb = sizeof(startup);
startup.dwFlags = STARTF_USESHOWWINDOW;
startup.wShowWindow = nCmdShow;
/* cmdline needs to be writable for CreateProcess */
if (!(cmdline = HeapAlloc( GetProcessHeap(), 0, strlen(lpCmdLine)+1 ))) return 0;
strcpy( cmdline, lpCmdLine );
if (CreateProcessA( NULL, cmdline, NULL, NULL, FALSE,
0, NULL, NULL, &startup, &info ))
{
/* Give 30 seconds to the app to come up */
if (wait_input_idle( info.hProcess, 30000 ) == WAIT_FAILED)
WARN("WaitForInputIdle failed: Error %d\n", GetLastError() );
ret = 33;
/* Close off the handles */
CloseHandle( info.hThread );
CloseHandle( info.hProcess );
}
else if ((ret = GetLastError()) >= 32)
{
FIXME("Strange error set by CreateProcess: %d\n", ret );
ret = 11;
}
HeapFree( GetProcessHeap(), 0, cmdline );
return ret;
}
33d is 0x21 so this actually just confirms the fruits of our earlier analysis.
In regards to the reason 0x21 is returned, my guess is that perhaps there exists more internal documentation which makes it more useful in some way.
Other than that this means success, the meaning of the return value is not defined. Perhaps it was chosen such that legacy applications will work well with this particular value. One thing is certain: there are more important things to worry about!
http://msdn.microsoft.com/en-us/library/windows/desktop/ms687393(v=vs.85).aspx
EDIT: This answer is wrong because the OP's result is not an error code. I mistakenly thought it was said that it was an error code. I still think the practical info below can be useful, plus that it can be useful to see what a wrong assumption can lead to, so I let this answer stand.
If you have installed Visual Studio (full or express edition), then you have a tool called errlook, which uses the FormatMessage API function to tell you what an error code or HRESULT value means.
In this case,
The process cannot access the file because another process has locked a portion of the file.
You can do much of the same manually by looking in the <winerror.h> file. For example, type an #include of that in a C++ source file in Visual Studio, then right click and ask it to open the header. Where you find that
//
// MessageId: ERROR_LOCK_VIOLATION
//
// MessageText:
//
// The process cannot access the file because another process has locked a portion of the file.
//
#define ERROR_LOCK_VIOLATION 33L
By the way, WinExec is just an old compatibility function. Preferably use ShellExecute or CreateProcess. The ShellExecute function is able to play more nicely with Windows Vista and 7 User Access Control, and it is simpler to use, so it is generally preferable.

Why do we allocate 12 bytes for each variable?

In visual Studio 2010 Professional (x86, Windows 7):
... more
00DC1362 B9 39 00 00 00 mov ecx,39h
00DC1367 B8 CC CC CC CC mov eax,0CCCCCCCCh
00DC136C F3 AB rep stos dword ptr es:[edi]
20: int a = 3;
00DC136E C7 45 F8 03 00 00 00 mov dword ptr [ebp-8],3
21: int b = 10;
00DC1375 C7 45 EC 0A 00 00 00 mov dword ptr [ebp-14h],0Ah
22: int c;
23: c = a + b;
00DC137C 8B 45 F8 mov eax,dword ptr [ebp-8]
00DC137F 03 45 EC add eax,dword ptr [ebp-14h]
00DC1382 89 45 E0 mov dword ptr [ebp-20h],eax
24: return 0;
Notice how the relative addressing variable A and B are not aligned by word size of 4?
What is happening here?
Also, why do we skip $ebp - 8 ?
Turning off the optimization will show the ideal addressing scheme.
Can someone please explain the reason? Thanks.
The offset of each variable is 12 bytes. A -> B -> C
I made a mistake. I meant why do we skip the first 8 bytes.
You are looking at the code generated by the default Debug build setting. Particularly the /RTC option (enable run-time error checks). Filling the stack frame with 0xcccccccc helps diagnose uninitialized variables, the gaps around the variables help diagnose buffer overflow.
There isn't much point in looking at this code, you are not going to ship that. It is purely a Debug build artifact, only there to help you get the bugs out of the code. None of it remains in the Release build.

Resources