I am not well acquainted to the compiler magic. The act of transforming human-readable code (or the not really readable Assembly instructions) into machine code is, for me, rocket science combined with sorcery.
I will narrow down the subject of this question to Win32 executables (.exe). When I open these files up in a specialized viewer, I can find strings (usually 16b per character) scattered at various places, but the rest is just garbage. I suppose the unreadable part (majority) is the machine code (or maybe resources, such as images etc...).
Is there any straightforward way of reading the machine code? Opening the exe as a file stream and reading it byte by byte, how could one turn these individual bytes into Assembly? Is there a straightforward mapping between these instruction bytes and the Assembly instruction?
How is the .exe written? Four bytes per instruction? More? Less? I have noticed some applications can create executable files just like that: for example, in ACD See you can export a series of images into a slideshow. But this does not necessarily have to be a SWF slideshow, ACD See is also capable of producing EXEcutable presentations. How is that done?
How can I understand what goes on inside an EXE file?
OllyDbg is an awesome tool that disassembles an EXE into readable instructions and allows you to execute the instructions one-by-one. It also tells you what API functions the program uses and if possible, the arguments that it provides (as long as the arguments are found on the stack).
Generally speaking, CPU instructions are of variable length, some are one byte, others are two, some three, some four etc. It mostly depends on the kind of data that the instruction expects. Some instructions are generalised, like "mov" which tells the CPU to move data from a CPU register to a place in memory, or vice versa. In reality, there are many different "mov" instructions, ones for handling 8-bit, 16-bit, 32-bit data, ones for moving data from different registers and so on.
You could pick up Dr. Paul Carter's PC Assembly Language Tutorial which is a free entry level book that talks about assembly and how the Intel 386 CPU operates. Most of it is applicable even to modern day consumer Intel CPUs.
The EXE format is specific to Windows. The entry-point (i.e. the first executable instruction) is usually found at the same place within the EXE file. It's all kind of difficult to explain all at once, but the resources I've provided should help cure at least some of your curiosity! :)
You need a disassembler which will turn the machine code into assembly language. This Wikipedia link describes the process and provides links to free disassemblers. Of course, as you say you don't understand assembly language, this may not be very informative - what exactly are you trying to do here?
You can use debug from the command line, but that's hard.
C:\WINDOWS>debug taskman.exe
-u
0D69:0000 0E PUSH CS
0D69:0001 1F POP DS
0D69:0002 BA0E00 MOV DX,000E
0D69:0005 B409 MOV AH,09
0D69:0007 CD21 INT 21
0D69:0009 B8014C MOV AX,4C01
0D69:000C CD21 INT 21
0D69:000E 54 PUSH SP
0D69:000F 68 DB 68
0D69:0010 69 DB 69
0D69:0011 7320 JNB 0033
0D69:0013 7072 JO 0087
0D69:0015 6F DB 6F
0D69:0016 67 DB 67
0D69:0017 7261 JB 007A
0D69:0019 6D DB 6D
0D69:001A 206361 AND [BP+DI+61],AH
0D69:001D 6E DB 6E
0D69:001E 6E DB 6E
0D69:001F 6F DB 6F
The executable file you see is Microsofts PE (Portable Executable) format. It is essentially a container, which holds some operating system specific data about a program and the program data itself split into several sections. For example code, resources, static data are stored in seperate sections.
The format of the section depends on what is in it. The code section holds the machine code according to the executable target architecture. In the most common cases this is Intel x86 or AMD-64 (same as EM64T) for Microsoft PE binaries. The format of the machine code is CISC and originates back to the 8086 and earlier. The important aspect of CISC is that its instruction size is not constant, you have to start reading at the right place to get something valuable out of it. Intel publishes good manuals on the x86/x64 instruction set.
You can use a disassembler to view the machine code directly. In combination with the manuals you can guess the source code most of the time.
And then there's MSIL EXE: The .NET executables holding Microsofts Intermediate Language, these do not contain machine specific code, but .NET CIL code. The specifications for that are available online at the ECMA.
These can be viewed with a tool such as Reflector.
The contents of the EXE file are described in Portable Executable. It contains code, data, and instructions to OS on how to load the file.
There is an 1:1 mapping between machine code and assembly. A disassembler program will perform the reverse operation.
There isn't a fixed number of bytes per instruction on i386. Some are a single byte, some are much longer.
Just relating to this question, anyone still read things like
CD 21?
I remembered Sandra Bullock in one show, actually reading a screenful of hex numbers and figure out what the program does. Sort of like the current version of reading Matrix code.
if you do read stuff like CD 21, how do you remember the different various combinations?
Win32 exe format on MSDN
I'd suggest taking an bit of Windows C source code and build and start debugging it in Visual Studio. Switch to the disassembly view and step over the commands. You can see how the C code has been compiled into machine code - and watch it run step-by-step.
If it's as foreign to you as it seems, I don't think a debugger or disassembler is going to help - you need to learn assembler programming first; study the architecture of the processor (plenty of documentation downloadable from Intel). And then since most machine code is generated by compilers, you'll need to understand how compilers generate code - the simplest way to write lots of small programs and then disassemble them to see what your C/C++ is turned into.
A couple of books that'll help you understand:-
Reversing
Hacking = The Art of Exploitation
To get an idea, set a breakpoint on some interesting code, and then go to the CPU window.
If you are interested in more, it is easier to compile short fragments with Free Pascal using the -al parameter.
FPC allows to output the generated assembler in a multitude of assembler formats (TASM,MASM,GAS ) using the -A parameter, and you can have the original pascal code interleaved in comments (and more) for easy crossreference.
Because it is compiler generated assembler, as opposed to assembler from disassembled .exe, it is more symbolic and easier to follow.
Familiarity with low level assembly (and I mean low level assembly, not "macros" and that bull) is probably a must. If you really want to read the raw machine code itself directly, usually you would use a hex editor for that. In order to understand what the instructions do, however, most people would use a disassembler to convert that into the appropriate assembly instructions. If you're one of the minority who wants to understand the machine language itself, I think you'd want the Intel® 64 and IA-32 Architectures Software Developer's Manuals. Volume 2 specifically covers the instruction set, which relates to your query about how to read machine code itself and how assembly relates to it.
Both your curiosity and your level of understanding is exactly where I was at one point. I highly recommend Code: The Hidden Language of Computer Hardware and Software. This will not answer all of the questions you ask here but it will shed light on some of the utterly black magic aspects of computers. It's a thick book but highly readable.
ACD See is probably taking advantage of the fact that .EXE files do no error checking on file length or anything beyond the length of the expected portion of the file. Because of this, you can make an .EXE file that will open its self and load everything beyond a given point as data. This is useful because you can then make a .EXE that works on a given set of data by just tacking that data on the end of a suitably written .EXE
(I have no idea what exactly ACD See is so take that with a big grain of salt but I do know that some program are generated that way.)
Every instruction is in machine code kept in a special memory area within the cpu. EARLY INTEL books gave the machine code for their instructions, so one should try to obtain such books so as to understand this. Obviously today machine codeis not easily available. What would be nice is a program which can reverse hex to machine code. Or do it manually _!!
tedious
Related
I'm disassembling "Test Drive III". It's a 1990 DOS game. The *.EXE has MZ format.
I've never dealt with segmentation or DOS, so I would be grateful if you answered some of my questions.
1) The game's system requirements mention 286 CPU, which has protected mode. As far as I know, DOS was 90% real mode software, yet some applications could enter protected mode. Can I be sure that the app uses the CPU in real mode only? IOW, is it guaranteed that the segment registers contain actual offset of the segment instead of an index to segment descriptor?
2) Said system requirements mention 1 MB of RAM. How is this amount of RAM even meant to be accessed if the uppermost 384 KB of the address space are reserved for stuff like MMIO and ROM? I've heard about UMBs (using holes in UMA to access RAM) and about HMA, but it still doesn't allow to access the whole 1 MB of physical RAM. So, was precious RAM just wasted because its physical address happened to be reserved for UMA? Or maybe the game uses some crutches like LIM EMS or XMS?
3) Is CS incremented automatically when the code crosses segment boundaries? Say, the IP reaches 0xFFFF, and what then? Does CS switch to the next segment before next instruction is executed? Same goes for SS. What happens when SP goes all the way down to 0x0000?
4) The MZ header of the executable looks like this:
signature 23117 "0x5a4d"
bytes_in_last_block 117
blocks_in_file 270
num_relocs 0
header_paragraphs 32
min_extra_paragraphs 3349
max_extra_paragraphs 65535
ss 11422
sp 128
checksum 0
ip 16
cs 8385
reloc_table_offset 30
overlay_number 0
Why does it have no relocation information? How is it even meant to run without address fixups? Or is it built as completely position-independent code consisting from program-counter-relative instructions? The game comes with a cheat utility which is also an MZ executable. Despite being much smaller (8448 bytes - so small that it fits into a single segment), it still has relocation information:
offset 1
segment 0
offset 222
segment 0
offset 272
segment 0
This allows IDA to properly disassemble the cheat's code. But the game EXE has nothing, even though it clearly has lots of far pointers.
5) Is there even such thing as 'sections' in DOS? I mean, data section, code (text) section etc? The MZ header points to the stack section, but it has no information about data section. Is data and code completely mixed in DOS programs?
6) Why even having a stack section in EXE file at all? It has nothing but zeroes. Why wasting disk space instead of just saying, "start stack from here"? Like it is done with BSS section?
7) MZ header contains information about initial values of SS and CS. What about DS? What's its initial value?
8) What does an MZ executable have after the exe data? The cheat utility has whole 3507 bytes in the end of the executable file which look like
__exitclean.__exit.__restorezero._abort.DGROUP#.__MMODEL._main._access.
_atexit._close._exit._fclose._fflush._flushall._fopen._freopen._fdopen
._fseek._ftell._printf.__fputc._fputc._fputchar.__FPUTN.__setupio._setvbuf
._tell.__MKNAME._tmpnam._write.__xfclose.__xfflush.___brk.___sbrk._brk._sbrk
.__chmod.__close._ioctl.__IOERROR._isatty._lseek.__LONGTOA._itoa._ultoa.
_ltoa._memcpy._open.__open._strcat._unlink.__VPRINTER.__write._free._malloc
._realloc.__REALCVT.DATASEG#.__Int0Vector.__Int4Vector.__Int5Vector.
__Int6Vector.__C0argc.__C0argv.__C0environ.__envLng.__envseg.__envSize
Is this some kind of debugging symbol information?
Thank you in advance for your help.
Re. 1. No, you can't be sure until you prove otherwise to yourself. One giveaway would be the presence of MOV CR0, ... in the code.
Re. 2. While marketing materials aren't to be confused with an engineering specification, there's a technical reason for this. A 286 CPU could address more than 1M of physical address space. The RAM was only "wasted" in real mode, and only if an EMM (or EMS) driver wasn't used. On 286 systems, the RAM past 640kb was usually "pushed up" to start at the 1088kb mark. The ISA and on-board peripherals' memory address space was mapped 1:1 into the 640-1024kb window. To use the RAM from the real mode needed an EMM or EMS driver. From protected mode, it was simply "there" as soon as you set up the segment descriptor correctly.
If the game actually needed the extra 384kb of RAM over the 640kb available in the real mode, it's a strong indication that it either switched to protected mode or required the services or an EMM or EMS driver.
Re. 3. I wish I remembered that. On reflection, I wish not :) Someone else please edit or answer separately. Hah, I did know it at some point in time :)
Re. 4. You say "[the code] has lots of instructions like call far ptr 18DCh:78Ch". This implies one of three things:
Protected mode is used and the segment part of the address is a selector into the segment descriptor table.
There is code there that relocates those instructions without DOS having to do it.
There is code there that forcibly relocates the game to a constant position in the address space. If the game doesn't use DOS to access on-disk files, it can remove DOS completely and take over, gaining lots of memory in the process. I don't recall whether you could exit from the game back to the command prompt. Some games where "play until you reboot".
Re. 5. The .EXE header does not "point" to any stack, there is no stack section you imply, the concept of sections doesn't exist as far as the .EXE file is concerned. The SS register value is obtained by adding the segment the executable was loaded at with the SS value from the header.
It's true that the linker can arrange sections contiguously in the .EXE file, but such sections' properties are not included in the .EXE header. They often can be reverse-engineered by inspecting the executable.
Re. 6. The SS and SP values in the .EXE header are not file pointers. The EXE file might have a part that maps to the stack, but that's entirely optional.
Re. 7. This is already asked and answered here.
Re. 8. This looks like a debug symbol list. The cheat utility was linked with the debugging information left in. You can have completely arbitrary data there - often it'd various resources (graphics, music, etc.).
I want to test some architecture changes on an already existing architecture (x86) using simulators. However to properly test them and run benchmarks, I might have to make some changes to the instruction set, Is there a way to add these changes to GCC or any other compiler?
Simple solution:
One common approach is to add inline assembly, and encode the instruction bytes directly.
For example:
int main()
{
asm __volatile__ (".byte 0x90\n");
return 0;
}
compiles (gcc -O3) into:
00000000004005a0 <main>:
4005a0: 90 nop
4005a1: 31 c0 xor %eax,%eax
4005a3: c3 retq
So just replace 0x90 with your inst bytes. Of course you wont see the actual instruction on a regular objdump, and the program would likely not run on your system (unless you use one of the nop combinations), but the simulator should recognize it if it's properly implemented there.
Note that you can't expect the compiler to optimize well for you when it doesn't know this instruction, and you should take care and work with inline assembly clobber/input/output options if it changes state (registers, memory), to ensure correctness. Use optimizations only if you must.
Complicated solution
The alternative approach is to implement this in your compiler - it can be done in gcc, but as stated in the comments LLVM is probably one of the best ones to play with, as it's designed as a compiler development platform, but it's still very complicated as LLVM is best suited for IR optimization stages, and is somewhat less friendly when trying to modify the target-specific backends.
Still, it's doable, and you have to do that if you also plan to have your compiler decide when to issue this instruction. I'd suggest to start from the first option though, to see if your simulator even works with this addition, and only then spending time on the compiler side.
If and when you do decide to implement this in LLVM, your best bet is to define it as an intrinsic function, there's relatively more documentation about this in here - http://llvm.org/docs/ExtendingLLVM.html
You can add new instructions, or change existing by modifying group of files in GCC called "machine description". Instruction patterns in <target>.md file, some code in <target>.c file, predicates, constraints and so on. All of these lays in $GCCHOME/gcc/config/<target>/ folder. All of this stuff using on step of generation ASM code from RTL. You can also change cases of emiting instructions by change some other general GCC source files, change SSA tree generation, RTL generation, but all of this a little bit complicated.
A simple explanation what`s happened:
https://www.cse.iitb.ac.in/grc/slides/cgotut-gcc/topic5-md-intro.pdf
It's doable, and I've done it, but it's tedious. It is basically the process of porting the compiler to a new platform, using an existing platform as a model. Somewhere in GCC there is a file that defines the instruction set, and it goes through various processes during compilation that generate further code and data. It's 20+ years since I did it so I have forgotten all the details, sorry.
I'd like to know how is it possible to write something as simple as an Hello World program just by using an Hex Editor. I know that I could use an assembler and assembly language to this at a near machine level but I just want to experiment with really writing machine code in a toy example such as Hello World.
This could be a simple DOS .COM file that I can run on DOSBox. But it would be nice if someone could provide an example for an .EXE file for running it directly on my Windows PC.
This is just pure curiosity. No... I'm not thinking of writing programs directly in binary machine code (I don't even usually write assembly code, I just use C/C++ as my most low level tools most of the time). I just want to see if that's possible to do it, because probably someone had to do it in the very early days of computers.
P.S.:
I know that there are similar questions about this topic around but none provide a working example. I just want a simple example so that it can help me understand how compilers and assemblers generate an executable file. I mean... someone must have done this by hand in the past for the very first programs. Also, for the Windows EXE format there must have been someone at Microsoft that wrote the first tools to generate the format and the way that Windows itself reads it and then executes it.
There's a quite minimalistic but fully working (on Win7, too) exe on corkami/wiki/PE101, every byte of it is explained in the nice graphic. You can type it all by hand in a hex editor, but the paddings may make that a little tedious.
As for the history, yes someone at Microsoft invented the exe format (the old DOS MZ exe format) and he (or someone else at Microsoft) wrote a loader for it and a linker, which is the thing that traditionally turns the output of a compiler ("object files") into executable files. It's possible (and even likely, I would say) that the first exe programs were written by hand, after all they were only meant to test the new loader.
Later, AT&T's COFF format was extended by Microsoft to the PE format, which still has the MZ header and typically (but optionally, it's not in the corkami example, and it can be anything really) includes a small DOS program just to print the message "This program cannot be run in DOS mode".
1) a .com file is the simplest place to start and will run on a dosbox, basically the program starts at something like offset 0x100 in the file, I think the first 0x100 can be whatever, dont remember
2) although true that first programs are often written and assembled by hand into machine code, we are talking about when you add two numbers save them in memory and are so happy that you take the rest of the day off. a "hello world" program that prints stuff to a video card is significantly more complicated. Now you can make a very simple one using dos system calls, and perhaps that is not what you are interested in, perhaps it is.
3) based on 2, anything more complicated than one or a few instructions at a time for testing back in the 1960s or 1970s, even when writing hand assembling a program you write your program in assembler by hand, then assemble it to machine code, then load it. Basically learn assembly language first, then learn how to generate the machine code for it, then start typing those bytes into a hex editor. It is not then 1960s, unless you enjoy excessive pain, learn the above by writing asm, using an assembler to generate the machine code, then use a disassembler to disassemble it and examine the assembly language and the machine code side by side to significantly improve the amount of time it is going to take you to get a working program. If you worked for a chip company before there were operating systems and instruction sets, you would still take advantage of other members of the team, the chip designers, etc for understanding how to make the machine code and arrange it. You wouldnt be coming at this with only high level language experience and doing it all on your own with a hope of success.
4) x86 is a horrible instruction set, if you dont know assembly I strongly discourage you to not learn it first. having an x86 is the worst excuse I have heard to learn x86 first. you already mentioned dosbox so are already planning to emulate/simulate so use a good instruction set and simulate it or buy that hardware (under $50 even under $20 will buy you a board with a much better instruction sets). I recommend simulate/emulate first and in parallel with the hardware if you choose to buy some. If you really want an education write your own simulator it is not difficult at all. Perhaps invent your own instruction set.
5) none of this will help you understand what a compiler does. Knowing assembly language then disassembling the compilers output is your best path toward that knowledge, machine code is not involved, no need to actually run the programs. A compiler goes from the higher level language to a lower level language (C to asm or C++ to asm for example). Then understand what an assembler does, there are many different solutions, both due to history and due to other reasons. The typical solution today is a separate compiler, assembler and linker (your compiler calls the assembler and linker for you unless you tell it not to, the three steps are hidden from view, in fact the compile process may be more than one program that is run to complete that task). Assemblers that output a binary will have to resolve the whole program, assemblers that output to an object will leave holes in the machine code for the linker to fill in. things like branching or calling items in another object that it cannot encode until the linker places things in the binary and knows the spacing/addressing. Also accessing variables that live in other objects.
You are likely not seeing actual examples on hex editing a program because first off it is such a broad question there isnt a simple answer (what operating, system, what system calls or are you creating those, what file format, what hex editor, etc). Also because it is a high level question and problem, the real questions are where do I learn assembly, where do I learn about the relationship between assembly and machine code, where do I learn about system calls (which are not an assembly question, they are unrelated to learning asm, you learn assembly language itself then you learn to USE it as a tool to perform system calls if you cannot perform the system calls directly using a higher language), where do I learn about executable file formats like .com, .exe, coff, elf, etc. What is a good or easy or some adjective, hex editor that runs on xyz operating system or environment. Ask those questions separately and you will find the answers and examples and once you have those answers you will know how to make a program using a hex editor typing in machine code. A shorter example is that you ARE seeing hex examples of complete programs when you see the disassembly of a program posted at SO, some of those are complete programs shown in hex. and if you know the file format you can simply type that stuff into a hex editor.
I make binaries by hand, but I think it's easier in assembly itself than a pure hex editor, where updating anything would be difficult.
The easiest is surely DOS COM format, which you can even type in notepad,
or at least, it's very easy even for a normal Hello World.
The EXE (non DOS format) doesn't require much either see here.
If you're trying to make a PE, you can make a TinyPE.
Most binaries should be available as PE, and EXE and COM.
Not spot on, but this tutorial should give you a better insight into how assembly maps to machinde code (x86 ELF): http://timelessname.com/elfbin/ (especially look at the lower half of the page)
This page is [...] about my attempts at creating the smallest x86 ELF binary that would execute saying Hello World on Ubuntu Linux My first attempts started with C then progressed to x86 assembly and finally to a hexeditor.
It's great to analyze really small executables like these because the mapping between assembly and machine code will be easier to spot. This is also a really interesting article on the subject (not exactly related to your question though): http://www.phreedom.org/research/tinype/ (x86 PE)
I wrote an article on creating executable DOS binary files just by using the ECHO at the command prompt. No other 3rd party HEX utilities or x86 IDEs required!
The technique uses a a combination of keypad - ALT ASCII codes which convert OPCODES to a binary format readable directly under MSDOS. The output is a fully runnable binary *.com file.
http://colinord.blogspot.co.uk/2015/02/extreme-programming-hand-coded.html
Excerpt:
Type the following key commands at the DOS prompt remembering to hold Left ALT.
c:\>Echo LALT-178 LALT-36 LALT-180 LALT-2 LALT-205 LALT-33 LALT-205 LALT-32 > $.com
The codes above are actually opcode values describing an X86 assembly program to print a dollar sign to the screen.
Your prompt should look something similar below when finished. Press enter to build!
c:\>Echo ▓$┤☻═!═ > $.com
Run the file '$.com' and you will see a single dollar ($) character displayed on the screen.
c:\>$.com
$
c:\>
Congratulations! You just created your first hand coded executable file called $.com.
you can do a disassembly and try figure out the machine code for the opcodes you use in your assembler
for example
org 0x100
mov dx,msg
mov ah,0x09
int 0x21
ret
msg db 'hello$'
compiled with nasm -fbin ./a.asm -o ./a.com
has ndisasm a.com deliver the following disassembly:
00000000 BA0801 mov dx,0x108
00000003 B409 mov ah,0x9
00000005 CD21 int 0x21
00000007 C3 ret
00000008 68656C push word 0x6c65
0000000B 6C insb
0000000C 6F outsw
0000000D 24 db 0x24
00000000 to 00000007 are the instructions
so you can play with the ba0801 machine code, using some hex editor, try changing it to ba0901, and only 'ello' will be printed, you can play around with your hex editor and pad stuff out with NOP, which is 0x90 in machine code, for example:
00000000: ba 50 01 90 90 90 90 90 90 90 90 90 90 90 90 90 .#..............
00000010: b4 09 90 90 90 90 90 90 90 90 90 90 90 90 90 90 ................
00000020: cd 21 90 90 90 90 90 90 90 90 90 90 90 90 90 90 .!..............
00000030: c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 ................
00000040: 71 77 65 72 74 79 75 69 61 73 64 66 67 68 6a 24 qwertyuiasdfghj$
00000050: 61 73 64 66 67 68 6a 6b 61 73 64 66 67 68 6a 24 asdfghjkasdfghj$
00000060: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- ----------------
if you save this with the extension .com you can run it in DosBox
I'd like to generate pseudo-random ARM instructions. Via assembler directives, I can tell gcc what mode I'm in, and it will complain if I try a set of opcodes and operands that's not legal in that mode, so it must have some internal listing of what can be done in which mode. Where does that live? Would it be easier to extract that info from LLVM?
Is this question "not even wrong"? Should I try a different approach entirely?
To answer my own question, this is actually really easy to do from arm.md and and constraints.md in gcc/config/arm/. I probably spent more time answering asking this question and answering comments for it than I did figuring this out. Turns out I just need to look for 'TARGET_THUMB1', until I get around to implementing thumb2.
For the ARM family the buck stops at the ARM ARM (ARM Architectural Reference Manual). There is an ARM instruction set section and a Thumb instruction set section. Within both each instruction tells you what generation (ARMvX where X is some number like 4 (arm7), or 5 (arm9 time frame) ,etc). Since the opcode and pseudo code is listed for each instruction you should be able to figure out what is a real instruction and, if any, are syntax to save typing on another (push and pop for example).
With the Cortex-m3 and thumb2 in particular you also need to look at the TRM (Technical Reference Manual) as well. ARM has, I forget the name, a universal syntax they are trying to use that should work on both Thumb and ARM. For example on an ARM you have three register instructions:
add r1,r1,r2
In thumb there are only two register operations
add r1,r2
The desire basically is to meet in the middle or I would say more accurately to encourage ARM assemblers to parse Thumb instructions and encode them with the equivalent ARM instruction without complaining. This may have started with thumb and not thumb2, I have always separated the two syntaxes in my code until recently (and I still generally use ARM syntax for ARM and Thumb for Thumb).
And then yes you have to see what the specific implementation of the assembler tool is, in your case binutils. And it sounds like you have found the binutils/gnu secret decoder ring.
I wrote a program about 10 years ago in Visual Basic 6 which was basically a full-screen game similar to Breakout / Arkanoid but had 'demoscene'-style backgrounds. I found the program, but not the source code. Back then I hard-coded the display mode to 800x600x24, and the program crashes whenever I try to run it as a result. No virtual machine seems to support 24-bit display when the host display mode is 16/32-bit. It uses DirectX 7 so DOSBox is no use.
I've tried all sorts of decompiler and at best they give me the form names and a bunch of assembly calls which mean nothing to me. The display mode setting was a DirectX 7 call but there's no clear reference to it in the decompilation.
In this situation, is there any pointers on how I can:
pin-point the function call in the program which is setting the display mode to 800x600x24 (ResHacker maybe?) and change the value being passed to it so it sets 800x600x32
view/intercept DirectX calls being made while it's running
or if that's not possible, at least
run the program in an environment that emulates a 24-bit display
I don't need to recover the source code (as nice as it would be) so much as just want to get it running.
One technique you could try in your disassembler is to do a search for the constants you remember, but as the actual bytes that would be contained within the executable. I guess you used the DirectDraw SetDisplayMode call, which is a COM object so can't be as easily traced to/from an entry point in a DLL. It takes parameters for width, height and bits per pixel and they are DWORDs (32-bit) so do a search for "58 02 00 00", "20 03 00 00" and "18 00 00 00". Hopefully that will narrow it down to what you need to change.
By the way which disassembler are you using?
This approach may be complicated somewhat if your VB6 program compiled to p-code rather than native code as you'll just get a huge chunk of data that represents the program rather than useful assembler instructions.
Check this:
http://www.sevenforums.com/tutorials/258-color-bit-depth-display-settings.html
If your graphics card doesn't have an entry for 24-bit display....I guess hacking your code's the only possibility. That or finding an old machine to throw windows 95 on :P.