(this my first question, excuse me for any mistakes)
I was messing around with debug.exe and tried to alter the BIOS date stored in address range FFFF:0005 to FFFF:000C.
-d FFFF:5 L 8
FFFF:0000 30 31 2F-30 31 2F 39 32 01/01/92
I finally figured out that to move to the address i want to modify i had to point the DS register to it and not the CS as erroneously stated in some sites(e.g. here)
-r DS
DS=073F
:FFFF
I also figured out that I can use the whole address to modify the exact memory address I want.
-e FFFF:000b
FFFF:000B 39.31 32.31
but then the output of dump command remained unchanged!!!
-d FFFF:5 L 8
FFFF:0000 30 31 2F-30 31 2F 39 32 01/01/92
I am suspecting that there are maybe some "protected" areas in memory I cannot modify, but I couldn't find any documentation about that is why I am asking. Can anyone possibly explain me why and how this is happening?
Thank you
P.S. Note that I am using DosBox to emulate this and to not brick my computer!(maybe this is the problem?)
As the comments suggest, you are writing to ROM, so the values there can't be changed by your code. On modern machines you would get some sort of error as feedback for doing this, but on old hardware it's very common for writes to ROM to be silently ignored. In other words, the CPU will perform the requested operation anyway, but that operation will have no effect on the memory.
Related
I'm currently dealing with a loss of 320gb of data.
During the installation of windows, I accidentally deleted a partition (currently in a "not allocated space" state) please see example picture.
I tried Autopsy and Foremost, but both are not detecting any data on this SATA HHD.
Is there any advice to proceed with?
The disk is untouched and nothing has been done after that operation
Stage of data loss
You have to recreate partition table. To do that you have to find where your deleted partition(s) started with some hex editor, write down where they start and then create new partition table.
Your picture says you're under windows. If partition is windows then it will start like this (hexdump -C of first 8 bytes):
00000000 eb 52 90 4e 54 46 53 20 |.R.NTFS |
which you can use as search pattern.
I would advise to create new partition table under linux as you can quickly check your work with mount. Unless you have some magical tools for windows. You can use any live linux you like (here is small list with desriptions) even in virtual box.
I did a spring cleaning in my code by splitting it up in more Go packages, mainly to help reusability (each "building block" in its own package).
After fixing the import errors, I discovered that my program suddenly won't build. Running "go build" returns a nosplit stack overflow error.
robot main.init: nosplit stack overflow
120 guaranteed after split check in main.init
112 on entry to robot/web.init
104 on entry to robot/controller.init
96 on entry to robot/slam.init
88 on entry to robot/slam/hector.init
80 on entry to hectormapping/map/mapimages.init
72 on entry to hectormapping/map/maprep.init
64 on entry to hectormapping/map/mapproccontainer.init
56 on entry to hectormapping/scanmatcher.init
48 on entry to hectormapping/map/gridmap/occbase.init
40 on entry to hectormapping/map/gridmap/base.init
32 on entry to hectormapping/map/gridmap.init
24 on entry to github.com/skelterjohn/go%2ematrix.init
16 on entry to math.init
8 on entry to math.init┬À1
0 on entry to runtime.panicindex
-8 on entry to runtime.morestack00
runtime.main: nosplit stack overflow
120 guaranteed after split check in runtime.main
128 after runtime.main uses -8
120 on entry to main.init
112 on entry to robot/web.init
104 on entry to robot/controller.init
96 on entry to robot/slam.init
88 on entry to robot/slam/hector.init
80 on entry to hectormapping/map/mapimages.init
72 on entry to hectormapping/map/maprep.init
64 on entry to hectormapping/map/mapproccontainer.init
56 on entry to hectormapping/scanmatcher.init
48 on entry to hectormapping/map/gridmap/occbase.init
40 on entry to hectormapping/map/gridmap/base.init
32 on entry to hectormapping/map/gridmap.init
24 on entry to github.com/skelterjohn/go%2ematrix.init
16 on entry to math.init
8 on entry to math.init┬À1
0 on entry to runtime.panicindex
-8 on entry to runtime.morestack00
Does anyone know what this is about? I can't find much documentation as to what might be causing it, except that for some cases this is a bug that supposedly is fixed.
Some of the code was split into a new folder in the "src" folder, so that the file structure is now:
src/robot/main.go (main() lives here)
src/robot/(...) (application-specific packages)
src/hectormapping/(...) (stand-alone package used in "robot")
I am using Go 1.0.3 on Windows 7 (x64).
This seems to be the same as described here which was said to be fixed in tip. The corresponding fix can be reviewed here.
To summarize the problem as I am seeing it:
Split stacking is used for growing stacks instead of the conventional fixed memory area. This has the benefit that more threads can be spawned, as only the needed stack memory is actually reserved. The problem here seems to be that the linker marks functions that don't use memory on the split stack accidentally as 'nosplit' because it doesn't find the split stack prologue. This leads to the linker calculating a wrong stack limit, which in turn lets the linker think there's no space and throws the error message at you.
Sadly, the only way of getting the tip version is to compile it by yourself. As Nick Craig-Wood already mentioned, you can find the instructions here. If you really really can't upgrade, you could try to work around this by allocation some arbitrary local variable in your init functions. But this is very messy of course.
I'd like to know how is it possible to write something as simple as an Hello World program just by using an Hex Editor. I know that I could use an assembler and assembly language to this at a near machine level but I just want to experiment with really writing machine code in a toy example such as Hello World.
This could be a simple DOS .COM file that I can run on DOSBox. But it would be nice if someone could provide an example for an .EXE file for running it directly on my Windows PC.
This is just pure curiosity. No... I'm not thinking of writing programs directly in binary machine code (I don't even usually write assembly code, I just use C/C++ as my most low level tools most of the time). I just want to see if that's possible to do it, because probably someone had to do it in the very early days of computers.
P.S.:
I know that there are similar questions about this topic around but none provide a working example. I just want a simple example so that it can help me understand how compilers and assemblers generate an executable file. I mean... someone must have done this by hand in the past for the very first programs. Also, for the Windows EXE format there must have been someone at Microsoft that wrote the first tools to generate the format and the way that Windows itself reads it and then executes it.
There's a quite minimalistic but fully working (on Win7, too) exe on corkami/wiki/PE101, every byte of it is explained in the nice graphic. You can type it all by hand in a hex editor, but the paddings may make that a little tedious.
As for the history, yes someone at Microsoft invented the exe format (the old DOS MZ exe format) and he (or someone else at Microsoft) wrote a loader for it and a linker, which is the thing that traditionally turns the output of a compiler ("object files") into executable files. It's possible (and even likely, I would say) that the first exe programs were written by hand, after all they were only meant to test the new loader.
Later, AT&T's COFF format was extended by Microsoft to the PE format, which still has the MZ header and typically (but optionally, it's not in the corkami example, and it can be anything really) includes a small DOS program just to print the message "This program cannot be run in DOS mode".
1) a .com file is the simplest place to start and will run on a dosbox, basically the program starts at something like offset 0x100 in the file, I think the first 0x100 can be whatever, dont remember
2) although true that first programs are often written and assembled by hand into machine code, we are talking about when you add two numbers save them in memory and are so happy that you take the rest of the day off. a "hello world" program that prints stuff to a video card is significantly more complicated. Now you can make a very simple one using dos system calls, and perhaps that is not what you are interested in, perhaps it is.
3) based on 2, anything more complicated than one or a few instructions at a time for testing back in the 1960s or 1970s, even when writing hand assembling a program you write your program in assembler by hand, then assemble it to machine code, then load it. Basically learn assembly language first, then learn how to generate the machine code for it, then start typing those bytes into a hex editor. It is not then 1960s, unless you enjoy excessive pain, learn the above by writing asm, using an assembler to generate the machine code, then use a disassembler to disassemble it and examine the assembly language and the machine code side by side to significantly improve the amount of time it is going to take you to get a working program. If you worked for a chip company before there were operating systems and instruction sets, you would still take advantage of other members of the team, the chip designers, etc for understanding how to make the machine code and arrange it. You wouldnt be coming at this with only high level language experience and doing it all on your own with a hope of success.
4) x86 is a horrible instruction set, if you dont know assembly I strongly discourage you to not learn it first. having an x86 is the worst excuse I have heard to learn x86 first. you already mentioned dosbox so are already planning to emulate/simulate so use a good instruction set and simulate it or buy that hardware (under $50 even under $20 will buy you a board with a much better instruction sets). I recommend simulate/emulate first and in parallel with the hardware if you choose to buy some. If you really want an education write your own simulator it is not difficult at all. Perhaps invent your own instruction set.
5) none of this will help you understand what a compiler does. Knowing assembly language then disassembling the compilers output is your best path toward that knowledge, machine code is not involved, no need to actually run the programs. A compiler goes from the higher level language to a lower level language (C to asm or C++ to asm for example). Then understand what an assembler does, there are many different solutions, both due to history and due to other reasons. The typical solution today is a separate compiler, assembler and linker (your compiler calls the assembler and linker for you unless you tell it not to, the three steps are hidden from view, in fact the compile process may be more than one program that is run to complete that task). Assemblers that output a binary will have to resolve the whole program, assemblers that output to an object will leave holes in the machine code for the linker to fill in. things like branching or calling items in another object that it cannot encode until the linker places things in the binary and knows the spacing/addressing. Also accessing variables that live in other objects.
You are likely not seeing actual examples on hex editing a program because first off it is such a broad question there isnt a simple answer (what operating, system, what system calls or are you creating those, what file format, what hex editor, etc). Also because it is a high level question and problem, the real questions are where do I learn assembly, where do I learn about the relationship between assembly and machine code, where do I learn about system calls (which are not an assembly question, they are unrelated to learning asm, you learn assembly language itself then you learn to USE it as a tool to perform system calls if you cannot perform the system calls directly using a higher language), where do I learn about executable file formats like .com, .exe, coff, elf, etc. What is a good or easy or some adjective, hex editor that runs on xyz operating system or environment. Ask those questions separately and you will find the answers and examples and once you have those answers you will know how to make a program using a hex editor typing in machine code. A shorter example is that you ARE seeing hex examples of complete programs when you see the disassembly of a program posted at SO, some of those are complete programs shown in hex. and if you know the file format you can simply type that stuff into a hex editor.
I make binaries by hand, but I think it's easier in assembly itself than a pure hex editor, where updating anything would be difficult.
The easiest is surely DOS COM format, which you can even type in notepad,
or at least, it's very easy even for a normal Hello World.
The EXE (non DOS format) doesn't require much either see here.
If you're trying to make a PE, you can make a TinyPE.
Most binaries should be available as PE, and EXE and COM.
Not spot on, but this tutorial should give you a better insight into how assembly maps to machinde code (x86 ELF): http://timelessname.com/elfbin/ (especially look at the lower half of the page)
This page is [...] about my attempts at creating the smallest x86 ELF binary that would execute saying Hello World on Ubuntu Linux My first attempts started with C then progressed to x86 assembly and finally to a hexeditor.
It's great to analyze really small executables like these because the mapping between assembly and machine code will be easier to spot. This is also a really interesting article on the subject (not exactly related to your question though): http://www.phreedom.org/research/tinype/ (x86 PE)
I wrote an article on creating executable DOS binary files just by using the ECHO at the command prompt. No other 3rd party HEX utilities or x86 IDEs required!
The technique uses a a combination of keypad - ALT ASCII codes which convert OPCODES to a binary format readable directly under MSDOS. The output is a fully runnable binary *.com file.
http://colinord.blogspot.co.uk/2015/02/extreme-programming-hand-coded.html
Excerpt:
Type the following key commands at the DOS prompt remembering to hold Left ALT.
c:\>Echo LALT-178 LALT-36 LALT-180 LALT-2 LALT-205 LALT-33 LALT-205 LALT-32 > $.com
The codes above are actually opcode values describing an X86 assembly program to print a dollar sign to the screen.
Your prompt should look something similar below when finished. Press enter to build!
c:\>Echo ▓$┤☻═!═ > $.com
Run the file '$.com' and you will see a single dollar ($) character displayed on the screen.
c:\>$.com
$
c:\>
Congratulations! You just created your first hand coded executable file called $.com.
you can do a disassembly and try figure out the machine code for the opcodes you use in your assembler
for example
org 0x100
mov dx,msg
mov ah,0x09
int 0x21
ret
msg db 'hello$'
compiled with nasm -fbin ./a.asm -o ./a.com
has ndisasm a.com deliver the following disassembly:
00000000 BA0801 mov dx,0x108
00000003 B409 mov ah,0x9
00000005 CD21 int 0x21
00000007 C3 ret
00000008 68656C push word 0x6c65
0000000B 6C insb
0000000C 6F outsw
0000000D 24 db 0x24
00000000 to 00000007 are the instructions
so you can play with the ba0801 machine code, using some hex editor, try changing it to ba0901, and only 'ello' will be printed, you can play around with your hex editor and pad stuff out with NOP, which is 0x90 in machine code, for example:
00000000: ba 50 01 90 90 90 90 90 90 90 90 90 90 90 90 90 .#..............
00000010: b4 09 90 90 90 90 90 90 90 90 90 90 90 90 90 90 ................
00000020: cd 21 90 90 90 90 90 90 90 90 90 90 90 90 90 90 .!..............
00000030: c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 ................
00000040: 71 77 65 72 74 79 75 69 61 73 64 66 67 68 6a 24 qwertyuiasdfghj$
00000050: 61 73 64 66 67 68 6a 6b 61 73 64 66 67 68 6a 24 asdfghjkasdfghj$
00000060: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- ----------------
if you save this with the extension .com you can run it in DosBox
I wrote a program about 10 years ago in Visual Basic 6 which was basically a full-screen game similar to Breakout / Arkanoid but had 'demoscene'-style backgrounds. I found the program, but not the source code. Back then I hard-coded the display mode to 800x600x24, and the program crashes whenever I try to run it as a result. No virtual machine seems to support 24-bit display when the host display mode is 16/32-bit. It uses DirectX 7 so DOSBox is no use.
I've tried all sorts of decompiler and at best they give me the form names and a bunch of assembly calls which mean nothing to me. The display mode setting was a DirectX 7 call but there's no clear reference to it in the decompilation.
In this situation, is there any pointers on how I can:
pin-point the function call in the program which is setting the display mode to 800x600x24 (ResHacker maybe?) and change the value being passed to it so it sets 800x600x32
view/intercept DirectX calls being made while it's running
or if that's not possible, at least
run the program in an environment that emulates a 24-bit display
I don't need to recover the source code (as nice as it would be) so much as just want to get it running.
One technique you could try in your disassembler is to do a search for the constants you remember, but as the actual bytes that would be contained within the executable. I guess you used the DirectDraw SetDisplayMode call, which is a COM object so can't be as easily traced to/from an entry point in a DLL. It takes parameters for width, height and bits per pixel and they are DWORDs (32-bit) so do a search for "58 02 00 00", "20 03 00 00" and "18 00 00 00". Hopefully that will narrow it down to what you need to change.
By the way which disassembler are you using?
This approach may be complicated somewhat if your VB6 program compiled to p-code rather than native code as you'll just get a huge chunk of data that represents the program rather than useful assembler instructions.
Check this:
http://www.sevenforums.com/tutorials/258-color-bit-depth-display-settings.html
If your graphics card doesn't have an entry for 24-bit display....I guess hacking your code's the only possibility. That or finding an old machine to throw windows 95 on :P.
I am not well acquainted to the compiler magic. The act of transforming human-readable code (or the not really readable Assembly instructions) into machine code is, for me, rocket science combined with sorcery.
I will narrow down the subject of this question to Win32 executables (.exe). When I open these files up in a specialized viewer, I can find strings (usually 16b per character) scattered at various places, but the rest is just garbage. I suppose the unreadable part (majority) is the machine code (or maybe resources, such as images etc...).
Is there any straightforward way of reading the machine code? Opening the exe as a file stream and reading it byte by byte, how could one turn these individual bytes into Assembly? Is there a straightforward mapping between these instruction bytes and the Assembly instruction?
How is the .exe written? Four bytes per instruction? More? Less? I have noticed some applications can create executable files just like that: for example, in ACD See you can export a series of images into a slideshow. But this does not necessarily have to be a SWF slideshow, ACD See is also capable of producing EXEcutable presentations. How is that done?
How can I understand what goes on inside an EXE file?
OllyDbg is an awesome tool that disassembles an EXE into readable instructions and allows you to execute the instructions one-by-one. It also tells you what API functions the program uses and if possible, the arguments that it provides (as long as the arguments are found on the stack).
Generally speaking, CPU instructions are of variable length, some are one byte, others are two, some three, some four etc. It mostly depends on the kind of data that the instruction expects. Some instructions are generalised, like "mov" which tells the CPU to move data from a CPU register to a place in memory, or vice versa. In reality, there are many different "mov" instructions, ones for handling 8-bit, 16-bit, 32-bit data, ones for moving data from different registers and so on.
You could pick up Dr. Paul Carter's PC Assembly Language Tutorial which is a free entry level book that talks about assembly and how the Intel 386 CPU operates. Most of it is applicable even to modern day consumer Intel CPUs.
The EXE format is specific to Windows. The entry-point (i.e. the first executable instruction) is usually found at the same place within the EXE file. It's all kind of difficult to explain all at once, but the resources I've provided should help cure at least some of your curiosity! :)
You need a disassembler which will turn the machine code into assembly language. This Wikipedia link describes the process and provides links to free disassemblers. Of course, as you say you don't understand assembly language, this may not be very informative - what exactly are you trying to do here?
You can use debug from the command line, but that's hard.
C:\WINDOWS>debug taskman.exe
-u
0D69:0000 0E PUSH CS
0D69:0001 1F POP DS
0D69:0002 BA0E00 MOV DX,000E
0D69:0005 B409 MOV AH,09
0D69:0007 CD21 INT 21
0D69:0009 B8014C MOV AX,4C01
0D69:000C CD21 INT 21
0D69:000E 54 PUSH SP
0D69:000F 68 DB 68
0D69:0010 69 DB 69
0D69:0011 7320 JNB 0033
0D69:0013 7072 JO 0087
0D69:0015 6F DB 6F
0D69:0016 67 DB 67
0D69:0017 7261 JB 007A
0D69:0019 6D DB 6D
0D69:001A 206361 AND [BP+DI+61],AH
0D69:001D 6E DB 6E
0D69:001E 6E DB 6E
0D69:001F 6F DB 6F
The executable file you see is Microsofts PE (Portable Executable) format. It is essentially a container, which holds some operating system specific data about a program and the program data itself split into several sections. For example code, resources, static data are stored in seperate sections.
The format of the section depends on what is in it. The code section holds the machine code according to the executable target architecture. In the most common cases this is Intel x86 or AMD-64 (same as EM64T) for Microsoft PE binaries. The format of the machine code is CISC and originates back to the 8086 and earlier. The important aspect of CISC is that its instruction size is not constant, you have to start reading at the right place to get something valuable out of it. Intel publishes good manuals on the x86/x64 instruction set.
You can use a disassembler to view the machine code directly. In combination with the manuals you can guess the source code most of the time.
And then there's MSIL EXE: The .NET executables holding Microsofts Intermediate Language, these do not contain machine specific code, but .NET CIL code. The specifications for that are available online at the ECMA.
These can be viewed with a tool such as Reflector.
The contents of the EXE file are described in Portable Executable. It contains code, data, and instructions to OS on how to load the file.
There is an 1:1 mapping between machine code and assembly. A disassembler program will perform the reverse operation.
There isn't a fixed number of bytes per instruction on i386. Some are a single byte, some are much longer.
Just relating to this question, anyone still read things like
CD 21?
I remembered Sandra Bullock in one show, actually reading a screenful of hex numbers and figure out what the program does. Sort of like the current version of reading Matrix code.
if you do read stuff like CD 21, how do you remember the different various combinations?
Win32 exe format on MSDN
I'd suggest taking an bit of Windows C source code and build and start debugging it in Visual Studio. Switch to the disassembly view and step over the commands. You can see how the C code has been compiled into machine code - and watch it run step-by-step.
If it's as foreign to you as it seems, I don't think a debugger or disassembler is going to help - you need to learn assembler programming first; study the architecture of the processor (plenty of documentation downloadable from Intel). And then since most machine code is generated by compilers, you'll need to understand how compilers generate code - the simplest way to write lots of small programs and then disassemble them to see what your C/C++ is turned into.
A couple of books that'll help you understand:-
Reversing
Hacking = The Art of Exploitation
To get an idea, set a breakpoint on some interesting code, and then go to the CPU window.
If you are interested in more, it is easier to compile short fragments with Free Pascal using the -al parameter.
FPC allows to output the generated assembler in a multitude of assembler formats (TASM,MASM,GAS ) using the -A parameter, and you can have the original pascal code interleaved in comments (and more) for easy crossreference.
Because it is compiler generated assembler, as opposed to assembler from disassembled .exe, it is more symbolic and easier to follow.
Familiarity with low level assembly (and I mean low level assembly, not "macros" and that bull) is probably a must. If you really want to read the raw machine code itself directly, usually you would use a hex editor for that. In order to understand what the instructions do, however, most people would use a disassembler to convert that into the appropriate assembly instructions. If you're one of the minority who wants to understand the machine language itself, I think you'd want the Intel® 64 and IA-32 Architectures Software Developer's Manuals. Volume 2 specifically covers the instruction set, which relates to your query about how to read machine code itself and how assembly relates to it.
Both your curiosity and your level of understanding is exactly where I was at one point. I highly recommend Code: The Hidden Language of Computer Hardware and Software. This will not answer all of the questions you ask here but it will shed light on some of the utterly black magic aspects of computers. It's a thick book but highly readable.
ACD See is probably taking advantage of the fact that .EXE files do no error checking on file length or anything beyond the length of the expected portion of the file. Because of this, you can make an .EXE file that will open its self and load everything beyond a given point as data. This is useful because you can then make a .EXE that works on a given set of data by just tacking that data on the end of a suitably written .EXE
(I have no idea what exactly ACD See is so take that with a big grain of salt but I do know that some program are generated that way.)
Every instruction is in machine code kept in a special memory area within the cpu. EARLY INTEL books gave the machine code for their instructions, so one should try to obtain such books so as to understand this. Obviously today machine codeis not easily available. What would be nice is a program which can reverse hex to machine code. Or do it manually _!!
tedious