(this my first question, excuse me for any mistakes)
I was messing around with debug.exe and tried to alter the BIOS date stored in address range FFFF:0005 to FFFF:000C.
-d FFFF:5 L 8
FFFF:0000 30 31 2F-30 31 2F 39 32 01/01/92
I finally figured out that to move to the address i want to modify i had to point the DS register to it and not the CS as erroneously stated in some sites(e.g. here)
-r DS
DS=073F
:FFFF
I also figured out that I can use the whole address to modify the exact memory address I want.
-e FFFF:000b
FFFF:000B 39.31 32.31
but then the output of dump command remained unchanged!!!
-d FFFF:5 L 8
FFFF:0000 30 31 2F-30 31 2F 39 32 01/01/92
I am suspecting that there are maybe some "protected" areas in memory I cannot modify, but I couldn't find any documentation about that is why I am asking. Can anyone possibly explain me why and how this is happening?
Thank you
P.S. Note that I am using DosBox to emulate this and to not brick my computer!(maybe this is the problem?)
As the comments suggest, you are writing to ROM, so the values there can't be changed by your code. On modern machines you would get some sort of error as feedback for doing this, but on old hardware it's very common for writes to ROM to be silently ignored. In other words, the CPU will perform the requested operation anyway, but that operation will have no effect on the memory.
I did a spring cleaning in my code by splitting it up in more Go packages, mainly to help reusability (each "building block" in its own package).
After fixing the import errors, I discovered that my program suddenly won't build. Running "go build" returns a nosplit stack overflow error.
robot main.init: nosplit stack overflow
120 guaranteed after split check in main.init
112 on entry to robot/web.init
104 on entry to robot/controller.init
96 on entry to robot/slam.init
88 on entry to robot/slam/hector.init
80 on entry to hectormapping/map/mapimages.init
72 on entry to hectormapping/map/maprep.init
64 on entry to hectormapping/map/mapproccontainer.init
56 on entry to hectormapping/scanmatcher.init
48 on entry to hectormapping/map/gridmap/occbase.init
40 on entry to hectormapping/map/gridmap/base.init
32 on entry to hectormapping/map/gridmap.init
24 on entry to github.com/skelterjohn/go%2ematrix.init
16 on entry to math.init
8 on entry to math.init┬À1
0 on entry to runtime.panicindex
-8 on entry to runtime.morestack00
runtime.main: nosplit stack overflow
120 guaranteed after split check in runtime.main
128 after runtime.main uses -8
120 on entry to main.init
112 on entry to robot/web.init
104 on entry to robot/controller.init
96 on entry to robot/slam.init
88 on entry to robot/slam/hector.init
80 on entry to hectormapping/map/mapimages.init
72 on entry to hectormapping/map/maprep.init
64 on entry to hectormapping/map/mapproccontainer.init
56 on entry to hectormapping/scanmatcher.init
48 on entry to hectormapping/map/gridmap/occbase.init
40 on entry to hectormapping/map/gridmap/base.init
32 on entry to hectormapping/map/gridmap.init
24 on entry to github.com/skelterjohn/go%2ematrix.init
16 on entry to math.init
8 on entry to math.init┬À1
0 on entry to runtime.panicindex
-8 on entry to runtime.morestack00
Does anyone know what this is about? I can't find much documentation as to what might be causing it, except that for some cases this is a bug that supposedly is fixed.
Some of the code was split into a new folder in the "src" folder, so that the file structure is now:
src/robot/main.go (main() lives here)
src/robot/(...) (application-specific packages)
src/hectormapping/(...) (stand-alone package used in "robot")
I am using Go 1.0.3 on Windows 7 (x64).
This seems to be the same as described here which was said to be fixed in tip. The corresponding fix can be reviewed here.
To summarize the problem as I am seeing it:
Split stacking is used for growing stacks instead of the conventional fixed memory area. This has the benefit that more threads can be spawned, as only the needed stack memory is actually reserved. The problem here seems to be that the linker marks functions that don't use memory on the split stack accidentally as 'nosplit' because it doesn't find the split stack prologue. This leads to the linker calculating a wrong stack limit, which in turn lets the linker think there's no space and throws the error message at you.
Sadly, the only way of getting the tip version is to compile it by yourself. As Nick Craig-Wood already mentioned, you can find the instructions here. If you really really can't upgrade, you could try to work around this by allocation some arbitrary local variable in your init functions. But this is very messy of course.
I'd like to know how is it possible to write something as simple as an Hello World program just by using an Hex Editor. I know that I could use an assembler and assembly language to this at a near machine level but I just want to experiment with really writing machine code in a toy example such as Hello World.
This could be a simple DOS .COM file that I can run on DOSBox. But it would be nice if someone could provide an example for an .EXE file for running it directly on my Windows PC.
This is just pure curiosity. No... I'm not thinking of writing programs directly in binary machine code (I don't even usually write assembly code, I just use C/C++ as my most low level tools most of the time). I just want to see if that's possible to do it, because probably someone had to do it in the very early days of computers.
P.S.:
I know that there are similar questions about this topic around but none provide a working example. I just want a simple example so that it can help me understand how compilers and assemblers generate an executable file. I mean... someone must have done this by hand in the past for the very first programs. Also, for the Windows EXE format there must have been someone at Microsoft that wrote the first tools to generate the format and the way that Windows itself reads it and then executes it.
There's a quite minimalistic but fully working (on Win7, too) exe on corkami/wiki/PE101, every byte of it is explained in the nice graphic. You can type it all by hand in a hex editor, but the paddings may make that a little tedious.
As for the history, yes someone at Microsoft invented the exe format (the old DOS MZ exe format) and he (or someone else at Microsoft) wrote a loader for it and a linker, which is the thing that traditionally turns the output of a compiler ("object files") into executable files. It's possible (and even likely, I would say) that the first exe programs were written by hand, after all they were only meant to test the new loader.
Later, AT&T's COFF format was extended by Microsoft to the PE format, which still has the MZ header and typically (but optionally, it's not in the corkami example, and it can be anything really) includes a small DOS program just to print the message "This program cannot be run in DOS mode".
1) a .com file is the simplest place to start and will run on a dosbox, basically the program starts at something like offset 0x100 in the file, I think the first 0x100 can be whatever, dont remember
2) although true that first programs are often written and assembled by hand into machine code, we are talking about when you add two numbers save them in memory and are so happy that you take the rest of the day off. a "hello world" program that prints stuff to a video card is significantly more complicated. Now you can make a very simple one using dos system calls, and perhaps that is not what you are interested in, perhaps it is.
3) based on 2, anything more complicated than one or a few instructions at a time for testing back in the 1960s or 1970s, even when writing hand assembling a program you write your program in assembler by hand, then assemble it to machine code, then load it. Basically learn assembly language first, then learn how to generate the machine code for it, then start typing those bytes into a hex editor. It is not then 1960s, unless you enjoy excessive pain, learn the above by writing asm, using an assembler to generate the machine code, then use a disassembler to disassemble it and examine the assembly language and the machine code side by side to significantly improve the amount of time it is going to take you to get a working program. If you worked for a chip company before there were operating systems and instruction sets, you would still take advantage of other members of the team, the chip designers, etc for understanding how to make the machine code and arrange it. You wouldnt be coming at this with only high level language experience and doing it all on your own with a hope of success.
4) x86 is a horrible instruction set, if you dont know assembly I strongly discourage you to not learn it first. having an x86 is the worst excuse I have heard to learn x86 first. you already mentioned dosbox so are already planning to emulate/simulate so use a good instruction set and simulate it or buy that hardware (under $50 even under $20 will buy you a board with a much better instruction sets). I recommend simulate/emulate first and in parallel with the hardware if you choose to buy some. If you really want an education write your own simulator it is not difficult at all. Perhaps invent your own instruction set.
5) none of this will help you understand what a compiler does. Knowing assembly language then disassembling the compilers output is your best path toward that knowledge, machine code is not involved, no need to actually run the programs. A compiler goes from the higher level language to a lower level language (C to asm or C++ to asm for example). Then understand what an assembler does, there are many different solutions, both due to history and due to other reasons. The typical solution today is a separate compiler, assembler and linker (your compiler calls the assembler and linker for you unless you tell it not to, the three steps are hidden from view, in fact the compile process may be more than one program that is run to complete that task). Assemblers that output a binary will have to resolve the whole program, assemblers that output to an object will leave holes in the machine code for the linker to fill in. things like branching or calling items in another object that it cannot encode until the linker places things in the binary and knows the spacing/addressing. Also accessing variables that live in other objects.
You are likely not seeing actual examples on hex editing a program because first off it is such a broad question there isnt a simple answer (what operating, system, what system calls or are you creating those, what file format, what hex editor, etc). Also because it is a high level question and problem, the real questions are where do I learn assembly, where do I learn about the relationship between assembly and machine code, where do I learn about system calls (which are not an assembly question, they are unrelated to learning asm, you learn assembly language itself then you learn to USE it as a tool to perform system calls if you cannot perform the system calls directly using a higher language), where do I learn about executable file formats like .com, .exe, coff, elf, etc. What is a good or easy or some adjective, hex editor that runs on xyz operating system or environment. Ask those questions separately and you will find the answers and examples and once you have those answers you will know how to make a program using a hex editor typing in machine code. A shorter example is that you ARE seeing hex examples of complete programs when you see the disassembly of a program posted at SO, some of those are complete programs shown in hex. and if you know the file format you can simply type that stuff into a hex editor.
I make binaries by hand, but I think it's easier in assembly itself than a pure hex editor, where updating anything would be difficult.
The easiest is surely DOS COM format, which you can even type in notepad,
or at least, it's very easy even for a normal Hello World.
The EXE (non DOS format) doesn't require much either see here.
If you're trying to make a PE, you can make a TinyPE.
Most binaries should be available as PE, and EXE and COM.
Not spot on, but this tutorial should give you a better insight into how assembly maps to machinde code (x86 ELF): http://timelessname.com/elfbin/ (especially look at the lower half of the page)
This page is [...] about my attempts at creating the smallest x86 ELF binary that would execute saying Hello World on Ubuntu Linux My first attempts started with C then progressed to x86 assembly and finally to a hexeditor.
It's great to analyze really small executables like these because the mapping between assembly and machine code will be easier to spot. This is also a really interesting article on the subject (not exactly related to your question though): http://www.phreedom.org/research/tinype/ (x86 PE)
I wrote an article on creating executable DOS binary files just by using the ECHO at the command prompt. No other 3rd party HEX utilities or x86 IDEs required!
The technique uses a a combination of keypad - ALT ASCII codes which convert OPCODES to a binary format readable directly under MSDOS. The output is a fully runnable binary *.com file.
http://colinord.blogspot.co.uk/2015/02/extreme-programming-hand-coded.html
Excerpt:
Type the following key commands at the DOS prompt remembering to hold Left ALT.
c:\>Echo LALT-178 LALT-36 LALT-180 LALT-2 LALT-205 LALT-33 LALT-205 LALT-32 > $.com
The codes above are actually opcode values describing an X86 assembly program to print a dollar sign to the screen.
Your prompt should look something similar below when finished. Press enter to build!
c:\>Echo ▓$┤☻═!═ > $.com
Run the file '$.com' and you will see a single dollar ($) character displayed on the screen.
c:\>$.com
$
c:\>
Congratulations! You just created your first hand coded executable file called $.com.
you can do a disassembly and try figure out the machine code for the opcodes you use in your assembler
for example
org 0x100
mov dx,msg
mov ah,0x09
int 0x21
ret
msg db 'hello$'
compiled with nasm -fbin ./a.asm -o ./a.com
has ndisasm a.com deliver the following disassembly:
00000000 BA0801 mov dx,0x108
00000003 B409 mov ah,0x9
00000005 CD21 int 0x21
00000007 C3 ret
00000008 68656C push word 0x6c65
0000000B 6C insb
0000000C 6F outsw
0000000D 24 db 0x24
00000000 to 00000007 are the instructions
so you can play with the ba0801 machine code, using some hex editor, try changing it to ba0901, and only 'ello' will be printed, you can play around with your hex editor and pad stuff out with NOP, which is 0x90 in machine code, for example:
00000000: ba 50 01 90 90 90 90 90 90 90 90 90 90 90 90 90 .#..............
00000010: b4 09 90 90 90 90 90 90 90 90 90 90 90 90 90 90 ................
00000020: cd 21 90 90 90 90 90 90 90 90 90 90 90 90 90 90 .!..............
00000030: c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 ................
00000040: 71 77 65 72 74 79 75 69 61 73 64 66 67 68 6a 24 qwertyuiasdfghj$
00000050: 61 73 64 66 67 68 6a 6b 61 73 64 66 67 68 6a 24 asdfghjkasdfghj$
00000060: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- ----------------
if you save this with the extension .com you can run it in DosBox
I wrote a program about 10 years ago in Visual Basic 6 which was basically a full-screen game similar to Breakout / Arkanoid but had 'demoscene'-style backgrounds. I found the program, but not the source code. Back then I hard-coded the display mode to 800x600x24, and the program crashes whenever I try to run it as a result. No virtual machine seems to support 24-bit display when the host display mode is 16/32-bit. It uses DirectX 7 so DOSBox is no use.
I've tried all sorts of decompiler and at best they give me the form names and a bunch of assembly calls which mean nothing to me. The display mode setting was a DirectX 7 call but there's no clear reference to it in the decompilation.
In this situation, is there any pointers on how I can:
pin-point the function call in the program which is setting the display mode to 800x600x24 (ResHacker maybe?) and change the value being passed to it so it sets 800x600x32
view/intercept DirectX calls being made while it's running
or if that's not possible, at least
run the program in an environment that emulates a 24-bit display
I don't need to recover the source code (as nice as it would be) so much as just want to get it running.
One technique you could try in your disassembler is to do a search for the constants you remember, but as the actual bytes that would be contained within the executable. I guess you used the DirectDraw SetDisplayMode call, which is a COM object so can't be as easily traced to/from an entry point in a DLL. It takes parameters for width, height and bits per pixel and they are DWORDs (32-bit) so do a search for "58 02 00 00", "20 03 00 00" and "18 00 00 00". Hopefully that will narrow it down to what you need to change.
By the way which disassembler are you using?
This approach may be complicated somewhat if your VB6 program compiled to p-code rather than native code as you'll just get a huge chunk of data that represents the program rather than useful assembler instructions.
Check this:
http://www.sevenforums.com/tutorials/258-color-bit-depth-display-settings.html
If your graphics card doesn't have an entry for 24-bit display....I guess hacking your code's the only possibility. That or finding an old machine to throw windows 95 on :P.
I am working with a shared memory application, and to delete the segments I use the following command:
ipcrm -M 0x0000162e (this is the key)
But I do not know if I'm doing the right things, because when I run ipcs I see the same segment but with the key 0x0000000. So is the memory segment really deleted? When I run my application several times I see different memory segments with the key 0x000000, like this:
key shmid owner perms bytes nattch status
0x00000000 65538 me 666 27 2 dest
0x00000000 98307 me 666 5 2 dest
0x00000000 131076 me 666 5 1 dest
0x00000000 163845 me 666 5 0
What is actually happening? Is the memory segment really deleted?
Edit: The problem was - as said below in the accepted answer - that there were two processes using the shared memory, until all the process were closed, the memory segment is not going to disappear.
I vaguely remember from my UNIX (AIX and HPUX, I'll admit I've never used shared memory in Linux) days that deletion simply marks the block as no longer attachable by new clients.
It will only be physically deleted at some point after there are no more processes attached to it.
This is the same as with regular files that are deleted, their directory information is removed but the contents of the file only disappear after the last process closes it. This sometimes leads to log files that take up more and more space on the file system even after they're deleted as processes are still writing to them, a consequence of the "detachment" between a file pointer (the zero or more directory entries pointing to an inode) and the file content (the inode itself).
You can see from your ipcs output that 3 of the 4 still have attached processes so they won't be going anywhere until those processes detach from the shared memory blocks. The other's probably waiting for some 'sweep' function to clean it up but that would, of course, depend on the shared memory implementation.
A well-written client of shared memory (or log files for that matter) should periodically re-attach (or roll over) to ensure this situation is transient and doesn't affect the operation of the software.
You said that you used the following command
ipcrm -M 0x0000162e (this is the key)
From the man page for ipcrm
-M shmkey
Mark the shared memory segment associated with key shmkey for
removal. This marked segment will be destroyed after the
last detach.
So the behaviour of -M options does exactly what you observed, ie set the segment to be destroyed only after the last detach.