I am trying to use Radare2 to save opcode sequences from PE32 files. Does anyone know how I can save the output Radare2 prints onto terminal after I call the "pd" command onto a text file? I am also wondering if there might be a way to do this in python as well using the r2pipe library.
I use Altera Quartus software which comes with its own Cygwin distribution and a dumb terminal which, according to the shortcut placed in my Start Menu by Altera, is run using
cmd.exe /c "c:\altera\15.1\nios2eds\NiosII Command Shell.bat"
where this batch file configures the environment for Quartus and launches bash. When I use this window to run Altera tools, their output comes out immediately (not buffered) and in color.
I also have my own Cygwin installation with an Xserver and terminals (i.e. lxterminal, mrxvt, xfce4-terminal, etc). I have adapted Altera's batch file to configure Altera's environment within my Cygwin and I can run all of Altera's tools. However, when I run these tools, their output is neither in color (not a big deal but surprising), and is buffered until the end of execution when all output appears at the same time.
Does anyone have any ideas on how to bypass this buffering?
Somehow, your tools are thinking that the output is not a terminal but a file. For files, the tool itself will create a buffer (4K on Linux). For terminals, the output is usually line buffered (i.e. the output code will collect all the characters until an end-of-line is detected and then print them all at once).
To help you further, we need more information how you "adapted Altera's batch file to configure Altera's environment within my Cygwin"
When compiling/interpreting a program from the command prompt, how does the command prompt know where to find the requested compiler/interpreter?
Are these files stored in a specific place or something like that? I'm just about getting a hang of high-level programming, but I find it pretty hard to wrap my head around what happens under the hood.
There are two parts: Where to find the file just from it's filename, and what to do with it.
Where files (programs) are searched if just the name is entered:
Windows (CMD):
There is a variable %PATH% which has a list of ;-separated directories, eg. C:\Windows;C:\Windows\system32;C.\somethingelse. It is saved somewhere in the registry and can be set either in CMD itself or with a GUI somewhere in the OS configs.
Linux etc. (Bash and many more):
Similar, there is a variable $PATH which can be set at least in the shell and various config files, and the entries are separated by :, eg. /bin:/usr/bin/even/more. Priority is from left to right.
Additionally, some shells (eg. Bash) cache lookup results in their own implementation-specific way (depending on the configuration), because it's faster than having to check every directory in the path variable (at least if the searched program is in the last dir, everything has to be checked).
What to do with the file once it's found:
Windows:
In Windows, everything works with the filename suffix.
.exe and some others are native programs to start.
.bat is a shell script which is executed like it's manually written to the shell.
For every other suffix, it's configurable which program belongs to the suffix (stored in the registry, how to comfortably change it depends heavily on the used Windows version). Eg. you could say that .py belongs to your Python interpreter, the a file foo.py will start the interpreter. Btw., the same suffix-program configuration is usen when a file is double-clicked in the GUI file explorer, and of course program installers can add their entries too without the user having to do it.
Linux:
For Linux, the suffix is not as important. The first relevant thing is a binary (yes/no) flag x (x like executable) which exists for each file on the file system, just like file name, creation timestamp etc.etc.
If the x flag is set to yes:
Linux tries to detect what kind of program it is from the content. The difference between a native compiled binary program and a not-compiled script of some scripting language is pretty clear.
A native linux program is started by the kernel, like expected. Additional binary program types could be configured, eg. there is the Wine software which runs some Windows programs on Linux, and one could add a specification how Windows exe`s can be recognized inside and that they should be started with Wine.
For a text file with the x flag, the next step is to look at the first line, which should start with a '#!' (called shebang), followed by the path of the interpreter (eg. #!/bin/bash). Shell scripts (like the bat files on Windows) are realized this way, but it's not limited to classical shell scripts: Nothing prevents anyone from making a #!/bin/python script which Python content (of course, Python has to be installed for this to work).
If the x flag is set to no:
Shells like bash with usual configuration won't do anything, independent if it is a real program just without flag or a jpg image etc. For the GUI file managers:
Again, the content (and possibly the file name suffix too) is inspected to get the type, like jpg images, mp3 music, C++ source code etc.etc. (Linux knows pretty many types), and then the fitting program is looked up in a list configurable by the user and/or program installations (mime file type id <-> program).
...
Note that in the case of eg. Python scripts (which are just normal text files, not something for the kernel to work with), it can be done with and without x flag: With flag and a shebang line, or without flag and a matching mime list entry. In the "without flag" case, the shebang won't hurt if it is there, because Python (and many other scripting languages) consider it a comment because of the #.
Regarding interpreters on unix-based systems, lots of scripts start with a so-called shebang or hashbang (#! on the first line of the script) that tells what interpreter to invoke, see https://en.wikipedia.org/wiki/Shebang_%28Unix%29 .
When you enter something like some-program some-arguments, the shell will look through each directory listed in the environment variable $PATH for an executable file named some-program (on Windows it's %PATH% and some-program.exe).
This is not specific to compilers and interpreters - it happens whether some-program is gcc, python, firefox or notepad.
I have compiled one clipper program using haarbour compiler and the c file produced was compiled using djgpp to produce final exe. This exe runs fine in console window of Windows 98.
However, when I exit to msdos prompt or try to run in pure dos, it does not give any error. But did not go further. however num lock and cap locks responds propery. When I press ctrl+al+del it gives me message hdpmi terminated by the user. I have tried cwsdpmi instead of Hdpmi32 but the problem remains same.
There is on exe HBRUN.exe which acts as an interpreter to .hrb files produced instead of stand alone exe. When I run HBRUn in pure do,it behaved in the same fashion. But when I runit another directory where no dbf files were there, it gave me error dbf file not found ! This also works fine in console window but compiled for pure DOS.
I think there is some problem with all the exes produced using harbour and djgpp if they are big.
one simple 5-6 line program's .hrb file was run correctly in pure dos by hbrun ( Size of hbrun is about 1700 K where as my exe file size is 950 K
Can anybody shade some light ?
Sadly, you're unlikely to find any help here. I would suggest approaching the Harbour Project itself for help. They can be found on github with their list of developers (with email addresses!)
Harbour is powerful.
http://harbour.github.io/
To run in pure DOS I would recommend you to try OPENWATCOM
Here is a copy of the message from Pritpal Bedi (one of the developers):
I could compile Harbour with OpenWatcom DOS.
Machine : WIndows 7 32 Bits
STEPS:
Downloaded: http://ftp.heanet.ie/pub/openwatcom/open-watcom-c-dos-1.9.exe
Installed in C:\WATCOM. Follow all default options when installing except making changes to AUTOEXEC.bat and CONFIG.sys.
WATCOM will save these two files with .DOS as extention.
Copy C:\Watcom\autoexec.dos C:\Watcom\m.bat
Redefine PATH statement as : SET PATH=C:\WATCOM\BINW;C:\Harbour\bin\dos\watcom;%PATH%
Keep all other SETs intact.
CD to C:\Harbour
Execute C:\Watcom\m.bat
Execute DOS-MAKE [ Do Not use "install" option, it will pollute your existing environment ]
Harbour binaries will be deposited in C:\Harbour\bin\dos\watcom
NOTE [ And it is strange ] that steps 7 and 8 will have to be repeated for few times
as console window from where you are initiating it will disappear, but do not worry,
it seems to be an issue with memory.
CD to harbour\tests
Issue : hbmk2 hello -workdir= -run
It appears long command line do not go through, so we have to shorten the path and hence command line length.
Be cheerful as you will see "Hello World" printed in current console.
Pritpal Bedi
I've tried
gdb --write --nx file
Whatever I do and even if I let the program exit normally it just won't write the 64-bit binary changes!
32-bit works and I use the exact same commands and order/sequence and "set {int} 0x0xxxxxx = 0xffffffff"
Then type quit (supposed to save with --write flag)
Permissions: rwx-r-x-r-x / 755
I've started to wonder if the BFD (Binary File Descriptor) or another internal component disallows it.
as stated (here):
Also, if the underlying BFD functionality supports it, you could use
gdb -write to patch object files using this technique. Note that gdb
can neither interpret nor modify relocations in this case, so branches
and some initialized variables will appear to go to the wrong place.
But this feature is still handy from time to time.