Ollydbg 1.10 "Back to user mode" doesn't work - debugging

I tried to learn "Lena's reversing for newbies", when some trouble arise.
I start Pixtopian Book with ollyDbg, then try to have MessageBox with message about uregistered version.
Then i switch to OllyDbg, stop program executing and press "Alt+F9" for "Back to user mode" which stop the program after it exit from DLL.
But after this program does not work, it's frozen and does not respond to my actions.
If i turn off "Back to user mode" program normally work.
What's the problem? Can i try to use "Back to user mode" in IDA (uses WinDbg) or some other debugger and How i can do this? Can i repair it function in OllyDbg?
P.S. It's like the program stopped and didn't run after use "ALT+F9".
/Sorry for my English, i'm just learning ;-)/

First of all, Ollydbg is meant for 32 bit platform. It means that it will run only on a 32 bit OS and can only debug 32 bit apps.
In x64 Operating Systems (specifically Windows) there is a feature called compatibility mode that lets you run 32 bit apps. The 32 bit code is either run by emulation or natively (when the hardware itself implements the instruction set and then it is called x86-64).
So, when you try to run Ollydbg in a x64 environment it will run, but you will experience problems like the one you are facing. It occurs because Ollydbg is run in an emulation mode. Being a debugger it needs access to the registers and other system structures, which it is denied. What it can see is a virtual image of the system.
So the solution to the problem is using a Virtual Machine.
You would install a 32 bit OS in it and debug the app using Ollydbg. As far as Virtual Machines are concerned, I would recommend VMWare. You can use either the workstation or player version. The latter is free but does not support snapshots.
Other solutions are Virtual Box, Parallels Workstation and Microsoft Virtual PC.
The disadvantages of them are that Virtual Box does not support hardware breakpoints, Parallels Workstation is no longer supported as of 2014 and moreover there you would get a BSOD if you try to single step through FPU instructions. I have not tested Virtual PC though.
Note : Ollydbg does not supports x64 but its author is working on a x64 version.

I just learned how to update changes to the exe file for ollydby v 2.01e.
suppose I wished to change a jl command to a jmp; do this by clicking the executable modules button, choose the file and right click to view the file. then record the changes and save file. The saved file also has a backup in case something goes wrong.

I just did a thorough learning of the pixtopian file. When I downloaded
that tutorial I get the file pixtopian107.exe only. Since it didn't agree with the tutorial I investigated further. That file is an installation file. It produces a regular pixtopian.exe file which is the file you wish to play with.
I also noticed that in running the file it never enters the main module.
this is because of the TLS callback. right now I am trying to learn how
to overcome this which is how i came to this site in the first place.
I am using ollydbg vs2.01e very effectively.
Another thing, in vs 2.01e I am still trying to save changed data. Until I succeed I am recording the changes in the exe files using hex editor.

From my experience I can tell that this functionality won't work on Windows 7.
On Windows 7 64bit --> Won't work at all.
On Windows 7 32bit --> Will work partially, but only when using option "File>Attach" in OllyDbg.
For me, best solution was to use Windows XP 32bit, then it worked fine.

Related

how can i run the debug command from windows 64x

I need to use the debug command in Windows 64x for learning purposes. When I type the command debug in the cmd, I get the following message:
'debug' is not recognized as an internal or external command,
operable program or batch file.
As I understand from some previous posts that debug does not work in 64x systems. Is there any work around for this issue?
EDIT:
I am trying to write assembly code for learning. I am not allowed to use any other option for writing assembly code. I have to use DEBUG.
debug.exe is not available in any of the 64 bit windows versions. What are you trying to accomplish? One option for you may be gnu debug - http://www.sourceware.org/gdb/
I know this is an ancient thread, but others might have the same question.
In general to use legacy software, the CLEANEST way to do it is to use the build in Hyper-V. And then have PC-DOS 3.30 (or any suitable 16 or 32 bit OS) on that.
Make sure to use a DYNAMIC disk (vhdx). This allows the disk to be mounted in Windows simply by clicking the vhdx (when not in use by Hyper-V - no sharing), so this allows for simple transfers, without complex net-setup.
There are other alternatives such as DOSBOX, though to my experience their emulation have some bugs (e.g. on the ancient FCB-system - older than file-handles)
I know it's a very long answer, but I just saw your post now. Use the vDosPlus (http://www.vdosplus.org/) or vDos (https://www.vdos.info/) software to run 16-bit (MS-DOS) programs on Windows 64-bit.

Best approach for debugging a Win 16-bit application?

I must reverse a legacy windows (16-bit, NE exec) application that controls an old DAQ that I must interface somehow with upgraded hardware. I've been able to disassemble the exec using W32Dasm (and WindowsCodeBack as well, the only two from many that I've tried that have worked) but the resulting asm file contains too many lines. I'd like to use a debugger and set some breakpoints to restrict the work. Could you advise which is the right approach to debug a Win16 app in 32-bit times? A VM running Windows98 for example? Which Win16 debugger could I use?
Many thanks
IDA can disassemble Win16 programs as well (though not the free version), and it's much more convenient than plain dead listing.
As for debuggers, I would try to find the Win16 Turbo Debugger (TDW.EXE). There's also OpenWatcom, which even supports remote debugging (so you can run the program in a VM and the debugger UI on your desktop).

Shell extension doesn't work in Windows-Explorer but works in other programs?

I made shell extension (implementing IContextMenu, IShellExtInit, IExtractIcon & IPersistFile) using ATL, "by the book".
The funny thing is that all is good in some files-managers such as "Total Commander" & "Free Commander", but completely fail in Windows' Explorer:
When I right click on my registered file (say *.000) in TC ("Total Commander"), the extension works: I see it in my logs and in the context menu: the entry with its icon appear.
But if I use Win-Explorer, nothing happens! no log entry comes up, no evidence in the context menu - nothing! even a complete PC restart doesn't help.
I of course registered the ShellEx\MyX\ContextMenuHandlers and ShellEx\IconHandlerkeys, as well as the Explorer's ...\Shell Extensions\Approved\, and they all point to my CLSID.
I use Windows 7.
What do I miss here ?
#SevaTitov answered my question but as a comment, so for future references for anyone who stumbles upon this issue:
#SevaTitov:
If your OS is 64 bit, the Explorer will be 64 bit as well, and all your shell extensions will have to be 64 bit as well. Looks like current version of Total Commander is 32 bit only
There's that thing, Registry Redirector, that when you're in 32-bit program, it hides ("redirects") the 64-bit reg-entries, and vice versa.
I was confused because in my 32-bit registry-viewer-app I could see those keys, but was unaware that I only see a part of the registry (the 64-bit part has to be seen with a 64-bit viewer like the default regedit.exe in a 64-bit Windows machine)

Delphi program & Windows 64-bit compatibility issue

I have some customers/candidate who complained that my program doesn't work on their Windows 7 64 bit version (confirmed with screenshots). The errors were strange, for example:
in the trial version i am
getting a error message whenever i
click on \"mark\" \"delete\" \"help\".
error msg is: Access violation at
address 0046C978 in module
\'ideduper.exe.\' read of address
00000004
windows 7 ultimate 64bit. i7 920
#2.67GHz 9gb or ram
'Mark', 'delete' and 'help' are just standard TToolButton on TToolbar.
The other example is failing to get a thumbnail from IExtractImage.
I have told them to try Compatibility mode but still doesn't work.
The problem is when I tested it on Windows 7 HP 64-bit on my computer (which I've done it before released it actually) it just works fine! So I don't know what causing it
Do you have any advice ?Are different Windows package (home basic,premium,ultimate,etc) treating 32 bit prog differently ?Are the newer version of Delphis (I use 2006) more compatible with 64 bit Windows ? Do I need to wait until 64 bit compiler out?
Thanks in advance
Your best bet in my opinion is to add MadExcept or EurekaLog or something similar to your application and give it to the customer to try again. MadExcept will generate log with stack trace, which will give you a clearer view of what is happening there.
To answer 2nd part of the question, 32bit Delphi programs work fine on 64bit Windows 7. I think it's more likely you have some memory management problems and the customer just happens to stumble upon them while you don't. Use FastMM4 to track those down.
Your applications is trying to access an invalid pointer. Changing environment may surface issues that are hidden in others. Check your application, and use FastMM + JCL+JCVL/MadExcept/EurekaLog to get a detailed trace of the issue. Some Windows APIs may have some stricter call requisites under 7 and/or 64 bit, but we would have to know what your app actually cals.
A free alternative to MadExcept is JCL Debug stuff. However it is less thorough and doesn't include the cool dialog box to send the stack trace to you via email, or as a file you can attach and manually email.
MadExcept is worth the money, and it is free for non-commercial use. You could try it first on your own PC, observe its functionality, and be sure it functions the way you want, and then buy it.
If buying Delphi is worth it (and it is!) then buying mad Except is a no brainer. But if you insist on rolling your own, JCLDebug (part of jedi code library) is also pretty nice.
Give them a stripped down version of your app and see when the problem goes away. I am betting it is your code as I never had any problems with my (hundreds of) W7/64 clients.
I'd be willing to bet it's an issue in your code. The reason it's failing on your customer's machine and not yours is that your machine probably has the default Data Execution Protection (DEP) enabled (which is turned on only for essential Windows programs and services), while your customer's computer is actually using DEP as intended (turned on for all programs and services).
The default setting (which is compatible with older versions of Windows, like 95/98/ME), allows software to execute code from what should be data segments. The more strict setting won't allow this, and raises a system-level exception instead.
You can check the settings between the two by looking at System Properties. I'm not at a Win7 machine right now, but on WinXP you get there by right-clicking on My Computer, choosing Properties, clicking on Performance Options, and then selecting the "Data Execution Prevention" tab. Find it on Vista/Win7 by using the Help; search for Data Execution Protection.
The solution, as previous answers have told you, is to install MadExcept or EurekaLog. You can also get a free version as part of JEDI, in JCLDebug IIRC. I haven't used it, so I can't vouch for it personally. I've heard it's pretty good, though.
If you don't want to go that route, set a breakpoint somewhere in the startup portion of your app (make sure to build with debugging info turned on). Run your app until the breakpoint is hit, and then use the IDE's Search->Goto Address (which is disabled until the breakpoint is hit). Enter the address from the exception dialog (not the one that's almost all zeros, but the 0046C978 address, prefixed with $ to indicate it's in hex) as in $0046C978. You'll probably end up in the CPU window looking at assembly code, but you can usually pick out a line of Delphi code of some sort that can sometimes give you a place to start looking.
In addition to all previous suggestions, I'll add the difference in accessing Registry under WOW64 compared to Win32. If your application is accessing Registry to read or write some settings, you should be aware of this. First, take a look at this and this page in the MSDN. On this page you will find 2 flags that determine the access you get to Registry from 32- or 64-bit application. KEY_WOW64_64KEY is the one that you should use.
In any case, I agree with others about using madExcept (or any other similar tool) to be able to find the exact cause of your problems.

Automating Win32 Driver Testing

Does anyone know ways of partially or fully automating driver test installation?
I am new to driver development and am used to more of a test-driven approach in higher level languages, so moving to the kind of environment where I can't easily test as I go has been a step up for me. I am using Virtual PC for my test environment and currently have to reset it, open device manager, choose the device, click through a bunch of "Are you really sure you wouldn't rather install one of these system drivers" type dialogs, then finally reset the test environment while restarting WinDbg in the host machine just as the test environment is booting up... argh.
After repeating this process many, many times already, surely there has to be a be a better way of doing this? What tools/methods/tricks do commercial driver developers use to run up their driver in a test environment?
Note, this isn't about unit testing drivers, I haven't got to that stage yet or know if it is even possible. This is just about firing up a test environment with WinDbg attached to make sure that some small change I may have done is doing what I expect.
It seems to me that a virtualization software + a "mock objects" (layering) approach (as suggested by Aaron Digulla) + scripts (as suggested by Sergius) can simplify device driver development.
But if you use Visual Studio to develop user-level applications, you can use it for kernel device driver development too with VisualDDK (+ VirtualKD to debug over a named pipe, which is faster than over a virtual COM port), which addresses specifically the annoyances that you mention; from its home page:
... This project brings the simplicity and
convenience of Windows application
development to the driver development
world. No more manual creation of
build scripts, copying of driver
files, installing drivers from INFs,
switching between WinDbg and the
source editor or waiting for seconds
after each step due to the extra-slow
virtual COM port. Just create a driver
project using a convenient Driver
Wizard, select a virtual machine, and
enjoy debugging your driver directly
from Visual Studio. Want to test a
change? Just normally press Shift-F5,
modify your driver, rebuild it and
launch again. VisualDDK will unload
the old driver, install the new one
and load it automatically and quickly.
Bored with WinDbg loading symbol files
for minutes and looking up symbols for
seconds? Just let VisualDDK optimize
this for you using its own DIA-based
symbol engine. Using C++/STLPort in
your drivers? VisualDDK will natively
visualize all STL containers and
strings, as good as Visual Studio does
for user-mode applications. ...
You can write some shell scripts (using sc.exe and devcon.exe) to automate deployment tasks (no opening device manager, clicking on buttons, etc). And make snapshot of the system ready to debug (needn't wait for system boot).
Don't forget to check your driver with DriverVerifier!
Example of my own script :)
sc create FsFilter type= filesys binPath= c:\FSFilterDrv.sys
sc start FsFilter
pause
sc stop FsFilter
sc delete FsFilter
Follow the advice I gave here. Basically, test as little as possible with the real system.
In your case, I've got another tip: Virtual PC is using a virtual hard disk (that's probably a file on your real hard disk).
You don't need to install your driver, you can simply replace the new files in the virtual hard disk. This is often not possible in the running system but in a virtual system, you can open the virtual disk file and change it (since Windows isn't locking the files in it).
I'm not sure about Virtual PC but other emulators come with tools to work with virtual disk images. If VPC can't do it, check out VirtualBox.
It all depends a little on what kind of driver you are writing. But in many cases, writing an appropriate makefile (or something similar) that handles driver installation, start/stop, and launching of a test harness can already be good enough.
I also configure all of my test machines to automatically logon (AutoAdminLogon), map net drives, and launch an appropriate command prompt after startup. Running a specific test is then a matter of typing in a single command only.
One word concerning VirtualPC: VirtualPC is very handy for kernel mode development, but do not forget that it emulates a uniprocessor machine only -- so be sure to regularly test the code on a multiprocessor machine as well. That said, the VHD trick may seem handy, but it somewhat ties you to Virtual PC -- writing appropriate scripts that equally work on VirtualPC as on a real machine therefore seems a better approach to me.
Finally, consider it a shameless plug, but if you are looking for a unit testing framework for Windows kernel mode code, I have written one: cfix.
I think the DevCon utility (described in this OSR Online article) will help you. You should be able to setup batch files that do the job on one click.
It's free to sign up with osronline.com, and you'll probably have to sign up to get to that article. And if you are writing drivers, you WANT to sign up. These guys have been doing this for a long time, and there's a LOT of really good info on that web site.

Resources