Debugger implementation - Step over issue - debugging

I am currently writing a debugger for a script virtual machine.
The compiler for the scripts generates debug information, such as function entry points, variable scopes, names, instruction to line mappings, etc.
However, and have run into an issue with step-over.
Right now, I have the following:
1. Look up the current IP
2. Get the source line from that
3. Get the next (valid) source line
4. Get the IP where the next valid source line starts
5. Set a temporary breakpoint at that instruction
or: if the next source line no longer belongs to the same function, set the temp breakpoint at the next valid source line after return address.
So far this works well. However, I seem to be having problems with jumps.
For example, take the following code:
n = 5; // Line A
if(n == 5) // Line B
{
foo(); // Line C
}
else
{
bar(); // Line D
--n;
}
Given this code, if I'm on line B and choose to step-over, the IP determined for the breakpoint will be on line C. If, however, the conditional jump evaluates to false, it should be placed on line D. Because of this, the step-over wouldn't halt at the expected location (or rather, it wouldn't halt at all).
There seems to be little information on debugger implementation of this specific issue out there. However, I found this. While this is for a native debugger on Windows, the theory still holds true.
It seems though that the author has not considered this issue, either, in section "Implementing Step-Over" as he says:
1. The UI-threads calls CDebuggerCore::ResumeDebugging with EResumeFlag set to StepOver.
This tells the debugger thread (having the debugger-loop) to put IBP on next line.
2. The debugger-thread locates next executable line and address (0x41141e), it places an IBP on that location.
3. It calls then ContinueDebugEvent, which tells the OS to continue running debuggee.
4. The BP is now hit, it passes through EXCEPTION_BREAKPOINT and reaches at EXCEPTION_SINGLE_STEP. Both these steps are same, including instruction reversal, EIP reduction etc.
5. It again calls HaltDebugging, which in turn, awaits user input.
Again:
The debugger-thread locates next executable line and address (0x41141e), it places an IBP on that location.
This statement does not seem to hold true in cases where jumps are involved, though.
Has anyone encountered this problem before? If so, do you have any tips on how to tackle this?

Since this thread comes in Google first when searching for "debugger implement step over". I'll share my experiences regarding the x86 architecture.
You start first by implementing step into: This is basically single stepping on the instructions and checking whether the line corresponding to the current EIP changes. (You use either the DIA SDK or the read the dwarf debug data to find out the current line for an EIP).
In the case of step over: before single stepping to the next instruction, you'll need to check if the current instruction is a CALL instuction. If it's a CALL instruction then put a temporary breakpoint on the instruction following it and continue execution till the execution stops (then remove it). In this case you effectively stepped over function calls literally in the assembly level and so in the source too.
No need to manage stack frames (unless you'll need to deal with single line recursive functions). This analogy can be applied to other architectures as well.

Ok, so since this seems to be a bit of black magic, in this particular case the most intelligent thing was to enumerate the instruction where the next line starts (or the instruction stream ends + 1), and then run that many instructions before halting again.
The only gotcha was that I have to keep track of the stack frame in case CALL is executed; those instructions should run without counting in case of step-over.

Related

Windows pintool mismatch between call/ret instructions

So i've been trying to write a pintool that monitors call/ret instructions but i've noticed that threre was a significant inconsistency between the two. For example ret instructions without previous call.
I've run the tool in a console application from which i got the following logs showing this inconsistency (this is an example, there are more inconsistencies like the one listed below in the other call/ret instructions):
1. Call from ntdll!LdrpCallInitRoutine+0x69, expected to return to 7ff88f00502a
2. RETURN to 7ff88f00502a
//call from ntdll!LdrpInitializeNode+0x1ac which is supposed to return at 7ff88f049385 is missing (the previous instruction)
3. RETURN to 7ff88f049385 (ntdll!LdrpInitializeNode+0x1b1)
The above are the first 3 log entries for the call/ret instructions. As one can see, the monitoring started a bit late, at the call found at ntdll!LdrpCallInitRoutine+0x69, it returned to the expected address but then returned to 7ff88f049385 without first tracking the call found in the previous instruction.
Any ideas of what could be the fault?
The program is traced with INS_AddInstrumentFunction with a callback that more or less does:
if INS_IsCall(ins) INS_InsertCall(ins,...
if INS_IsRet(ins) INS_InsertCall(ins,...
I've tried the same program on Linux which worked as expected, without any mismatch.
Any ideas of the reason behind this behavior?

MPLAB X + XC8 break at wrong line

I've encountered a very annoying problem that has cost a lot of time for several months now.
I have a project in MPLAB X. When I use a line-breakpoint it does not break on the right line at all when debugging my project.
I am using MPLAB X v4.15
This is what actually happens:
No matter where the breakpoint is, the debugger never breaks at the right place.
if I put a breakpoint somewhere, it always breaks at the wrong position
if I then restart the debugging it breaks at the same wrong position
If I change the breakpoint location, the position where the program actually breaks is different, but stays the same again when i restart the program.
Some more info:
Why does this happen?
Are there more people with this problem?
How do i solve this?
EDIT
Sadly, the solution suggested by K_Trenholm did not work for me. I put 3 "NOPs" in one function, but it didn't work as you suggested. See the picture below:
but what I got:
I would like to add that I tried various combinations of breakpoints for the NOPs. No matter what i do, the program always halts at the same PC for this case, seen in the picture above.
Thank you for your reply, it is very helpfull to even have ANY ideas on how to solve it.
If you have any other ideas, I would be very grateful if you would share them!
Two things come to mind:
1) Compiler optimizations can cause problems with breakpoint locations/values when debugging. When debugging, turn optimizations off (if possible, it looks like in your example you're bumping up on the ceiling in terms of code size).
2) Breakpoint "Skidding". See http://microchipdeveloper.com/tls0201:skid-effect#top-of-page
One way to work around this from what I've seen is to put a couple NOP instructions after the line where you plan on placing the breakpoint. This will ensure that any "skidding" will not execute more code.
The instruction where the break occurs will always execute completely, and anything pending in the pipeline will execute as well. For single cycle instructions, this adds a one instruction skid. For multiple cycle instructions and branches, it adds multiple cycles. So if you want to avoid
to jump the debugger into a subroutine you had to include some Nop behind the breakpoint.
example:
void main (void)
{
int x = 0;
x++1; //put Breakpoint here
Nop();
Nop();
Nop(); //Debugger will stop here
foo(x); //so foo() is not called
}
Depending on the MCU being used the debugger will introduce a 'skid effect' upon hitting a breakpoint. The debug session will execute up to two extra instructions before halting.

Assembly - Why this CALL function doesn't work?

I don't understand why CALL function in this code doesn't work:
#include<stdio.h>
void main() {
__asm {
jmp L1
L2:
mov eax, 8
ret
L1:
call L2
}
}
If i debug the code step by step, the line 'call L1' is not processed, and program directly skips to the end. What is wrong? I'm working on VisualStudio2015 with Intel 32-bit registers.
The problem
You've stumbled on the difference between step over F10 and step into F11.
When you use (the default) step over, call appears to be ignored.
You need to step into the code and then the debugger will behave as you'd expect.
Step over
The way this works with step over is that the debugger sets a breakpoint on the next instruction, halts there and moves the breakpoint to the next instruction again.
Step over knows about (conditional) jumps and accounts for that, but disregards (steps over) call statements; it interprets a call as a jump to another subroutine and 'assumes' you want to stay within the current context.
These automatic breakpoints are ephemeral, unlike manual breakpoints which persist until you cancel them.
Step into
Step into does the same, but also sets a breakpoint at every call destination; in effect leading you deep into the woods traversing every subroutine.
Step out
If you've stepped too deep 'into' a subroutine Visual Studio allows you to step out using ShiftF11; this will take you back to the next instruction after the originating call.
Some other debuggers name this feature "run until return".
Debugging high level code
When the debugger is handling higher language source code (e.g. C) it keeps a list of target addresses for every line of source code. It will plan its breakpoints per line of source code.
Other than the fact that every line of high level code translates to zero or more lines of assembly it works the same as stepping through raw assembly code.

how to force gdb to stop right after the start of program execution?

I've tried to set breakpoint on every function that makes any sense but program exit before reaching any of those. Is there a way to make program run in step-by-step mode from the start so I can see what's going on?
I'm trying to debug /usr/bin/id if it's important (we have custom plugin for it and it's misbehaved)
P.S. Start command doesn't work for me here(it should be a comment, but I don't have enough rep for it)
Get the program entry point address and insert a breakpoint at that address.
One way to do this is to do info files which gives you for example "Entry point: 0x4045a4". Then do "break *0x4045a4". After run-ning program, it will immediately stop.
From here on you can use single stepping instructions (like step or stepi) to proceed.
You did not tell what system you are trying to debug. If code is in read-only memory you may need to use hardware breakpoints (hbreak) if they are supported by that system.
Use start command
The ‘start’ command does the equivalent of setting a temporary breakpoint at the beginning of the main procedure and then invoking the ‘run’ command.
e.g.
a program with debug info main, and usage like this: main arg1 arg2
gdb main
(gdb) start arg1 arg2
Use starti. Unlike start this stops at the actual first instruction, not at main().
You can type record full right after running the program. This will record all instructions and make them possible for replaying/going back.
For main function, you'd need to type this before reaching the breakpoint so you can set an earlier one by break _start -> _start is a function always called before the standard main function. (apparently applies only to the gcc compiler or similar)
Then continue to main breakpoint and do reverse-stepi to go exactly one instruction back
For more info about recording look here: link

implementing step over, dwarf

Im working on a source level debugger. The debug info available in elf
format. How could be 'step over' implemented?
The problem is at 'Point1', anyway I can wait for the
next source line (reading it from the .debug_line table).
Thanks
if (a == 1)
x = 1; //Point1
else if (a == 2)
x = 1;
z = 1;
I'm not sure I understand the question entirely, but I can tell you how GDB implements its step command.
Once control has entered a particular compilation unit, GDB reads that CU's debugging information; in particular, it reads the CU's portion of the .debug_line section and builds a table that maps instruction addresses to source code positions.
When the step begins, GDB looks up the source location for the current PC. Then it steps by machine instruction, looking up the source location of the new PC each time, until the source location changes. When the source location changes, the step is complete.
It also computes the frame ID—the base address of the stack frame, and the start address of the function—after each step, and checks if that has changed. If it has, that means that we've stepped into or returned from a recursive call, and the step is complete.
To see why it's necessary to check the frame ID as well as the source location, consider stepping through a call to the following function:
int fact(n) { if (n > 0) { return n * fact(n-1); } else return 1; }
Since this function is defined entirely on the same source line, stepping by instruction until the source line changes would step you through all the recursive calls without stopping. However, when we enter a new call to fact, the stack frame base address will have changed, indicating that we should stop. This gives us the following behavior:
fact (n=10) at recurse.c:4
(gdb) step
fact (n=9) at recurse.c:4
(gdb) step
fact (n=8) at recurse.c:4
GDB's next command combines this general behavior with appropriate logic for recognizing function calls and letting them return to completion. As before, one must use frame IDs in deciding when calls have truly returned to the original frame; and there are other complications.
It's worth thinking a bit about how to treat inlined instances of functions (which DWARF does describe). But that's a bit much for this question.
Not to discourage experimentation, but if I were beginning a debugger project, I would want to look at Apple's work-in-progress debugger, lldb, which is open source.

Resources