(I'm coding a debugger. But my doubt is also from the point of view of a debugger user)
Many debuggers in many languages (GDB, Eclipse) implement a STEP_OVER command that permits to execute one statement at a time; the difference with STEP_INTO is that it does not perform the stepping down in stack (i.e., called functions), which is often a good thing.
10 : y = f1(x);
11 : z = y + 1;
Now, suppose I step over line 10 above, but a breakpoint is hit inside function f1 (perhaps several levels deep in the call stack). It's not clear what should happen when I resume: should the debugger pause at line 11 (effectively "completing the step over" command)? Or should it forget about it? I believe most (all?) debuggers do the later. Is that the standard/expected behaviour? I myself have found this a little frustrating. Is there a way (in some debugger) to resume execution from the inside breakpoint to the outside stepped-over statement? Or is there some way to do a step-over-ignoring breakpoints?
WinDbg does the latter, and I believe this is standard behavior. If you are worried about a different breakpoint occurring during a step-over command, you could always manually set a breakpoint on line 11 and continue running until line 11 is hit. Alternatively, you could temporarily disable the other breakpoints, but note that the debugger still may break for other reasons as well (such as raising an exception), depending on its configuration.
Related
I've encountered a very annoying problem that has cost a lot of time for several months now.
I have a project in MPLAB X. When I use a line-breakpoint it does not break on the right line at all when debugging my project.
I am using MPLAB X v4.15
This is what actually happens:
No matter where the breakpoint is, the debugger never breaks at the right place.
if I put a breakpoint somewhere, it always breaks at the wrong position
if I then restart the debugging it breaks at the same wrong position
If I change the breakpoint location, the position where the program actually breaks is different, but stays the same again when i restart the program.
Some more info:
Why does this happen?
Are there more people with this problem?
How do i solve this?
EDIT
Sadly, the solution suggested by K_Trenholm did not work for me. I put 3 "NOPs" in one function, but it didn't work as you suggested. See the picture below:
but what I got:
I would like to add that I tried various combinations of breakpoints for the NOPs. No matter what i do, the program always halts at the same PC for this case, seen in the picture above.
Thank you for your reply, it is very helpfull to even have ANY ideas on how to solve it.
If you have any other ideas, I would be very grateful if you would share them!
Two things come to mind:
1) Compiler optimizations can cause problems with breakpoint locations/values when debugging. When debugging, turn optimizations off (if possible, it looks like in your example you're bumping up on the ceiling in terms of code size).
2) Breakpoint "Skidding". See http://microchipdeveloper.com/tls0201:skid-effect#top-of-page
One way to work around this from what I've seen is to put a couple NOP instructions after the line where you plan on placing the breakpoint. This will ensure that any "skidding" will not execute more code.
The instruction where the break occurs will always execute completely, and anything pending in the pipeline will execute as well. For single cycle instructions, this adds a one instruction skid. For multiple cycle instructions and branches, it adds multiple cycles. So if you want to avoid
to jump the debugger into a subroutine you had to include some Nop behind the breakpoint.
example:
void main (void)
{
int x = 0;
x++1; //put Breakpoint here
Nop();
Nop();
Nop(); //Debugger will stop here
foo(x); //so foo() is not called
}
Depending on the MCU being used the debugger will introduce a 'skid effect' upon hitting a breakpoint. The debug session will execute up to two extra instructions before halting.
I am currently writing a debugger for a script virtual machine.
The compiler for the scripts generates debug information, such as function entry points, variable scopes, names, instruction to line mappings, etc.
However, and have run into an issue with step-over.
Right now, I have the following:
1. Look up the current IP
2. Get the source line from that
3. Get the next (valid) source line
4. Get the IP where the next valid source line starts
5. Set a temporary breakpoint at that instruction
or: if the next source line no longer belongs to the same function, set the temp breakpoint at the next valid source line after return address.
So far this works well. However, I seem to be having problems with jumps.
For example, take the following code:
n = 5; // Line A
if(n == 5) // Line B
{
foo(); // Line C
}
else
{
bar(); // Line D
--n;
}
Given this code, if I'm on line B and choose to step-over, the IP determined for the breakpoint will be on line C. If, however, the conditional jump evaluates to false, it should be placed on line D. Because of this, the step-over wouldn't halt at the expected location (or rather, it wouldn't halt at all).
There seems to be little information on debugger implementation of this specific issue out there. However, I found this. While this is for a native debugger on Windows, the theory still holds true.
It seems though that the author has not considered this issue, either, in section "Implementing Step-Over" as he says:
1. The UI-threads calls CDebuggerCore::ResumeDebugging with EResumeFlag set to StepOver.
This tells the debugger thread (having the debugger-loop) to put IBP on next line.
2. The debugger-thread locates next executable line and address (0x41141e), it places an IBP on that location.
3. It calls then ContinueDebugEvent, which tells the OS to continue running debuggee.
4. The BP is now hit, it passes through EXCEPTION_BREAKPOINT and reaches at EXCEPTION_SINGLE_STEP. Both these steps are same, including instruction reversal, EIP reduction etc.
5. It again calls HaltDebugging, which in turn, awaits user input.
Again:
The debugger-thread locates next executable line and address (0x41141e), it places an IBP on that location.
This statement does not seem to hold true in cases where jumps are involved, though.
Has anyone encountered this problem before? If so, do you have any tips on how to tackle this?
Since this thread comes in Google first when searching for "debugger implement step over". I'll share my experiences regarding the x86 architecture.
You start first by implementing step into: This is basically single stepping on the instructions and checking whether the line corresponding to the current EIP changes. (You use either the DIA SDK or the read the dwarf debug data to find out the current line for an EIP).
In the case of step over: before single stepping to the next instruction, you'll need to check if the current instruction is a CALL instuction. If it's a CALL instruction then put a temporary breakpoint on the instruction following it and continue execution till the execution stops (then remove it). In this case you effectively stepped over function calls literally in the assembly level and so in the source too.
No need to manage stack frames (unless you'll need to deal with single line recursive functions). This analogy can be applied to other architectures as well.
Ok, so since this seems to be a bit of black magic, in this particular case the most intelligent thing was to enumerate the instruction where the next line starts (or the instruction stream ends + 1), and then run that many instructions before halting again.
The only gotcha was that I have to keep track of the stack frame in case CALL is executed; those instructions should run without counting in case of step-over.
How does a debugger set breakpoints if the image is in read-only memory? I know there are hardware breakpoints, but in the debugger I use (OllyDbg) those have to be set specially using a different dialog than normal breakpoints.
Explanation:
Here is a routine in a debugger that is comparing itself to a copy of itself. EDX points to the running image, EBX points to the known good copy of the image. The breakpoint on 4010CE only is reached if there is a mismatch. The character being compared is in the AL register. As you can see the debugger shows EB F6 at 10CE, but this is false. 10CE actually has CC in it, as you can see by looking at the AL register. This is because the debugger has secretely inserted the CC to perform the breakpoint.
The debugger first has to change the memory protection of the page it wants to write to. This can be done with VirtualProtectEx. After that it is able to write with WriteProcessMemory and then set the protection back to the original value.
Let me preface this with a disclaimer that I'm not familiar with your particular toolset.
If you haven't enabled hardware breakpoints, the only remaining breakpoint type is a software breakpoint. These are only hit (on x86 because that's what I'm most familiar with) when you replace the first byte of an instruction with a trap instruction, and will only be routed through the breakpoint mechanism of your OS to your debugger if the correct trap instruction for your OS is used and the debugger has already registered itself with the OS as a debugger for this process. In order to cause the software breakpoint to happen at the correct moment, the trap instruction must be written into your code segment over the first byte of your correct instruction.
The two answers that got here first explain the two scenarios which could get you here (at least, the only two I can think of):
The kernel always has write access everywhere, except for hardware-protected pages (ie on some sort of ROM), which your process' memory is almost certainly not. It has the ability to write the breakpoint instruction regardless of the permissions exposed to the user process being debugged.
The debugger must use some syscall to change the access rights on the memory of the target process before inserting the breakpoint.
Personally, I'm guessing the first thing is happening. The segment permissions are only in place to protect your target process from itself, not from a debugger process or from the kernel. Debugging mechanisms in operating systems pretty regularly violate "normal" permissions to allow the debugger to do whatever it wants to the target process. This, of course, is why some operating systems require you to enter a password before you're allowed to use the debugger in certain scenarios.
However, you can test if it's the second one by attempting to write to the code segment from inside the target process after a breakpoint has been set. If the write succeeds, you know the permissions have been lowered by the OS (to allow the process to be debugged). It would be pretty awkward for the OS to require a debugger to jump through this hoop since it can already insert arbitrary code into the writeable parts of memory and then force a jump to it by generating a stack frame overflow.
The debugger takes advantage of the WriteProcessMemory() function to alter the instruction in place. It'll keep a copy of the instruction. When the bp is hit it will reset the old byte value and set EIP back to the previous instruction so the real instruction can execute.
This is definitely obsessive, but I am fascinated by how cleverly the compiler translates C code into machine instructions. Since I am a little fuzzy on some instructions, it would be helpful if I could watch the operation of a compiled program at the level of individual machine instructions - "below" the level of a C statement (which might generate several CPU instructions). In other words, can I watch the registers/memory change after a single machine instruction?
I'm sure it's possible with some other debuggers, but I am only using Xcode.
It's possible to use Xcode's step over, step in, and step out commands with the debugger. The trick is to hold the ctrl (Control) key while you mouse over the debugger step icons to change the context. Underneath each step icon, the line will change to a dot and you can then step at the instruction level.
See the attached screenshot highlighted with a red oval.
si is "Step Instruction", and ni is "Next Instruction". They have the same semantics as "step" and "next" do for lines of code, just on the instruction level.
In the Debugger window, enable assembly: Run/Debugger display/Source and disassembly. Wait till you hit a breakpoint. Then use Run/Step into(over) instruction to step in assembly.
Ooops, didn't check closely enough - Step Into w/Option (Cmd-Opt-Shft-I = ⌘⌥⇧I)
Here I searched that:
Trap Flag (T) – This flag is used for on-chip debugging. Setting trap
flag puts the microprocessor into single step mode for debugging. In
single stepping, the microprocessor executes a instruction and enters
into single step ISR.
If trap flag is set (1), the CPU automatically generates an internal
interrupt after each instruction, allowing a program to be inspected
as it executes instruction by instruction.
If trap flag is reset (0), no function is performed.
https://en.wikipedia.org/wiki/Trap_flag
Now I am coding on emu-8086. As explained, TF must be set in order to debugger work.
Should I set a TF always myself or it is set automatically?
If I somehow set a TF to 0, will the whole computer systems debuggers work or just emu-8086 wont debug?
I've never used emu8086 but by looking at some screenshot of it and judging by its name it's probably an emulator - this means it is not running the code natively.
Each instruction is changing the state of a virtual 8086 CPU (represented as a data structure in memory) and not the state of your real CPU.
With this emulation, emu8086 doesn't need to rely on the TF flag to single-step your program, it just needs to stop after one step of emulation and wait for you to hit another button.
This is also why you can find a thing such as "Step back".
If you were wondering what would happen if a debugged program (and not an emulated one) sets the TF flag then the answer is that it depends on the debugger.
The correct behaviour is the one where the debuggee receives the exceptions but this is hard to handle correctly (since the debugger itself uses the TF flag).
Some debugger just don't care and swallow the exception (i.e. they don't forward it to the program under debug) assuming that a well written program doesn't need to use the TF flag.
Unfortunately malwares routinely use a set of anti-debug technique including setting the TF and checking it back/waiting for exceptions to detect the presence of a debugger.
A truly transparent debugger has to handle the RFLAGS register carefully.
When debugging with breakpoints the TF is not set while the program is executing, so there is nothing to worry about.
However when single stepping the TF is set during the next instruction, this is problematic during a pushfd/q and the debugger must explicitly handle that case to avoid detection.
If the debuggee sets the TF the debugger must pass the debug exception to the program - under current OS the TF won't last more than an instruction because the OS will catch the exception,
trasnform it in a signal and dispatch it to the program while clearing the TF. So the debugger can simply do a check before stepping into a popfd/q instruction.
Where the TF doesn't get cleared by the OS the debugger must effectively emulate RFLAGS with a copy.
The debugger sets TF according to what it needs to do. The code being debugged should not modify TF.