I want to debug into the implementation of a [MethodImpl(MethodImplOptions.InternalCall)] BCL method, which is presumably implemented in C++. (In this particular case, I'm looking at System.String.nativeCompareOrdinal.) This is mainly because I'm nosy and want to know how it's implemented.
However, the Visual Studio debugger is refusing to step into that method. I can set a breakpoint on this call:
"Hello".Equals("hello", StringComparison.OrdinalIgnoreCase);
then bring up Debug > Windows > Disassembly, step into the Equals call, and step until it gets to the call x86 instruction. But when I try to use "Step Into" on that call (which I know from Reflector is the nativeCompareOrdinal call), it doesn't step to the first instruction inside nativeCompareOrdinal like I want -- it steps over instead, and goes straight to the next x86 instruction in Equals.
I'm building as x86, since mixed-mode debugging isn't supported for x64 apps. I've unchecked "Just My Code" in Tools > Options > Debugging, and I have "Enable unmanaged code debugging" checked in project properties > Debug tab, but it still steps over the call. I also tried starting the process and then attaching the debugger, and explicitly attaching both the managed and native debuggers, but it still won't step into that InternalCall method.
How can I get the Visual Studio debugger to step into an unmanaged method?
Yes, it is tricky. The offset you see for the CALL instruction is bogus. Plus it won't let you navigate to an unmanaged code address when the current focus is on a managed function.
Start by enabling unmanaged code debugging and setting a breakpoint on the call. Run the code and when the break point hits use Debug + Windows + Disassembly:
"Hello".Equals("hello", StringComparison.OrdinalIgnoreCase);
00000025 call 6E53D5D0
0000002a nop
The debugger tries to display the absolute address but gets it wrong because it uses the bogus incremental address instead of the real instruction address. So first recover the true relative value: 0x6E53D5D0 - 0x2A = 0x6E53D5A6.
Next you need to find the real code address. Debug + Windows + Registers and look at the value of the EIP register. 0x009A0095 in my case. Add 5 to get to the nop, then add the relative offset: 0x9A0095 + 5 + 0x6E53D5A6 = 0x6EEDD640. The real address of the function.
Debug + Windows + Call Stack and double-click an unmanaged stack frame. Now you can enter the calculated address in the Disassembly window's Address box, prefix with 0x.
6EEDD640 push ebp
6EEDD641 mov ebp,esp
6EEDD643 push edi
6EEDD644 push esi
6EEDD645 push ebx
6EEDD646 sub esp,18h
etc...
Bingo, you're know you're good if you see the stack frame setup code. Set a breakpoint on it and press F5.
Of course, you'll be stepping machine code since there's no source code available. You'll get much better insight in what this code is doing by looking at the SSCLI20 source code. No guarantee that it will be a match for the actual code in your current version of the CLR but my experience is that these low-level code chunks that have been around since 1.0 are highly conserved. The implementation is in clr\src\classlibnative\nls, not sure which source code file. It won't be named "nativeCompareOrdinal", that's just an internal name used by ecall.cpp.
Related
The windbg command tct executes a program until it reaches a call instruction or a ret instruction. I am wondering how the debugger implements this functionality under the hood.
I could imagine that the debugger scans the instructions from the current instructions for the next call or ret and sets according breakpoints on the found instructions. However, I think this is unlikely because it would also have to take into account jmp instructions so that there are an arbitrary number of possible call or ret instructions where such a breakpoint would have to be set.
On the other hand, I wonder if the x86/x64 CPU provides a functionality that raises an exception to be caught by the debugger whenever the CPU is about to process a call or ret instruction. Yet, I have not heard of such a functionality.
I'd guess that it single-steps repeatedly, until the next instruction is a call or ret, instead of trying to figure out where to set a breakpoint. (Which in the general case could be as hard as solving the Halting Problem.)
It's possible it could optimize that by scanning forward over "straight line" code and setting a breakpoint on the next jmp/jcc/loop or other control-transfer instruction (e.g. xabort), and also catching signals/exceptions that could transfer control to an SEH handler.
I'm also not aware of any HW support for breaking on a certain type of instruction or opcode: the x86 debug registers DR0..7 allow hardware breakpoints at code addresses without rewriting machine code to int3, and also hardware watchpoints (to trap data load/store to a specific address or range of addresses). But not filtering by opcode.
I am writing some multi-thread C program. I tried to modify the few instructions at the beginning of a function's body to redirect the execution to somewhere else.
But I noticed that when debugging within Visual Studio 2015, some memory location seems to be unchangeable as displayed in the Memory window.
For example:
In below picture, a function ApSignalMceToOs() begins at 0x7FFBBEE51360. I have unprotected the memory range 0x7FFBBEE51360 to 0x7FFBBEE5136E to modify it.
Line 305 to 312 modify the address range 0x7FFBBEE51360 ~ 0x7FFBBEE5136E.
Everything is fine until 0x7FFBBEE51369. At line 311, the (uint32_t(((uintptr_t)dst) >> 32 is 0x00007ffb.
After line 311 is executed, I was expecting the memory range in 0x7FFBBEE51369 ~ 0x7FFBBEE5136C will be filled as fb 7f 00 00. But as shown below, Visual Studio says it is 48 7f 00 00, where the 48 is the old value.
Then I went to check the disassembly code of the function ApSignalMceToOs(). And not surprisingly, the instruction at 00007FFBBF171365 is mov dword ptr [rsp+4], 7F48h, which should be 7FFB. As shown below in the red box below.
So until this point, Visual Studio 2015 is telling me that my modification would fail.
But as the yellow arrow in above picture shows, after the mov dword ptr [rsp+4], 7F48h is executed, I checked the content in the stack area. Surprisingly it is indeed 7f fb got moved onto the stack (shown in the green box in above picture).
And after the ret instruction is executed, the RIP register does change to 00007FFBBEEAD940, which is no surprise. See below:
And in another function, the same location is being read. Shown as below:
The code[len] or byte ptr [rax] is the memory location holding 48 or fb. But it reads 0xcc, which is neither 0x48 nor 0xfb.
Visual Studio disassembly code is decoded from the memory content. So the memory content or how VS2015 read/refresh it is the key point.
Based on above observation, I came to 2 conclusions with VS 2015 debug mode:
Some memory content is not correctly shown (or refreshed in GUI).
Some memory read operation doesn't work correctly.
But the program runs smoothly when not debugging.
Does anyone know why this is happening?
ADD 1 - 5:08 PM 10/14/2019
Thanks to #MichaelBurr. I guess I can explain it now.
The root cause is I added a breakpoint at 0x00007FFB...369 at the disassembly code level, not the C source level.
When I did this, the VS Debugger did add a 0xCC instruction at the location 0x00007FFB...369. But it seems Visual Studio 2015 goes to great lengths to hide this fact. Below is the show of the memory content with the breakpoint at 0x00007FFB...369, we can see 0x00007FFB...369 still holds the old value 0x48.
But after I manually copied the memory from 0x00007FFB...360 to 0x00007FFB...36e to somewhere else. The 0xCC instruction at the offset 0x9 is unveiled. See below:
When I modify the content at 0x00007FFB...369, Visual Studio seemed to be alerted and it just restored the content to the old preserved one, i.e. 0x48. Not my newly written one.
But I think this very restoration doesn't make any sense. The restoration of the preserved byte content shouldn't be triggered at this moment in any way. A more reasonable action is to update the breakpoint's location a little bit and insert the 0xCC instruction to a new location. Because the newly modified code may change the "instruction boundary". This way, the debug experience of the self-modifying code can be best preserved. But this will require the Visual Studio to disassemble the new code in the nearby. And the new instruction content could be invalid if the programmer made a mistake.
I think you are essentially fighting with the debugger's breakpoint/single step handling. Breakpoints are often implemented with the int 3 instruction which has the encoding 0xCC. When the debugger sets the 0xCC for the breakpoint it has to save the original value, then replace it when the debugger has stopped program execution.
In a normal situation (code that isn't self-modified) this makes things appear as you expect when examining the code memory region. However if your program modifies the memory that is being managed by the debugger you can get confusing results since the debugger will restore the value it had saved when it set the breakpoint (overwriting your modification).
I am reading this article about the SEH on Windows.
and here is the source code of myseh.cpp
I debugged myseh.cpp. I set 2 breakpoints at printf("Hello from an exception handler\n"); at line:24 and DWORD handler = (DWORD)_except_handler; at line: 36 respectively.
Then I ran it and it broke at line:36. I saw the stack trace as follows.
As going, AccessViolationException occurred because of mov [eax], 1
Then it broke at line:24. I saw the stack trace as follows.
The same thread but the frame of main was gone! Instead of _except_handle. And ESP jumped from 0018f6c8 to 0018ef34;it's a big gap between 0018f6c8 and 0018ef34
After Exception handled.
I know that _except_handle must be run at user mode rather than kernel mode.
After _except_handle returned, the thread turned to ring0 and then windows kernel modified CONTEXT EAX to &scratch & and then returned to ring3 . Thus thread ran continually.
I am curious about the mechanism of windows dealing with exception:
WHY the frame calling main was gone?
WHY the ESP jumped from 0018f6c8 to 0018ef34?(I mean a big pitch), Do those ESP address belong to same thread's stack??? Did the kernel play some tricks on ESP in ring3??? If so, WHY did it choose the address of 0018ef34 as handler callback's frame? Many thanks!
You are using the default debugger settings, not good enough to see all the details. They were chosen to help you focus on your own code and get the debug session started as quickly as possible.
The [External Code] block tells you that there are parts of the stack frame that do not belong to code that you have written. They don't, they belong to the operating system. Use Tools > Options > Debugging > General and untick the "Enable Just My Code" option.
The [Frames below might be incorrect...] warning tells you that the debugger doesn't have accurate PDBs to correctly walk the stack. Use Tools > Options > Debugging > Symbols and tick the "Microsoft Symbol Servers" option and choose a cache location. The debugger will now download the PDBs you need to debug through the operating system DLLs. Might take a while, it is only done once.
You can reason out the big ESP change, the CONTEXT structure is quite large and takes up space on the stack.
After these changes you ought to now see something resembling:
ConsoleApplication1942.exe!_except_handler(_EXCEPTION_RECORD * ExceptionRecord, void * EstablisherFrame, _CONTEXT * ContextRecord, void * DispatcherContext) Line 22 C++
ntdll.dll!ExecuteHandler2#20() Unknown
ntdll.dll!ExecuteHandler#20() Unknown
ntdll.dll!_KiUserExceptionDispatcher#8() Unknown
ConsoleApplication1942.exe!main() Line 46 C++
ConsoleApplication1942.exe!invoke_main() Line 64 C++
ConsoleApplication1942.exe!__scrt_common_main_seh() Line 255 C++
ConsoleApplication1942.exe!__scrt_common_main() Line 300 C++
ConsoleApplication1942.exe!mainCRTStartup() Line 17 C++
kernel32.dll!#BaseThreadInitThunk#12() Unknown
ntdll.dll!__RtlUserThreadStart() Unknown
ntdll.dll!__RtlUserThreadStart#8() Unknown
Recorded on Win10 version 1607 and VS2015 Update 2. This isn't the correct way to write SEH handlers, find a better example in this post.
I don't understand why CALL function in this code doesn't work:
#include<stdio.h>
void main() {
__asm {
jmp L1
L2:
mov eax, 8
ret
L1:
call L2
}
}
If i debug the code step by step, the line 'call L1' is not processed, and program directly skips to the end. What is wrong? I'm working on VisualStudio2015 with Intel 32-bit registers.
The problem
You've stumbled on the difference between step over F10 and step into F11.
When you use (the default) step over, call appears to be ignored.
You need to step into the code and then the debugger will behave as you'd expect.
Step over
The way this works with step over is that the debugger sets a breakpoint on the next instruction, halts there and moves the breakpoint to the next instruction again.
Step over knows about (conditional) jumps and accounts for that, but disregards (steps over) call statements; it interprets a call as a jump to another subroutine and 'assumes' you want to stay within the current context.
These automatic breakpoints are ephemeral, unlike manual breakpoints which persist until you cancel them.
Step into
Step into does the same, but also sets a breakpoint at every call destination; in effect leading you deep into the woods traversing every subroutine.
Step out
If you've stepped too deep 'into' a subroutine Visual Studio allows you to step out using ShiftF11; this will take you back to the next instruction after the originating call.
Some other debuggers name this feature "run until return".
Debugging high level code
When the debugger is handling higher language source code (e.g. C) it keeps a list of target addresses for every line of source code. It will plan its breakpoints per line of source code.
Other than the fact that every line of high level code translates to zero or more lines of assembly it works the same as stepping through raw assembly code.
I'm having an issue where my application is failing a debug assertion (_CrtIsValidHeapPointer) before anything is even executed. I know this because I added a breakpoint on the first statement of my main function, and it fails the assertion before the breakpoint is reached.
Is there a way to somehow "step through" everything that happens before my main function is called? Things like static member initializations, etc.
I should note that my program is written in C++/CLI. I recently upgraded to VS2015 and am targeting the v140 toolset. The C++ libraries I'm using (ImageMagick, libsquish, and one of my own C++ libraries) have been tested individually, and I do not receive the assertion failure with these libraries, so it has to be my main application.
I haven't changed any of the code since I upgraded from VS2013, so I'm a little stumped on what is going on.
EDIT:
Here is the call stack. This is after I click "Retry" in the assertion failed window. I then get a multitude of other exceptions being thrown, but they are different each time I run the program.
> ucrtbased.dll!527a6853()
[Frames below may be incorrect and/or missing, no symbols loaded for ucrtbased.dll]
ucrtbased.dll!527a7130()
ucrtbased.dll!527a69cb()
ucrtbased.dll!527c8116()
ucrtbased.dll!527c7eb3()
ucrtbased.dll!527c7fb3()
ucrtbased.dll!527c84b0()
PathCreator.exe!_onexit(int (void)* const function) Line 268 + 0xe bytes C++
PathCreator.exe!atexit(void (void)* const function) Line 276 + 0x9 bytes C++
PathCreator.exe!std::`dynamic initializer for '_Fac_tidy_reg''() Line 65 + 0xd bytes C++
[External Code]
mscoreei.dll!7401cd87()
mscoree.dll!741fdd05()
kernel32.dll!76c33744()
ntdll.dll!7720a064()
ntdll.dll!7720a02f()
You have to debug the C runtime initialization code. Not intuitive to do because the debugger tries hard to avoid it and get you into the main() entrypoint instead. But still possible, use Debug > New Breakpoint > Function Breakpoint.
Enter _initterm for the function name, Language = C.
Press F5 and the breakpoint will hit. You should see the C runtime source code. You can now single-step through the initialization functions of your program one-by-one, every call to (**it)() executes one.
That's exactly what you asked for. But not very likely what you actually want. The odds that your code produces this error are very low. Much more likely is that one of these libraries causes this problem. They are likely to be built targeting another version of the C runtime library. And therefore have their own _initterm() function.
Having more than one copy of the C runtime library in a process is generally very unhealthy. And highly likely to generate heap corruption. If you can't locate it from the stack trace (be sure to change the Debugger Type from Auto to Mixed, always post the stack trace in an SO question) then the next thing you should strongly consider is rebuilding those libraries with the VS version you use.