Infinite loop when debugging in Visual Studio 2010 - visual-studio-2010

I have a strange issue with VS2010 (building a large C++ project). When starting to debug, the execution goes in something like an infinite loop. I put the a breakpoint at the first line of main(), but the breakpoint is never reached. I also tried F11 to step into main(), but no effect. The task manager shows an instance of my application, and the console says nothing but "xxx.dll Symbols loaded.". I tried to pause the execution, but I get thrown into some assembly loop, here is the assembly if someone can read it:
0000000077226129 lea rdx,[rsp+88h]
0000000077226131 xor ecx,ecx
0000000077226133 call 0000000077231650
0000000077226138 mov dword ptr [rsp+30h],eax
000000007722613C test eax,eax
000000007722613E js 000000007725E73F
0000000077226144 cmp dword ptr [7731201Ch],r14d
000000007722614B je 0000000077226129
Can someone tell me or at least point me to how to approach this problem?
Edit: I found out that when removing one of the shared libraries (FlyCapture2 developed by Point Grey Research), the application starts normally. It seems that the library has some kind of loading routite, which is called before the execution of main(). Even though I solved my current problem, I still would like to know: how to detect such kind of problems?

I think I found the problem.
In one of my DLLs I had a singleton class. In the header file i had an getter LogManager::instance() and an destroyer void LogManager::destroyInstance(). The instance was defined in the .cpp file but not statically but just as global variable
LogManager* sInstance = new LogManager;
and the instance() function just returned that variable and the destroyInstance() function deleted it. So I removed the global variable and created that instance within the instance() function
void LogManager::instance()
{
static LogManager* sInstance = 0;
if(!sInstance)
sInstance = new LogManager;
return sInstance;
}
and the problem was gone. So I think maybe the global variable in that DLL caused an infinite loop during loading of that DLL?
Maybe thats an hint for people with similar problems.

How to connect to DLL(s) correctly is a big problem.It is before applications so we are hard to find it and solve it.But when I hava a question about DLL(s),I would also look for answers on the Internet.And every error has its own serial number.That's the key.It also says "LNK**** ........" in the "Error List".So I can find it ,and even solve it by myself according to the information from others.
PS:Your code is called "Machine Instruction".It's part of Principles of Computer.

Related

MPLAB X + XC8 break at wrong line

I've encountered a very annoying problem that has cost a lot of time for several months now.
I have a project in MPLAB X. When I use a line-breakpoint it does not break on the right line at all when debugging my project.
I am using MPLAB X v4.15
This is what actually happens:
No matter where the breakpoint is, the debugger never breaks at the right place.
if I put a breakpoint somewhere, it always breaks at the wrong position
if I then restart the debugging it breaks at the same wrong position
If I change the breakpoint location, the position where the program actually breaks is different, but stays the same again when i restart the program.
Some more info:
Why does this happen?
Are there more people with this problem?
How do i solve this?
EDIT
Sadly, the solution suggested by K_Trenholm did not work for me. I put 3 "NOPs" in one function, but it didn't work as you suggested. See the picture below:
but what I got:
I would like to add that I tried various combinations of breakpoints for the NOPs. No matter what i do, the program always halts at the same PC for this case, seen in the picture above.
Thank you for your reply, it is very helpfull to even have ANY ideas on how to solve it.
If you have any other ideas, I would be very grateful if you would share them!
Two things come to mind:
1) Compiler optimizations can cause problems with breakpoint locations/values when debugging. When debugging, turn optimizations off (if possible, it looks like in your example you're bumping up on the ceiling in terms of code size).
2) Breakpoint "Skidding". See http://microchipdeveloper.com/tls0201:skid-effect#top-of-page
One way to work around this from what I've seen is to put a couple NOP instructions after the line where you plan on placing the breakpoint. This will ensure that any "skidding" will not execute more code.
The instruction where the break occurs will always execute completely, and anything pending in the pipeline will execute as well. For single cycle instructions, this adds a one instruction skid. For multiple cycle instructions and branches, it adds multiple cycles. So if you want to avoid
to jump the debugger into a subroutine you had to include some Nop behind the breakpoint.
example:
void main (void)
{
int x = 0;
x++1; //put Breakpoint here
Nop();
Nop();
Nop(); //Debugger will stop here
foo(x); //so foo() is not called
}
Depending on the MCU being used the debugger will introduce a 'skid effect' upon hitting a breakpoint. The debug session will execute up to two extra instructions before halting.

Why is ExitProcess necessary under Win32 when you can use a RET?

I've noticed that many assembly language examples built using straight Win32 calls (no C Runtime dependency) illustrate the use of an explicit call to ExitProcess() to end the program at the end of the entry-point code. I'm not talking about using ExitProcess() to exit at some nested location within the program. There are surprisingly fewer examples where the entry-point code simply exits with a RET instruction. One example that comes to mind is the famous TinyPE, where the program variations exit with a RET instruction, because a RET instruction is a single byte. Using either ExitProcess() or a RET both seem to do the job.
A RET from an executable's entry-point returns the value of EAX back to the Windows loader in KERNEL32, which ultimately propagates the exit code back to NtTerminateProcess(), at least on Windows 7. On Windows XP, I think I remember seeing that ExitProcess() was even called directly at the end of the thread-cleanup chain.
Since there are many respected optimizations in assembly language that are chosen purely on generating smaller code, I wonder why more code floating around prefers the explicit call to ExitProcess() rather than RET. Is this habit or is there another reason?
In its purest sense, wouldn't a RET instruction be preferable to a direct call to ExitProcess()? A direct call to ExitProcess() seems akin to exiting your program by killing it from the task manager as this short-circuits the normal flow of returning back to where the Windows loader called your entry-point and thus skipping various thread cleanup operations?
I can't seem to locate any information specific to this issue, so I was hoping someone could shed some light on the topic.
If your main function is being called from the C runtime library, then exiting will result in a call to ExitProcess() and the process will exit.
If your main function is being called directly by Windows, as may well be the case with assembly code, then exiting will only cause the thread to exit. The process will exit if and only if there are no other threads. That's a problem nowadays, because even if you didn't create any threads, Windows may have created one or more on your behalf.
As far as I know this behaviour is not properly documented, but is described in Raymond Chen's blog post, "If you return from the main thread, does the process exit?".
(I have also tested this myself on both Windows 7 and Windows 10 and confirmed that they behaved as Raymond describes.)
Addendum: in recent versions of Windows 10, the process loader is itself multi-threaded, so there will always be additional threads present when the process first starts.

What exactly happened with the Lisp REPL on JPL's DS-1?

I've heard the Google talk (http://www.youtube.com/watch?v=_gZK0tW8EhQ) by Ron Garret and read the paper (http://www.flownet.com/gat/jpl-lisp.html), but I'm not understanding how it worked to "correct" supposedly running code with a REPL. Was the DS-1's Lisp code running is some sort of virtual machine? Or was it "live" in the REPL's actual world? Or was the Lisp code an executable that got replaced? What exactly happened/happens when you dynamically change running Lisp code through a REPL?
Whereas most programs are built and distributed as an executable that contains only the necessary components to run the program, Lisp can be distributed as an image that contains not just the components for the specific program, but also much or all of the Lisp runtime and development environment.
The REPL is the quintessential mechanism for providing interactive access to a running Lisp environment. The two key components of the REPL, Read, and Eval, expose much of the Lisp runtime system. For example, many Lisp systems today implement Eval by compiling the provided form (that is read by the Reader), compiling the form to machine code, and then executing the result. This is in contrast to interpreting the form. Some systems, especially in the past, contained both an interpreter that executes quickly and is suitable for interactive access, and a compiler that produces better code. But modern systems are fast enough that the compiler phase isn't noticeable and simply forgo the interpreter.
Of course, you can do very similar things today. A simple example is running SSH to your Linux box that's hosting PHP. Your PHP server is up and running and live, serving pages and requests. But you login through SSH, go over and fix a PHP file, and as soon as you save that file, all of your users see the new result in real time -- the system updated itself on the fly.
The fact that PHP is running on a Linux runtime vs Lisp running on a Lisp runtime, is a detail. The effect is the same. The fact that PHP isn't compiled is a detail also. For example, you can do the same thing on a Java server: modify a JSP, save it, and the JSP is converted in to a Servlet as Java source code, then compiled on the fly by the Java runtime, then loaded in to the executing container, replacing the old code.
Lisps capability to do this is very nice, and it was very interesting far back in the day. Today, it's less so, as there are different system providing similar capabilities.
Addenda:
No, Lisp is not a virtual machine, there's no need for it to be that complicated.
The key to the concept is dynamic dispatch. With dynamic dispatch there is some lookup involved before a function is invoked.
In a static language like C, locations of things are pretty much set in stone once the linker and loader have finished processing the executable in preparation to start executing.
So, in C if you have something simple like:
int add(int i) {
return i + 1;
}
void main() {
add(1);
}
After compiling and linking and loading of the program, the address of the add function will be set in stone, and thus thing referring to that function will know exactly where to find it.
So, in assembly language: (note this is a contrived assembly language)
add: pop r1 ; pop R1 from the stack, loading the i parameter
add r1, 1; Add 1 to the parameter.
push r1 ; push result of function call
rts ; return from subroutine
main: push 1 ; Push parameter to function
call add ; call function
pop r1 ; gather (and ignore) the result
So, you can see here that add is fixed in place.
In something like Lisp, function are referred to indirectly.
int add(int i) {
return i + 1;
}
int *add_ptr() = &add;
void main() {
*(add_ptr)(1);
}
In assembly you get:
add: pop r1 ; pop R1 from the stack, loading the i parameter
add r1, 1; Add 1 to the parameter.
push r1 ; push result of function call
rts ; return from subroutine
add_ptr: dw add ; put the address of the add routine in add_ptr
main: push 1 ; Push parameter to function
mov r1, add_ptr ; Put the contents of add_ptr into R1
call (r1) ; call function indirectly through R1
pop r1 ; gather (and ignore) the result
Now, you can see here that rather than calling add directly, it is called indirectly through the add_ptr. In a Lisp runtime, it has the capability of compiling new code, and when that happens, add_ptr would be overwritten to point to the newly compiled code. You can see how the code in main never has to change, it will call whatever function add_ptr is pointing to.
Since most all of the functions in Lisp are indirectly referenced through their symbols, a lot can change "behind the back" of a running system, and the system, will continue to run.
When a function is recompiled, the old function code (assuming no other references) become eligible for garbage collection, and will, typically, eventually go away.
You can also see that when the system is garbage collected, any code that is moved (such as the code for the add function) can be moved by the runtime, and it's new location updated in the add_ptr so the system will continue operating even after code and been relocated by the garbage collector.
So, the key to it all, is to have your functions invoked through some lookup mechanism. Having this gives you a lot of flexibility.
Note, you can also do this is a running C system, for example. You can put code in a dynamic library, load the library, execute the code, and if you want you can build a new dynamic library, close the old one, open the new one, and invoke the new code -- all in a "running" system. The dynamic library interface provides the lookup mechanism that isolates the code.

call immediate versus call dword near [dword addr]

So recently I've been wanting to call some win32 calls from assembly, and I've been using NASM as my external assembler. I was calling SendMessage in my code in the following way:
call __imp__SendMessageW#16
This was assembled into a relative jump (0xE8 opcode) and the result was an access violation. In the debugger, the computed jump offset seemed to be the correct one (in that __imp__SendMessageW#16 really did seem to reside there) but nonetheless it did not work. Examining the assembly produced by Visual Studio when I called the function from C++, I noticed that it wasn't a relative immediate jump it was using, but instead (in the language of MASM) a call dword ptr [__imp__SendMessageW#16], corresponding to an 0xFF15 opcode. After some futzing around I figured out that NASM syntax encodes this as call dword near [dword __imp__SendMessageW#16], and making the change my code suddenly worked.
My question is, why does one work and not the other? Is there some relocation of code going on that causes the relative immediate call to jump somewhere unfriendly? I've never been much of an assembly programmer but my impression was always that the two calls should do the same thing and the main difference is that one is position independent and the other is not (assuming that they move the IP to the same place). The relocation of code theory makes sense given that, but then how do you explain the debugger showing the right address?
Also: what's the logic behind the [] syntax in this call? The offset is still an immediate (just little endian encoded immediately after 0xFF15), there's no memory access going on here beyond the instruction fetch (I tend to think of [] as a dereference outside the context of lea).
call dword[__imp__SendMessageW#16]
_imp_SendMessageW#16 is an address to your imports section that contains the address of the API function. You use the square brackets to deference (call the address STORED by this address)

How can I debug into an unmanaged BCL (InternalCall) method?

I want to debug into the implementation of a [MethodImpl(MethodImplOptions.InternalCall)] BCL method, which is presumably implemented in C++. (In this particular case, I'm looking at System.String.nativeCompareOrdinal.) This is mainly because I'm nosy and want to know how it's implemented.
However, the Visual Studio debugger is refusing to step into that method. I can set a breakpoint on this call:
"Hello".Equals("hello", StringComparison.OrdinalIgnoreCase);
then bring up Debug > Windows > Disassembly, step into the Equals call, and step until it gets to the call x86 instruction. But when I try to use "Step Into" on that call (which I know from Reflector is the nativeCompareOrdinal call), it doesn't step to the first instruction inside nativeCompareOrdinal like I want -- it steps over instead, and goes straight to the next x86 instruction in Equals.
I'm building as x86, since mixed-mode debugging isn't supported for x64 apps. I've unchecked "Just My Code" in Tools > Options > Debugging, and I have "Enable unmanaged code debugging" checked in project properties > Debug tab, but it still steps over the call. I also tried starting the process and then attaching the debugger, and explicitly attaching both the managed and native debuggers, but it still won't step into that InternalCall method.
How can I get the Visual Studio debugger to step into an unmanaged method?
Yes, it is tricky. The offset you see for the CALL instruction is bogus. Plus it won't let you navigate to an unmanaged code address when the current focus is on a managed function.
Start by enabling unmanaged code debugging and setting a breakpoint on the call. Run the code and when the break point hits use Debug + Windows + Disassembly:
"Hello".Equals("hello", StringComparison.OrdinalIgnoreCase);
00000025 call 6E53D5D0
0000002a nop
The debugger tries to display the absolute address but gets it wrong because it uses the bogus incremental address instead of the real instruction address. So first recover the true relative value: 0x6E53D5D0 - 0x2A = 0x6E53D5A6.
Next you need to find the real code address. Debug + Windows + Registers and look at the value of the EIP register. 0x009A0095 in my case. Add 5 to get to the nop, then add the relative offset: 0x9A0095 + 5 + 0x6E53D5A6 = 0x6EEDD640. The real address of the function.
Debug + Windows + Call Stack and double-click an unmanaged stack frame. Now you can enter the calculated address in the Disassembly window's Address box, prefix with 0x.
6EEDD640 push ebp
6EEDD641 mov ebp,esp
6EEDD643 push edi
6EEDD644 push esi
6EEDD645 push ebx
6EEDD646 sub esp,18h
etc...
Bingo, you're know you're good if you see the stack frame setup code. Set a breakpoint on it and press F5.
Of course, you'll be stepping machine code since there's no source code available. You'll get much better insight in what this code is doing by looking at the SSCLI20 source code. No guarantee that it will be a match for the actual code in your current version of the CLR but my experience is that these low-level code chunks that have been around since 1.0 are highly conserved. The implementation is in clr\src\classlibnative\nls, not sure which source code file. It won't be named "nativeCompareOrdinal", that's just an internal name used by ecall.cpp.

Resources