WinDbg not showing useful information - debugging

First let me say I am a total WinDbg noob, so this might be an easy question...
I have an application ("MyApp" - name changed to protect the innocent!) that I am trying to debug because it is throwing an exception. This only happens on user machines - I have not been able to reproduce it on my development machine. So I set up DebugDiag on the users machine and captured a Full Dump. Then I loaded the dump in WinDbg and did an analyze -v and a kp to try to figure out what was going on... but neither of these seem to give me the information that I'm looking for - the function (and hopefully the line number) of the line that is causing the problem... I think I have the symbol file loaded by specifying the path to 'MyApp.pdb' in the Symbol File Path:
srv*c:\symcache*http://msdl.microsoft.com/download/symbols;srv*c:\symcache*C:\dev\Customer\MyAppSln\MyApp\Debug
First, here's the output from kp:
0:004> kp
ChildEBP RetAddr
WARNING: Stack unwind information not available. Following frames may be wrong.
0502f474 7c347966 MyApp!DllMain+0x3e8a6
0502f4bc 7c3a2448 msvcr71!_nh_malloc(unsigned int size = <Memory access error>, int nhFlag = <Memory access error>)+0x24 [f:\vs70builds\3052\vc\crtbld\crt\src\malloc.c # 117]
0502f57c 7c3416b3 msvcp71!std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> >::_Tidy(bool _Built = <Memory access error>, unsigned int _Newsize = <Memory access error>)+0x45 [f:\vs70builds\3077\vc\crtbld\crt\src\xstring # 1520]
0502f610 7c3a32de msvcr71!_heap_alloc(unsigned int size = <Memory access error>)+0xe0 [f:\vs70builds\3052\vc\crtbld\crt\src\malloc.c # 212]
0502f620 7c3b3f63 msvcp71!wmemcpy(wchar_t * _S1 = 0x04e463b9 "Ҹ???", wchar_t * _S2 = 0xffffffff "--- memory read error at address 0xffffffff ---", unsigned int _N = 0x4e25212)+0x14 [f:\vs70builds\3077\vc\crtbld\crt\src\wchar.h # 843]
0502f640 04e463b9 msvcp71!std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> >::assign(class std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > * _Right = 0xffffffff, unsigned int _Roff = 0x4e25212, unsigned int _Count = 2)+0x7c [f:\vs70builds\3077\vc\crtbld\crt\src\xstring # 601]
0502f770 04df1077 MyApp!DllMain+0x65329
0502f824 04e01b35 MyApp!DllMain+0xffe7
0502ff08 04dfe034 MyApp!DllMain+0x20aa5
0502ff48 04dfde4f MyApp!DllMain+0x1cfa4
0502ff88 7648d0e9 MyApp!DllMain+0x1cdbf
0502ffc4 773499f9 kernel32!BaseThreadInitThunk+0xe
0502ffd4 7738198e ntdll!RtlQueryInformationAcl+0x8b
0502ffec 00000000 ntdll!_RtlUserThreadStart+0x1b
the line I'm specifically trying to decode is the 'MyApp!DllMain+0x65329' as this is the last line that seems to be executing, and the error is occurring within the malloc call, which is apparently where the exception is being thrown from. What am I doing wrong that makes it only display the module and offset instead of source file and line number?
I'm also not sure why the line above the malloc call is back in MyApp again - maybe someone can explain that too.
Just in case, here's the output from 'analyze -v':
0:004> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
*** WARNING: Unable to verify checksum for MyApp.exe
*** ERROR: Module load completed but symbols could not be loaded for MyApp.exe
*** WARNING: Unable to verify checksum for ThirdPartyDll.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for ThirdPartyDll.dll -
*** WARNING: Unable to verify checksum for mdnsNSP.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for mdnsNSP.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for SLC.dll -
FAULTING_IP:
MyApp!DllMain+3e8a6
04e1f936 8b16 mov edx,dword ptr [esi]
EXCEPTION_RECORD: ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 04e1f936 (MyApp!DllMain+0x0003e8a6)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 00000000
Parameter[1]: 00000000
Attempt to read from address 00000000
PROCESS_NAME: MyApp.exe
ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The memory could not be "%s".
EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The memory could not be "%s".
EXCEPTION_PARAMETER1: 00000000
EXCEPTION_PARAMETER2: 00000000
READ_ADDRESS: 00000000
FOLLOWUP_IP:
msvcr71!_heap_alloc+e0 [f:\vs70builds\3052\vc\crtbld\crt\src\malloc.c # 212]
7c3416b3 e88e0c0000 call msvcr71!__SEH_epilog (7c342346)
NTGLOBALFLAG: 0
APPLICATION_VERIFIER_FLAGS: 0
LAST_CONTROL_TRANSFER: from 00000000 to 773bbb33
FAULTING_THREAD: ffffffff
BUGCHECK_STR: APPLICATION_FAULT_ACTIONABLE_HEAP_CORRUPTION_heap_failure_freelists_corruption_NULL_POINTER_READ_SHUTDOWN
PRIMARY_PROBLEM_CLASS: ACTIONABLE_HEAP_CORRUPTION_heap_failure_freelists_corruption_SHUTDOWN
DEFAULT_BUCKET_ID: ACTIONABLE_HEAP_CORRUPTION_heap_failure_freelists_corruption_SHUTDOWN
STACK_TEXT:
773bbb33 ntdll!RtlpAllocateHeap+0x7ad
773a6e0c ntdll!RtlAllocateHeap+0x1e3
7c3416b3 msvcr71!_heap_alloc+0xe0
FAULTING_SOURCE_CODE:
No source found for 'f:\vs70builds\3052\vc\crtbld\crt\src\malloc.c'
SYMBOL_STACK_INDEX: 2
SYMBOL_NAME: msvcr71!_heap_alloc+e0
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: msvcr71
IMAGE_NAME: msvcr71.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 3e561eac
STACK_COMMAND: dds 7740c078 ; kb
FAILURE_BUCKET_ID: ACTIONABLE_HEAP_CORRUPTION_heap_failure_freelists_corruption_SHUTDOWN_c0000005_msvcr71.dll!_heap_alloc
BUCKET_ID: APPLICATION_FAULT_ACTIONABLE_HEAP_CORRUPTION_heap_failure_freelists_corruption_NULL_POINTER_READ_SHUTDOWN_msvcr71!_heap_alloc+e0

If you believe the PDB should be in your symbol path, you should run something like this:
!sym noisy
.reload MyApp.dll
kp
!sym noisy causes the debugger to give out more detailed information on why it couldn't load symbols - no MyApp.pdb found, found but does not match, etc. This will help you find out why it is not loading symbols. !sym noisy again turns off the verbose symbol output.

When you set the path for symbols, did you reload them?
.reload
I'm not sure your adding
srv*c:\symcache*C:\dev\Customer\MyAppSln\MyApp\Debug
to the symbol path has the desired effect.
I usually list all local paths in the .sympath first, and as the last step, I do .symfix+ to configure the public symbols using the microsoft symbol server:
.sympath C:\dev\Customer\MyAppSln\MyApp\Debug
.symfix+ c:\symcache
the rationale behind listing local paths first being that the debugger would not have to check the remote server for pdbs (that are not there anyways) as opposed to simply retrieving them locally.
Anyways, your problem is that the symbols for MyApp are not loaded therefore stack walking does not quite work.
Debugger walks the stack backwards, starting from the top, that's why you're seeing MyApp - this is where the access violation occurred.
Now, since debugger does not have the symbols at this point, it can only guess what invocation chain has led to the function on top.
And it guesses wrong by following a misleading path.

Related

How to fix VB6 APPCRASH ntdll.dll error on make

I am making an exe from an existing VB6 project. During Make, VB crashes with the following message
Visual Basic has stopped working
Problem Event Name: APPCRASH
Application Name: vb6.exe
Application Version: 6.0.97.82
Fault Module Name: ntdll.dll
Exception Code: c0000005
I am able to run the project from VB6 without any trouble. The problem occurs when I try to make the exe.
Please could you let me know what could be wrong?
Thanks!
This all needs to be done on the computer with the fault. I cannot load my ntdll.dll as it a different version and the addresses will be different to yours.
Download and install Debugging Tools for Windows
http://msdn.microsoft.com/en-us/windows/hardware/hh852363
Install the Windows SDK but just choose the debugging tools.
Create a folder called Symbols in C:\
This allows WinDbg to get the symbols for your version of ntdll.dll. Start Windbg. File menu - Symbol File Path and enter
srv*C:\symbols*http://msdl.microsoft.com/download/symbols
then
Open ntdll in WinDbg as a crashdump.
It will show the load address.
Type in WinDbg
ln <modloadaddress> + 7c911780
This will give you the nearest symbol to the crash. It probably isn't useful but lets see.
You can also run VB6 under WinDbg (make sure WinDbg is run as admin). When you crash do a stack trace.
Also do an !Analyze when you crash. It is meant for blue screens but will give info on appcrash.
Type in the WinDbg command prompt
!analyze -v
-v stands for Verbose and if the crash was originated by a program, as opposed to hardware or a driver, it will appear in the middle of the listing.
eg
PROCESS_NAME: java.exe
IMAGE_NAME: ntkrnlmp.exe
PROCESS_NAME only appears in the analyze -v output and only if a program originated the call that faulted.
WinDbg Commands
Open as Executable.
windbg -o -g -G c:\windows\system32\cmd.exe /k batfile.bat
You can press F12 to stop it and kb will show the call stack (g continues the program). If there's errors it will also stop and show them.
There is a breakpoint after loading but before any code is run. Press g to continue. Likewise there is a breakpoint after all code has run but before it is unloaded.
Type lm to list loaded modules, x *!* to list the symbols and bp symbolname to set a breakpoint
If programming in VB6 then this environmental variable link=/pdb:none stores the symbols in the dll rather than separate files. Make sure you compile the program with No Optimisations and tick the box for Create Symbolic Debug Info. Both on the Compile tab in the Project's Properties.
Sample output from a nearest symbol search.
Loading Dump File [C:\Windows\System32\ntdll.dll] Symbol search path
is: srvc:\symbolshttp://msdl.microsoft.com/download/symbols
Executable search path is: ModLoad: 4b280000 4b3f9000
C:\Windows\System32\ntdll.dll eax=00000000 ebx=00000000 ecx=00000000
edx=00000000 esi=00000000 edi=00000000 eip=4b280000 esp=00000000
ebp=00000000 iopl=0 nv up di pl nz na pe nc cs=0000 ss=0000
ds=0000 es=0000 fs=0000 gs=0000 efl=00000000
ntdll!__guard_fids_table (ntdll+0x0): 4b280000 4d
dec ebp 0:000> ln 4b280000 + 65534 (4b2e5520)
ntdll!RtlInitializeBitMap+0x14 | (4b2e5540)
ntdll!TpCallbackUnloadDllOnCompletion
Sample stack trace.
You follow what function called what functions. So you read it from the bottom up. It has the first 4 parameters that were passed to the function. You find the debugger starts additional threads so we need to find our program's one.
~
Lists all threads
~<threadid> e <command>
Do a KB on all threads until you find the main one.
0:004> ~0 e kb
ChildEBP RetAddr Args to Child 04bdfc30
75ae325a 04bdfc70 00000000 00000000 USER32!NtUserGetMessage+0xc
04bdfc4c 00895eb6 04bdfc70 00000000 00000000 USER32!GetMessageW+0x2a
04bdfc8c 008a5b41 00890000 00000000 04e2336f notepad!WinMain+0xe6
04bdfd20 74ad3744 7f229000 74ad3720 10fde46e
notepad!WinMainCRTStartup+0x151 04bdfd34 7755a064 7f229000 b0c1107f
00000000 KERNEL32!BaseThreadInitThunk+0x24 04bdfd7c 7755a02f ffffffff
7757d7c9 00000000 ntdll!__RtlUserThreadStart+0x2f 04bdfd8c 00000000
008a59f0 7f229000 00000000 ntdll!_RtlUserThreadStart+0x1b
Assume that 04bdfc70 is an HWnd. Which it is because the documentation says so. But assume it an address of a string. This displays what is there.
ds 775a1300
or to look at the values
db 775a1300

Why is Windows Safe Mode Magic?

Everyone.
I help out with I.T. for a medical practice. It turns out there’s a bug in the EMR software which shows up once in a while. (All the computers run on Windows 10 currently)
The program dies and I get the popup “A problem has caused the program to stop working…”
I hook up a debugger. It looks like a heap issue in their code according to the stack trace. I turn on GFlags and it hits every time. I tell the EMR Software folks but they don’t like to claim bugs and want to prove it’s not them.
They boot the computer to Safe Mode with Networking and low and behold there’s no heap problem so they say they’re off the hook. It’s not them.
I’ve reproduced the problem on all the machines (32 and 64 bit) and in Clean Mode and logged in under the activated Administrator account but Safe Mode works every time.
I've reproduced the problem by running in a command window:
gflags /p /enable emrprogram.exe /full
cdb -g -x emrname.exe
Questions:
How can Safe Mode make a heap bug disappear?
Any suggestions for reproducing the bug in Safe Mode, i.e. what’s really happening in Safe Mode Can I make it more like Normal Mode?
P.S. Here's the stack trace:
VERIFIER STOP 00000010: pid 0x238: corrupted start stamp
04F61000 : Heap handle
0014F09C : Heap block
00000000 : Block size
00000000 : Corrupted stamp
0:000> kb
ChildEBP RetAddr Args to Child
0014ed20 5fa0aac9 00000010 5fa01bc8 04f61000 verifier!VerifierStopMessage+0x27e
0014ed84 5fa0ae8a 04f61000 00000004 0014f09c
verifier!AVrfpDphReportCorruptedBlock+0x239
0014ede0 5fa0b3c2 04f61000 0014f09c 00000004 verifier!AVrfpDphCheckNormalHeapBlock+0x11a
0014ee00 5fa09cf3 04f61000 05230000 01000002 verifier!AVrfpDphNormalHeapFree+0x22
0014ee24 77d76c42 04f60000 01000002 0014f09c verifier!AVrfDebugPageHeapFree+0xe3
0014ee84 77cca934 0014f09c 014b1fe2 04f60000 ntdll!RtlDebugFreeHeap+0x3c
0014ef40 77cc9238 00000000 0014f09c 44ce6ff8 ntdll!RtlpFreeHeap+0xb4
*** ERROR: Module load completed but symbols could not be loaded for image00400000
0014ef68 01f3d902 04f60000 00000000 0014f09c ntdll!RtlFreeHeap+0x268
WARNING: Stack unwind information not available. Following frames may be wrong.
0014f0b8 01f3c6c6 0014f1dc 08f58f70 0dab2f88 image00400000+0x1b3d902
0014f21c 01f3c4b9 0dab2f88 0014f2d8 08f58f70 image00400000+0x1b3c6c6
0014f320 02053935 0dab2f88 0014f454 0014f500 image00400000+0x1b3c4b9
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\SYSTEM32\MSVBVM60.DLL -
0014f4f4 66051d33 08f58f70 0014f510 0051c783 image00400000+0x1c53935
0014f510 66052034 0051c783 0014f57c 00000002 MSVBVM60!IID_IVbaHost+0x236f3
0014f528 6605211a 08f58fe8 0014f6c4 0014f57c MSVBVM60!IID_IVbaHost+0x239f4
0014f6cc 77b3b3cc 00000009 0659b040 00000000 MSVBVM60!IID_IVbaHost+0x23ada
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\Program Files\emrname\IGThreed40.ocx -
0014f72c 2411bead 0c7e0fac 00000016 241343f8 GDI32!ScriptStringAnalyzeGlyphs+0x2cc
0014f98c 2410ae97 0c71fd60 2413794c 0c71fff0 IGThreed40!DLLGetDocumentation+0x27d1
0014f9a4 24116fda 09f4b6fc 16fa9fd4 759fb0ff IGThreed40+0xae97
0014f9c8 759dc807 0c71fff0 000001cc 00000004 IGThreed40+0x16fda
0014fc70 2411b679 0cac5638 0c71fff0 fffffdd9 OLEAUT32!CTypeInfo2::Invoke+0x517
0014fc9c 24106d11 0c71fd60 fffffdd9 0cac5638 IGThreed40!DLLGetDocumentation+0x1f9d
0014fcd0 241179f6 0c71fd60 fffffdd9 6601aea8 IGThreed40+0x6d11
0014fcfc 66049039 0c71fd60 fffffdd9 6601aea8 IGThreed40+0x179f6
0014fd44 66049a8a 0c7e0e8c 08baaf0c 0000000d MSVBVM60!IID_IVbaHost+0x1a9f9
0014fd74 66083900 0c7e0e8c 000a037e 0000104d MSVBVM60!IID_IVbaHost+0x1b44a
0014fd9c 66083d58 08b8ce6c 0014fe50 00000001 MSVBVM60!IID_IVbaHost+0x552c0
0014fdd8 6601ca5e 08b8ce6c 00c8030c 7552d390 MSVBVM60!IID_IVbaHost+0x55718
0014fe08 6600a782 0014fe50 ffffffff 6600a72e MSVBVM60!Zombie_Release+0xe005
0014fe34 6600a6b0 07097f8c 0014fe50 ffffffff MSVBVM60!_vbaStrToAnsi+0x3ab
0014fe78 6600a63f ffffffff 07097f8c 07080000 MSVBVM60!_vbaStrToAnsi+0x2d9
0014febc 6600a51d 0709dfcc ffffffff 00000238 MSVBVM60!_vbaStrToAnsi+0x268
0014fed8 6600a4e8 07097f88 0709dfcc ffffffff MSVBVM60!_vbaStrToAnsi+0x146
0014fefc 66003644 ffffffff 03f40670 03f40670 MSVBVM60!_vbaStrToAnsi+0x111
0014ff78 00489fca 004a30b4 779995f4 0020d000 MSVBVM60!ThunRTMain+0xa0
0014ff94 77cb241a 0020d000 014b0f7e 00000000 image00400000+0x89fca
0014ffdc 77cb23e9 ffffffff 77d339e7 00000000 ntdll!__RtlUserThreadStart+0x2b
0014ffec 00000000 03f40670 0020d000 00000000 ntdll!_RtlUserThreadStart+0x1b
0:000>

LoadLibraryA with User32.dll crash in ntdll.dll (x64 assembly)

So I have a block of assembly code that initializes a program, resolves kernel32, finds GetProcAddress, then finds LoadLibarayA to load User32.dll. It works up to the point of LoadLibraryA. It crashes in the function call but I can see User32.dll loaded in the debugger. If I try to use LoadLibraryA on a different module such as Kernel32.dll it returns and succeeds.
Here is the full source if you want to look it over
https://gist.github.com/mojobojo/921a5af897e86bb940a2
Exception thrown at 0x00007FFAFAE8E91C (ntdll.dll) in Small.exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.
Here is the snippet that trys to load user32.
mov rcx, ActualAddress + User32DllStr ; ActualAddress is the program address in memory
call rax ; LoadLibararyA
cmp rax, 0
je EndFunction ; Failed to open user32.dll
LoadLibraryAStr:
db "LoadLibraryA", 0
Here is a look at the call stack.
ntdll.dll!RtlDosPathNameToRelativeNtPathName() Unknown
ntdll.dll!LdrpResolveDllName() Unknown
ntdll.dll!LdrpFindLoadedDll() Unknown
ntdll.dll!LdrGetDllHandleEx() Unknown
ntdll.dll!LdrGetDllHandle() Unknown
KernelBase.dll!00007ffaf82d2984() Unknown
KernelBase.dll!00007ffaf82d29ef() Unknown
user32.dll!00007ffaf934e7e8() Unknown
user32.dll!00007ffaf934dc92() Unknown
ntdll.dll!LdrpCallInitRoutine() Unknown
ntdll.dll!LdrpInitializeNode() Unknown
ntdll.dll!LdrpInitializeGraph() Unknown
ntdll.dll!LdrpPrepareModuleForExecution() Unknown
ntdll.dll!LdrpLoadDll() Unknown
ntdll.dll!LdrLoadDll() Unknown
KernelBase.dll!00007ffaf82d8e4a() Unknown
KernelBase.dll!00007ffaf82d97e5() Unknown
kernel32.dll!00007ffaf8b5499a() Unknown
Small.exe!0000000140010253() Unknown
I figured it out. My stack wasn't 16 byte aligned.

MEM_FREE pages identification

I want to allocate a specific 512MB page which is MEM_FREE, and I want
to change that page to MEM_RESERVE and PAGE_NOACCESS.
Hence, with Windbg, I found a page and I called to
NtAllocateVirtualMemory on that page address with PAGE_RESERVED and
PAGE_NOACCESS.
After this call - I didn't notice that the page got PAGE_NOACCESS
flag(with !address command), but I couldn't change a memory address
inside that page - (I got memory access error for eb command).
So the operation succeeded because I couldn't change the memory.
Do you have an idea why I don't see in windbg PAGE_NOACCESS for
that page after change it's permissions?
Next Step, I called to VirtualQuery on that free page, and the
function failed with error 998 (Invalid access to memory location)
Finally I want to identify free pages and reveal their size.
Do you have an idea how can I get this information it VirtualQuery fails?
Thanks in Advance!
MEM_RESERVE reserves a range of addresses in the VAD tree. It doesn't commit pages that can have protection applied to them. Read the description of the PAGE_* constants. The phrase "committed region of pages" is written repeatedly. The Protect parameter of NtAllocateVirtualMemory says it applies to a "committed region of pages". Similarly flProtect of VirtualAllocEx is valid when "the pages are being committed".
Here's a Python script the commits a page of memory that's protected with PAGE_NOACCESS. Then it attempts to read the first byte, which of course raises an access violation exception.
test.py:
from ctypes import *
MEM_COMMIT = 0x1000
PAGE_NOACCESS = 1
VirtualAlloc = WinDLL('kernel32').VirtualAlloc
VirtualAlloc.restype = c_void_p
addr = VirtualAlloc(None, 4096, MEM_COMMIT, PAGE_NOACCESS)
array = (c_char * 4096).from_address(addr)
array[0] # access violation
demo:
(test) C:\>cdb -xi ld python test.py
Microsoft (R) Windows Debugger Version 6.12.0002.633 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
CommandLine: python test.py
Symbol search path is: symsrv*symsrv.dll*C:\Symbols*
http://msdl.microsoft.com/download/symbols
Executable search path is:
(d70.d08): Break instruction exception - code 80000003 (first chance)
ntdll!LdrpDoDebuggerBreak+0x30:
00000000`77b78700 cc int 3
0:000> g
(d70.d08): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
python34!PyBytes_FromStringAndSize+0x70:
00000000`64bd5cc0 0fb601 movzx eax,byte ptr [rcx]
ds:00000000`00190000=??
0:000> !address 190000
Usage: <unclassified>
Allocation Base: 00000000`00190000
Base Address: 00000000`00190000
End Address: 00000000`00191000
Region Size: 00000000`00001000
Type: 00020000 MEM_PRIVATE
State: 00001000 MEM_COMMIT
Protect: 00000001 PAGE_NOACCESS

How to get a call stack for an intermittently crashing devenv.com?

I have a nightly build DOS batch script that invokes devenv.exe to build a solution file. Intermittently I observe a devenv.com crash. I get a DW20.exe "share your pain" dialog.
1) If Debug button is pressed, I am not presented with a usual "Choose your debugger" window. Rather, it does nothing.
2) If I launch a Visual Studio and try to attach to devenv.com, it says that the application appears to be deadlocked or waiting for an operation to finish. (I think because it's debugged by DW20.exe)
What would you recommend I do to get a decent call stack?
EDIT
I was successful in getting WinDBG to attach. Executed the k command. Does the following look like something meaningful that can be passed on to the compiler team? Is there a way to look at the current exception?
(e90.fb8): Break instruction exception - code 80000003 (!!! second chance !!!)
eax=0012ccb8 ebx=04ed2750 ecx=0111bdc4 edx=5a57f004 esi=00000000 edi=14ed1000
eip=77e4bef7 esp=0012ccb4 ebp=0012cd08 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\system32\kernel32.dll -
kernel32!RaiseException+0x3c:
77e4bef7 5e pop esi
Missing image name, possible paged-out or corrupt data.
Missing image name, possible paged-out or corrupt data.
Missing image name, possible paged-out or corrupt data.
Missing image name, possible paged-out or corrupt data.
0:000> k
ChildEBP RetAddr
WARNING: Stack unwind information not available. Following frames may be wrong.
0012cd08 5a760cf1 kernel32!RaiseException+0x3c
0012cd58 5a766105 cslangsvc!InMemoryCompile+0x4c6c1
0012cd7c 5a767375 cslangsvc!InMemoryCompile+0x51ad5
0012cd84 5a767637 cslangsvc!InMemoryCompile+0x52d45
00000000 00000000 cslangsvc!InMemoryCompile+0x53007
I am trying to get !analyze -v to work.
0:000> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
***** OS symbols are WRONG. Please fix symbols to do analysis.
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: ntdll!_PEB ***
*** ***
*************************************************************************
c0000005 Exception in ext.analyze debugger extension.
PC: 014d7875 VA: 00000000 R/W: 0 Parameter: 0001003f
EDIT 2
I found out that there are these wonderful SOS extensions to WinDBG. Apparently I can use them to debug C# Compiler...
0:000> !PrintException
There is no current managed exception on this thread
0:000> !clrstack
OS Thread Id: 0x9fc (0)
ESP EIP
0012ee08 77e4bef7 [ComPlusMethodFrameGeneric: 0012ee08] Microsoft.Build.Tasks.Hosting.ICscHostObject.Compile()
0012ee18 6be671ab Microsoft.Build.Tasks.Csc.CallHostObjectToExecute()
0012ee48 6c0aed17 Microsoft.Build.Utilities.ToolTask.Execute()
0012ee7c 6bcbb348 Microsoft.Build.BuildEngine.TaskEngine.ExecuteTask(ExecutionMode, System.Collections.Hashtable, Microsoft.Build.BuildEngine.BuildPropertyGroup, Boolean ByRef)
0012ef24 6bcadf87 Microsoft.Build.BuildEngine.Target.ExecuteAllTasks(Microsoft.Build.BuildEngine.DependencyAnalysisResult, System.Collections.Hashtable, System.Collections.Hashtable, Microsoft.Build.BuildEngine.ItemBucket, System.Collections.ArrayList, Microsoft.Build.BuildEngine.BuildPropertyGroup)
0012efa0 6bcac7c0 Microsoft.Build.BuildEngine.Target.ExecuteAllTasks()
0012efec 6bcad05b Microsoft.Build.BuildEngine.Target.Build(System.Collections.IDictionary)
0012f070 6bcacfa6 Microsoft.Build.BuildEngine.Target.Build(System.Collections.IDictionary)
0012f0f4 6bcacfa6 Microsoft.Build.BuildEngine.Target.Build(System.Collections.IDictionary)
0012f178 6bcacfa6 Microsoft.Build.BuildEngine.Target.Build(System.Collections.IDictionary)
0012f1fc 6bc9b20d Microsoft.Build.BuildEngine.Project.DoBuild(System.String[], System.Collections.IDictionary, Boolean)
0012f250 6bca2134 Microsoft.Build.BuildEngine.Engine.BuildProject(Microsoft.Build.BuildEngine.Project, System.String[], System.Collections.IDictionary, Microsoft.Build.BuildEngine.BuildSettings, Boolean)
0012f2ac 6bc9af03 Microsoft.Build.BuildEngine.Project.Build(System.String[], System.Collections.IDictionary)
0012f2c0 04711a36 Microsoft.VisualStudio.Build.ComInteropWrapper.ProjectShim.BuildTarget(System.String, System.Collections.IDictionary)
0012f4f4 79f68cde [GCFrame: 0012f4f4]
0012f650 79f68cde [ComMethodFrame: 0012f650]
I am still looking for a way to examine the exception contents. !PrintException did not find any exceptions.
You might try installing Debugging Tools for Windows then use the -iae option (for NTSD/CDB) or the -I option (for WinDBG) to have one of the debuggers from that package be the JIT/AeDebug debugger.
I finally arrived at an exception type and callstack for the faulting compiler. I did this by saving a full dump file (.dump /ma c:\myCSCdump.dmp), then opening the dump file as you would open any other project file in VS2005. Pressing the run button produced the following in the call stack window:
kernel32.dll!_RaiseException#16() + 0x3c bytes
cslangsvc.dll!ALLOCHOST::ThrowOutOfMemoryException() + 0x10 bytes
cslangsvc.dll!PAGEHEAP::AllocPages() + 0xc1fd8 bytes
cslangsvc.dll!SYMTBL::GrowTable() + 0x5b bytes
cslangsvc.dll!BSYMMGR::AddChild() + 0x3b bytes
cslangsvc.dll!BSYMMGR::CreateGlobalSym() + 0x2ba bytes
cslangsvc.dll!BSYMMGR::CreateMethod() + 0xc bytes
cslangsvc.dll!IMPORTER::ImportMethod() + 0x1d9 bytes
cslangsvc.dll!IMPORTER::DefineImportedType() + 0x3b2 bytes
cslangsvc.dll!CLSDREC::prepareAggregate() + 0x8d bytes
cslangsvc.dll!COMPILER::ForceAggStates() + 0x8e bytes
cslangsvc.dll!COMPILER::ForceAggStates() + 0x127 bytes
cslangsvc.dll!COMPILER::EnsureState() + 0x2e bytes
cslangsvc.dll!CLSDREC::CheckForTypeErrors() + 0x23 bytes
cslangsvc.dll!CLSDREC::CheckForTypeErrors() + 0x2ba bytes
cslangsvc.dll!CLSDREC::CheckForTypeErrors() + 0x80 bytes
cslangsvc.dll!CLSDREC::CheckForTypeErrors() + 0x8a bytes
cslangsvc.dll!CLSDREC::CheckForTypeErrors() + 0x8a bytes
cslangsvc.dll!CLSDREC::CheckForTypeErrors() + 0x8a bytes
cslangsvc.dll!COMPILER::CheckForTypeErrors() + 0x100 bytes
cslangsvc.dll!COMPILER::CompileAll() + 0x546 bytes
cslangsvc.dll!COMPILER::Compile() + 0x7c bytes
cslangsvc.dll!CController::RunCompiler() + 0x177 bytes
cslangsvc.dll!CController::Compile() + 0x16 bytes
cslangsvc.dll!CCSharpProjectSite::BuildProjectCore() + 0x5a bytes
cslangsvc.dll!CProjectSite::BuildProject() + 0x11 bytes
csproj.dll!CCSharpBuildCompiler::DoMainBuild() + 0x88 bytes
csproj.dll!CCscMSBuildHostObject::Compile() + 0x4e bytes
mscorwks.dll!_CLRToCOMWorker#8() + 0x171 bytes
This is a bit wordy, so I'll post it as a reply. It's taken from Microsoft Connect site, in response to my issue. Reader might find it relevant:
Date: Thu, 7 May 2009 19:14:52 -0700
Greetings from Microsoft Connect!
This notification was generated for feedback item: Nightly build troubles (bucket 365749762) which you submitted at the Microsoft Connect site.
Thanks for reporting this issue you've encountered with Visual Studio 2005, and thanks for the dumps you've provided!
It looks like the issue is in cslangsvc.dll (the in-process C# Compiler), which is DevEnv is calling into to perform your build.
Our suggestion to work around this issue would
be to try using MSBuild instead to do your nightly builds, as this should be more robust than automating DevEnv /build. Specifically, you'll likely avoid this particular issue as MSBuild will use csc.exe for compilation rather than cslangsvc.dll.
We're unlikely to have a further VS 2005 release to get out a fix to the cslangsvc issue, so I'm resolving this bug Won't Fix, but please reactivate it if you do still see an issue when using MSBuild.
Thanks again for all the information on this issue!
Alex Turner
Program Manager
Visual C# Compiler

Resources