I forced a complete kernel dump for an application that appears to be waiting on a deadlocked thread in a kernel driver. I believe I have identified the offending thread as it has 1920031 ticks (0:08:19:12.675) which is roughly how long the application has been waiting. The thread has also only spent .015 milliseconds in user and kernel time:
THREAD 85e18cf8 Cid 0988.1160 Teb: 7ffa8000 Win32Thread: 00000000 WAIT: (UserRequest) UserMode Non-Alertable
85dac938 NotificationEvent
85da5350 SynchronizationEvent
IRP List:
b7d67818: (0006,0094) Flags: 00060000 Mdl: 00000000
85d95c70: (0006,0094) Flags: 00060000 Mdl: 00000000
Not impersonating
DeviceMap 8c808980
Owning Process ba224730 Image: MyApplication.exe
Attached Process N/A Image: N/A
Wait Start TickCount 2447201 Ticks: 1920031 (0:08:19:12.675)
Context Switch Count 326 IdealProcessor: 1
UserTime 00:00:00.015
KernelTime 00:00:00.015
Win32 Start Address 0x6b19f28e
Stack Init c5463fd0 Current c5463748 Base c5464000 Limit c5461000 Call 0
Priority 9 BasePriority 8 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5
Kernel stack not resident.
ChildEBP RetAddr
c5463760 830ce8d5 nt!KiSwapContext+0x26 (FPO: [Uses EBP] [0,0,4])
c5463798 830cd733 nt!KiSwapThread+0x266
c54637c0 830c94e4 nt!KiCommitThreadWait+0x1df
c546393c 8327d1db nt!KeWaitForMultipleObjects+0x535
c5463bc8 8327cf48 nt!ObpWaitForMultipleObjects+0x262
c5463d18 8308da16 nt!NtWaitForMultipleObjects+0xcd
c5463d18 777870d4 nt!KiSystemServicePostCall (FPO: [0,3] TrapFrame # c5463d34)
WARNING: Frame IP not in any known module. Following frames may be wrong.
012af6f8 00000000 0x777870d4
It has two pending IRPs to the same device and same file.
2: kd> !irp 85d95c70
Irp is active with 1 stacks 1 is current (= 0x85d95ce0)
No Mdl: No System Buffer: Thread 85e18cf8: Irp stack trace.
cmd flg cl Device File Completion-Context
>[IRP_MJ_DEVICE_CONTROL(e), N/A(0)]
0 1 8886d6b8 a1b90a48 00000000-00000000 pending
\Driver\nsiproxy
Args: 00000014 00000014 0012003f 012af5b0
2: kd> !object a1b90a48
Object: a1b90a48 Type: (85b36518) File
ObjectHeader: a1b90a30 (new version)
HandleCount: 1 PointerCount: 10
!findhandle on that file confirms MyApplication.exe has access to it.
600: Entry ba707c00 Granted Access 100080
The file is not locked:
2: kd> dt -r nt!_OBJECT_HEADER a1b90a30
+0x000 PointerCount : 0n10
+0x004 HandleCount : 0n1
+0x004 NextToFree : 0x00000001 Void
+0x008 Lock : _EX_PUSH_LOCK
+0x000 Locked : 0y0
+0x000 Waiting : 0y0
+0x000 Waking : 0y0
+0x000 MultipleShared : 0y0
+0x000 Shared : 0y0000000000000000000000000000 (0)
+0x000 Value : 0
+0x000 Ptr : (null)
How can one thread have two nearly-identical pending IRPs to the same device and file?
How can the file object be open in two IRPs but only have one open handle?
Thanks!
Related
I am not able to get the right stack for on my own C++ x64 compiled apps. I tried multiple versions of Visual Studio (VS2013, VS2015, VS2017). VS2013 worked fine, stacks were correct in Windbg KD, but VS2015 and VS2017 stacks were incorrect in Windbg KD.
To simply reproduce this
[optional] Enable windows debugging and restart PC
bcdedit -debug on
Open Visual Studio.
Create new console app project. Replace main with this:
#include "stdafx.h"
#include <Windows.h>
class CSymbolTest
{
public:
void TestSymbols(const char* param1, unsigned int param2)
{
printf("%s %u\n", param1, param2);
system("PAUSE");
}
};
int main()
{
CSymbolTest o;
o.TestSymbols("Hello world is ", 0);
return 0;
}
Compile x64/debug
Run app
Run Windbg (I have latest 10.0.17134.12) with admin rights
File->Kernel Debug...->Local (must be lokal kernel debugging enabled - step 1.)
Here are Windbg commands and output of my testing app (SymbolTest.exe)
lkd> !process 0 0 SymbolTest.exe
PROCESS ffffc68d3f536580
SessionId: 1 Cid: 1cc8 Peb: 2371da000 ParentCid: 2ba4
DirBase: 264500000 ObjectTable: ffffa30237269540 HandleCount: 43.
Image: SymbolTest.exe
lkd> .process /P ffffc68d3f536580
Implicit process is now ffffc68d`3f536580
lkd> .reload /user
Loading User Symbols
.......
lkd> !process ffffc68d3f536580 7
PROCESS ffffc68d3f536580
SessionId: 1 Cid: 1cc8 Peb: 2371da000 ParentCid: 2ba4
DirBase: 264500000 ObjectTable: ffffa30237269540 HandleCount: 43.
Image: SymbolTest.exe
VadRoot ffffc68d3dbc3890 Vads 22 Clone 0 Private 118. Modified 2. Locked 0.
DeviceMap ffffa3022c2669b0
Token ffffa3023bbdc060
ElapsedTime 00:00:51.609
UserTime 00:00:00.000
KernelTime 00:00:00.000
QuotaPoolUsage[PagedPool] 24064
QuotaPoolUsage[NonPagedPool] 3256
Working Set Sizes (now,min,max) (712, 50, 345) (2848KB, 200KB, 1380KB)
PeakWorkingSetSize 690
VirtualSize 4141 Mb
PeakVirtualSize 4148 Mb
PageFaultCount 777
MemoryPriority BACKGROUND
BasePriority 8
CommitCharge 147
Job ffffc68d3eb26600
THREAD ffffc68d3f161080 Cid 1cc8.23e0 Teb: 00000002371db000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Non-Alertable
ffffc68d3c3cb580 ProcessObject
Not impersonating
DeviceMap ffffa3022c2669b0
Owning Process ffffc68d3f536580 Image: SymbolTest.exe
Attached Process N/A Image: N/A
Wait Start TickCount 493631 Ticks: 3333 (0:00:00:52.078)
Context Switch Count 56 IdealProcessor: 2
UserTime 00:00:00.000
KernelTime 00:00:00.000
*** WARNING: Unable to verify checksum for c:\users\petr.pospisil\documents\visual studio 2015\Projects\SymbolTest\x64\Debug\SymbolTest.exe
Win32 Start Address SymbolTest!ILT+260(mainCRTStartup) (0x00007ff737361109)
Stack Init fffff60366c81c90 Current fffff60366c816c0
Base fffff60366c82000 Limit fffff60366c7c000 Call 0000000000000000
Priority 9 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
GetContextState failed, 0x80004001
Unable to get current machine context, HRESULT 0x80004001
Child-SP RetAddr : Args to Child : Call Site
fffff603`66c81700 fffff802`2e2fbd76 : fffff603`00000000 ffffc68d`3f161080 fffff603`66c818d0 fffff603`00000000 : nt!KiSwapContext+0x76
fffff603`66c81840 fffff802`2e2fb56b : ffffc68d`3ddfd0f0 00000000`00000000 00000000`00000000 fffff802`2e77194d : nt!KiSwapThread+0x2c6
fffff603`66c81910 fffff802`2e2fac8f : 00000000`000000b4 fffff802`00000000 00007ffe`71eb8800 ffffc68d`3f1611c0 : nt!KiCommitThreadWait+0x13b
fffff603`66c819b0 fffff802`2e7887bc : ffffc68d`3c3cb580 fffff802`00000006 00000000`00000001 00000000`00000000 : nt!KeWaitForSingleObject+0x1ff
fffff603`66c81a90 fffff802`2e455223 : ffffc68d`3f161080 00000000`00000000 00000000`00000000 ffffc68d`3c3cb580 : nt!NtWaitForSingleObject+0xfc
fffff603`66c81b00 00007ffe`74d8a014 : 00007ffe`71e8e0e2 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13 (TrapFrame # fffff603`66c81b00)
00000002`372ff918 00007ffe`71e8e0e2 : 00000000`00000000 00000000`00000000 00000000`00000000 00000002`372ffa50 : ntdll!NtWaitForSingleObject+0x14
00000002`372ff920 00007ffe`35edf728 : 00000000`000000ac 00000002`372ffa30 00000002`00000000 00000000`000000a4 : KERNELBASE!WaitForSingleObjectEx+0xa2
00000002`372ff9c0 00007ffe`35edef6b : 00000132`4df81d20 00000002`372ffa10 00000002`372ffb98 00000000`00000000 : ucrtbased!execute_command<char>+0x264 [minkernel\crts\ucrt\src\desktopcrt\exec\spawnv.cpp # 247]
00000002`372ffb00 00007ffe`35ee0969 : 00000000`00000000 00000132`4df81d20 00000000`00000000 00000000`00000000 : ucrtbased!common_spawnv<char>+0x233 [minkernel\crts\ucrt\src\desktopcrt\exec\spawnv.cpp # 328]
(Inline Function) --------`-------- : --------`-------- --------`-------- --------`-------- --------`-------- : ucrtbased!_spawnve+0x14 (Inline Function # 00007ffe`35ee0969) [minkernel\crts\ucrt\src\desktopcrt\exec\spawnv.cpp # 405]
(Inline Function) --------`-------- : --------`-------- --------`-------- --------`-------- --------`-------- : ucrtbased!__crt_char_traits<char>::tspawnve+0x14 (Inline Function # 00007ffe`35ee0969) [minkernel\crts\ucrt\inc\corecrt_internal_traits.h # 106]
00000002`372ffb60 00007ff7`3736175b : 00007ff7`37369ca4 00000000`00000000 00000000`00000000 00000002`372ffcb8 : ucrtbased!common_system<char>+0x101 [minkernel\crts\ucrt\src\desktopcrt\exec\system.cpp # 58]
00000002`372ffbd0 00007ff7`37369ca4 : 00000000`00000000 00000000`00000000 00000002`372ffcb8 cccccccc`cccccccc : SymbolTest!CSymbolTest::TestSymbols+0x5b [c:\users\petr.pospisil\documents\visual studio 2015\projects\symboltest\symboltest\symboltest.cpp # 14]
00000002`372ffbd8 00000000`00000000 : 00000000`00000000 00000002`372ffcb8 cccccccc`cccccccc cccccccc`cccccccc : SymbolTest!`string'
As you can see the stack ends with the SymbolTest!`string', which is wrong because windbg did not take SymbolTest!CSymbolTest::TestSymbols function param count into account to get next right stack function.
I tried almost any configuration in the C++ compiler and linker in VS2015 to find an workaround for this. There must be something because VS2013 pdb symbols work fine for me.
Any idea what compiler/VS option to use to fix this to workaround this?
Thx in advance.
I have a problem with the remote desktop connection on Windows 7 professional 64 bits (6.1 version 7601).
When I type the password of the server and click on connect button, it crashs.
I know that if the printers checkbox is checked, it causes this type of problems but I disable all local resources.
Here is the dump files :
https://www.dropbox.com/s/xvthjbldyr1ncl2/LocalDumps.zip?dl=0
Thanks.
The debug symbols are now online and I see in Windbg that the BLEtokenCredentialProvider.dll from CSR Harmony Wireless Software Stack causes your crash:
APPLICATION_VERIFIER_HEAPS_CORRUPTED_HEAP_BLOCK_EXCEPTION_RAISED_FOR_PROBING (c)
Exception raised while verifying the heap block.
This situation happens if we really cannot determine any particular
type of corruption for the block. For instance you will get this if
during a heap free operation you pass an address that points to a
non-accessible memory area.
This can also happen for double free situations if we do not find the
block among full page heap blocks and we probe it as a light page heap block.
Arguments:
Arg1: 0000000025701000, Heap handle used in the call.
Arg2: f0f0f0f0f0f0f0f0, Heap block involved in the operation.
Arg3: 0000000000000000, Size of the heap block.
Arg4: 00000000c0000005, Reserved.
DUMP_QUALIFIER: 400
CONTEXT: (.ecxr)
rax=000000000d65cee0 rbx=0000000000000001 rcx=000007fffff94000
rdx=000000000000fffd rsi=0000000000000000 rdi=000000000000000c
rip=000007fee9f9a668 rsp=000000001a20d9a0 rbp=0000000000000000
r8=000000001a203000 r9=0000000040010006 r10=0000000000000000
r11=000000001a20c528 r12=0000000000000000 r13=000000000000005d
r14=f0f0f0f0f0f0f0f0 r15=0000000000000000
iopl=0 nv up ei pl nz na pe nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
verifier!VerifierStopMessage+0x1f0:
000007fe`e9f9a668 cc int 3
Resetting default scope
FAULTING_IP:
verifier!VerifierStopMessage+1f0
000007fe`e9f9a668 cc int 3
EXCEPTION_RECORD: (.exr -1)
ExceptionAddress: 000007fee9f9a668 (verifier!VerifierStopMessage+0x00000000000001f0)
ExceptionCode: 80000003 (Break instruction exception)
ExceptionFlags: 00000000
NumberParameters: 1
Parameter[0]: 0000000000000000
DEFAULT_BUCKET_ID: BREAKPOINT_AVRF
PROCESS_NAME: mstsc.exe
ERROR_CODE: (NTSTATUS) 0x80000003 - {AUSNAHME} Haltepunkt Im Quellprogramm wurde ein Haltepunkt erreicht.
EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - Mindestens ein Argument ist ung ltig.
EXCEPTION_CODE_STR: 80000003
EXCEPTION_PARAMETER1: 0000000000000000
WATSON_BKT_PROCSTAMP: 524b5b3d
WATSON_BKT_PROCVER: 6.3.9600.16415
BUILD_VERSION_STRING: 6.1.7601.24000 (win7sp1_ldr.171231-1547)
THREAD_ATTRIBUTES:
OS_LOCALE: FRA
PROBLEM_CLASSES:
ID: [0n300]
Type: [#APPLICATION_FAULT_STRING]
Class: Primary
Scope: DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
BUCKET_ID
Name: Omit
Data: Add
String: [BREAKPOINT]
PID: [Unspecified]
TID: [Unspecified]
Frame: [0]
ID: [0n92]
Type: [AVRF]
Class: Addendum
Scope: DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
BUCKET_ID
Name: Add
Data: Omit
PID: [0x2fe0]
TID: [0x1c98]
Frame: [0] : verifier!VerifierStopMessage
BUGCHECK_STR: BREAKPOINT_AVRF
PRIMARY_PROBLEM_CLASS: BREAKPOINT
LAST_CONTROL_TRANSFER: from 000007fee9f994f2 to 000007fee9f9a668
STACK_TEXT:
00000000`1a20d9a0 000007fe`e9f994f2 : 00000000`1a20ea28 000007fe`e9f91988 000007fe`fd649270 000007fe`e9f91610 : verifier!VerifierStopMessage+0x1f0
00000000`1a20da50 000007fe`e9fb5863 : 000007fe`e9fb6604 00000000`1a20e9f0 000007fe`fd470000 00000000`1a20e9f0 : verifier!AVrfpDphReportCorruptedBlock+0x32a
00000000`1a20db10 00000000`76d67398 : 00000000`1a20dc70 00000000`1a20dc40 00000000`00000000 00000000`76d58468 : verifier!_chkstk+0xf3
00000000`1a20db40 00000000`76d7bf9d : 00000000`1a210000 00000000`1a20e9f0 00000000`1a20e9f0 000007fe`e9fde0fc : ntdll!_C_specific_handler+0x8c
00000000`1a20dbb0 00000000`76d504ca : 00000000`1a210000 00000000`01001002 000007fe`00001950 00000000`02991710 : ntdll!RtlpExecuteHandlerForException+0xd
00000000`1a20dbe0 00000000`76d7b63e : 00000000`1a20e7b0 00000000`1a20e2c0 00000000`00000000 00000000`00000000 : ntdll!RtlDispatchException+0x45a
00000000`1a20e2c0 000007fe`e9f96be1 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!KiUserExceptionDispatch+0x2e
00000000`1a20e9f0 000007fe`e9f98b87 : 00000000`25701000 f0f0f0f0`f0f0f0f0 00000000`00000000 00000000`00000000 : verifier!AVrfpDphFindBusyMemoryNoCheck+0x91
00000000`1a20ea50 00000000`76dd4a14 : 00000000`25700000 00000000`1a20f2e0 00000000`01001002 00000000`00000000 : verifier!AVrfDebugPageHeapSize+0x5b
00000000`1a20ea90 00000000`76d9427f : 00000000`25700000 00000000`00000000 00000000`25700000 f0f0f0f0`f0f0f0f0 : ntdll!RtlDebugSizeHeap+0x34
00000000`1a20eae0 000007fe`e9fb093f : 00000000`00000000 f0f0f0f0`f0f0f0f0 00000000`02a80000 000007fe`d87e9a63 : ntdll! ?? ::FNODOBFM::`string'+0xd26f
00000000`1a20eb30 000007fe`d87eed84 : 00000000`25700000 00000000`00000000 f0f0f0f0`f0f0f0f0 00000000`76c11a00 : verifier!AVrfpHeapFree+0x57
00000000`1a20ebc0 00000000`25700000 : 00000000`00000000 f0f0f0f0`f0f0f0f0 00000000`76c11a00 00000000`00000000 : BLEtokenCredentialProvider+0xed84
00000000`1a20ebc8 00000000`00000000 : f0f0f0f0`f0f0f0f0 00000000`76c11a00 00000000`00000000 000007fe`d87e6b46 : 0x25700000
SYMBOL_NAME: bletokencredentialprovider+ed84
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: BLEtokenCredentialProvider
IMAGE_NAME: BLEtokenCredentialProvider.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 4f686c3b
STACK_COMMAND: .ecxr ; kb
BUCKET_ID: X64_BREAKPOINT_AVRF_bletokencredentialprovider+ed84
FAILURE_EXCEPTION_CODE: 80000003
FAILURE_IMAGE_NAME: BLEtokenCredentialProvider.dll
BUCKET_ID_IMAGE_STR: BLEtokenCredentialProvider.dll
FAILURE_ID_HASH_STRING: um:breakpoint_avrf_80000003_bletokencredentialprovider.dll!unknown
FAILURE_ID_HASH: {2a6d23c0-cb20-73ec-3d92-f208d9f741cc}
Followup: MachineOwner
---------
0:020> lmvm BLEtokenCredentialProvider
Browse full module list
start end module name
000007fe`d87e0000 000007fe`d885d000 BLEtokenCredentialProvider T (no symbols)
Loaded symbol image file: BLEtokenCredentialProvider.dll
Image path: C:\Program Files\CSR\CSR Harmony Wireless Software Stack\BLEtokenCredentialProvider.dll
Image name: BLEtokenCredentialProvider.dll
Browse all global symbols functions data
Timestamp: Tue Mar 20 12:38:35 2012 (4F686C3B)
CheckSum: 000826D1
ImageSize: 0007D000
File version: 2.1.63.0
Product version: 2.1.63.0
Look for an update and if there is no one, remove this tool.
The section object from a 3thParty vendor is named rpsPdf10.mutex and it's intended use is to mimic a semaphore by writing a Boolean flag to it.
Using LiveKd and with a lot of help from SO, I've issued following command's trying to get detailed info of this Section object.
0: kd>!process 0 0 3thParty.exe
...
PROCESS fffffa800ea80060
SessionId: 0 Cid: 0a00 Peb: fffdf000 ParentCid: 014c
DirBase: 99349000 ObjectTable: fffff8a004448bf0 HandleCount: 338.
Image: 3thParty.exe
...
0: kd> !handle 0 7 fffffa800ea80060
...
08 fffff8a012e26710 Section rpsPdf10.mutex
...
0: kd> !object fffff8a012e26710
Object: fffff8a012e26710 Type: (fffffa800cd7cea0) Section
ObjectHeader: fffff8a012e266e0 (new version)
HandleCount: 38 PointerCount: 39
Directory Object: fffff8a00a980080 Name: rpsPdf10.mutex
0: kd> dt -r1 nt!_SECTION_OBJECT 0xfffff8a012e26710
+0x000 StartingVa : 0x00000022`00000100 Void
+0x008 EndingVa : 0x00000000`0008dfb0 Void
+0x010 Parent : 0xfffffa80`c0000001 Void
+0x018 LeftChild : (null)
+0x020 RightChild : 0xfffffa80`00000034 Void
+0x028 Segment : 0xfffff8a0`102d7820 _SEGMENT_OBJECT
+0x000 BaseAddress : 0xfffffa80`0fbed900 Void
+0x008 TotalNumberOfPtes : 1
+0x010 SizeOfSegment : _LARGE_INTEGER 0x1
+0x018 NonExtendedPtes : 0x1000
+0x01c ImageCommitment : 0
+0x020 ControlArea : (null)
+0x028 Subsection : (null)
+0x030 MmSectionFlags : 0xfffffa80`10987b10 _MMSECTION_FLAGS
+0x038 MmSubSectionFlags : 0x00000000`03400000 _MMSUBSECTION_FLAGS
All this looks correct to me but how can I find the content the section?
points to note as in earlier answer machine is 32 bit and os is win7 and windbg version is insider preview 16278 commands are arch agnostic and pointer arithmetic if any are arch dependent
and the walk through is on live binary not in a dump as there is a very fair chance that the page might have been paged out in the dump and demo might be inconclusive i might add to this answer later
getting contents form section is kinda convoluted
(there are several types of Section like
1) ALPC section(com objects )
2) File backed Section
3) PageFileBacked Section etc
the walkthrough below is for a pagefile backed section (most common type)
Assuming you compiled and executed the code below
the exe will create a SectionObject in global namespace
and the contents will be backed by PagingFile and will be waiting for a
keypress
#include <windows.h>
#include <stdio.h>
#define bsize 256
int main(){
char szMsg[]={"Message from blabb to lieven from Stack Overflow."};
int ret = NULL;
HANDLE hMap = CreateFileMapping((HANDLE)-1,NULL,4,0,bsize,"Global\\MyMap");
if(hMap){
PCHAR buff = (PCHAR) MapViewOfFile(hMap,0xf001f,0,0,bsize);
if(buff){
CopyMemory(buff, szMsg, sizeof(szMsg));
ret = getchar();
UnmapViewOfFile(buff);
}
CloseHandle(hMap);
}
return ret;
}
assuming the process is waiting for a keypress launch livekd or setup a live kernel debugging connection if it is running in a remote machine / vm
C:>livekd -k "c:\Program Files\Windows Kits\10\Debuggers\x86\cdb.exe"
LiveKd v5.62 - Execute kd/windbg on a live system
Launching c:\Program Files\Windows Kits\10\Debuggers\x86\cdb.exe:
Microsoft (R) Windows Debugger Version 10.0.16278.1000 X86
get the the _EPROCESS and set context
kd> !process 0 0 secobj.exe
PROCESS 8605ab28 SessionId: 1 Cid: 0fbc Peb: 7ffd9000 ParentCid: 0af4
DirBase: 7e2712e0 ObjectTable: c288ba00 HandleCount: 9.
Image: secobj.exe
kd> .process /p /r 8605ab28
Implicit process is now 8605ab28
kd> ? #$proc
Evaluate expression: -2046448856 = 8605ab28
kd> ?? (char *)#$proc->ImageFileName
char * 0x8605ac94
"secobj.exe"
look for a handle of type Section in the current process notice since our
section is named in global namespace windbg deciphers that for us
kd> !handle 0 3 #$proc Section
Searching for handles of type Section
PROCESS 8605ab28 SessionId: 1 Cid: 0fbc Peb: 7ffd9000 ParentCid: 0af4
DirBase: 7e2712e0 ObjectTable: c288ba00 HandleCount: 9.
Image: secobj.exe
Handle table at c288ba00 with 9 entries in use
0024: Object: c238e9c8 GrantedAccess: 000f0007 Entry: c37b7048
Object: c238e9c8 Type: (84ec6040) Section
ObjectHeader: c238e9b0 (new version)
HandleCount: 1 PointerCount: 2
Directory Object: 98a0f170 Name: MyMap
dumping the SectionObject
kd> dt nt!_SECTION_OBJECT c238e9c8
+0x000 StartingVa : 0xc227e2c8 Void
+0x004 EndingVa : 0x00d3db6c Void
+0x008 Parent : 0xb0d3db20 Void
+0x00c LeftChild : (null)
+0x010 RightChild : 0x00000003 Void
+0x014 Segment : 0xc36aba20 _SEGMENT_OBJECT
kd> $$ Notice the Last Segment member it is not SEGMENT_OBJECT but
nt!_segment or actually a pointer to ControlArea for this Section
kd> dt nt!_SEGMENT 0xc36aba20
+0x000 ControlArea : 0x85182d08 _CONTROL_AREA
+0x004 TotalNumberOfPtes : 1
+0x008 SegmentFlags : _SEGMENT_FLAGS
+0x00c NumberOfCommittedPages : 1
+0x010 SizeOfSegment : 0x1000
+0x018 ExtendInfo : (null)
+0x018 BasedAddress : (null)
+0x01c SegmentLock : _EX_PUSH_LOCK
+0x020 u1 : <unnamed-tag>
+0x024 u2 : <unnamed-tag>
+0x028 PrototypePte : 0xc36aba50 _MMPTE
+0x030 ThePtes : [1] _MMPTE
kd> $$ you can expand the union u2 and dump the FirstMappedVa of the union to look at the contents of this Section
kd> dt nt!_SEGMENT u2.FirstMappedVa 0xc36aba20
+0x024 u2 :
+0x000 FirstMappedVa : 0x000e0000 Void
dumping the contents
kd> da 0xe0000
000e0000 "Message from blabb to lieven fro"
000e0020 "m Stack Overflow."
kd>
or do !ca to get the FirstMappedVa which points to the first page
if the contents are greater than one page boundary fetching them is
kinda tedious as they may have been paged out and would need an execution
operation and thus a page fault handled to get them into view
kd> !ca poi(0xc36aba20)
ControlArea # 85182d08
Segment c36aba20 Flink 00000000 Blink 00000000
Section Ref 1 Pfn Ref 0 Mapped Views 1
User Ref 2 WaitForDel 0 Flush Count 0
File Object 00000000 ModWriteCount 0 System Views 0
WritableRefs 0
Flags (2000) Commit
Pagefile-backed section
Segment # c36aba20
ControlArea 85182d08 ExtendInfo 00000000
Total Ptes 1
Segment Size 1000 Committed 1
CreatingProcess 8605ab28 FirstMappedVa e0000 <-------------
ProtoPtes c36aba50
Flags (80000) ProtectionMask
Subsection 1 # 85182d58
ControlArea 85182d08 Starting Sector 0 Number Of Sectors 0
Base Pte c36aba50 Ptes In Subsect 1 Unused Ptes 0
Flags 8 Sector Offset 0 Protection 4
kd>
Adding another answer to show another pagefile backed section contents
where you may need to go to the Creating Process and look at its virtual address instead of the process that is current
you may know shell keeps some global counters in a shared section
0104: Object: c2255450 GrantedAccess: 00000006 Entry: c5d9b208
Object: c2255450 Type: (84ec6040) Section
ObjectHeader: c2255438 (new version)
HandleCount: 6 PointerCount: 7
Directory Object: 9d662520 Name: windows_shell_global_counters
code to retrieve them
#include <stdio.h>
#include <windows.h>
#include <stdlib.h>
#include <shlwapi.h>
#pragma comment(lib,"shlwapi.lib")
//add all items in the ... place holder in arrays below
PCHAR enuname[] = { "GLOBALCOUNTER_SEARCHMANAGER",...};
int foo [] = { GLOBALCOUNTER_SEARCHMANAGER,...};
void main (void) {
long cval = NULL;
for(int i =0; i < _countof(foo) ;i++) {
cval = SHGlobalCounterGetValue((SHGLOBALCOUNTER)foo[i]);
printf ("%65s = %0x\n" ,enuname[i], cval);
}
}
compiling and executing this you may get a result like this (keep in mind the counters are volatile so you may need to be quick enough for comparing)
C:\secobj\shglob>shglob.exe | head -n 13
GLOBALCOUNTER_SEARCHMANAGER = 0
GLOBALCOUNTER_SEARCHOPTIONS = 0
GLOBALCOUNTER_FOLDERSETTINGSCHANGE = 0
GLOBALCOUNTER_RATINGS = 0
GLOBALCOUNTER_APPROVEDSITES = 0
GLOBALCOUNTER_RESTRICTIONS = 5
GLOBALCOUNTER_SHELLSETTINGSCHANGED = 2
GLOBALCOUNTER_SYSTEMPIDLCHANGE = 0
GLOBALCOUNTER_OVERLAYMANAGER = 0
GLOBALCOUNTER_QUERYASSOCIATIONS = 0
GLOBALCOUNTER_IESESSIONS = 0
GLOBALCOUNTER_IEONLY_SESSIONS = 0
GLOBALCOUNTER_APPLICATION_DESTINATIONS = 83
looking at current process
kd> ? #$proc
Evaluate expression: -2033240552 = 86cf3618
kd> ?? (char *)#$proc->ImageFileName
char * 0x86cf3784
"cdb.exe"
kd> !handle 0 3 #$proc Section
0164: Object: c2255450 GrantedAccess: 00000006 Entry: c3df32c8
Object: c2255450 Type: (84ec6040) Section
ObjectHeader: c2255438 (new version)
HandleCount: 11 PointerCount: 12
Directory Object: 9d662520 Name: windows_shell_global_counters
kd> dc ##c++(((nt!_segment *)((nt!_section_object *) 0xc2255450)->Segment)->u2.FirstMappedVa)
00290000 ???????? ???????? ???????? ???????? ????????????????
00290010 ???????? ???????? ???????? ???????? ????????????????
00290020 ???????? ???????? ???????? ???????? ????????????????
00290030 ???????? ???????? ???????? ???????? ????????????????
00290040 ???????? ???????? ???????? ???????? ????????????????
00290050 ???????? ???????? ???????? ???????? ????????????????
00290060 ???????? ???????? ???????? ???????? ????????????????
00290070 ???????? ???????? ???????? ???????? ????????????????
it may be that this current process obtained a handle with OpenFileMapping but didn't map it yet or the page is paged out (livekd can't use pagein ) whatever we cant seem to view this section content
lets look who created this shared section
kd> ? ##c++(((nt!_segment *)((nt!_section_object *) 0xc2255450)->Segment)->u1.CreatingProcess)
Evaluate expression: -2050635184 = 85c5ca50
kd> !process 85c5ca50 0
PROCESS 85c5ca50 SessionId: 1 Cid: 0af4 Peb: 7ffd9000 ParentCid: 0704
DirBase: 7e271420 ObjectTable: c5d873c8 HandleCount: 888.
Image: explorer.exe
it looks logical explorer.exe seemed to have created this shared section
lets check by dumping 0n13 dwords from the creating process virtual address
kd> .process /p /r 85c5ca50
kd> dc ##c++(((nt!_segment *)((nt!_section_object *) 0xc2255450)->Segment)->u2.FirstMappedVa) l0n13
00290000 00000000 00000000 00000000 00000000 ................
00290010 00000000 00000005 00000002 00000000 ................
00290020 00000000 00000000 00000000 00000000 ................
00290030 00000083 ....
I'm debugging a process which is like frozen:
I suspect the root cause is the thread below THREAD 877f4030 Cid 0568.0fb8 that is stuck on the user-mode call to GetOverlappedResult.
I have opened the dump with kd.exe.
Namely, I'm interested into knowing more about the NotificationEvent which obviously is never releasing our thread.
In the thread info we have:
879f6fdc NotificationEvent
In what type should I cast address 879f6fdc ? or in which structure field should I search for it, so as to understand, or have a clue to what is blocking the situation ?
As far as the Thread Infos goes, this thread currently does not list any IRP that would be in undesired or unfinished state.
Below entire Thread Information for the corresponding thread:
THREAD 877f4030 Cid 0568.0fb8 Teb: 7ff3d000 Win32Thread: 00000000 WAIT: (UserRequest) UserMode Non-Alertable
879f6fdc NotificationEvent
Not impersonating
DeviceMap 89809fc8
Owning Process 87950030 Image: OurProduct.exe
Attached Process N/A Image: N/A
Wait Start TickCount 1472232 Ticks: 5394 (0:00:01:24.146)
Context Switch Count 2791788 IdealProcessor: 0
UserTime 00:00:06.848
KernelTime 00:00:09.890
Win32 Start Address MSVCR120!_threadstartex (0x721fbfb4)
Stack Init 8c761fd0 Current 8c761bc8 Base 8c762000 Limit 8c75f000 Call 0
Priority 8 BasePriority 8 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5
Kernel stack not resident.
ChildEBP RetAddr Args to Child
8c761be0 824cfced 877f4030 00000000 8ab36120 nt!KiSwapContext+0x26 (FPO: [Uses EBP] [0,0,4])
8c761c18 824ceb4b 877f40f0 877f4030 879f6fdc nt!KiSwapThread+0x266
8c761c40 824c856f 877f4030 877f40f0 00000000 nt!KiCommitThreadWait+0x1df
8c761cb8 8267ae07 879f6fdc 00000006 826bca01 nt!KeWaitForSingleObject+0x393
8c761d20 8248f8a6 00001018 00000000 00000000 nt!NtWaitForSingleObject+0xc6
8c761d20 774f7094 00001018 00000000 00000000 nt!KiSystemServicePostCall (FPO: [0,3] TrapFrame # 8c761d34)
09f9f61c 774f6a24 758b179c 00001018 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])
09f9f620 758b179c 00001018 00000000 00000000 ntdll!NtWaitForSingleObject+0xc (FPO: [3,0,0])
09f9f68c 758b7841 00001018 ffffffff 00000000 KERNELBASE!WaitForSingleObjectEx+0x98 (FPO: [Non-Fpo])
09f9f6a0 758cb9e1 00001018 ffffffff 064f3d10 KERNELBASE!WaitForSingleObject+0x12 (FPO: [Non-Fpo])
09f9f6b8 745be159 00001018 0639ee0c 09f9f6ec KERNELBASE!GetOverlappedResult+0x57 (FPO: [Non-Fpo])
What is the correct way to proceed and know which event or synchronisation mechanism is faulting ?
some commands on the NotificationEvent address:
0: kd> !object 879f6fdc
879f6fdc: Not a valid object (ObjectType invalid)
0: kd> dt nt!_KEVENT 879f6fdc
+0x000 Header : _DISPATCHER_HEADER
and then:
0: kd> dt nt!_DISPATCHER_HEADER 879f6fdc
+0x000 Type : 0 ''
+0x001 TimerControlFlags : 0 ''
+0x001 Absolute : 0y0
+0x001 Coalescable : 0y0
+0x001 KeepShifting : 0y0
+0x001 EncodedTolerableDelay : 0y00000 (0)
+0x001 Abandoned : 0 ''
+0x001 Signalling : 0 ''
+0x002 ThreadControlFlags : 0x4 ''
+0x002 CpuThrottled : 0y0
+0x002 CycleProfiling : 0y0
+0x002 CounterProfiling : 0y1
+0x002 Reserved : 0y00000 (0)
+0x002 Hand : 0x4 ''
+0x002 Size : 0x4 ''
+0x003 TimerMiscFlags : 0 ''
+0x003 Index : 0y0
+0x003 Processor : 0y00000 (0)
+0x003 Inserted : 0y0
+0x003 Expired : 0y0
+0x003 DebugActive : 0 ''
+0x003 ActiveDR7 : 0y0
+0x003 Instrumented : 0y0
+0x003 Reserved2 : 0y0000
+0x003 UmsScheduled : 0y0
+0x003 UmsPrimary : 0y0
+0x003 DpcActive : 0 ''
+0x000 Lock : 0n262144
+0x004 SignalState : 0n0
+0x008 WaitListHead : _LIST_ENTRY [ 0x877f40f0 - 0x877f40f0 ]
from a former investigation I remember that if +0x003 DpcActive was 1, it would mean we'd be waiting for some hardware operation to put it to 0. But in this case it is 0.
So right now, I just don't know what this NotificationEvent is waiting for.
Any idea ?
Events do not wait, Threads do. NotificationEvents are signaled by whoever would perform the operation and then notify the the waiters about the completion of the operation. In other words your stack is an example of a Async IO where we pass an overlapped structure with an hEvent set. reference https://msdn.microsoft.com/en-us/library/windows/desktop/ms684342(v=vs.85).aspx
You should be inspecting the source which has scheduled this IO or the type of IO we are waiting for instead of dumping out the event. The event will be signaled when the operation is completed.
I'm hoping someone here can help.
I have a new Windows 10 machine (all parts by EVGA).
I get random BSOD, so I've grabbed a mini dump, installed the SDK and looked into it. I just don't understand what it is reporting.
Can someone point me in the direction of a guide, or decode this mini dump.
Note : Each dump looks very similar. e.g. almost the same report from 'irp'
Here is the dump....
Microsoft (R) Windows Debugger Version 10.0.10586.567 X86
Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [C:\Windows\Minidump\033016-4718-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available
Symbol search path is: srv*
Executable search path is:
Windows 10 Kernel Version 10586 MP (8 procs) Free x64
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 10586.162.amd64fre.th2_release_sec.160223-1728
Machine Name:
Kernel base = 0xfffff8018d674000 PsLoadedModuleList = 0xfffff8018d952cd0
Debug session time: Wed Mar 30 18:15:33.639 2016 (UTC + 1:00)
System Uptime: 0 days 2:47:26.264
Loading Kernel Symbols
.
Press ctrl-c (cdb, kd, ntsd) or ctrl-break (windbg) to abort symbol loads that take too long.
Run !sym noisy before .reload to track down problems loading symbols.
..............................................................
................................................................
................................
Loading User Symbols
Loading unloaded module list
.............
*
Bugcheck Analysis *
*
Use !analyze -v to get detailed debugging information.
BugCheck 9F, {3, ffffe000935ea880, fffff8018f25a890, ffffe00092718bd0}
Probably caused by : ACPI.sys
Followup: MachineOwner
0: kd> !analyze -v
*
Bugcheck Analysis *
*
DRIVER_POWER_STATE_FAILURE (9f)
A driver has failed to complete a power IRP within a specific time.
Arguments:
Arg1: 0000000000000003, A device object has been blocking an Irp for too long a time
Arg2: ffffe000935ea880, Physical Device Object of the stack
Arg3: fffff8018f25a890, nt!TRIAGE_9F_POWER on Win7 and higher, otherwise the Functional Device Object of the stack
Arg4: ffffe00092718bd0, The blocked IRP
Debugging Details:
DUMP_CLASS: 1
DUMP_QUALIFIER: 400
BUILD_VERSION_STRING: 10586.162.amd64fre.th2_release_sec.160223-1728
SYSTEM_MANUFACTURER: EVGA INTERNATIONAL CO.,LTD
SYSTEM_PRODUCT_NAME: Default string
SYSTEM_SKU: Default string
SYSTEM_VERSION: Default string
BIOS_VENDOR: American Megatrends Inc.
BIOS_VERSION: 1.07
BIOS_DATE: 01/04/2016
BASEBOARD_MANUFACTURER: EVGA INTERNATIONAL CO.,LTD
BASEBOARD_PRODUCT: 111-SS-E172
BASEBOARD_VERSION: 1.0
DUMP_TYPE: 2
DUMP_FILE_ATTRIBUTES: 0x8
Kernel Generated Triage Dump
BUGCHECK_P1: 3
BUGCHECK_P2: ffffe000935ea880
BUGCHECK_P3: fffff8018f25a890
BUGCHECK_P4: ffffe00092718bd0
DRVPOWERSTATE_SUBCODE: 3
IMAGE_NAME: ACPI.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 56cbf9c9
MODULE_NAME: ACPI
FAULTING_MODULE: fffff800d5de0000 ACPI
CPU_COUNT: 8
CPU_MHZ: d50
CPU_VENDOR: GenuineIntel
CPU_FAMILY: 6
CPU_MODEL: 5e
CPU_STEPPING: 3
CPU_MICROCODE: 6,5e,3,0 (F,M,S,R) SIG: 33'00000000 (cache) 33'00000000 (init)
CUSTOMER_CRASH_COUNT: 1
DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT
BUGCHECK_STR: 0x9F
PROCESS_NAME: System
CURRENT_IRQL: 2
ANALYSIS_SESSION_HOST: Q-PC
ANALYSIS_SESSION_TIME: 03-30-2016 20:04:47.0460
ANALYSIS_VERSION: 10.0.10586.567 x86fre
STACK_TEXT:
fffff8018f25a858 fffff8018d854e42 : 000000000000009f 0000000000000003 ffffe000935ea880 fffff8018f25a890 : nt!KeBugCheckEx
fffff8018f25a860 fffff8018d854d62 : ffffe00096133010 fffff8018f252070 0000000000000000 fffff8018d73e0a6 : nt!PopIrpWatchdogBugcheck+0xde
fffff8018f25a8c0 fffff8018d6e22c6 : ffffe00096133048 fffff8018f25aa10 0000000000000001 0000000000000002 : nt!PopIrpWatchdog+0x32
fffff8018f25a910 fffff8018d7b951a : 0000000000000000 fffff8018d991180 fffff8018da07740 ffffe00096723800 : nt!KiRetireDpcList+0x5f6
fffff8018f25ab60 0000000000000000 : fffff8018f25b000 fffff8018f254000 0000000000000000 0000000000000000 : nt!KiIdleLoop+0x5a
STACK_COMMAND: kb
THREAD_SHA1_HASH_MOD_FUNC: 81a7ba75a791115b4f55c8910c64a260d525502e
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 936d5c51c0ad2157bf4c85af575dd55cea2c0947
THREAD_SHA1_HASH_MOD: f08ac56120cad14894587db086f77ce277bfae84
FOLLOWUP_NAME: MachineOwner
IMAGE_VERSION: 10.0.10586.122
FAILURE_BUCKET_ID: 0x9F_3_POWER_DOWN_i8042prt_IMAGE_ACPI.sys
BUCKET_ID: 0x9F_3_POWER_DOWN_i8042prt_IMAGE_ACPI.sys
PRIMARY_PROBLEM_CLASS: 0x9F_3_POWER_DOWN_i8042prt_IMAGE_ACPI.sys
TARGET_TIME: 2016-03-30T17:15:33.000Z
OSBUILD: 10586
OSSERVICEPACK: 0
SERVICEPACK_NUMBER: 0
OS_REVISION: 0
SUITE_MASK: 272
PRODUCT_TYPE: 1
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
OSEDITION: Windows 10 WinNt TerminalServer SingleUserTS
OS_LOCALE:
USER_LCID: 0
OSBUILD_TIMESTAMP: 2016-02-24 05:48:00
BUILDDATESTAMP_STR: 160223-1728
BUILDLAB_STR: th2_release_sec
BUILDOSVER_STR: 10.0.10586.162.amd64fre.th2_release_sec.160223-1728
ANALYSIS_SESSION_ELAPSED_TIME: 3d7
ANALYSIS_SOURCE: KM
FAILURE_ID_HASH_STRING: km:0x9f_3_power_down_i8042prt_image_acpi.sys
FAILURE_ID_HASH: {22a3ff34-49ca-8d37-715b-ae023b6cc9fb}
Followup: MachineOwner
0: kd> !irp ffffe00092718bd0
Irp is active with 8 stacks 6 is current (= 0xffffe00092718e08)
No Mdl: No System Buffer: Thread 00000000: Irp stack trace. Pending has been returned
cmd flg cl Device File Completion-Context
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[IRP_MJ_POWER(16), IRP_MN_WAIT_WAKE(0)]
0 0 ffffe000935ea880 00000000 fffff800d6a81ec0-00000000
\Driver\ACPI i8042prt!I8xPowerUpToD0Complete
Args: 00000000 00000000 00000000 00000002
[IRP_MJ_POWER(16), IRP_MN_SET_POWER(2)]
0 e1 ffffe00093f936f0 00000000 fffff800d6ab1060-00000000 Success Error Cancel pending
\Driver\i8042prt kbdclass!KeyboardClassPowerComplete
Args: 00051100 00000001 00000001 00000002
[IRP_MJ_POWER(16), IRP_MN_SET_POWER(2)]
0 e1 ffffe00093dc95f0 00000000 fffff8018d7840b8-ffffe00096133010 Success Error Cancel pending
\Driver\kbdclass nt!PopRequestCompletion
Args: 00051100 00000001 00000001 00000002
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-ffffe00096133010
Args: 00000000 00000000 00000000 00000000
I'm also adding a BlueScreen screen shot, incase that helps.
Now adding output from some extra commands after Martins comments...
0: kd> !devstack ffffe000935ea880
!DevObj !DrvObj !DevExt ObjectName
ffffe00093dc95f0 \Driver\kbdclass ffffe00093dc9740 InfoMask field not found for _OBJECT_HEADER at ffffe00093dc95c0
ffffe00093f936f0 \Driver\i8042prt ffffe00093f93840 InfoMask field not found for _OBJECT_HEADER at ffffe00093f936c0
> ffffe000935ea880 \Driver\ACPI ffffe000923fa8d0 Cannot read info offset from nt!ObpInfoMaskToOffset
!DevNode ffffe000935d6af0 :
DeviceInst is "ACPI\PNP0303\0"
ServiceName is "i8042prt"
!process 0 7
**** NT ACTIVE PROCESS DUMP ****
GetPointerFromAddress: unable to read from fffff8018d9f3200
Error in reading nt!_EPROCESS at 0000000000000000
0: kd> !poaction
PopAction: fffff8018d94efe0
State..........: 0 - Idle
Updates........: 0
Action.........: None
Lightest State.: Unspecified
Flags..........: 10000003 QueryApps|UIAllowed
Irp minor......: ??
System State...: Unspecified
Hiber Context..: 0000000000000000
Allocated power irps (PopIrpList - fffff8018d94f4f0)
IRP: ffffe00092718bd0 (set/D0,), PDO: ffffe000935ea880, CURRENT: ffffe00093f936f0
IRP: ffffe000971aa990
Irp worker threads (PopIrpThreadList - fffff8018d94e100)
THREAD: ffffe00091515040 (static)
THREAD: ffffe00091501800 (static)
Error resolving nt!_POP_CURRENT_BROADCAST...
Summary: Error was caused by my 10 year old Razor mouse with Windows 10.
The driver when entering power save state was freaking out and causing the blue screen.
I purchased a new mouse, removed the driver & 2 months in no more BSOD.
I usually use BlueScreenView by Nirsoft. It will get you a list of last BSOD and will show a nice view of the components. "Normally" the first mentioned component could be the reason.
Not sure, if you are looking for a solution on a specific problem or the minidump usage in general.
Some driver got problems with power state change. Make sure, you have the current Drivers installed.