Windows update 1903 causing file system driver to crash - windows

Question

I have a serious problem with a windows file system KMDF driver. the problem occurred after Windows 10 ver 1903 update (may latest update).
the driver was running smoothly before the update at any giving windows 10 versions.
When the driver start running the system CARSH (Blue Screen) with "WDF_VIOLATION" Error.
I opened the system dump file with the "Visual Studio windbg" tool, and i found this Error log:
WDF_VIOLATION (10d)
The Kernel-Mode Driver Framework was notified that Windows detected an error
in a framework-based driver. In general, the dump file will yield additional
information about the driver that caused this bug check.
Arguments:
Arg1: 0000000000000005, A framework object handle of the incorrect type was passed to
a framework object method.
Arg2: 0000000000000000, The handle value passed in.
Arg3: 0000000000001023, Reserved.
Arg4: ffffd808fc533e00, Reserved.
Debugging Details:
------------------
KEY_VALUES_STRING: 1
Key : Analysis.CPU.Sec
Value: 7
Key : Analysis.Elapsed.Sec
Value: 23
Key : Analysis.Memory.CommitPeak.Mb
Value: 72
PROCESSES_ANALYSIS: 1
SERVICE_ANALYSIS: 1
STACKHASH_ANALYSIS: 1
TIMELINE_ANALYSIS: 1
DUMP_CLASS: 1
DUMP_QUALIFIER: 401
BUILD_VERSION_STRING: 18362.1.amd64fre.19h1_release.190318-1202
SYSTEM_MANUFACTURER: LENOVO
SYSTEM_PRODUCT_NAME: 20DCA020IV
SYSTEM_SKU: LENOVO_MT_20DC_BU_Think_FM_ThinkPad E450
SYSTEM_VERSION: ThinkPad E450
BIOS_VENDOR: LENOVO
BIOS_VERSION: J5ET63WW (1.34 )
BIOS_DATE: 09/26/2018
BASEBOARD_MANUFACTURER: LENOVO
BASEBOARD_PRODUCT: 20DCA020IV
BASEBOARD_VERSION: Not Defined
DUMP_TYPE: 1
BUGCHECK_P1: 5
BUGCHECK_P2: 0
BUGCHECK_P3: 1023
BUGCHECK_P4: ffffd808fc533e00
BUGCHECK_STR: 0x10D_5
CPU_COUNT: 4
CPU_MHZ: 893
CPU_VENDOR: GenuineIntel
CPU_FAMILY: 6
CPU_MODEL: 3d
CPU_STEPPING: 4
CPU_MICROCODE: 6,3d,4,0 (F,M,S,R) SIG: 2B'00000000 (cache) 2B'00000000 (init)
ADDITIONAL_DEBUG_TEXT: 
You can run '.symfix; .reload' to try to fix the symbol path and load symbols.
FAULTING_MODULE: fffff8007c800000 nt
DEBUG_FLR_IMAGE_TIMESTAMP:  5cf4fafd
BUGCHECK_STR:  0x10D_5
DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT
CURRENT_IRQL:  0
ANALYSIS_VERSION: 6.3.9600.17336 (debuggers(dbg).150226-1500) amd64fre
LAST_CONTROL_TRANSFER:  from fffff8007f58b828 to fffff8007c9bc810
STACK_TEXT: 
ffff9b0c`b3d0ee98 fffff800`7f58b828 : 00000000`0000010d 00000000`00000005 00000000`00000000 00000000`00001023 : nt!KeBugCheckEx
ffff9b0c`b3d0eea0 fffff800`7f559e27 : 0000001e`571ef000 00000000`00000000 00000000`00000001 00000000`00000001 : Wdf01000+0x5b828
ffff9b0c`b3d0eee0 fffff800`885f2bfc : 00000000`00000000 ffff9e0e`04604380 ffff9e0d`fec5f8b0 ffff9b0c`b3d0f280 : Wdf01000+0x29e27
ffff9b0c`b3d0ef20 fffff800`885f34c6 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : Cyber20UNCProvider!CA_EstablishConnection+0x40 [c:\_dev\workspace\agent-supdriver\supdriver_x64\sentineluncprovider\commagent.c # 99]
ffff9b0c`b3d0f170 fffff800`7f31cd8a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : Cyber20UNCProvider!CS_SUPMessage+0x66 [c:\_dev\workspace\agent-supdriver\supdriver_x64\sentineluncprovider\commservice.c # 502]
ffff9b0c`b3d0f630 fffff800`7f34bf18 : ffff9b0c`b3d0f740 00000000`00000001 ffff9e0e`02efaaa0 00000000`00000000 : FLTMGR!FltGetIoPriorityHintFromCallbackData+0x15a
ffff9b0c`b3d0f690 fffff800`7f34bfd6 : ffff9e0e`02efab70 0000001e`575ff210 00000000`00000000 00000000`00000000 : FLTMGR!FltRemoveOpenReparseEntry+0x418
ffff9b0c`b3d0f6f0 fffff800`7f313f4f : ffff9e0d`faf716b0 00000000`00000002 00000000`00000000 fffff800`7ce1da15 : FLTMGR!FltRemoveOpenReparseEntry+0x4d6
ffff9b0c`b3d0f760 fffff800`7c827da9 : ffff9e0e`02efaaa0 00000000`00000000 00000000`0008801b 00000000`00000001 : FLTMGR!FltDecodeParameters+0x11ef
ffff9b0c`b3d0f7c0 fffff800`7ce15dd5 : ffff9e0e`02efaaa0 00000000`00000000 00000000`00000000 ffff9e0e`04cccbf0 : nt!IofCallDriver+0x59
ffff9b0c`b3d0f800 fffff800`7ce1572a : 00000000`00000000 00000000`0008801b 00000000`00000000 ffff9b0c`b3d0fb40 : nt!NtDeviceIoControlFile+0xce5
ffff9b0c`b3d0f8a0 fffff800`7ce15146 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!NtDeviceIoControlFile+0x63a
ffff9b0c`b3d0f9e0 fffff800`7c9cde98 : 00000000`00000000 ffff9b0c`b3d0fb40 ffff9e0e`0404a330 fffff800`7ce2288d : nt!NtDeviceIoControlFile+0x56
ffff9b0c`b3d0fa50 00007ffe`e453c144 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!setjmpex+0x7af8
0000001e`575febe8 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x00007ffe`e453c144
STACK_COMMAND:  kb
FOLLOWUP_IP:
Cyber20UNCProvider!CA_EstablishConnection+40 [c:\_dev\workspace\agent-supdriver\supdriver_x64\sentineluncprovider\commagent.c # 99]
fffff800`885f2bfc 48833d4c45000000 cmp     qword ptr [Cyber20UNCProvider!CA_agentFileObject (fffff800`885f7150)],0
FAULTING_SOURCE_LINE:  c:\_dev\workspace\agent-supdriver\supdriver_x64\sentineluncprovider\commagent.c
FAULTING_SOURCE_FILE:  c:\_dev\workspace\agent-supdriver\supdriver_x64\sentineluncprovider\commagent.c
FAULTING_SOURCE_LINE_NUMBER:  99
FAULTING_SOURCE_CODE: 
    95:   NTSTATUS                    status;
    96:
    97:   WdfWaitLockAcquire(CA_agentFileObjectLock, NULL);
    98:
>   99: if (CA_agentFileObject != NULL)
   100: {
   101:              LoggerLog(LOGGER_LS_WARN, L"CommAgent.c", L"CA_EstablishConnection", L"Communication with the Agent is already established");
   102:
   103:              WdfWaitLockRelease(CA_agentFileObjectLock);
   104:
SYMBOL_STACK_INDEX:  3
SYMBOL_NAME:  Cyber20UNCProvider!CA_EstablishConnection+40
FOLLOWUP_NAME:  MachineOwner
MODULE_NAME: Cyber20UNCProvider
IMAGE_NAME:  Cyber20UNCProvider.sys
BUCKET_ID:  WRONG_SYMBOLS
FAILURE_BUCKET_ID:  WRONG_SYMBOLS
ANALYSIS_SOURCE:  KM
FAILURE_ID_HASH_STRING:  km:wrong_symbols
FAILURE_ID_HASH:  {70b057e8-2462-896f-28e7-ac72d4d365f8}
Followup: MachineOwner
The windbg marking line 99 as the crashing command, i did checked it and i found that the line above 99 is causing the problem:
"WdfWaitLockAcquire(CA_agentFileObjectLock, NULL);"
THE CODE:
The problem occurred in this function at the third line
NTSTATUS CA_EstablishConnection()
{
UNICODE_STRING agentIrpBusName;
NTSTATUS status;
WdfWaitLockAcquire(CA_agentFileObjectLock, NULL);
if (CA_agentFileObject != NULL)
{
LoggerLog(LOGGER_LS_WARN, L"CommAgent.c", L"CA_EstablishConnection", L"Communication with the Agent is already established");
WdfWaitLockRelease(CA_agentFileObjectLock);
return STATUS_SUCCESS;
}
RtlUnicodeStringInit(
&agentIrpBusName,
AS_IRPBUS_SUP_AGENT_NAME
);
// Connect to the Agent.
status = IoGetDeviceObjectPointer(
&agentIrpBusName,
FILE_ALL_ACCESS,
&CA_agentFileObject,
&CA_agentDeviceObject
);
if (status != STATUS_SUCCESS)
{
wchar_t strBuff[256];
swprintf_s(strBuff, 256, L"Cannot connect to the Agent. Reason: 0x%x", status);
LoggerLog(LOGGER_LS_ERROR, L"CommAgent.c", L"CA_EstablishConnection", strBuff);
}
WdfWaitLockRelease(CA_agentFileObjectLock);
return status;
}
More notes about the problem
KMDF 1.29 is not released yet
Microsoft mention that there is no functional changes in the WDF
As i mention before the code had no bugs until this last Windows 10 update (the software is in production)

Answer 1

you past only partial dump output. you need open dump in windbg and run !analyze -v command. i sure that you get something like:
WDF_VIOLATION (10d)
The Kernel-Mode Driver Framework was notified that Windows detected an error
in a framework-based driver. In general, the dump file will yield additional
information about the driver that caused this bug check.
Arguments:
Arg1: 0000000000000005, A framework object handle of the incorrect type was passed to
a framework object method.
Arg2: 0000000000000000, The handle value passed in.
Arg3: 0000000000001023, Reserved.
why i think so ? because view next line
00000000`0000010d 00000000`00000005 00000000`00000000 00000000`00001023 : nt!KeBugCheckEx
this mean that KeBugCheckEx(10d, 5, 0, 1023, *) is called
where 10d is WDF_VIOLATION, 5 is WDF_INVALID_HANDLE, 1023 is FX_TYPE_WAIT_LOCK. so
KeBugCheckEx(WDF_VIOLATION, WDF_INVALID_HANDLE, 0, FX_TYPE_WAIT_LOCK, *)
is called.
this mean that you pass invalid lock object handle (because WDF_INVALID_HANDLE, FX_TYPE_WAIT_LOCK) with 0 value to call WdfWaitLockAcquire.
you also must install pdb files (for Wdf01000.sys and ntoskrnl.exe) - in this case you will view FxVerifierBugCheckWorker call instead Wdf01000+0x5b828
so in call WdfWaitLockAcquire(CA_agentFileObjectLock, NULL);
CA_agentFileObjectLock == NULL, why is this - of course not visible from information which you post