Windbg and Symbol Files - windows

I have a problem with symbol files. I experimented with the symbol file path and set the path as follows:
srv*c:\symbols*http://msdl.microsoft.com/download/symbols;C:\Users\myuser\Desktop\driver2\objchk_win7_x86\i386
But afterwards I changed it to the following:
srv*c:\symbols*http://msdl.microsoft.com/download/symbols;C:\Users\myuser\Desktop\mydriver\objchk_win7_x86\i386
I changed the driver2 with mydriver in the path: this is the path where the .pdb file for my driver is located. The problem is that .sympath prints the right path as shown below:
kd> .sympath
Symbol search path is: srv*c:\symbols*http://msdl.microsoft.com/download/symbols;C:\Users\myuser\Desktop\mydriver\objchk_win7_x86\i386
Expanded Symbol search path is: srv*c:\symbols*http://msdl.microsoft.com/download/symbols;c:\users\myuser\desktop\mydriver\objchk_win7_x86\i386
But the symbols for the driver are still not found. If I run .reload command, we can see that WinDbg is looking for .pdb in driver2/ directory instead of mydriver/ directory.
kd> .reload /f mydriver.sys
SYMSRV: c:\symbols\mydriver.pdb\3D655E533B0449A38D7AB0AF637CE9201\mydriver.pdb not found
SYMSRV: http://msdl.microsoft.com/download/symbols/mydriver.pdb/3D655E533B0449A38D7AB0AF637CE9201/mydriver.pdb not found
SYMSRV: c:\users\myuser\desktop\mydriver\objchk_win7_x86\i386\mydriver.pdb\3D655E533B0449A38D7AB0AF637CE9201\mydriver.pdb not found
DBGHELP: c:\users\myuser\desktop\driver2\objchk_win7_x86\i386\mydriver.pdb - file not found
*** ERROR: Module load completed but symbols could not be loaded for mydriver.sys
DBGHELP: mydriver - no symbols loaded
I've deleted all the workspaces, closed WinDbg, restarted Windows, but the driver2/ entry is still there: it must be in the default workspace's cache or somewhere. How can I delete the whole personal settings of WinDbg including those caches, so I can restart WinDbg and be gone with the driver2/ path and make it use mydriver/.
I could also solve the problem with renaming the mydriver/ directory back into driver2/, but I don't want to solve the problem like that. I want to understand what's going on and solve it the best I can.

!sym noisy
will tell you why it did not want to load the pdb. Perhaps you did rebuild your driver and the pdb guid or pdb age does no longer match.
If you are sure that you have built the same source files you can force loading of your pdb by
.reload /i /f yourdriver.sys
/i is the magic switch to load also mismatched pdbs. This switch will not load any driver from your symbol server but it will consider only local file paths to load your driver. Also symbol store directories (SRV*) are not considered since there would be many versions to choose from. But if your .sympath directly points to your pdb it will be loaded.

Is the driver originally compiled & built in the driver2 path? What is the location of mydriver.sys?
For example assume I have symbol path 'c:\users\rahulsundar\desktop' set and try to load ntdll.dll, then it displays below error,
0:000> .reload ntdll.dll
DBGHELP: c:\users\rahulsundar\desktop\ntdll.pdb - file not found
DBGHELP: c:\users\rahulsundar\desktop\dll\ntdll.pdb - file not found
DBGHELP: c:\users\rahulsundar\desktop\symbols\dll\ntdll.pdb - file not found
DBGHELP: C:\Windows\SYSTEM32\ntdll.pdb - file not found
DBGHELP: ntdll.pdb - file not found
*** ERROR: Symbol file could not be found. Defaulted to export symbols for ntdll.dll -
DBGHELP: ntdll - export symbols
Note: Windbg by default searches ntdll.pdb from the same location 'C:\Windows\SYSTEM32'
One way to solve the current issue, from the log its clear that windbg expects pdb file in directory 'c:\users\myuser\desktop\mydriver\objchk_win7_x86\i386\mydriver.pdb\3D655E533B0449A38D7AB0AF637CE9201\mydriver.pdb'.
So manually create directory till
'mydriver.pdb\3D655E533B0449A38D7AB0AF637CE9201' and place pdb file
there.
This is just a standard way( binaryfoldername\hashid\pdbfile ) that Windows expects a symbol for a binary.

Better way to solve this - turn on sim noisy and look at path
at .reload /f my_driver.sys
or add new path to .sympath[+] path/to/pdb and do the same

Related

Download symbols from Microsoft symbol server

I read MSDN "using symbol server"
I am trying to download symbols from microsoft symbol server.
My symbol search path: srv*C:\symbols*http://msdl.microsoft.com/download/symbols
In WinDbg I always get error:
************* Symbol Loading Error Summary **************
Module name Error
ntdll PDB not found :
srv*c:\symbols*http://msdl.microsoft.com/download/symbols
Unable to locate the .pdb file in this location
PDB not found : > srv*c:\symbols*https://msdl.microsoft.com/download/symbols
Unable to locate the .pdb file in this location
How can I obtain the symbols from MS symbol server?
because the location site is crashed!
http://msdl.microsoft.com/download/symbols
open that in your browser,you will see a crashed page,normally it will show a hint page if the site is good.
i try to report this but cannot find the way.-_-!!!
that brought me some trouble too.
question:is there a site cached MS symbols and opened for everyone?thanks.

*** ERROR: Symbol file could not be found. Defaulted to export symbols for ntkrnlmp.exe

Hello Stackoverflow community,
I am trying to debug an application on Windows 10 and require windows debugger framework to load symbols to run certain commands in the (windows kernel debugger) kd.
Each time I load the kernel debugger (kd) on the command prompt by typing kd -kl (NOTE: you would need to enable bcdedit -debug on for it to work), I get the below stack trace:
Microsoft (R) Windows Debugger Version 10.0.14321.1024 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
Connected to Windows 10 10240 x64 target at (Tue May 2 18:26:51.800 2017 (UTC - 7:00)), ptr64 TRUE
Symbol search path is: srv*
Executable search path is:
*** ERROR: Symbol file could not be found. Defaulted to export symbols for ntkrnlmp.exe -
Windows 10 Kernel Version 10240 MP (6 procs) Free x64
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 10240.17354.amd64fre.th1_st1.170327-1827
Machine Name:
Kernel base = 0xfffff803`1da07000 PsLoadedModuleList = 0xfffff803`1dd2c070
Debug session time: Tue May 2 18:26:53.740 2017 (UTC - 7:00)
System Uptime: 0 days 0:50:43.754
lkd>
At first glance, it appears that my sympath is not configured.
I configure it to point to a path on my file system (see next point).
.sympath srv*C:\symbols*https://msdl.microsoft.com/download/symbols
Now, I set !sym noisy and do a .reload and I get the following trace
lkd> !sym noisy
noisy mode - symbol prompts off
lkd> .reload
Connected to Windows 10 10240 x64 target at (Tue May 2 18:41:38.542 2017 (UTC - 7:00)), ptr64 TRUE
SYMSRV: BYINDEX: 0x3
c:\symbols*https://msdl.microsoft.com/download/symbols
ntkrnlmp.pdb
30D698E116494C24A48409E2A73883CF1
SYMSRV: c:\symbols\ntkrnlmp.pdb\30D698E116494C24A48409E2A73883CF1\ntkrnlmp.pdb - file not found
SYMSRV: HTTPGET: /download/symbols/ntkrnlmp.pdb/30D698E116494C24A48409E2A73883CF1/ntkrnlmp.pdb
SYMSRV: HttpQueryInfo: 404 - HTTP_STATUS_NOT_FOUND
SYMSRV: HTTPGET: /download/symbols/ntkrnlmp.pdb/30D698E116494C24A48409E2A73883CF1/ntkrnlmp.pd_
SYMSRV: HttpQueryInfo: 404 - HTTP_STATUS_NOT_FOUND
SYMSRV: HTTPGET: /download/symbols/ntkrnlmp.pdb/30D698E116494C24A48409E2A73883CF1/file.ptr
SYMSRV: HttpQueryInfo: 404 - HTTP_STATUS_NOT_FOUND
SYMSRV: c:\symbols\ntkrnlmp.pdb\30D698E116494C24A48409E2A73883CF1\ntkrnlmp.pdb not found
SYMSRV: https://msdl.microsoft.com/download/symbols/ntkrnlmp.pdb/30D698E116494C24A48409E2A73883CF1/ntkrnlmp.pdb not found
DBGHELP: ntkrnlmp.pdb - file not found
*** ERROR: Symbol file could not be found. Defaulted to export symbols for ntkrnlmp.exe -
DBGHELP: nt - export symbols
Loading Kernel Symbols
...............................................................
................................................................
.........
Loading User Symbols
************* Symbol Loading Error Summary **************
Module name Error
SharedUserData No error - symbol load deferred
Symbol loading has been deferred because this symbol is not needed
at this time. Use reload /f to force load symbols.
ntkrnlmp The system cannot find the file specified
The SYMSRV client failed to find a file in the UNC store, or there
is an invalid UNC store (an invalid path or the pingme.txt file is
not present in the root directory), or the file is present in the
symbol server exclusion list.
I also tried performing the steps explained in ERROR: Symbol file could not be found. windbg.exe and Error:Symbol File not found in WinDbg but no luck. I get errors that indicate several 404 errors.
It always points to this file ntkrnlmp.exe and says its missing(or not found).
Has anyone faced similar issues in the past? If so, what can I do to get this fixed?
The sympath you configured is correct. I think it is very likely because Microsoft has not uploaded the PDB symbol to their symbol servers. Typically new symbols will be available a few days after each Patch Tuesday. (It may become faster in the future.[1]) For your situation, I suggest you to report this issue to Microsoft WinDbg team at windbgfb#microsoft.com with the trace info you posted here.
[1] https://twitter.com/aluhrs13/status/842590084952088580
[2] The email address comes from this page.

Is there a known issue relating to Windows 7 Kernel Symbols?

I have a few Windows 7 machines that I am not able to read their memory dumps. I found something that I suspect may be related, but am not positive:
https://twitter.com/aionescu/status/634028737458114560
I also found this: http://support.microsoft.com/kb/2528507
However, the scenario message regarding wow64exts given in the doc is not seen in any of my dumps. I also cannot apply that hotfix at this time to test it. So I'm just looking for some more information or opinions.
I'm able to open any other OS dump as well as my own system's Windows 7 dump, but there are 2 other machines that run Win 7 and it's telling me I have the wrong kernel symbols.
I have tried clearing out my symbol cache, reinstalled the Windows SDK, and also tried to open the dumps on two other machines with the same result. If it matters, the crash is manually created using the scroll lock method.
Symbol path: SRV*c:\symbols*http://msdl.microsoft.com/download/symbols;
Seeing these errors: followed by "Type referenced: nt!_KPRCB"
Does anyone know about the issue mentioned by Alex in the twitter link and if it's possibly related to what I'm seeing?
Update 2015-10-22:
With the Microsoft patch day (2015-10-13) and KB3088195, symbols are available again.
However, symbols for the broken version have not been provided, so below may still be useful.
Microsoft has already published "good" symbols for ntdll in the past, containing type information like _TEB or _KPRCB. Starting from mid of July 2015, Microsoft has still published symbols for ntdll, but not containing that information.
So it depends on the version of ntdll whether you get type information or not. Old dumps referencing an old version of ntdll will download old PDBs containing type information while new dumps reference new versions of ntdll and WinDbg (or any other debugger) downloads PDBs without type information.
Could Microsoft remove type information of "good" symbols retroactively, thus making them "bad"?
Yes. As described in this answer, there is a tool to remove type information from existing PDBs. Doing that and replacing the PDB would result in such an effect.
Can Microsoft publish the "good" version of those PDBs which are currently "bad"?
That's hard to tell, since we don't know whether Microsoft has kept a copy of the "good" version so they could replace the "bad" version on the symbol server with the "good" one. Rebuilding ntdll from the same source code and thus creating new PDBs sounds possible, but the PDB gets a new time stamp and checksum. This can potentially be corrected manually, especially be Microsoft, since they should have the knowledge about the PDB internal format, but IMHO it's unlikely they'll do that. Things may go wrong and MS will hardly have tests to guarantee the correctness of such a thing.
So what can I do?
IMHO you can do nothing to really correct the situation.
You could assume that the types in ntdll have not changed so much. This would allow you to take an older version of wntdll.pdb and the new version of ntdll.dll and apply ChkMatch -m to it. This will copy the timestamp and checksum from the DLL to the PDB. After you did that (in an empty folder), replace the existing wntdll.pdb in your symbols directory with the hacked one.
WinDbg walkthrough (with output shortened to relevant things). I am using the latest version of wntdll.pdb I could find on my PC.
WARNING: doing the following may fix the type information but will likely destroy the correctness of the callstacks. Since any changes in the implementation (which are likely for security fixes) will change the method offsets.
0:005> dt nt!_PEB
*************************************************************************
*** ***
*** Either you specified an unqualified symbol, or your debugger ***
...
*** Type referenced: nt!_PEB ***
*** ***
*************************************************************************
Symbol nt!_PEB not found.
0:005> lm m ntdll
start end module name
773f0000 77570000 ntdll (pdb symbols) e:\debug\symbols\wntdll.pdb\FA9C48F9C11D4E0894B8970DECD92C972\wntdll.pdb
0:005> .shell cmd /c copy C:\Windows\SysWOW64\ntdll.dll e:\debug\temp\ntdllhack\ntdll.dll
1 file(s) copied.
0:005> .shell cmd /c copy "E:\Windows SDk\8.0\Debuggers\x86\sym\wntdll.pdb\B081677DFC724CC4AC53992627BEEA242\wntdll.pdb" e:\debug\temp\ntdllhack\wntdll.pdb
1 file(s) copied.
0:005> .shell cmd /c E:\debug\temp\ntdllhack\chkmatch.exe -m E:\debug\temp\ntdllhack\ntdll.dll E:\debug\temp\ntdllhack\wntdll.pdb
...
Executable: E:\debug\temp\ntdllhack\ntdll.dll
Debug info file: E:\debug\temp\ntdllhack\wntdll.pdb
Executable:
TimeDateStamp: 55a69e20
Debug info: 2 ( CodeView )
TimeStamp: 55a68c18 Characteristics: 0 MajorVer: 0 MinorVer: 0
Size: 35 RVA: 000e63e0 FileOffset: 000d67e0
CodeView format: RSDS
Signature: {fa9c48f9-c11d-4e08-94b8-970decd92c97} Age: 2
PdbFile: wntdll.pdb
Debug info: 10 ( Unknown )
TimeStamp: 55a68c18 Characteristics: 0 MajorVer: 565 MinorVer: 6526
Size: 4 RVA: 000e63dc FileOffset: 000d67dc
Debug information file:
Format: PDB 7.00
Signature: {b081677d-fc72-4cc4-ac53-992627beea24} Age: 4
Writing to the debug information file...
Result: Success.
0:005> .shell cmd /c copy E:\debug\temp\ntdllhack\wntdll.pdb E:\debug\symbols\wntdll.pdb\FA9C48F9C11D4E0894B8970DECD92C972\wntdll.pdb
1 file(s) copied.
0:005> .reload
Reloading current modules
.............................
0:005> dt nt!_PEB
ntdll!_PEB
+0x000 InheritedAddressSpace : UChar
+0x001 ReadImageFileExecOptions : UChar
...
0:005> !heap -s
LFH Key : 0x219ab08b
Termination on corruption : DISABLED
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-----------------------------------------------------------------------------
Virtual block: 00920000 - 00920000 (size 00000000)
Virtual block: 02c60000 - 02c60000 (size 00000000)
Virtual block: 02e10000 - 02e10000 (size 00000000)
...
Note: using ChkMatch like this has the benefit that you do not need to turn on .symopt- 100, since that option would affect all PDB files, and you would not find potential other symbol issues. If you don't mind using .symopt, you could simply copy an old wntdll.PDB over the new one.
The issue is now fixed according to Microsoft and Microsoft told me that you should clear your symbol cache to get the new PDBs, otherwise Windbg would use the old Symbols which miss the information.

Teamcity as Symbol Server no pdbs found

I am trying to use Teamcity as Symbol Server. I set Up TC like I should and TC is indexing my files.
I found THIS question so I tried it to find the symbols on the server. I get some .dll listed with the command symchk /r * /s SRV*http://outserver.acp.net/app/symbols. I Only see other dlls but not my libraries or pdbs.
What did i miss?
A short Log:
[14:45:48]Indexing sources appeared in file C:\BuildAgent\work\8652c4bdd8dd2154\***\File.pdb
[14:45:48]Information about 7 source files was updated
[14:45:53]Collecting symbol files signatures.
[14:45:53]Running command C:\BuildAgent\plugins\symbol-server\bin\JetBrains.CommandLine.Symbols.exe dumpSymbolSign /o="C:\BuildAgent\temp\buildTmp\symbol-signatures-8701232224987713334.xml" /i="C:\BuildAgent\temp\globalTmp\dumpSymbolSign5535309674029756055.input"

how windbg determine which OS symbols to download for a dump file

When I do a dump file analysis, I setup symbol path to point to Microsoft symbol server. How does windbg knows that the dump file was generated under what OS and how does it downloads correct symbols for that OS?
To match symbols to binary, WinDbg looks at the thing called Debug Directories. Debug Directories are sections in PE modules (which is a file format used by Windows for all types of executables). Debug Directories simply contain links to types of debug information. If you type command in cmd window link /dump /headers <my_module_name>, it will print out something like this:
...
Debug Directories
Time Type Size RVA Pointer
-------- ------ -------- -------- --------
4CC78FB1 cv 22 00102588 101988 Format: RSDS, {30976E0B-FBF7-45EF-8608-99932F2B791F}, 2, ntdll.pdb
4CC78FB1 ( A) 4 00102584 101984 BB03197E
...
This is output for ntdll.dll. You can see that CV (for CodeView) debug information is contained in ntdll.pdb, and GUID of that PDB has to match the one in the link. That GUID is generated randomly for each module at build time.
Command !lmi in WinDbg will also dump this information, in different format.
What WinDbg does when loading symbols from symbol server is it sends request to symbol server to get file named 'ntdll.pdb' with matching GUID.

Resources