I'm working on a custom symbols/source server.
I've been able to produce pdb files which reference our sources. The most of our sources can be retrieved by Visual Studio. But sometimes, SRCSRV fails to retrieve them.
If I inspect the Visual Studio output window, I can get the following message
SRCSRV: Source server cannot retrieve the source code for file 'e:\SoftwareFactory\Projects\Product.Net Trunk\WorkingDirectory\Services\ErpWebServices\ErpServiceLegacyHost\Threading\ErpTransactionsSynchronizationContext.cs' in module 'C:\Program Files (x86)\Product\ProductCommon\ePgiStarterCS\server\Product.Erp.Services.LegacyHost.dll'. Données non valides.
The web server hosting the sources hasn't received any request for such file. So this must be an issue in the record concerning this precise file.
Is there any way to get more information by SRCSRV?
Apparently the answer is no: there is no way to obtain more information from srcsrv.dll.
The Microsoft forums moderators have told me that the message "Données non valides", "Invalid data" is possibly related to the length of the path of the file to be downloaded.
This path is combined to the temporary symbol path you've specified in the Visual Studio/WinDBG settings.
e.g. If you've specified
%APPDATA%\Symbols
as local symbol storage, and you're downloading a source file hosted by an HTTP server at the address
http://nightlybuilds.int/sources/get.svc/path/file.cs
the path
%APPDATA%\Symbols\sources\get.svc\path\file.cs
shall not be longer than 255 characters.
Other factors that can affect the behavior of SRCSRV:
The presence of characters invalid in a classic dos PATH (i.e. other than [0-9 A-Za-z\.])
The client debugger settings. (e.g. In the Native mode, the symbols for Managed code won't be downloaded. The Modules window will give you some hint about the symbols currently loaded.)
This is where SymChk can help. Using the /v switch, you get detailed output on how the symbols are resolved and what the symbol server responds with. Use this in conjunction with a tool like Fiddler which captures HTTP traffic, and you can analyze where your server is not responding with the expected protocol.
Related
I've set up Artifactory and uploaded some (s)nupkg packages. Then i've added the artifactory feed as described in the documentation. https://jfrog.com/knowledge-base/artifactory-how-to-configure-artifactory-as-symbol-server-and-integrate-with-visual-studio/
This nuget packages can be downloaded from artifactory as expected and all works well well. Yet for some reason, i can't seem to download the pdb symbols.
When i debug my code and look in the modules tab, i can see that the symbols are not loaded. In the log i can find an HTTP_STATUS_BAD_METHOD error:
SYMSRV: HTTPGET: /artifactory/api/nuget/v3/etm-nuget-local-release/ETM.WCCOA.Basics.pdb/6264D37419404FE5A4A845AF52F44612ffffffff/ETM.WCCOA.Basics.pd_
SYMSRV: HttpQueryInfo(HTTP_QUERY_CONTENT_LENGTH): 800C2F76 - ERROR_HTTP_HEADER_NOT_FOUND
SYMSRV: HttpQueryInfo: 80190195 - HTTP_STATUS_BAD_METHOD
SYMSRV: RESULT: 0x80190195
If i copy the string into my browser and add the address of the artifactory server like this:
https://artifactory.etm.at:8445/artifactory/api/nuget/v3/etm-nuget-local-release/ETM.WCCOA.Basics.pdb/6264D37419404FE5A4A845AF52F44612ffffffff/ETM.WCCOA.Basics.pdb
i can download the pdb and then add it manually to the the modules and everything works as expected. So it seems like there's a problem with the automatic download in studio from artifactory.
i don't know what's causing this, but the authentication should not be the problem, since i've also tried it with my admin account and the normal nuget packages are fine
if the symbol file does not exist in the cache, the VS debugger tries to download it in 3 different ways/formats:
The actual symbol file (.pdb extension), which is supported in Artifactory. In your case (as it worked for you in your browser), the request path would be:
ETM.WCCOA.Basics.pdb/6264D37419404FE5A4A845AF52F44612ffffffff/ETM.WCCOA.Basics.pdb
From a Pointer file (.ptr extension), that is currently not supported in Artifactory. In your case, the request path would be: ETM.WCCOA.Basics.pdb/6264D37419404FE5A4A845AF52F44612ffffffff/file.ptr
From a compressed file (.pd_ extension), which is not supported in Artifactory either. In your case (same as you added in your question in the error part), the request path would be:
ETM.WCCOA.Basics.pdb/6264D37419404FE5A4A845AF52F44612ffffffff/ETM.WCCOA.Basics.pd_
It is very common that the debugger fails to download the desired symbol file using the second (.pd_) or third (file.ptr) format (even with other Symbol Servers, like Nuget.org for example), but it should have tried to download the symbol file using the first format, which is the same request that worked for you manually using the browser.
Can you add the full output from the debugger? Or alternatively, can you check in your "artifactory-request.log" which requests you see that contain "/artifactory/api/nuget/v3/etm-nuget-local-release/ETM.WCCOA.Basics.pdb/6264D37419404FE5A4A845AF52F44612ffffffff/" ?
Those details would be very helpful.
I have a program which is instrumented to generate mini-dumps on exceptions. I have archived copies of the .exe, .pdb and the source files. The only way that I have found to get Visual Studio to find the .pdb file and analyze a dump when I receive one from a client, is to place the archived files in the exactly the same location that the original build took place on the disk.
I have tried adding the path to the .pdb file to Visual Studio's debug symbol directories, but the path is always ignored. The path in the .exe file seems to be used instead.
This is terribly inconvenient, since it means moving the code which is currently under development to some temporary location, while the archived code takes its place for crash dump analysis.
Is there some simple way (i.e. without setting up symbol and source servers) to direct Visual Studio to access the debugging context in some location other than the original build location?
What you need is a symbol server or at least a directory that has the same structure. If you have TFS, you just might need to configure it correctly.
If not, you have the following options:
a) add symbols manually for each delivered version using symstore
b. add symbols automatically for each build using symstore in a post build step
c) either of a) or b) and publish the result onto a webserver that acts as a HTTP symbol server.
You can do a) or b) if you're working alone. You should really consider c) if you're working in a team.
The things are not so simple and Stack Overflow is not thought for writing fully-fledged tutorials. Therefore I give you the following hints:
You need to understand that a symbol path can have several tiers. You are currently using a 0-tier symbol store, which is a flat directory. This is the worst option. Good news: if you have the symbols, you can still get other tier types set up.
Once you understood point 1. about the tiers and want to go for option c) without TFS, build an HTTP server.
IMHO you should find all necessary information in How to get a symbol server set up. If you don't want it on the network, you can also put it on local disk.
We have source server enabled, and source indexing is implemented according Using SrcSrv (MSDN). Debugger intelligently Copies the file from the server to the local cache. This works fine on developer machine.
Using IdebugSymbols Interface, we are able to retrieve information from pdb files. We have IdebugSymbols API functions for retrieving file name, source line number, module name etc.
Reference: IDebugSymbols interface (MSDN)
My query is whether we can retrieve Source File revision number from .pdb file using some Microsoft API?
Please let me know if the problem is not understood or you require some more inputs from my side. Thanks :)
I found one more general approach using Microsoft tools called "pdbstr.exe"and "Srctool.exe". These tools can be found at this location %PROGRAMFILES%\Debugging Tools for Windows (x86)\srcsrv
(from: Source Server (MSDN))
These tools simply interacts with pdb symbols file and fetches the information. So I was interested in fetching following info and yes that answers my question.
- What are all files which are Indexed with their revision number?
Every piece of documentation I've found (references 1 through 5) talks about setting up a symbol server by using a shared UNC path, and then putting the correct settings available to the local debugger instance (whether _NT_SYMBOL_PATH or the Visual Studio IDE Debugging settings).
Microsoft provides a symbol server (reference 6) available via http for their public symbol stores.
I want to create, for my own code, a symbol server accessible over http transport, instead of over UNC file sharing. The Mozilla folks appear to have done so (reference 7), but it is no longer functional.
Are there better references available for performing this task than I have found so far?
References
https://msdn.microsoft.com/en-us/library/b8ttk8zy(v=vs.80).aspx
http://msdn.microsoft.com/en-us/library/ms680693(v=vs.85).aspx
http://stackhash.com/blog/post/Setting-up-a-Symbol-Server.aspx
http://entland.homelinux.com/blog/2006/07/06/…
http://msdn.microsoft.com/en-us/windows/hardware/gg462988
http://support.microsoft.com/kb/311503
http://developer.mozilla.org/en/Using_the_Mozilla_symbol_server
I believe the answer is a very simple, "Just share the directory via some sort of http path." According to Chad Austin's entry on "Creating Your Very Own Symbol Server", this will just work.
In other words, the directory which symstore.exe uses to store the symbols, when served up as http://symbols.example.com/public_symbols/ , will be usable as the symbol server target for the Windows Debugging Tools.
Be careful when having multiple users use Symstore.exe directly against the same symbol store. Microsoft's white papers on this subject make it sound like you simply create a share and have everyone update through the SYMSTORE.EXE program delivered as part of Debugging Tools for Windows. The white papers advised you to have this done by each build.
And it works great with single users or when funneling all updates through a single person who is updating the symbol server for a team.
Unfortunately, the "fine print" at the bottom of some of the white papers says that only one user running symstore.exe can update the shared symbol server at the same time without breaking the content.
(Example: At http://msdn.microsoft.com/en-us/library/ms681417(VS.85).aspx, Microsoft says: "Note SymStore does not support simultaneous transactions from multiple users. It is recommended that one user be designated "administrator" of the symbol store and be responsible for all add and del transactions.")
So there is no inherent mechanism to serialize updates to the symbol store. It appears that multiple, simultaneous attempts to update the symbol store can break the symbol store and/or its index.
We cannot have builds for our entire multi-thousand man, international corporation in all time zones dependent upon coordination thru one man in one location.
Based on those white papers, I raised this issues with Microsoft in March of 2009; who confirmed this was a possible issue. After that discussion, we chose to implement a symbol update service which serializes the updates via direct Windows Debugging Tools SDK DbgEng.DLL SymbolSrvStoreFile() API calls so there is never a possibility of two simultaneous updates against the same area of symbols at the same time. Users have a build action that queues their symbols through the service instead of directly updating the symbol store. The service then serializes the updates to make sure true concurrent update attempts never happen.
The limited documentation available about using SymSrvStoreFile was not very clear at the time. I did get it working. Hopefully it has been improved since then. if not, the most crucial issue was the that the input path must be specified in a format similar to _NT_SYMBOL_PATH. So instead of, for example, using "C:\Data\MyProject\bin" as the input path, you would instead specify "srv*C:\Data\MyProject\bin".
Our service now also logs the updates through a database. The database both serves as a backup to the symbol store (in case it ever gets corrupted and must be rebuilt) and also creates a reporting point so that managers and support people know who is actually saving their symbols and who is not. We generate a weekly "symbol check-in" report which is auto-EMailed to stakeholders.
A symbol server served via HTTP has the same structure as a symbol server served via a UNC file path, so the simplest thing to do would be to use symstore.exe to store the files in a folder somewhere and then use a simple HTTP server which exposes that folder via HTTP (even running python -m SimpleHTTPServer in the symbols dir would work).
A small gotcha is that if a symbol file does not exist, the HTTP server must return a 404 error code (tested under Visual Studio 2013 at least). I ran into an issue where an HTTP server returning 403 for missing files caused Visual Studio to stop making requests after the first failed request.
symstore.exe creates a number of auxilliary files and folders (the 000Admin/ folder, refs.ptr and files.ptr files). None of these are needed for the symbol server to work.
If you want to create a symbol store without using symstore.exe, you can upload the files with this structure:
BinaryName.pdb/$BUILD_ID/BinaryName.pdb
BinaryName.exe/$LINK_ID/BinaryName.exe
Where BUILD_ID is a GUID embedded in the PDB file and executable and LINK_ID is a combination of build timestamp and file size in the executable. These can be obtained by reading the output of the dump_syms.exe tool from the breakpad library. See http://www.chromium.org/developers/decoding-crash-dumps
Our (Mozilla's) symbol server works fine, AFAICT. We're not doing anything particularly complicated, we just put the PDB files into the right directory structure (we have a script for that, but you could use symstore.exe) and serve it up via Apache. I think the only special thing we have are some Rewrite rules to allow accessing the files in a non-case-sensitive manner, because Microsoft's tools are really inconsistent about filename/GUID case.
There is also Electron's variant of this, which sits in front of S3.
It has the additional helpers of converting 403's to 404's (to not upset the debugger), and converting all paths to lowercase, so that incoming requests are case-insensitive.
https://github.com/electron/symbol-server
When using WinDbg, where should the private symbol files (pdb?) be placed?
My situation is: I have a DLL which I want to debug. I have the source code and symbol files for this DLL. This DLL is called by another DLL (which I don't have symbols or source for) which, in turn, is called by an EXE (which I also don't have symbols or source for).
My problem is that I am getting a warning that says
*** WARNING: Unable to verify checksum for C:\TheProgram\SomeSubfolder\AnotherSubfolder\MyDll.dll
This warning I think is the reason why I am getting the following type of messages in the call stack:
MyDll!AClass::AFunction+SomeHexAddress
My file structure looks something like this:
The exe: C:\TheProgram\program.exe
The calling dll: C\TheProgram\SomeSubfolder\caller.???
My DLL that I want to debug: C:\TheProgram\SomeSubfolder\AnotherSubfolder\MyDll.dll
Note: I set Symbol File path and the Source file path to where the debug DLL was generated, in my workspace on a different drive from the exe.. But I did copy the pdb + map files and put it on the dll that I wanted to debug..
Sorry for the late reply.
In your post you mention that you are seeing the following error message.
*** WARNING: Unable to verify checksum for C:\TheProgram\SomeSubfolder\AnotherSubfolder\MyDll.dll
You also ask the question, "where do I put my symbols for my DLL in the symbol path?"
Here is a response for the first problem:
Steps to identify mismatched symbols.
!sym noisy
.reload
x MyDll!*class*
*This reloads your dll, alternatively you can type kb to display the call stack of the DLL which should load it as well.
!sym quiet
*Reset's back to original quiet symbol loading
Also you can run
0:001> lmv m myDll *(and examine the Checksum)
Note: If you have a checksum, then Windbg can match the checksum of the DLL against the checksum of the PDB. Every development environment has a different way to generate a checksum.
Here is the response for the questions about where to put the PDBs
If you have MyDll.pdb added to a symbol store then you can use the following syntax
.sympath SRV*c:\symcache*http://msdl.microsoft.com/download/symbols
As Roger has suggested above...
However if you just have the PDB locally, you may want to put the path to the PDB first before going out to the symbol server like this
.sympath C:\TheProgram\SomeSubfolder\AnotherSubfolder\;SRV*c:\symcache*http://msdl.microsoft.com/download/symbols
This way Windbg should look local to your SomSubFolder dir before trying to use the Symbols Server cache.
Thanks,
Aaron
It does not matter where you put private symbol files as long as you're able to tell the debugger where they are.
The warning you're seeing does not have any effect on the stack trace, but the fact you're missing symbols for caller.DLL and app.EXE does.
Configuring symbols in windbg (locally) is as simple as using:
.sympath[+] path_to_pdbs
*and
.symfix+ path_to_system_pdb_store
You seeing:
MyDll!AClass::AFunction+SomeHexAddress
actually means nothing as long as SomeHexAddress is reasonable (and provided that MyDll.pdb has been found and loaded!) - it looks like a proper call stack entry.
Now, my question would be, what is the problem that you're stuck with?
P.S. you don't need .map file with windbg.
As part of our build process, we copy the private PDB files and the released EXE/DLL files to a symbol server. At its simplest, this is just a UNC path, but you can configure it for access using HTTP.
To copy your output files, use the SYMSTORE.EXE program.
Then, configure your debugger (we use Visual Studio and WinDbg) to look in that path. For WinDbg, the simplest way to do this is to set an environment variable:
_NT_SYMBOL_PATH=
SRV*C:\WebSymbols*http://msdl.microsoft.com/download/symbols;
\\symsvr\Symbols
(that should all be on one line)
This configures WinDbg to look on the Microsoft Symbol Server (caching the files in C:\WebSymbols) and also to look in a local symbol store (\\symsvr\Symbols).
We also use the Source Server tools to store SVN details in the PDB file, meaning that we can get back to the exact source file used to build a particular release. Look in ...\Debugging Tools for Windows (x86)\srcsrv.
One option is to leave the symbol files where they are (i.e. in the build output folder) and then use -y WinDbg command line option to locate these files. Using this approach should guarantee that the symbol files are always be up to date.
From the Microsoft Help:
-y SymbolPath
Specifies the symbol search path. Separate multiple paths with a
semicolon (;). If the path contains spaces, it should be enclosed
in quotation marks. For details, and for other ways to change this
path, see Symbol Path.
As it turned out, my target machine - provisioned inside Visual Studio - did not get the latest build upon deploying the to it, hence a "driver.sys has mismatched symbols" error.
Basically deploying did not replace the driver with the modified version of it for me. Use devcon tool to properly install it and Windbg will be happy again.