Setting up a public (or private) symbol server over http - visual-studio

Question

Every piece of documentation I've found (references 1 through 5) talks about setting up a symbol server by using a shared UNC path, and then putting the correct settings available to the local debugger instance (whether _NT_SYMBOL_PATH or the Visual Studio IDE Debugging settings).
Microsoft provides a symbol server (reference 6) available via http for their public symbol stores.
I want to create, for my own code, a symbol server accessible over http transport, instead of over UNC file sharing. The Mozilla folks appear to have done so (reference 7), but it is no longer functional.
Are there better references available for performing this task than I have found so far?
References
https://msdn.microsoft.com/en-us/library/b8ttk8zy(v=vs.80).aspx
http://msdn.microsoft.com/en-us/library/ms680693(v=vs.85).aspx
http://stackhash.com/blog/post/Setting-up-a-Symbol-Server.aspx
http://entland.homelinux.com/blog/2006/07/06/…
http://msdn.microsoft.com/en-us/windows/hardware/gg462988
http://support.microsoft.com/kb/311503
http://developer.mozilla.org/en/Using_the_Mozilla_symbol_server

Answer 1

I believe the answer is a very simple, "Just share the directory via some sort of http path." According to Chad Austin's entry on "Creating Your Very Own Symbol Server", this will just work.
In other words, the directory which symstore.exe uses to store the symbols, when served up as http://symbols.example.com/public_symbols/ , will be usable as the symbol server target for the Windows Debugging Tools.

Answer 2

Be careful when having multiple users use Symstore.exe directly against the same symbol store. Microsoft's white papers on this subject make it sound like you simply create a share and have everyone update through the SYMSTORE.EXE program delivered as part of Debugging Tools for Windows. The white papers advised you to have this done by each build.
And it works great with single users or when funneling all updates through a single person who is updating the symbol server for a team.
Unfortunately, the "fine print" at the bottom of some of the white papers says that only one user running symstore.exe can update the shared symbol server at the same time without breaking the content.
(Example: At http://msdn.microsoft.com/en-us/library/ms681417(VS.85).aspx, Microsoft says: "Note SymStore does not support simultaneous transactions from multiple users. It is recommended that one user be designated "administrator" of the symbol store and be responsible for all add and del transactions.")
So there is no inherent mechanism to serialize updates to the symbol store. It appears that multiple, simultaneous attempts to update the symbol store can break the symbol store and/or its index.
We cannot have builds for our entire multi-thousand man, international corporation in all time zones dependent upon coordination thru one man in one location.
Based on those white papers, I raised this issues with Microsoft in March of 2009; who confirmed this was a possible issue. After that discussion, we chose to implement a symbol update service which serializes the updates via direct Windows Debugging Tools SDK DbgEng.DLL SymbolSrvStoreFile() API calls so there is never a possibility of two simultaneous updates against the same area of symbols at the same time. Users have a build action that queues their symbols through the service instead of directly updating the symbol store. The service then serializes the updates to make sure true concurrent update attempts never happen.
The limited documentation available about using SymSrvStoreFile was not very clear at the time. I did get it working. Hopefully it has been improved since then. if not, the most crucial issue was the that the input path must be specified in a format similar to _NT_SYMBOL_PATH. So instead of, for example, using "C:\Data\MyProject\bin" as the input path, you would instead specify "srv*C:\Data\MyProject\bin".
Our service now also logs the updates through a database. The database both serves as a backup to the symbol store (in case it ever gets corrupted and must be rebuilt) and also creates a reporting point so that managers and support people know who is actually saving their symbols and who is not. We generate a weekly "symbol check-in" report which is auto-EMailed to stakeholders.

Answer 3

A symbol server served via HTTP has the same structure as a symbol server served via a UNC file path, so the simplest thing to do would be to use symstore.exe to store the files in a folder somewhere and then use a simple HTTP server which exposes that folder via HTTP (even running python -m SimpleHTTPServer in the symbols dir would work).
A small gotcha is that if a symbol file does not exist, the HTTP server must return a 404 error code (tested under Visual Studio 2013 at least). I ran into an issue where an HTTP server returning 403 for missing files caused Visual Studio to stop making requests after the first failed request.
symstore.exe creates a number of auxilliary files and folders (the 000Admin/ folder, refs.ptr and files.ptr files). None of these are needed for the symbol server to work.
If you want to create a symbol store without using symstore.exe, you can upload the files with this structure:
BinaryName.pdb/$BUILD_ID/BinaryName.pdb
BinaryName.exe/$LINK_ID/BinaryName.exe
Where BUILD_ID is a GUID embedded in the PDB file and executable and LINK_ID is a combination of build timestamp and file size in the executable. These can be obtained by reading the output of the dump_syms.exe tool from the breakpad library. See http://www.chromium.org/developers/decoding-crash-dumps

Answer 4

Our (Mozilla's) symbol server works fine, AFAICT. We're not doing anything particularly complicated, we just put the PDB files into the right directory structure (we have a script for that, but you could use symstore.exe) and serve it up via Apache. I think the only special thing we have are some Rewrite rules to allow accessing the files in a non-case-sensitive manner, because Microsoft's tools are really inconsistent about filename/GUID case.

Answer 5

There is also Electron's variant of this, which sits in front of S3.
It has the additional helpers of converting 403's to 404's (to not upset the debugger), and converting all paths to lowercase, so that incoming requests are case-insensitive.
https://github.com/electron/symbol-server