Is there any way to automatically generate the DEF file for a DLL in Visual Studio? I've always just manually created them before, but there's gotta be an easier way.
I have found a place to generate the .DEF file for you here: expdef - def file generator
This works amazing and has a list of options you could also generate besides the method names of the functions and symbols.
Simply put: write a script that suits your needs. I for one created a small Perl script at some to create a .def file from an existing DLL that would then be used to create an import library (.lib) from it.
The reason there is no read-to-use tool that does it for you is because none of the tools involved can guess or somehow telepathically determine which functions or variables or constants or classes you want to export and under which names.
As you may know module definition files allow you to alter how an internal name appears in the export table, allow to export by ordinal (instead of name) or by name and ordinal. Of course they also allow to export a function multiple times (e.g. an old name due to legacy support purposes and a newer one because it follows some convention now).
Also see this question. You will notice that the answer also involves a custom script. In this case the requirement is apparently only to demangle and export the names of any and all non-static functions.
See answer with demo project in here:
https://stackoverflow.com/a/64302521/2338477
I have included python script to generate .def file from static library, but that unfortunately does not work with variables / data.
Related
I'm currently trying to make splint available as an external tool in Visual Studio 2010.
It has problems with finding all includes for the file, since it seems that the INCLUDE variable is only set at build time and I haven't found any other possibility to extract the include files in any way.
My question: Would there be any way to extract the IncludeDir field from the current file's project's Properties page, ideally with the VC++'s AdditionalIncludeDirectories?
Note also that AdditionalIncludeDirectories is per file, as it can be changed for individual source files as well as on the project level, and if it contains macros it can evaluate differently for each source file too!
I'm not familiar with driving the MSBuild objects via the API, but that's used by the IDE. Whether that way or by simply running MSBuild.exe, you need to get it to figure out all the properties, conditions, etc. and then tell you the result. If everything is well behaved, you could create a target that also uses the ClCompile item array and emits the %(AdditionalIncludeDirectories) metadata somehow such as writing it to a file or passing it to your other tool somehow. That's what's used to generate the /I parameters to CL, and you can get the same values.
If things are not well behaved in that necessary values are changed during the detailed build process, you would need to get the same prelims done just like the ClCompile target normally does, too. Or just override ClCompile with your own (last definition of a target is used) so it certainly is in the same context.
Either way, there are places where build script files can be automatically included into all projects, so you can add your stuff there or use a command argument (I think) to MSBuild to add another Include.
—John
I have a question about MSBuild.exe and Environment Variables. First, the development environment:
The code is mostly C++ with some C#. There are over 5,000 classes, 340 projects and 200 solutions arranged in a deep source tree. There is a solution at the root of the tree and at various other points in the tree. We use TFS and maintain multiple active branches for a series of future releases. Each developer uses a local view to modify and test code. Some developers work on multiple branches concurrently. Sometimes developers build solutions from different TFS branches concurrently.
We use about 70 environment variables for locating shared header files, libraries, etc. In VS2008 we used a file with the same base name as the solution file and an extension of .slnenv for defining the environment variables. All variables are defined relative to the base of the source tree. This .slnenv file is read by a custom VS2008 AddIn that creates Environment Variables in the VS2008 process space, i.e.
MyProjectDir=$(SolutionDir)\..\..
MyRoot=$(MyProjectDir)\..\..
MyInstallDir=$(MyRoot)\Install
MySourceDir=$(MyRoot)\Source
MyUnmanagedSourceDir=$(MySourceDir)\My\Unmanaged
MyIncludeDirs=$(MyProjectDir);$(MyUnmanagedSourceDir)
This AddIn does not work correctly with VS2010 because MSBuild does not inherit the environment variables that are created after the solution is loaded.
My question is how do I get these environment variables to MSBuild? I have found two methods that work, but are not as convenient as the AddIn we had been using.
VS2010 can be started with a command script that first sets Environment Variables in the process space and then starts VS2010. MSBuild does inherit these Environment Variables. This is unsatisfactory because the scripts would need to be customized for the various points where a solution can be loaded.
The second method I have tried is defining the Environment Variables as Properties in property sheets and .vcxproj files. We have a base property sheet that all .vcxproj files load. In that property sheet:
<PropertyGroup>
<MyRoot>$(MyProjectDir)\..\..</MyRoot>
<MyInstallDir>$(MyRoot)\Install</MyInstallDir>
<MySourceDir>$(MyRoot)\Source</MySourceDir>
<MyUnmanagedSourceDir>$(MySourceDir)\My\Unmanaged</MyUnmanagedSourceDir>
<MyIncludeDirs>$(MyProjectDir);$(MyUnmanagedSourceDir)</MyIncludeDirs>
</PropertyGroup>
Then I can define the base directory in each .vcxproj file:
<PropertyGroup>
<MyProjectDir>..\..</MyProjectDir>
</PropertyGroup>
This method uses relative path names where the AddIn resolved all Environment Variables to absolute path names. I'd rather not have to edit 340 .vcxproj files where the definition of "MyProjectDir" would vary depending on how far from the source root the project file exists. So far I've tried this method in just one project.
I have tried to modify the AddIn to create Properties rather than Environment Variables. I tried using ENV2.get_Properties(), but that seems to work only for Properties that are defined in VS2010, not for Properties I've defined.
Thank you,
Dan Kary
After discussion via email, we figured what were Dan needs and following message helps him with question issue. He agrees that this could be helpfull to other SO members:
Aha, I get it now.
As I did understand from that topic - you really need all that stuff only for build step (paths for include clause, paths to built tool to resolve dll\exe dependency), and no regular developer activities depends on it.
It's good, cause we don't need to fight VStudio about not reinitializing env vars after first load. We just need somehow edit and implement your solution for huge amount of proj files ;) Task become much easier :D
Small disclaimer - I'm writing this on the road and have no access to fully fledged dev env to check and bulletproof all further recomendations, so take my apologizes in advance for any mistakes I will make ;)
I would like to bring your attention to some msbuild features:
http://msdn.microsoft.com/en-us/library/ms164309.aspx
You have predefined properties, that available for all scripts and projects, it looks like this will help you solve first issue - absolute paths instead of relative. This is small excerpt from the link for further reference
$(MSBuildProjectDirectory) -
The absolute path of the directory where the project file is located, for example, C:\MyCompany\MyProduct.
$(MSBuildProjectFile) -
The complete file name of the project file, including the file name extension, for example,
MyApp.proj.
$(MSBuildProjectExtension)
The file name extension of the project file, including the period, for example, .proj.
$(MSBuildProjectFullPath)
The absolute path and complete file name of the project file, for example, C:\MyCompany\MyProduct\MyApp.proj.
$(MSBuildProjectName)
The file name of the project file without the file name extension, for example, MyApp.
So you should be able to use this defined properties in your Property sheets.
Now, with msbuild 4.0 toolset (toolset defined by attribute ToolsVersion="" on the root script element) there is interesting trick
Microsoft.Common.Targets define 2 variables to conditionally import custom msbuild script before or after itself
CustomBeforeMicrosoftCommonTargets and CustomAfterMicrosoftCommonTargets
You can use it the following way ( I think it will possible help you to avoid editing of all proj files or at least - reduce editing to very simplified copy-paste part)
Define it using relative path to include your property sheet files
<CustomBeforeMicrosoftCommonTargets>$(MSBuildProjectFile)\ConcreteProjectCustomProperties.propz</CustomBeforeMicrosoftCommonTargets>
Take a note that CustomBeforeMicrosoftCommonTargets should be global scoped property to be inherited by all secondary
And if you properly define your ConcreteProjectCustomProperties.propz I think you could achieve what you need.
"Properly" is that you will include there your global properties file (with relative or absolute path) and then define all your project-level properties.
NB:
Also take a note that later defined properties not available for reference to early defined
Your example defines $(MyIncludeDirs) with reference to $(MyProjectDir), but $(MyProjectDir) declared and defined a bit later.
It could be due to "exampling" and fast churning that code, but if you define your vars same way in production - it could lead to subtle error.
I hope I got your problem properly and my explanations will help you to quickly move all your properties from custom add-on to simpler and native msbuild script ;)
I have learned that you can:
Convert a .DLL file into a .DEF file, which includes its exports
(Edit: This doesn't work with many conventions)
Convert a .DEF file into a .LIB file, which you can use to link to the DLL
Why can't (most) linkers link to a DLL given only a .DEF file, instead of a .LIB file?
Ultimately, the answer here is 'because noone wanted it badly enough and it doesn't really help anything'.
The DEF file is an input file that creates an import lib for the DLL. And then, later, when the DLL is being consumed by another link, the importlib is itself an input. The importlib looks like something special on the outside, but when you look at the inside it's really just a slightly special lib with objects in it.
It totally would be possible to modify the linker to take a def file (or a DLL, for that matter) directly.
But the design centre of the linker is that it takes objects as inputs and outputs a PE executable. So taking a DEF or DLL as an input goes outside the design pattern.
Beyond that it'd be rather pointless - allowing the linker to take a DEF file or DLL as input would neither enable any important new scenarios, nor does leaving this feature out block anything. Converting a DEF file you have (even without the actual DLL) into a usable importlib is the work of a few moments (simply create fake empty function for each DEF entry and link that). So there's no reason to add the ability to link a DEF file directly.
Martyn
In terms of MSVC, .lib files are always static libraries. They get linked in as a compilation unit along with all your compiled .c/.cpp files, so all of the library's code is included in your final executable.
Some .lib files, however, (in particular most of the Windows system ones) merely contain stubs which tell the OS to load the desired DLL at loadtime and then the stubs route function calls to the DLL. But, those stubs are statically linked into your executable. Your program will then use DLLs (and gain all the advantages and disadvantages thereof), but since the named DLL functions it requires are happily located in the .lib (and thus actually located in the executable itself), your code doesn't have to know it's using a DLL (specifically using declspec(dllimport)).
A .def file is merely used as a sort of "settings" or "configuration" file during the creation of a .dll to specify what functions the file should export. It cannot be linked to, as it doesn't really describe anything that the linker understands.
You do not convert a dll to a DEF file. The DEF just indicates which dll functions will be accessible from the outside, exported.
From the docs:
A DLL file has a layout very similar to an .exe file, with one
important difference — a DLL file contains an exports table. The
exports table contains the name of every function that the DLL exports
to other executables. These functions are the entry points into the
DLL; only the functions in the exports table can be accessed by other
executables. Any other functions in the DLL are private to the DLL.
The exports table of a DLL can be viewed by using the DUMPBIN tool
with the /EXPORTS option.
You can export functions from a DLL using two methods:
Create a module definition (.def) file and use the .def file when
building the DLL. Use this approach if you want to export functions
from your DLL by ordinal rather than by name.
Use the keyword __declspec(dllexport) in the function's definition.
When exporting functions with either method, make sure to use the
__stdcall calling convention.
Use the provided link to learn more about exporting from your dll's.
I think you got down voted because your point is not really clear, at least not to me. Also check this. It explains how to choose the export method.
Mehrdad, this isn't always a question of how to LINK to a DLL, as I, personally have NEVER linked a DLL using a .DEF file. What I HAVE done is take someone else's DLL, and very painstakingly constructed a header file, or rather, function prototypes that I could use with
LoadLibrary() in C, Declare Function ... Lib "Foo.dll" Alias "OrdinalName" in VB, and
[DllImport()] in C#.
Of course, this is RARELY done, as if you are using a DLL for something, normally you have permission to do so, and the authors provide the .lib's, and the headers to go with the binary DLL file.
I've never done the exact techniques you speak of, by converting a .DEF info a .LIB, etc... But, I suppose it would be easy to take a lib, or the DLL itself and export .DEF from it. Now, THAT I actually HAVE done, in a project where the DLL code was built with a vbScript that took code from the main project, and created an API out of all the existing, compiled, and tested code. This level of complication was only done because I had no idea what functions were going to BE in the DLL, as the main project could change at any time, so a static .DEF file would have never worked. So, I had to build the DLL once, capture the dimpbin /exports, undecorate the functions, and then build the .DEF file, and re-link the DLL.
If you find yourself in that type of situation, perhaps you need to re-think your original designs, and fix the problem from there...
As for .LIB files, USUALLY you'd only NEED those for static linkage, but they are also used when the .H file is available, often making debugging a just a little nicer...
EDIT:
What I want is to distinguish statically linked library functions and user self-written functions within a compiled file (e.g. PE file).
How to achieve that? (I am thinking of database comparison but I do not know any database.)
By the way, (I have already known long before I asked this question) for dynamically linked library functions, they are just an entry in the import table (of PE).
By library functions, I mean those defined in libraries, such as STL (I know this is a bad name).
By user-defined functions, I mean those written by individual programmers.
Is there any programmatic way to achieve this goal?
Right now I am thinking about comparing binaries with a database, but I do not know any database so far.
Please recommend a database or a different way as an answer. Thank you.
This answer is assuming you want to analyze a standard Windows executable that is dynamically linked against other import libraries (.lib and assoicated .dll files that are not statically linked), and if this is the case, you want to interperet the PE (Portable Executable) file structure.
Here's a good article to get you started, with sample code on dumping the PE header.
You will want to focus on the Import table (.idata section) for external library calls, and the Export table (.edata section) for calls defined inside the executable and marked as exportable (usually this only exists in .dll files).
For static libraries, their format is called COFF, and there is the DUMPBIN utility that ships with Visual Studio that you can use to quickly peer into your lib files and even dump the disassembly of the code if you wanted.
The DUMPBIN utility, which is provided with the 32-bit version of
Microsoft Visual C++, combines the abilities of the LINK, LIB, and
EXEHDR utilities. The combination of these tools features the ability
to provide information about the format and symbols provided in
executable, library, and DLL files.
For information on the structure of COFF files, see this article.
Figuring out if a function call is from a lib or not would be tricky, but from what I remember, most static lib calls in code are actually thunk calls (simple jmp calls to the actual object code copied in from the lib) and are small in size (usually around 5 bytes), while "user defined" ones are not thunks, and are bp-based framed calls.
When your programm is linked, static functions and user-define functions are
include file by file.
So if you dump the header of a PE file, and look at the symbols
table (using objdump -x if you run with mingw32, or anything else)
you will see the name of a file and then all functions import from this one,
after an other file name and its functions...
Or if you have debug information, may be this can be easier.
So after you link functions with a file you can sort the functions by analysing their file name. Looking for extention (.c / .lib / .a) or check in a list of file you have somwhere.
Be carefull to eliminate crt0 files...
However this is kind a tricky solution and I'm not sure this'll work for every program.
I need to dynamically generate some macros into a .h configuration file that C programs can include in order to check which options are enabled, in a fashion similar to what is possible with CMake's CONFIGURE_FILE macro. But after looking in the doc and the web, I could not find something useful. Is it possible to generate such a file from bjam and have the dependencies handled correctly? If so, how would you do it?
Yes it's possible.. The way to do it boils down to defining a make target for the header and using the #() file output action support in bjam. You would set up a set of configuration variables on the header target and the action would use them to generated the file. That is what I do in one of the library extensions I wrote (see GIF lib extension). I also wrote some basic support for automating some of the tasks, but it still ends up being functionally the same, to create text files in the ext.jam utility. To allow easier definition of header configuration files that change based on Boost Build features (see Irrlicht 3D lib extension). Basically you can do just about anything you can think of with the make target since it's implementation is entirely up to you.