I have a program do so some graphics. When I run it interactively, I want it to use OpenGL from the system to provide hardware accelerated graphics. When I run it in batch, I want to be able to redirect it to use the Mesa GL library so that I can use OSMesa functionality to render to an offscreen buffer. The OSMesa functionality is enabled by doing a LoadLibrary/GetProcAddress if the batch start up option is selected.
On Linux, its fairly easy to make this work. By using a wrapper script to invoke the program, I can do something like this:
if [ "$OPTION" = "batch" ]; then
export LD_LIBRARY_PATH=$PATHTO/mesalibs:$LD_LIBRARY_PATH
fi
It is possible to do something this in Windows?
When I try adding a directory to the PATH variable, the program continues to go to the system opengl32.dll. The only way I can get the program to use the Mesa GL/OSMesa shared libraries is to have them reside in the same directory as my program. However, when I do that, the program will never use the system opengl32.dll.
If I've understood what you're saying correctly, the wrong version of opengl32.dll is being loaded when your process starts up, i.e., load-time dynamic linking. There is probably no good way to solve your problem without changing this.
You say you can't use conveniently use run-time dynamic linking (LoadLibrary/GetProcAddress) for opengl32.dll because the calls to it are coming from the Qt library. I presume that the Qt library is itself dynamically linked, however, so you should be able to solve your problem by using run-time linking for it. In this scenario, provided you load opengl32.dll before you load the Qt library, you should be able to explicitly choose which version of opengl32.dll you want to load.
You might want to consider using delayed loading in order to simplify the process of moving from load-time to run-time linking. In this scenario, the first call into the Qt library causes it to be loaded automatically, and you'll just need to explicitly load opengl32.dll first.
There are a few ways you could handle this, depending on the libraries and their names/locations:
If both have the same name (opengl32.dll), then you need to add the Mesa DLL location to the search path such that it is searched before the system directory. The order directories are checked in is detailed here. As you can see, $PATH comes last, after system, so you can't just add the directory to that. However, you can make use of the second step ("The current directory") by setting the working directory to a path containing the mesa files. Generally this means starting the application using an absolute path while in the directory containing the files.
That's still not particularly pleasant, though. If you can, you should use LoadLibrary and check for an environment variable (OPENGL_LIBRARY_PATH) when your app starts up. Assuming the exports from opengl32.dll and Mesa's DLL are the same, you can do something like:
void LoadExports()
{
char location[MAX_PATH];
getenv("OPENGL_LIBRARY_PATH", location);
HMODULE oglLib = LoadLibrary(location);
function1 = GetProcAddress(oglLib, "glVertex2f");
...
}
This will work perfectly fine, doing almost exactly what you want.
However, if you want to do that, you can't import opengl32.dll, which you're probably doing, you have to dynamically link throughout. Make sure not to link against opengl32.lib and you should be fine. Depending on how many functions you use, it may be a pain to set up, but the code can easily be scripted and only needs done once, you can also use static variables to cache the results for the lifetime of the program. It's also possible to use different function names for different libraries, although that takes a bit more logic, so I'll leave the details to you.
Though this should be possible in the cmd window, it seems you're having no luck.
Try: set a variable in your script (RUNNING_IN_SCRIPT=Y) and then parse for that variable in your executable and LoadLibrary from the absolute path of installation - be sure to clear the variable when you exit.
Windows used to search different paths for dynamic libraries, but due to security consideration, the system path is searched first.
You could, however use Delay Load Imports to get a workaround:
If you're using MSVC, you could single-out the DLLs you're interested in loading on your own with /DELAYIMPORT flag to the linker.
Then, override the delay load helper function and use LoadLibrary to find the proper DLL (and not trust it to the system).
After loading the correct DLL, have your helper function just call the original one that will do all the GetProcAddress business by itself.
Related
I've been writing a Python extension use the Python/C API to read data out of a .ROOT file and store it in a list of custom objects. The extension itself works just fine, however when I tried to use it on a different machine I ran into some problems.
The code depends upon several libraries written for the ROOT data manipulation program. The compiler is linking these libraries dynamically, which means I cannot use my extension on a machine that does not have ROOT installed.
Is there a set of flags that I can add to my compilation commands to make these libraries statically linked? Obviously this would make the file size much larger but that isn't much of an issue providing that the code runs at the same speed.
I did think about collating all of the ROOT libraries that I need into an 'archive' file. I'm not too familiar with this so I don't know if that's a good idea or not.
Any advice would be great, I've never really dealt with the static/dynamic library issue before.
Thanks, Sean.
I am aware that implicitly linking to libraries at load time can lead to performance increases and as such I was wondering if it was good practice to link in this way at compile time thus increasing executable size (admittedly this is only marginal) compared to linking explicitly at runtime. My question is when linking against Microsoft Windows dll files located in System32, is it 'better' to link at load time as you can be mostly certain that the libraries will be present or follow the explicit approach?
Language used is Delphi (pascal) and the library in question is the WTsAPI32.dll - Terminal Services.
EDIT: As pointed out - my choice of language was incorrect and has been amended. Also, due to having only really every extensively linked to libraries in Unix, my comments about executable size can be omitted, I believed at the time I WAS in fact referring to static linking which bundles the library code into the executable and I now realise this is impossible when using dll files (DUH!). Thanks all.
The two forms of DLL linking are perhaps better named implicit and explicit. Implicit linking is what you refer to as static linking. And explicit linking is what you refer to as runtime linking
For implicit linking the linker writes entries into the import table of the executable file. This import table is metadata that is used by the loader to resolve DLL imports at module load time. A stub function is included for each implicit import that is only a few bytes in size. The executable size implications of implicit linking are negligible.
With explicit linking the imported function's address is resolved by a call to GetProcAddress. This call is made when the programmer chooses. If the DLL or the function cannot be resolved, the programmer can code fall back behaviour. There are size implications to explicit linking that I estimate to be similar to implicit linking. If the function address is evaluated once and remembered between calls then the performance characteristics are similar to implicit linking.
My advice is as follows:
Prefer implicit linking. It is more convenient to code.
If the DLL may not be present, use explicit linking.
If the DLL must be loaded using a full path, use explicit linking.
If you want to unload the DLL during program execution, use explicit linking.
You specifically mention Windows DLLs. You can safely assume that they will be present. Don't try to code to allow your program to run in case user32.dll is missing. Some functions may not be present in older versions of Windows. If you support those older versions you'll need to use explicit linking and provide a fallback. Decide which version you support and use MSDN to be sure that a function is available on your minimum supported platform.
If your only two options are static linking and run-time dynamic linking, then the latter is the best choice for linking with Windows DLLs because it's your only choice. You cannot link statically to a DLL because DLLs are exclusively for dynamic linking; that's what the D stands for. Microsoft does not provide static libraries for the OS modules, so you cannot link to them statically.
But those typically aren't your only two options. There's a third, namely load-time dynamic linking.
In Delphi, you use load-time dynamic linking by marking a function declaration external and specifying the name of the DLL where the function resides. If you use the function, then an entry is created in your module's import table, and when the OS loads your module, it reads the table, loads the referenced DLL, looks up the address of the function, and stores the address in your program's memory image so that your program can call it directly.
You use run-time dyanmic linking by declaring a function pointer, and then using LoadLibrary and GetProcAddress to look up the function's address prior to calling it. In newer Delphi versions, you can also declare a function in the same style that load-time dynamic linking uses, but then mark it with delay. In that case, the Delphi run-time library will call LoadLibrary and GetProcAddress on your behalf the first time you call the function.
The size differences are negligible. Run-time dynamic linking requires your program to contain code to load and link to libraries, but load-time dynamic linking stores more function references in the import table.
Run-time dynamic linking offers more flexibility in the face of uncertain DLL availability. With load-time dynamic linking, if a DLL is missing, or if it doesn't have all the functions mentioned in your import table, then the OS will fail to load your program — none of your code will run. With run-time dynamic linking, however, you have the opportunity to recover from the problem. You can disable certain parts of your program that the missing DLL depends on, or you can search for DLLs in non-standard places, or you can provide alternative implementations of missing functions.
If the functions you're calling are integral to your program's ability to operate, and there's ample reason to expect the functions to be present wherever your program is installed, then you should choose to link at load time. It allows you to write simpler code. You can be confident that you'll have the required functions if they are available on a certain version of windows that you check for in your installer, or if they're provided by DLLs that you distribute with your program.
On the other hand, if the functions you're calling are optional, then you should prefer to link at run time. Use that for loading plug-ins, or for taking advantage of advanced OS features while maintaining backward compatibility. (For example, you might want to take advantage of Windows Vista theme support when it's present, but still allow your program to run on Windows XP.)
Why do you think that compile-time linking to dynamic libraries would increase EXE size ? I believe you are mislead by somewhat poor choice of terms, used in windows programming from far ago. Let us better use relative terms "early binding" and "late binding" instead for the choice who should search for procedure names, compiler/loader or programmer's custom code.
Using early binding (aka static linking against dynamic library) your EXE contains the values (in a special tables):
DLL1 Name:
procedure "aaaaa" into the variable $1234
procedure "bbbbb" into the variable $5678
.
DLL2 Name:
procedure "ccccc" into the variable $4567
...et cetera.
Now, when you turn this into runtime loading (dynamic linking against dynamic libraries) it would look like
VarH1 := SafeLoatLibrary(DLL1 Name);
if Error-Loading-DLL then do-error-handling;
Var1234 := GetProcAfdress(VarH1, "aaaaa");
if Error-Searching-For-Function then do-error-handling;
Var5678 := GetProcAfdress(VarH1, "bbbbb");
if Error-Searching-For-Function then do-error-handling;
et cetera.
Obviously in the latter case your EXE contains all those values like in the 1st case, but more so - it contains a lot of code to deal with those values, that was just absent before.
So, while EXE size difference is not really large for today memory sizes, it is still in favor of early binding (static compilation against dynamic library).
Then what are the benefits for late binding? For example you can load different DLLs from different paths, determined in runtime by configuration - the flexibility and avoiding of DLL Hell (funny, concept of avoiding DLL Hell is against concept of volume saving). You can make your application work with limited functionality, if DLL load failed while statically binded EXE would just not load - graceful degradation concept. And at least you may give user much better, full of semantics, error messages than Windows could ever do.
And the last word, where you got that concept of EXE size from. I believe you mistaken it from talks about - attention! - static linking against static libraries. That is when OBJ/LIB/DCU files are not the part of distribution, but are just temporary code containers, that ultimately takes its place inside the monolythic EXE. Then yes - then your EXE has all those libraries insideitself and thus grows larger. However this case have nothing about dynamic libraries - DLLs.
The wording chosen once ago overuses static/dynamic terms in two closely related topics: how the library is loaded (compile-time vs runtime) and how functions inside the library are located (or bound. By developer's custom codeing ro by some OS-provided or compiler-provided toolset way before 1st line of your sources started execution).
Due to that ambiguity those close but different concepts start overlapping and sometimes this leads to a total confusion.
Now, what more static linking may give you in modern Windows versions. That is WinSxS folder Novadays Windows tends to keep multiple versions of each system DLL and your program may ask for the specific version of it (while in System32 folder there would be the most recent version that your program may be not get used to. Then you can make a special MANIFEST resource and compile it into EXE asking windows to load not DLLs not be name, but by name+version instead. You can replicaty that functionality with dynamic loading as well, but using Windows-provided toolset it is much easier.
Now you can decide which of those options do or do not have importance for your particular case and make somewhat better informed choice.
HTH.
When I would like to compile a program which uses a dynamic library, do I have to install (i.e. copy to a specific place, say, /usr/share/lib) this library? Or is it ok, if I put this library to any place somewhere and later during linking I point the linker to it, e.g. '-L ./thelibfolder'?
do I have to install (i.e. copy to a specific place, say, /usr/share/lib) this library?
No.
For a UNIX shared library, you need to arrange for two things:
You have to make the library known to the static linker, while linking main executable. Usually this is achieved by adding -L/path/to/directory -lfoo link flags to the link line.
You have to make runtime loader search /path/to/directory as well. This is system-specific. On many systems, setting LD_LIBRARY_PATH environment variable achieves the desired result, though this is usually not the preferred method. Another method is to encode this path into the application itself, e.g. on Linux one would add -Wl,-rpath=/path/to/directory to the application link line.
I'm trying to load a framework (or dylib) on mac where I know the path only at runtime.
Windows solution:
Link library with /DELAYLOAD
Just before a function from the library is used, change the current directory in the program to the directory where the dll resides
Call some function from this library. The dll is loaded from the current directory
On mac, I can use weak linking to make the application startup without the library being available yet. However, as soon as some function from the library is needed, I get "image not found", and the application is aborted.
How can I tell the Mac dynamic linker during runtime where to look for the library? "dlopen" does not work, since it only loads the library and does not resolve the symbols. Setting rpath to "." (the current directory) and changing the current directory does not work. Setting typical environment variables (DYLD_LIBRARY_PATH) only works when done before running the executable, not during runtime.
Any other ideas?
Write a wrapper script or executable that discovers the path at runtime, add the path to DYLD_LIBRARY_PATH, then calls execve to the real executable.
Use dlopen() to open the library and then dlsym() to find the symbols. If you are relying on the dynamic linker, you must know the path in advance and set it either via the rpath or with environment variables. The rpath can take relative paths, so that may work for you... but most likely, dlopen() and friends is the best solution here (although it may take some work to convert to using function pointers).
The advantage of dlopen() etc, is the same (or similar) code will work on other *nix's.
You can also look at NSAddImage(), which is OSX specific but should also do what you want.
When one should implicitly or explicitly link to a DLL and what are common practices or pitfalls?
It is fairly rare to explicitly link a DLL. Mostly because it is painful and error prone. You need to write a function pointer declaration for the exported function and get the LoadLibrary + GetProcAddress + FreeLibrary code right. You'd do so only if you need a runtime dependency on a plug-in style DLL or want to select from a set of DLLs based on configuration. Or to deal with versioning, an API function that's only available on later versions of Windows for example. Explicit linking is the default for COM and .NET DLLs.
More background info in this MSDN Library article.
I'm assuming you refer to linking using a .lib vs loading a DLL dynamically using LoadLibrary().
Loading a DLL statically by linking to its .lib is generally safer. The linking stage checks that all the entry points exist in compile time and there is no chance you'll load a DLL that doesn't have the function you're expecting. It is also easier not to have to use GetProcAddress().
So generally you should use dynamic loading only when it is absolutely required.
I agree with other who answered you already (Hans Passant and shoosh). I want add only two things:
1) One common scenario when you have to use LoadLibrary and GetProcAddress is the following: you want use some new API existing in new versions of Windows only, but the API are not critical in your application. So you test with LoadLibrary and GetProcAddress whether the function which you need exist, and use it in the case. What your program do if the functions not exist depend total from your implementation.
2) There are one important options which you not included in your question: delayed loading of DLLs. In this case the operating system will load the DLL when one of its functions is called and not at the application start. It allows to use import libraries (.lib files) in some scenarios where explicitly linking should be used at the first look. Moreover it improve the startup time of the applications and are wide used by Windows itself. So the way is also recommended.