Operating system is MacOS X, specifically 10.5 (Leopard) on a PowerPC G4, but I have the same problem on an x86 running 10.6.
I am writing an application which dynamically loads a DLL. The DLL (let's call it foo.dylib) is part of another application, located elsewhere on the harddisk; my application finds foo.dylib programmatically (exact emplacement may change, possibly the user designates the DLL path through a GUI from the running application itself). For instance, assume that my application is located in directory /Application/MyApp.app/Contents/MacOS, and foo.dylib happens to be in /Application/OtherApp.app/Contents/MacOS. DLL loading uses dlopen().
Now, it turns out that foo.dylib itself needs a bunch of other DLL, which are in the same directory, but about which I do not know anything beforehand. Each such extra DLL is registered in foo.dylib with a path such as #executable_path/bar.dylib. The semantics of #executable_path are that it should be replaced by the directory in which the current process executable was found. This works great for OtherApp, not for me: when I open foo.dylib, it tries to load bar.dylib, and it looks for it in /Application/MyApp.app/Contents/MacOS/bar.dylib, which is not the right directory.
A workaround is to set the DYLD_FALLBACK_LIBRARY_PATH environment variable to /Application/OtherApp.app/Contents/MacOS, but this must be done before launching my application (that environment variable is read only once by the dynamic linker; changing its value programmatically with setenv() or putenv() has no effect). This is not compatible with the dynamic discovery of the location of the foo.dylib file.
Is there a programmatic way to override the effect of #executable_path ?
From reading the dyld source (search for #executable_path), I would say the answer is unequivocally "no". #executable_path is replaced with the main executable path, which is stored as a global string in the dyld module.
And yes, your suspicion is correct, dyld reads and saves its environment variables on startup, so you can't change them on the fly (you can search that same source file I linked for DYLD_LIBRARY_PATH). You could have a stub application that sets the environment variables and then launches your real application. dyld doesn't offer you many solutions here, it's not really designed to let you link in arbitrary private third-party libraries.
if you maintain OtherApp you could use #loader_path instead of #executable_path to locate dependencies: #loader_path always resolve to the path of the module (i.e. library or executable) which requires to load the library, so recursive dependencies are always found.
This is available from Mac Os 10.5 onwards.
See "man dyld" for detailed information.
Another option would be dlopening dependencies before main library.
Related
I'm working on a macOS tool which uses Apple's Safari framework. When running in macOS 10.13, the tool links to and loads it from
/System/Library/PrivateFrameworks/Safari.framework
and all works fine. But when running in macOS 10.12.6, some behaviors are missing. Based on some probing with DTrace, I think that this is because my tool needs to load instead the latest Staged framework, which is here:
/System/Library/StagedFrameworks/Safari/Safari.framework
This is apparently what Safari does, because if I attach to Safari with lldb and run image list, in 10.13 the list includes only the former path, and in 10.12.6 only the latter.
I tried the following:
NSBundle* stagedBundle = [NSBundle bundleWithPath:#"/System/Library/StagedFrameworks/Safari/Safari.framework"];
That returns nil in 10.13 because there is, at this time, no such directory. However, in 10.12.6, I get a stagedBundle, and then:
NSBundle* privateBundle = [NSBundle bundleForClass:[BookmarksController class]];
[privateBundle unload];
[stagedBundle load];
The unloading and loading apparently works, because if I log -description of those two bundles, before running that code the Private bundle is (loaded) and the Staged bundle is (not yet loaded), but after running that code those states are swapped, as desired.
But it is not effective. (1) If I again invoke -bundleForClass:, passing a class known to be in both frameworks, it gives me the Private bundle. (2) If I invoke -respondsToSelector:, passing a selector which is known to exist only in the Staged framework, I get NO.
I tried calling _CFBundleFlushBundleCaches(), as suggested here, but that did not help.
I've also tried changing my target's FRAMEWORK_SEARCH_PATHS, and installing the Staged framework on my Mac and linking to it, but since this post is already too long I'll just say that this resulted in more heat than light.
How can one selectively load a framework in this situation?
UPDATE
I've tried another approach. After re-reading Apple's Framework Programming Guide, even though it seems really dated, I decided that this framework needs to be weakly linked. Did this:
In the code, removed those NSBundle -load and -unload calls
In my tool's target,
In Build Phases > Link Binary with Libraries, removed path to the Safari Private framework, because this was a strong link.
In Build Settings > Other Linker Flags added -weak_framework Safari
In Build Settings > Framework Search Paths, listed paths to both frameworks' parent directories, with the Staged path before the Private path, because I want this one to load in macOS 10.12.6, where both exist.
It makes sense to me, builds and runs in both 10.13 and 10.12.6, but it is apparently still loading the undesired Private framework in 10.12.6. NSLog reports that as the bundle's path, and a class does not respond to a selector known to be in Staged framework only.
Any other ideas?
First, a disclaimer: I'd strongly suggest you don't rely on loading private frameworks in any application that you ship to users. It's fragile and unsupported.
That said, if you really want to do this, my suggestion would be to use the same technique that Safari itself uses to select between the two copies of the framework, which is dyld's DYLD_VERSIONED_FRAMEWORK_PATH environment variable.
To quote the dyld man page:
This is a colon separated list of directories that contain potential override frameworks. The dynamic linker searches these directories for frameworks. For each framework found dyld looks at its LC_ID_DYLIB and gets the current_version and install name. Dyld then looks for the framework at the install name path. Whichever has the larger current_version value will be used in the process whenever a framework with that install name is required. This is similar to DYLD_FRAMEWORK_PATH except instead of always overriding, it only overrides is the supplied framework is newer. Note: dyld does not check the framework's Info.plist to find its version. Dyld only checks the -current_version number supplied when the framework was created.
In short, this results in dyld performing a version check between the framework being loaded and the one in the versioned framework path, with the higher version being loaded. If the versioned framework path doesn't exist or the framework in question doesn't exist within it, the original framework path will be used.
Safari makes use of a second dyld feature to simplify its use of DYLD_VERSIONED_FRAMEWORK_PATH, the LC_DYLD_ENVIRONMENT load command. This load command allows DYLD_* environment variables to be specified at link time that will be applied by dyld at runtime prior to it attempting to load any dependent libraries. Without this trick you'd need to set DYLD_VERSIONED_FRAMEWORK_PATH as an environment variable prior to your application being launched, which typically requires a cumbersome re-exec to achieve.
Putting these two building blocks together, you end up adding a configuration setting like:
OTHER_LDFLAGS = -Wl,-dyld_env -Wl,DYLD_VERSIONED_FRAMEWORK_PATH=/System/Library/StagedFrameworks/Safari;
You can then either link statically against /S/L/PrivateFrameworks/Safari.framework, or attempt to load it dynamically at runtime. Either should result in the appropriate framework being loaded at runtime.
To address some of the misunderstandings your question reveals:
The unloading and loading apparently works, because if I log -description of those two bundles, before running that code the Private bundle is (loaded) and the Staged bundle is (not yet loaded), but after running that code those states are swapped, as desired.
Unloading shared libraries containing Objective-C code isn't supported. I suspect the only thing it does is result in a "loaded" flag being toggled on the NSBundle instance, since at dyld's level it is ignored.
In Build Settings > Framework Search Paths, listed paths to both frameworks' parent directories, with the Staged path before the Private path, because I want this one to load in macOS 10.12.6, where both exist.
Framework search paths are a concept that's only used at compile-time. At runtime, the library's install name is what tells dyld where to find the binary to load.
I'm trying to package a pre built binary into a Mac OS application bundle and I need to change the dylib and framework search paths for the executable. Unfortunately I can't rebuild the binaries at this point which makes means I have to find a work arround rather than using the correct search paths from the outset. Does anybody know how I can do this?
I assume that you want a permanent change to the executable. You can get temporary changes using the environment variables described in dyld(1). But for a permanent change, you can modify the executable using install_name_tool(1). See Creating Working dylibs for a good short writeup on how to do it. See the dyld(1) page for the replaceable variables you can use, such as #executable_path and #loader_path.
I'm trying to load a framework (or dylib) on mac where I know the path only at runtime.
Windows solution:
Link library with /DELAYLOAD
Just before a function from the library is used, change the current directory in the program to the directory where the dll resides
Call some function from this library. The dll is loaded from the current directory
On mac, I can use weak linking to make the application startup without the library being available yet. However, as soon as some function from the library is needed, I get "image not found", and the application is aborted.
How can I tell the Mac dynamic linker during runtime where to look for the library? "dlopen" does not work, since it only loads the library and does not resolve the symbols. Setting rpath to "." (the current directory) and changing the current directory does not work. Setting typical environment variables (DYLD_LIBRARY_PATH) only works when done before running the executable, not during runtime.
Any other ideas?
Write a wrapper script or executable that discovers the path at runtime, add the path to DYLD_LIBRARY_PATH, then calls execve to the real executable.
Use dlopen() to open the library and then dlsym() to find the symbols. If you are relying on the dynamic linker, you must know the path in advance and set it either via the rpath or with environment variables. The rpath can take relative paths, so that may work for you... but most likely, dlopen() and friends is the best solution here (although it may take some work to convert to using function pointers).
The advantage of dlopen() etc, is the same (or similar) code will work on other *nix's.
You can also look at NSAddImage(), which is OSX specific but should also do what you want.
I have a program do so some graphics. When I run it interactively, I want it to use OpenGL from the system to provide hardware accelerated graphics. When I run it in batch, I want to be able to redirect it to use the Mesa GL library so that I can use OSMesa functionality to render to an offscreen buffer. The OSMesa functionality is enabled by doing a LoadLibrary/GetProcAddress if the batch start up option is selected.
On Linux, its fairly easy to make this work. By using a wrapper script to invoke the program, I can do something like this:
if [ "$OPTION" = "batch" ]; then
export LD_LIBRARY_PATH=$PATHTO/mesalibs:$LD_LIBRARY_PATH
fi
It is possible to do something this in Windows?
When I try adding a directory to the PATH variable, the program continues to go to the system opengl32.dll. The only way I can get the program to use the Mesa GL/OSMesa shared libraries is to have them reside in the same directory as my program. However, when I do that, the program will never use the system opengl32.dll.
If I've understood what you're saying correctly, the wrong version of opengl32.dll is being loaded when your process starts up, i.e., load-time dynamic linking. There is probably no good way to solve your problem without changing this.
You say you can't use conveniently use run-time dynamic linking (LoadLibrary/GetProcAddress) for opengl32.dll because the calls to it are coming from the Qt library. I presume that the Qt library is itself dynamically linked, however, so you should be able to solve your problem by using run-time linking for it. In this scenario, provided you load opengl32.dll before you load the Qt library, you should be able to explicitly choose which version of opengl32.dll you want to load.
You might want to consider using delayed loading in order to simplify the process of moving from load-time to run-time linking. In this scenario, the first call into the Qt library causes it to be loaded automatically, and you'll just need to explicitly load opengl32.dll first.
There are a few ways you could handle this, depending on the libraries and their names/locations:
If both have the same name (opengl32.dll), then you need to add the Mesa DLL location to the search path such that it is searched before the system directory. The order directories are checked in is detailed here. As you can see, $PATH comes last, after system, so you can't just add the directory to that. However, you can make use of the second step ("The current directory") by setting the working directory to a path containing the mesa files. Generally this means starting the application using an absolute path while in the directory containing the files.
That's still not particularly pleasant, though. If you can, you should use LoadLibrary and check for an environment variable (OPENGL_LIBRARY_PATH) when your app starts up. Assuming the exports from opengl32.dll and Mesa's DLL are the same, you can do something like:
void LoadExports()
{
char location[MAX_PATH];
getenv("OPENGL_LIBRARY_PATH", location);
HMODULE oglLib = LoadLibrary(location);
function1 = GetProcAddress(oglLib, "glVertex2f");
...
}
This will work perfectly fine, doing almost exactly what you want.
However, if you want to do that, you can't import opengl32.dll, which you're probably doing, you have to dynamically link throughout. Make sure not to link against opengl32.lib and you should be fine. Depending on how many functions you use, it may be a pain to set up, but the code can easily be scripted and only needs done once, you can also use static variables to cache the results for the lifetime of the program. It's also possible to use different function names for different libraries, although that takes a bit more logic, so I'll leave the details to you.
Though this should be possible in the cmd window, it seems you're having no luck.
Try: set a variable in your script (RUNNING_IN_SCRIPT=Y) and then parse for that variable in your executable and LoadLibrary from the absolute path of installation - be sure to clear the variable when you exit.
Windows used to search different paths for dynamic libraries, but due to security consideration, the system path is searched first.
You could, however use Delay Load Imports to get a workaround:
If you're using MSVC, you could single-out the DLLs you're interested in loading on your own with /DELAYIMPORT flag to the linker.
Then, override the delay load helper function and use LoadLibrary to find the proper DLL (and not trust it to the system).
After loading the correct DLL, have your helper function just call the original one that will do all the GetProcAddress business by itself.
On Mac OS X, binary executables in the DYLD format contain "loader commands" that instruct the library loading system how to handle the contents of the file. In particular, the loader command instruct the system where dependent libraries should be searched for, etc.
You can see the complete list of loader commands for any binary on your system by running "otool -l /path/to/your/app".
Generally speaking these loader commands are set by the "ld" tool during the link phase of a project's compilation.
My question is, what do I need to do to add loader commands for publicized types that are not supported (apparently) by ld?
In particular, I would like to take advantage of the LC_DYLD_ENVIRONMENT loader commmand, which can be used to specify a string in the loader commands table of a binary that should be loaded and evaluated as environment variable settings in the context of the executable.
I don't see any argument to ld that would facilitate this. Something like "-sectcreate" but for specifically adding to the content of the loader commands, is what I'm after.
I know this is possible because at least one standard app on Mac OS X uses it: Safari. But I don't know whether they achieve this by some kind of post-link massage of the binary, if they're using a custom version of ld that knows how to build and chain the custom loader command in, or if they are exploiting a general-purpopse feature of the ld command that I haven't been able to figure out.
It looks like you can use -dyld_env, like so: "-dyld_env DYLD_FRAMEWORK_PATH=/". This isn't documented in the man page, but can be found in ld64's Options.cpp and mentioned in the Changelog file. If you're trying to do it from Xcode, you'll probably have to do it like this: "-Xlinker -dyld_env -Xlinker DYLD_FRAMEWORK_PATH=/".
One thing to note: if you look at dyld's dyld.cpp, you'll see that it only allows environment variables that start with DYLD_ and ends with _PATH.
If your executable is structured as part of a standard OS X application bundle (i.e. a .app that can be launched by a user), the conventional way to specify application specific environment variables is through its plist file using the LSEnvironment key. See here for more information.