This file: MoltenVK Runtime User Guide offers a nice description of how to link MoltenVK to a project that uses it, within XCode.
I prefer using makefiles, as it removes the need to boot up XCode to build (+ is a consistent workflow with the same project on Linux).
The problem is there's quite a bit of OSX/XCode specific terminology in the instructions, and I'm not quite sure how it translates to running clang via command line.
If you assume I'm looking to link only with the minimum specified to use MoltenVK, what would such a command line argument look like?
Here is my minimal attempt:
MOLTENVKDIR = /blah/vulkansdk-macos-1.1.108.0
a.out: my_headers.h my_src_files.cpp
clang -I$(MOLTENVKDIR)/MoltenVK/include -L$(MOLTENVKDIR)/MoltenVK/macOS/dynamic -lMoltenVK my_src_files.cpp
More specific questions:
How should I be setting LD_RUNPATH_SEARCH_PATHS? I assume this is a PATH-style variable that gets embedded in the executable with a list of places to look for the MoltenVK.dylib file?
In step 4 of the user guide, it says "drag (MoltenVK/macOS/dynamic/libMoltenVK.dylib) to the Link Binary With Libraries list"- is that essentially the -lMoltenVK line?
Where does step 5 come in? Should I append -framework Metal -framework Foundation -framework ...? What should I do with the .tbd file?
Is step 6 just ensuring that I copy the libMoltenVK.dylib file relative to a.out consistent with LD_RUNPATH_SEARCH_PATHS?
Is step 7 safe to ignore, as I'm not using XCode?
TLDR; I use g++, use the flags -lMoltenVK and -L$(MOLTENVK_DIR), and at the top define VK_ICD_FILENAMES using export VK_ICD_FILENAMES=$(ICD_DIR)
Header Search Paths is -I$(PATH)
Library Search Paths is -L$(PATH)
Runpath Search Paths is -rpath $(PATH) (with the space)
What I did was copy the .dylibs into a folder called lib in my project folder, set Library Search Path with -Llib, and link them with -lvulkan and -lMoltenVK. Since these dynamic libraries are hardwired to use #rpath when linking, I set -rpath lib.
You can see the vulkan tutorial for linux which uses a Makefile which shows a very basic version of making a Makefile for Vulkan on linux (no MoltenVK)
Most importantly, to not get a VkResult -9 when calling vkCreateInstance (MoltenVK not installed properly) make sure to define the VK_ICD_FILENAMES environment variable! I did this by writing export VK_ICD_FILENAMES=lib/MoltenVK_icd.json near the top of my Makefile, and copied the MoltenVK_icd.json file to my project! This is a file which points to where the dylib is, and is necesarry for MoltenVK to work. When installing MoltenVK, it was inside the same folder as libMoltenVK.dylib
Similarily, you must define VK_LAYER_PATH for validation layers to work.
Minimally, all you need for MoltenVK is:
export VK_ICD_FILENAMES=lib/MoltenVK_icd.json
main:
g++ test.cpp -o App -Llib -rpath lib -lMoltenVK
I'm not an expert in best practices and conventions, I do what works. Hope this helped
Related
I'm a student doing research involving extending the TM capabilities of gcc. My goal is to make changes to gcc source, build gcc from the modified source, and, use the new executable the same way I'd use my distro's vanilla gcc.
I built and installed gcc in a different location (not /usr/bin/gcc), specifically because the modified gcc will be unstable, and because our project goal is to compare transactional programs compiled with the two different versions.
Our changes to gcc source impact both /gcc and /libitm. This means we are making a change to libitm.so, one of the shared libraries that get built.
My expectation:
when compiling myprogram.cpp with /usr/bin/g++, the version of libitm.so that will get linked should be the one that came with my distro;
when compiling it with ~/project/install-dir/bin/g++, the version of libitm.so that will get linked should be the one that just got built when I built my modified gcc.
But in reality it seems both native gcc and mine are using the same libitm, /usr/lib/x86_64-linux-gnu/libitm.so.1.
I only have a rough grasp of gcc internals as they apply to our project, but this is my understanding:
Our changes tell one compiler pass to conditionally insert our own "function builtin" instead of one it would normally use, and this is / becomes a "symbol" which needs to link to libitm.
When I use the new gcc to compile my program, that pass detects those conditions and successfully inserts the symbol, but then at runtime my program gives a "relocation error" indicating the symbol is not defined in the file it is searching in: ./test: relocation error: ./test: symbol _ITM_S1RU4, version LIBITM_1.0 not defined in file libitm.so.1 with link time reference
readelf shows me that /usr/lib/x86_64-linux-gnu/libitm.so.1 does not contain our new symbols while ~/project/install-dir/lib64/libitm.so.1 does; if I re-run my program after simply copying the latter libitm over the former (backing it up first, of course), it does not produce the relocation error anymore. But naturally this is not a permanent solution.
So I want the gcc I built to use the shared libs that were built along with it when linking. And I don't want to have to tell it where they are every time - my feeling is that it should know where to look for them since I deliberately built it somewhere else to behave differently.
This sounds like the kind of problem any amateur gcc developer would have when trying to make a dev environment and still be able to use both versions of gcc, but I had difficulty finding similar questions. I am thinking this is a matter of lacking certain config options when I configure gcc before building it. What is the right configuration to do this?
My small understanding of the instructions for building and installing gcc led me to do the following:
cd ~/project/
mkdir objdir
cd objdir
../source-dir/configure --enable-languages=c,c++ --prefix=/home/myusername/project/install-dir
make -j2
make install
I only have those config options because they seemed like the ones closest related to "only building the parts I need" and "not overwriting native gcc", but I could be wrong. After the initial config step I just re-run make -j2 and make install every time I change the code. All these steps do complete without errors, and they produce the ~/project/install-dir/bin/ folder, containing the gcc and g++ which behave as described.
I use ~/project/install-dir/bin/g++ -fgnu-tm -o myprogram myprogram.cpp to compile a transactional program, possibly with other options for programs with threads.
(I am using Xubuntu 16.04.3 (64 bit), within VirtualBox on Windows. The installed /usr/bin/gcc is version 5.4.0. Our source at ~/project/source-dir/ is a modified version of 5.3.0.)
You’re running into build- versus run-time linking differences. When you build with -fgnu-tm, the compiler knows where the library it needs is found, and it tells the linker where to find it; you can see this by adding -v to your g++ command. However when you run the resulting program, the dynamic linker doesn’t know it should look somewhere special for the ITM library, so it uses the default library in /usr/lib/x86_64-linux-gnu.
Things get even more confusing with ITM on Ubuntu because the library is installed system-wide, but the link script is installed in a GCC-private directory. This doesn’t happen with the default GCC build, so your own GCC build doesn’t do this, and you’ll see libitm.so in ~/project/install-dir/lib64.
To fix this at run-time, you need to tell the dynamic linker where to find the right library. You can do this either by setting LD_LIBRARY_PATH (to /home/.../project/install-dir/lib64), or by storing the path in the binary using -Wl,-rpath=/home/.../project/install-dir/lib64 when you build it.
According to the manual page for ld (and gcc used for linking by extension), if a -L option appears on the command line, it applies to all libraries specified by -l and takes precedence over the default search locations. However, that is not working in my link step. I have this on the command line:
-L /users/me/mylib -lpcre -lz
and /users/me/mylib contains (copies) of libpcre.so and libz.so
These libraries exist in other locations on the system (although not necessarily the same versions) and what I see (with ldd on Linux and otool on Mac) is a path that references the libraries in those locations. Some of those locations are on the LD_LIBRARY_PATH (which I cannot control in the build environment I am running in) and it appears that somehow those locations are being picked up in preference to my explicit setting with -L.
Just to be clear, this a link step problem and not a runtime problem. There is a lot of info on the web on how to affect/override library locations when executing and I am familiar with all that. In some sense what I am trying to do with the -L is create a completely specified setup. I know I can fix things up with install_name_tool on MacOS but I'd really like to understand why -L isn't doing what it claims to.
One thing I learned using gcc -Wl,-v is that gcc appears to forward all the LD_LIBRARY_PATH directories to ld. However, it places them after the ones explicitly listed by me and man ld says they are searched in order they appear on the line.
Just to be clear, this a link step problem and not a runtime problem.
From what you describe as the problem, I don't think you are right about this - it sounds like a runtime problem for which you are (justifiably) looking for a solution that you can employ while linking that will solve the problem you have at runtime.
The reason I say it does not appear to be a problem with linking is that it sounds like your linking is working as it is intended. LD (or GCC) are not complaining about the linking, and your linked executables are being produced just fine. The issue you are having is that when you subsequently go to run those executables, the loader is finding libraries other than the ones you intend. The purpose of the -L flag during linking is to let the linker know where it can find suitable libraries to use in preparing the linked binary. That is completely separate from where the loader will search for the required libraries at runtime.
As you say, you are already aware that there are ways you could employ at runtime (such as changing LD_LIBRARY_PATH) that would avoid the issue by changing the set of paths that the loader searches for libraries, but you'd rather not have to do that because for whatever reason you won't necessarily have control over the runtime environment, which is fair enough.
Luckily, there is a facility that I believe will get you what you want. Take a look at the ld option called -rpath (see the GNU ld man page for full documentation). Basically, if you add paths during linking using the -rpath option, those paths gets stored in the linked executable as preferred locations to find the libraries at runtime, in much the same way they would be searched if included in LD_LIBRARY_PATH. This should work on Linux or Mac OS X (at least since 10.5).
Passing the -rpath option to ld via gcc requires using the -Wl option to pass the flag through. To obtain an ld command line that contains ld -rpath /custom/path/to/libs requires a gcc invocation something like: gcc -Wl,-rpath,/custom/path/to/libs
In short, try replacing what you currently have: -L/users/me/mylib -lpcre -lz
With: -L/users/me/mylib -Wl,-rpath,/users/me/mylib -lpcre -lz
The resulting executable (or library) will then have /users/me/mylib stored as the place to go to find libraries, and it should find libpcre.so and libz.so there without needing to control LD_LIBRARY_PATH.
We know that the command for making the executable is :
gcc -L/home/username/foo -Wall -o test main.c -lfoo
But if run ./test ,it will give error :
/test
./test: error while loading shared libraries: libfoo.so: cannot open shared object file: No such file or directory
There are 2 solutionss for it:
1)
LD_LIBRARY_PATH=/home/username/foo
2)
gcc -L/home/username/foo -Wl,-rpath=/home/username/foo -Wall -o test main.c -lfoo
my question is why do need to provide the library path to the loader??While making the execuatble I have explicilty mentioned the path such that the linker could create the excutable and I beleive there must be a refernce of the location of the library embedded in th execuatble
This a conceptual doubt i have ,plz clear it out.Thank you
It's just a choice that the linker designers made -- but a good one.
The reason it is good is that it lets you build a library, link against it, and then install the executable and library without requiring a re-link.
The cost is that you have to use LD_LIBRARY_PATH or the like to run the executable in the build tree; but normally this is no problem, as you can add the needed setting to make check or what have you.
If the default were flipped, then you would have to either relink at install time (which is a bit unfriendly since it is reasonably common to make install as root); or provide a flag to turn off the behavior at build time (but then you're back to the above...).
-Wl,rptath: Its actually build the executable feeding the information that where the loader will check for the library for loading.
Lets me give an example with 2 cases:
1)
If I transfer the library and executable to another machine and place them in 2 different location then I need to mention the path where the library is located by setting LD_LIBRARY_PATH.Then the loader will understand where to look for the loader while executing.
2)If i make the make the executable with -Wl,rpath= then I need to place the execuatble in the same path in another machine as the value of -Wl,rpath=.
So in simple terms we can say that we have the independency of placing the library with LD_LIBRARY_PATH
My main aim is to get the GSL Shell working on my OSX 10.7 system. So far I have the correct version of lua with the correct patches running. I also have a working version of GSL which compiles and runs example programs. I can build agg perfectly and also run their example programs by running make in the macosx_sdl folder.
My first question is how on earth do I create my own project with agg? I know that you are supposed to simply add the files to your project file and go, but this does not seem to want to compile for me. Is it simply a case of adding the include directory and the libagg.a?
Finally, how do I build gsl shell? Currently it complains about the agg-plot folder a lot, so where do I put the agg files to make this build, then when i've done it where do I place the agg files so that the lua scripts can get to them?!
Hope someone can help!
In general to use the AGG library you need to make sure that the compiler is able to find the headers files and, during the linking, the libraries, either in form of a static or dynamic libraries.
To make the headers files and the libraries available you need to take into account the system that is used to build the software. If a traditional makefile sistem is used you need to add some flags to make sure that the headers file can be found. This can be achieved by adding into the makefile something like:
CFLAGS += -I/path/to/agg/headers
and for the linker:
LIBS += -L/path/to/agg/library -lagg -lm
In the specific case of GSL Shell 1.1 the file "make-packages" is used in the Makefile to configure the required packages. You can add here the flags required to locate the AGG library:
AGG_INCLUDES = -I/usr/include/agg2
AGG_LIBS = -lagg -lX11 -lpthread -lsupc++
you should just modify the path provided with the "-I" option and, in AGG_LIBS, add an option "-L/path/to/agg/library" to specify the path where the AGG libraries are located.
Please note also that the agg libraries depends on other libraries. For example on linux it needs at least the X11 library. The libraries supc++ may be needed if the linking is made by invoking gcc instead of g++ because if gcc is used the C++ runtime libraries are not included.
I've set LIBRARY_SEARCH_PATHS to /opt/local/lib, and verified that the library in question is there (I'm linking to GLEW):
$ls /opt/local/lib
libGLEW.1.5.1.dylib libfreetype.a libz.a
libGLEW.1.5.dylib libfreetype.dylib libz.dylib
libGLEW.a libfreetype.la pkgconfig
libGLEW.dylib libz.1.2.3.dylib
libfreetype.6.dylib libz.1.dylib
but Xcode gives me the linker error
library not found for -lGLEW
I'm generating the Xcode project with CMake, so I don't want to explicitly modify the Xcode project (if someone suggests adding it as a framework, or something like that). Xcode recognizes USER_HEADER_SEARCH_PATHS fine (as in this question); why doesn't it work here?
Perhaps adding something like this to your CMakeLists.txt?
find_library(GLEW_LIB GLEW /opt/local/lib)
if(NOT ${GLEW_LIB})
message(FATAL_ERROR "Could not find GLEW")
endif()
target_link_libraries(myprogram ${GLEW_LIB} ...)
Where myprogram is the name of the target executable that needs to link with the library. You would replace the ... with the other libraries you are using on that executable.
This way CMake would handle the library path details for you.
Xcode works on potentially multiple SDK's, so whenever your define these kinds of things (like HEADER_SEARCH_PATHS or LIBRARY_SEARCH_PATHS) the current SDK root is prepended to the actual path that's getting passed to the linker.
So, one way to make this work would be to add your directory to the SDK. For example, assuming you're building with the Mac OS X 10.5 sdk, you could add your opt dir:
ln -s /opt /Developer/SDKs/MacOSX10.5.sdk/opt
Your library would now be found on your system.
If you don't want to do this, then you will have to look at CMake and find out how to get it to generate a library requirement for your actual library (I don't know anything about CMake, so I can't help you there). This is also why you see a difference between USER_HEADER_SEARCH_PATHS and HEADER_SEARCH_PATHS re your other question.
As another option, you could also specify this path with the OTHER_LDFLAGS build variable:
OTHER_LDFLAGS=-L/opt/local/lib
This would cause the linker to search /opt/local/lib as well as its standard paths and wouldn't require you to generate a different project file.