correct usage of rpath (relative vs absolute) - gcc

When building a binary or library, specifying the rpath, i.e.
-Wl,rpath,<path/to/lib>
tells the linker where to find the required library at runtime of the binary.
What is the UNIX philosphy regarding absolute and relative paths here? Is it better to use an absolute path so the lib can be found from everywhere? Or is it better to make it relative so copying an entire directory or renaming a higher level path won't render the binary unusable?
Update
Using $ORIGIN is usually the preferred way of building binaries. For libraries I like to put in the absolute path, because otherwise you won't be able to link to the library. A symbolic link will change the $ORIGIN to point to the path of the link and not of the link target.

In the case of rpath, it makes no sense to use a relative path, since a relative path will be relative to the current working directory, NOT relative to the directory where the binary/library was found. So it simply won't work for executables found in $PATH or libraries in pretty much any case.
Instead, you can use the $ORIGIN "special" path to have a path relative to the executable with-Wl,-rpath,'$ORIGIN' -- note that you need quotes around it to avoid having the shell interpret it as a variable, and if you try to do this in a Makefile, you need $$ to avoid having make interpret the $ as well.

What is the UNIX philosphy regarding absolute and relative paths here?
Using relative path makes an executable that only works when invoked from a particular directory, which is almost never what you want. E.g. if the executable is in /app/foo/bin/exe and has DT_RUNPATH of lib/, and a dependent library is in /app/foo/lib/libfoo.so, then the exe would only run when invoked from /app/foo, and not when invoked from any other directory.
Using absolute path is much better: you can do cd /tmp; /app/foo/bin/exe and have the executable still work. This is still however less than ideal: you can't easily have multiple versions of the binary (important during development), and you dictate to end-users where they must install the package.
On systems that support $ORIGIN, using DT_RUNPATH of $ORIGIN/../lib would give you an executable what works when installed anywhere and invoked from any directory, so long as relative paths to bin/ and lib/ are preserved.

From The Linux Programming Interface:
Using $ORIGIN in rpath
Suppose that we want to distribute an application that uses some of
its own shared libraries, but we don’t want to require the user to
install the libraries in one of the standard directories. Instead, we
would like to allow the user to unpack the application under an
arbitrary directory of their choice and then immediately be able to
run the application. The problem is that the application has no way of
determining where its shared libraries are located, unless it requests
the user to set LD_LIBRARY_PATH or we require the user to run some
sort of installation script that identifies the required directories.
Neither of these approaches is desirable. To get around this
problem, the dynamic linker is built to understand a special string,
$ORIGIN (or, equivalently, ${ORIGIN}), in an rpath
specification. The dynamic linker interprets this string to mean “the
directory containing the application.” This means that we can, for
example, build an application with the following command:
$ gcc -Wl,-rpath,'$ORIGIN'/lib ...
This presumes that at run time the application’s shared libraries will
reside in the subdirectory lib under the directory that contains the
application executable. We can then provide the user with a simple
installation package that contains the application and associated
libraries, and the user can install the package in any location and
then run the application (i.e., a so-called “turn-key application”).

Related

Appimage problems

I would like to release my program that wrote in ruby language, I need to pack ruby to appimage file and send it to my client ubuntu PC first.
so I create the folder "ruby-img", then copy my compiled ruby which in "/app/ruby" folder to "ruby-img/app/ruby" and then made a link as "ln -r -s app/ruby/bin/ruby usr/bin/." in "ruby-img" folder.
then I create the desktop file and put png file to "ruby-img", using appimagetool to create ruby-x86_64.AppImage. sadly it can not run, AFAIK that ruby.AppImage still using /app/ruby/lib folder to find some library of ruby but not in "ruby-img/app/ruby/lib" related folder.
so I tried re-compile ruby as --prefix=/tmp/ruby or --prefix=/usr/local/ruby, then copy them to "ruby-img/usr/local/ruby" or "ruby-img/tmp/ruby" then maka some link as above, and repack to AppImage but ruby.AppImage still not working...
any idea can help me ?
AppImages contain of a filesystem with all the content you provide plus a small executable stub that will mount the AppImage filesystem, then run the AppRun executable to be found inside.
With that knowledge it is utmost important that you provide an executable in the root directory along with the .desktop and icon files. I suggest you do not create AppRun yourself. Use the precompiled one from https://github.com/AppImage/AppImageKit/releases/tag/continuous (do not forget to rename it to exactly 'AppRun').
Now when this AppRun gets invoked, it will perform a few checks, cd into the /usr directory and try to start the executable specified in the .desktop file. Check it's source code and you can see that it also sets a few environment variables.
Therefore it is best you provide your entrypoint as /usr/bin/ruby.sh and register that in the desktop file. Remember if /usr/bin/ruby.sh gets called, the current work directory is /usr. So ruby.sh can set further environment variables such as LD_LIBRARY_PATH so that the libraries you configured for /usr/lib will actually be loaded.
With that I hope you have at least as much success as I had.

How to get the name of GNU install directories outside of CMake or make?

I'm using CMake to install software as defined by GNUInstallDirs which in turn are supposed to follow these standards.
It turns out that these are not entirely uniform across distributions however - libdir becomes lib under ubuntu whereas under alpine it resolves to lib64.
I need to reference these directories outside of CMake in a portable manner - specifically I'm adding a path containing libdir to $PYTHONPATH in a bash script.
How can I find the actual directory name that libdir is resolving to on the current system within bash?
Criteria using which the module GNUInstallDirs chooses between lib and lib64 are described in the module itself:
# Override this default 'lib' with 'lib64' iff:
# - we are on Linux system but NOT cross-compiling
# - we are NOT on debian
# - we are on a 64 bits system
# reason is: amd64 ABI: https://github.com/hjl-tools/x86-psABI/wiki/X86-psABI
# For Debian with multiarch, use 'lib/${CMAKE_LIBRARY_ARCHITECTURE}' if
# CMAKE_LIBRARY_ARCHITECTURE is set (which contains e.g. "i386-linux-gnu"
# and CMAKE_INSTALL_PREFIX is "/usr"
# See http://wiki.debian.org/Multiarch
In a simple form, your Python script could check:
Whether it runs on 64-bit Linux.
Whether it runs on Debian-based OSes.
If the first check is true but the second is false, then libdir is lib64/.
Otherwise libdir is lib/.
Alternative. Searching for the library
Your script could check whether lib/ or lib64/ contains a specific library, and for $PYTHONPATH chose the directory which actually contains a library.
This approach has an advantage, that it is safe against (future) changing rules in GNUInstallDirs module for select between lib/ and lib64/.
Alternative. Shipping your installation with an information about installing dirs
When install your project, you could also create a file which contains value of CMAKE_INSTALL_FULL_LIBDIR variable, so this value could be extracted by your Python script. As for the file which contains this information, it could be stored under predefined directory: e.g. datarootdir is always a share/.
This approach has an advantage, that it works even when the value for libdir differs from both lib/ and lib64/.
To add a further alternative to Tsyvarev's answer:
Return the dir name from CMake on demand
At time of referring to the installed path, we can get the specific directory by printing the variable from within CMake itself:
echo "include(GNUInstallDirs)\nmessage(\${CMAKE_INSTALL_LIBDIR})\n" > /tmp/gnuinstall &&
cmake -P /tmp/gnuinstall

Consistent builds / remove personal information from binaries

I've now realized that Go saves absolute paths to source code in binaries for the purpose of printing stack-traces and the likes. I don't want to completely remove this information, however, this also means that every developer building the same program will produce an executable with a different checksum. Before I try to reimplement the build using chroot or something like that: isn't there any way to tell Go not to use absolute paths for this purpose?
I know it doesn't directly address what you asked, but #JimB's suggestion does indicate a class of solutions to the problem you seem to be having.
One of the easier ones (I think) would be to have your developers install Docker and create an alias so that the go command runs:
docker run --rm --tty --volume $GOPATH:/go golang:1.7.1(-$YOUR_PLATFORM) go
Then, every build (and test and run) thinks it's using a GOPATH of /go and your developers' checksums won't disagree based on that.
See here for more info.
isn't there any way to tell Go not to use absolute paths for this purpose?
Nowadays there is: -trimpath.
https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies explains:
-trimpath
remove all file system paths from the resulting executable.
Instead of absolute file system paths, the recorded file names
will begin either a module path#version (when using modules),
or a plain import path (when using the standard library, or GOPATH).

how to set path in bashrc for multiple locations

I am installing same library (maybe with different release versions) at two different locations. Now I am exporting the path in bashrc for both. In linux which one is taken if I call the library in some program?
For example:
mylib_version1 is installed in /home/PATH1/lib ,
mylib_version2 is installed in /home/PATH2/lib
in bashrc I do,
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/PATH1/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/PATH2/lib
which path is actually taken by some other program when calling this library? How does the ordering work in bashrc?
Similarly what happens when PATH1 is just /usr/local/lib (which I don't export in bashrc)
and PATH2 is some user define path.
What I experience for some program is, if I install it in /usr/local/bin
and if I install using some prefix like /home/PATH/bin and export it in bashrc like
export PATH=$PATH:/home/PATH/bin
it always takes from /usr/local/bin.
If I understand you correctly, you have library.so with 2 versions and you have a binary that might use library.so version 1 or 2.
To deal with that problem you must first understand the meaning of libraries version mechanism. All libraries should be placed in the same place and you might have something like that:
/usr/lib/library.so.1.0.0
/usr/lib/library.so.2.0.0
Your binary would be linked to the correct library based on the API used and linking during the build process.
Please read more information regarding libraries here

Relative or independent paths in libtool .la file

My .la file has full pathnames in both the dependency_libs= section and the libdir= section which makes it difficult to copy my libraries to a different machine (same arch but different path structure). What is the solution to this, besides having some script to hack the .la file to adjust for the paths on the new machine?
==Details==
When I ./configure; make; make install my libfoo, depending on how I use the --prefix, --exec-prefix, and DESTDIR= flags, I'll get an entry in the libfoo.la file that reads libdir=/dir1/lib and I'll have the actual .so files in the same dir as libfoo.la. All is well (in terms of linking something with libfoo as it is) until I package these up and put them on another machine.
Let's say I have libbar on my second machine which depends on libfoo. When I use my -L/dir2/lib flag to look for -lfoo, the libbar compilation/linking fails because of the libfoo.la file is expecting foo to be installed in /dir1/lib (from the first machine) when it's actually in /dir2/lib. I then need to replace all of the dir1's with the correct path, both of which can be long and complicated.
The dependency_libs= line also comes into play in a similar manner.
How do I avoid this problem?

Resources