create symbolic link in bitbake recipe - symlink

I have a .bbappend recipe that I need to create a symbolic link in my system.
This is how it looks like now:
bernardo#bernardo-ThinkCentre-Edge72:~/yocto/genericx86-64-rocko-18.0.0/meta-datavision/recipes-devtools/oracle-java$ cat oracle-jse-jdk_1.7.0.bbappend
FILES_${PN} += "/lib64/ld-linux-x86-64.so.2"
do_install_append() {
install -d ${D}/lib64
ln -s ${D}/lib/ld-2.26.so ${D}/lib64/ld-linux-x86-64.so.2
}
However, only the directory /lib64 is created in the sysroot. The symlink /lib64/ld-linux-x86-64.so.2 is not being generated.
What changes should I make in my recipe in order to have this symlink correctly created?

The cleanest solution is to use the "-r" flag:
do_install_append() {
install -d ${D}/lib64
ln -s -r ${D}/lib/ld-2.26.so ${D}/lib64/ld-linux-x86-64.so.2
}
From the gnu ln man page:
-r, --relative create symbolic links relative to link location

Try to avoid usage of absolute paths:
do_install_append() {
install -d ${D}/lib64
cd ${D}/lib64
ln -s ../lib/ld-2.26.so ld-linux-x86-64.so.2
}

For Yocto 2.3 to 3.4:
do_install_append() {
install -d ${D}/lib64
lnr ${D}/lib/ld-2.26.so ${D}/lib64/ld-linux-x86-64.so.2
}
For Yocto 4.0+ (or if your host system has coreutils 8.16+):
do_install_append() {
install -d ${D}/lib64
ln --relative --symbolic ${D}/lib/ld-2.26.so ${D}/lib64/ld-linux-x86-64.so.2
}
Alternatively, you can also inherit relative_symlinks which will turn any absolute symlinks into relative ones, but this is less commonly used than lnr.
Cf:
Yocto 4.0 Migration Guide (Thanks to #parsley72 for the info!)
Yocto 2.3 Migration Guide

I had a look at how other recipes create links in the rootfs, and most seem to do it this way:
ln -sf /data/etc/bluetooth/main.conf ${D}/${sysconfdir}/bluetooth/main.conf
This command in the recipe will create the following link on the device:
/# ls -al /etc/bluetooth/main.conf
lrwxrwxrwx 1 root root 29 Sep 11 15:34 /etc/bluetooth/main.conf -> /data/etc/bluetooth/main.conf
You use the full, Yocto-generated path when creating the link, but you make it point to the "final" location in the rootfs.
This way you can use "absolute" paths and won't have to change the working directory in the recipe.

As of 2022-01-19 this seems to be the only way to get it to work (adapt to the filenames needed):
do_install() {
install -d ${D}${libdir}
install -m 0644 ${S}/libmine.so.0 ${D}${libdir}/
lnr ${D}${libdir}/libmine.so.0 ${D}${libdir}/libmine.so
}
FILES_${PN} += " \
${libdir}/libmine.so.0 \
${libdir}/libmine.so \
"
FILES_SOLIBSDEV = ""
INSANE_SKIP_${PN} += "dev-so"

You can do:
ln -s ../lib/ld-2.26.so ${D}/lib64/ld-linux-x86-64.so.2
or if you don't require the symbolic-link until after your target system has booted up (i.e. it's not a dependency of other packages you are building) you can also do:
ln -s /lib/ld-2.26.so ${D}/lib64/ld-linux-x86-64.so.2
ln doesn't care if your target is valid or exists when a symbolic-link is created. It will become valid after you boot your target-system (or somehow mount this filesystem to /). But indeed, relative links are recommended.

do_install_append () {
install -d 0755 ${D}/dir
install -d 0755 ${D}/dir/subdir
cd ${D}/dir/subdir
ln -sf /source_so_the_symbilic_link <name_of_the_symbolic_link>
}
FILES_${PN} += "/dir"

Related

Building and Installing BZip2 with shared library dylib on MacOS

'Trying to build/install BZip2 on MacOS (10.13.4 - High Sierra), but all the instructions I've been able to find [including from the README] have me at the following:
wget -c https://sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz
tar -zxf bzip2-1.0.8.tar.gz
cd bzip2-1.0.8
sudo make install PREFIX=/usr/local
This installs the package without the shared library file with .dylib extension in the ./lib directory. The included instructions in the package are for Linux environment.
How do I install the shared libraries?
It turns out that doing this requires an update to the Makefile.
The package [bzip2 v1.0.8] comes with a Makefile-libbz2_so file that is for creating shared library files for Linux. To do same for MacOS a seperate Makefile is required.
Below are the updated instructions which worked. Follow the link for Makefile-libbz2_dylib for contents of the Makefile.
# Download BZip2
wget -c https://sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz
# Extract and enter directory
tar -zxf bzip2-1.0.8.tar.gz
cd bzip2-1.0.8
# create variable
export PREFIX="/usr/local"
# install - change PREFIX is required
sudo make install PREFIX=$PREFIX
# Make dynamic libraries for MACOS
wget -c https://gist.githubusercontent.com/obihill/3278c17bcee41c0c8b59a41ada8c0d35/raw/3bf890e2ad40d0af358e153395c228326f0b44d5/Makefile-libbz2_dylib
make -f Makefile-libbz2_dylib
# Do below only if your PREFIX is not /usr/local
# Create symlinks for bin
sudo ln -s $PREFIX/bin/bunzip2 /usr/local/bin/
sudo ln -s $PREFIX/bin/bzcat /usr/local/bin/
sudo ln -s $PREFIX/bin/bzcmp /usr/local/bin/
sudo ln -s $PREFIX/bin/bzdiff /usr/local/bin/
sudo ln -s $PREFIX/bin/bzegrep /usr/local/bin/
sudo ln -s $PREFIX/bin/bzfgrep /usr/local/bin/
sudo ln -s $PREFIX/bin/bzgrep /usr/local/bin/
sudo ln -s $PREFIX/bin/bzip2 /usr/local/bin/
sudo ln -s $PREFIX/bin/bzip2recover /usr/local/bin/
sudo ln -s $PREFIX/bin/bzless /usr/local/bin/
sudo ln -s $PREFIX/bin/bzmore /usr/local/bin/
# Create symlinks for lib
sudo ln -s $PREFIX/lib/libbz2.a /usr/local/lib/
sudo ln -s $PREFIX/lib/libbz2.dylib /usr/local/lib/
# Create symlinks for include
sudo ln -s $PREFIX/include/bzlib.h /usr/local/include/
If you don't have wget on your Mac, you can find instructions to set it up on this community post.
Inspiration for Makefile from here and here.

Script is running perfectly on co-workers devices but gives me 'Invalid cross-device link'

My script is running perfectly on co-workers devices (MacOSX with Docker Desktop same as me), but gives me every time the same error and it does not move or only half, the libraries in the deps directory:
OSError: [Errno 18] Invalid cross-device link: '/tmp/pip-target-dzwe_2kc/lib/python/numpy' ->
'/foo/python/numpy'
My script :
#!/bin/bash
export PKG_DIR='python'
export SIDE_DEPS_DIR='deps'
rm -rf ${PKG_DIR} && mkdir -p ${PKG_DIR}
rm -rf ${SIDE_DEPS_DIR} && mkdir -p ${SIDE_DEPS_DIR}
docker run --rm -v $(pwd):/foo -w /foo lambci/lambda:build-python3.8 \
pip3 install -r requirements.txt -t ${PKG_DIR}
# move stuff to deps
find /${PKG_DIR} -maxdepth 1 -type d \
\( -name "pandas*" -o -name "numpy*" -o -name "numpy.libs*" -o -name "scipy*" -o -name "scipy.libs*" \) -exec mv '{}' ${SIDE_DEPS_DIR} \;
# zip side dependencies
zip -r ge_deps.zip deps
# zip layer
zip -r layers-python38-great-expectations.zip python
It's a script which uses a public lambda docker image to create a lambda layer (basically a zip that contains libraries) and which removes unwanted libraries to put them in another folder deps.
The above code will use the public Docker image lambci / lambda and will install in the empty python directory, libraries which come from a python package which is called 'great-expectations' and which helps to test pipelines of data (which is specified in requirements.txt and is great-expectations==0.12.7)
I have been stuck with this problem for a while and have not found a solution.
Had this exact problem just now.
/tmp and /foo are different devices - /tmp is within the docker OS and /foo is mapped to your local OS.
pip seems to be using shutil.rename() to move the built package from tmp to the final output location (/foo). This fails because they are different devices. Ideally pip would use shutil.move() instead, which will deal with a cross-device move.
As a workaround, you can change the temp folder used by PIP by setting TMPDIR before invoking the pip command. i.e. export TMPDIR=/foo/tmp before calling pip in the docker image. So, the whole command might be something like
docker run --rm -v $(pwd):/foo -w /foo lambci/lambda:build-python3.8 \
/bin/bash -c "export TMPDIR=/foo/tmp && pip3 install -r requirements.txt -t ${PKG_DIR}"
(multiple commands soln taken from https://www.edureka.co/community/10736/how-to-run-multiple-commands-in-docker-at-once - open to better suggestions!)
This will likely be slower because it's using the local OS for temp files, but it avoids the attempted 'rename' across devices from the temp folder to the final output folder.

Singularity 3.6.2 Installation

I have problems with installation of singularity 3.6.2 in linux mint, I followed the instructions of https://sylabs.io/guides/3.0/user-guide/installation.html.
I installed the dependencies and Go.
Then I run the command for install the latest version:
export VERSION=3.6.2 && # adjust this as necessary \
mkdir -p $GOPATH/src/github.com/sylabs && \
cd $GOPATH/src/github.com/sylabs && \
wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \
tar -xzf singularity-${VERSION}.tar.gz && \
cd ./singularity && \
./mconfig
The error is:
Configuring for project `singularity' with languages: C, Golang
=> running pre-basechecks project specific checks ...
=> running base system checks ...
checking: host C compiler... cc
checking: host C++ compiler... c++
checking: host Go compiler (at least version 1.13)... not found!
mconfig: could not complete configuration
I have go (go version)
go version go1.15.2 linux/amd64
I don't know what happend!
Thanks so much!
I was struggling with the same error. All the suggestions say that probably you have an older version of Go and that's why. But turned out it's even more important to place Go and Singularity in the right locations.
I found these docs https://github.com/hpcng/singularity/blob/release-3.5/INSTALL.md are the most useful and correct about where to put what in terms of directories.
The key is to clone Singularity in a directory which is GOPATH:
You won't have this directory by default so create it first
$ mkdir -p ${GOPATH}/src/github.com/sylabs && \
cd ${GOPATH}/src/github.com/sylabs && \
git clone https://github.com/sylabs/singularity.git && \
cd singularity
Make sure your singularity is here: {GOPATH}/src/github.com/sylabs/singularity
To summarize:
The Go itself is located here /usr/local/go
GOPATH would be something like home/your_username/go and the singularity will be located inside in e.g. home/your_username/go/src/github.com/sylabs/singularity
The issue was reported in 5099.
# 5320 also mentions:
I deleted the PPO python 3.6 and this worked fine!
Make sure nothing is executed as root, which would have a $PATH different from your current user.
If someone faces this issue, follow this installation guide.
sudo apt-get update && \
sudo apt-get install -y build-essential \
libseccomp-dev pkg-config squashfs-tools cryptsetup
sudo rm -r /usr/local/go
export VERSION=1.13.15 OS=linux ARCH=amd64 # change this as you need
wget -O /tmp/go${VERSION}.${OS}-${ARCH}.tar.gz https://dl.google.com/go/go${VERSION}.${OS}-${ARCH}.tar.gz && \
sudo tar -C /usr/local -xzf /tmp/go${VERSION}.${OS}-${ARCH}.tar.gz
echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
source ~/.bashrc
curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh |
sh -s -- -b $(go env GOPATH)/bin v1.21.0
mkdir -p ${GOPATH}/src/github.com/sylabs && \
cd ${GOPATH}/src/github.com/sylabs && \
git clone https://github.com/sylabs/singularity.git && \
cd singularity
git checkout v3.6.3
cd ${GOPATH}/src/github.com/sylabs/singularity && \
./mconfig && \
cd ./builddir && \
make && \
sudo make install
singularity version

rpmbuild - /usr/sbin Symlink not installing

I'm setting up a Redis RPM for a local, unnetworked box. I'm trying to create a symlink: /usr/sbin/redis-server -> /opt/redis/redis-server
However, when I do an rpm -Uvh redis-3.2.7-1.rpm, it installed fine to /opt/redis/redis-server but never creates the symlink. Here's the relevant part of my spec file:
%build
# Empty section.
%install
rm -rf %{buildroot}
rm -f /usr/sbin/redis-server
mkdir -p %{buildroot}
# in builddir
cp -a * %{buildroot}
ln -sf /opt/redis/redis-server /usr/sbin/redis-server
%clean
rm -rf %{buildroot}
%files
/opt/redis/*
/etc/init.d/redis
ln -sf /opt/redis/redis-server /usr/sbin/redis-server needs to be ln -sf /opt/redis/redis-server %{buildroot}/usr/sbin/redis-server and then /usr/sbin/redis-server needs to be added to the %files section. Also remove that rm in %install.
The fact that the ln did not fail tells me you really made the symlink, and you're building RPMs as root which is a spectacularly bad idea.
I'm assuming that the tarball expands with opt at the top level; if not your cp is incorrect as well.

How to install directory structure recursively in OpenEmbedded BitBake recipe?

I'd like to simplify a BitBake recipe that installs a large directory structure by using some sort of recursive install routine rather than calling install many times. The source directory layout is frequently changing during development, which is causing far more recipe revisions than I want to deal with.
As an example, how would the following do_install() be simplified from this:
do_install() {
install -d ${D}/foo
install -m 0644 ${S}/foo/*.* ${D}/foo
install -d ${D}/foo/a
install -m 0644 ${S}/foo/a/*.* ${D}/foo/a
install -d ${D}/foo/b
install -m 0644 ${S}/foo/b/*.* ${D}/foo/b
install -d ${D}/foo/c
install -m 0644 ${S}/foo/c/*.* ${D}/foo/c
install -d ${D}/bar
install -m 0644 ${S}/bar/*.* ${D}/bar
install -d ${D}/bar/a
install -m 0644 ${S}/bar/a/*.* ${D}/bar/a
install -d ${D}/bar/a/bananas
install -m 0644 ${S}/bar/a/bananas/*.* ${D}/bar/a/bananas
}
To something more like this pseudocode:
do_install() {
for each subdir in ${S}/foo/
install subdir recursively to ${D}/foo/subdir
end
for each subdir in ${S}/bar/
install subdir recursively to ${D}/bar/subdir
end
}
The top-level directories in our source files (foo & bar in the above example) rarely change, so calling them out in the recipe is fine. It's the lower level directories that are frequently changing.
It may be that cp -r ends up being the way to go, but I believe I've read that it's frowned upon in BitBake recipes, so I'm wondering if BitBake provides some alternate mechanism, or if there's
some other reasonably standardized way to do this.
We used to do it in that way:
do_install() {
find ${WORKDIR}/ -type f -exec 'install -m 0755 "{}" ${D}/var/www/' \;
}
The canonical form in OE is
cp -R --no-dereference --preserve=mode,links -v SOURCE DESTINATION
see the answer here (while they looks a bit different in code, the questions are semantically equivalent)

Resources