Docker doesn't see a binary in PATH inside container - bash

I'm trying to build a Docker image to help automatize code generation from Google Protocol Buffers spec files.
My current Dockerfile:
FROM alpine:latest
WORKDIR /protobuf
VOLUME /dst
RUN apk update \
&& apk add --no-cache bash \
&& apk add --virtual .build-deps curl unzip \
&& curl -k -L https://github.com/google/protobuf/releases/download/v3.6.0/protoc-3.6.0-linux-x86_64.zip -o protoc.zip \
&& unzip protoc.zip \
&& rm protoc.zip \
&& chmod +x ./bin/protoc \
&& apk --purge del .build-deps
COPY . spec/
RUN chmod +x spec/entrypoint.sh
ENV SRC_DIR=/usr/src/protoc/spec DST_DIR=/dst PATH=/protobuf/bin:${PATH}
ENTRYPOINT [ "/bin/bash" ]
When I build and run it with TTY enabled, it runs without issue. However, shell inside the container is somehow unable to see the /protobuf/bin/protoc binary, even though it should be able to:
bash-4.4# echo $PATH
/protobuf/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
bash-4.4# ls -l /protobuf/bin
total 4444
-rwxr-xr-x 1 root root 4548312 Jun 15 23:40 protoc
bash-4.4# protoc
bash: /protobuf/bin/protoc: No such file or directory
What can be the issue here?

protobuf is linked against glibc library and alpine uses musl (and older alpine used uclibc). They are not compatible. The error you are seeing comes from the linker not finding the libc standard library /lib64/ld-linux-x86-64.so.2. You can check that, by issuing ldd command:
$ ldd /bin/protoc
/lib64/ld-linux-x86-64.so.2 (0x7faa1a641000)
libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7faa1a641000)
libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7faa1a641000)
libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7faa1a641000)
Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /bin/protoc)
The file exists and the PATH is ok.
To fix this you may install glibc on alpine (not recommended) or just move to a normal linux container.
#edit: My old answer was not true, bash nor sh does not source /etc/profile by default, only when supplied with -l option.

Related

How to build raspbian with custom kernel 5.10 (PREEMPT RT)

I am trying to enable PREEMPT RT (Fully preemptive model) in Linux kernel 5.10. However, I get a black screen when booting from the custom kernel image and I have no clue why that happens. Any help is greatly appreciated. Here are all my steps:
All below files are available at https://github.com/remusmp/rpi-rt-kernel. Just clone and run make.
I build with docker. Here is my Dockerfile:
FROM ubuntu:20.04
ENV TZ=Europe/Copenhagen
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt-get update
RUN apt-get install -y git make gcc bison flex libssl-dev bc ncurses-dev kmod
RUN apt-get install -y crossbuild-essential-arm64
RUN apt-get install -y wget zip unzip fdisk nano
WORKDIR /rpi-kernel
RUN git clone https://github.com/raspberrypi/linux.git -b rpi-5.10.y --depth=1
RUN wget https://mirrors.edge.kernel.org/pub/linux/kernel/projects/rt/5.10/patch-5.10.59-rt52.patch.gz
WORKDIR /rpi-kernel/linux/
ENV KERNEL=kernel8
ENV ARCH=arm64
ENV CROSS_COMPILE=aarch64-linux-gnu-
RUN gzip -cd ../patch-5.10.59-rt52.patch.gz | patch -p1 --verbose
RUN make bcm2711_defconfig
ADD .config ./
RUN make Image modules dtbs
WORKDIR /raspios
RUN apt -y install
RUN wget https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2021-05-28/2021-05-07-raspios-buster-armhf-lite.zip
RUN unzip 2021-05-07-raspios-buster-armhf-lite.zip && rm 2021-05-07-raspios-buster-armhf-lite.zip
RUN mkdir /raspios/mnt && mkdir /raspios/mnt/disk && mkdir /raspios/mnt/boot
ADD build.sh ./
ADD config.txt ./
Script build.sh:
#!/bin/sh
mount -t ext4 -o loop,offset=$((532480*512)) 2021-05-07-raspios-buster-armhf-lite.img /raspios/mnt/disk
mount -t vfat -o loop,offset=$((8192*512)),sizelimit=$((524288*512)) 2021-05-07-raspios-buster-armhf-lite.img /raspios/mnt/boot
cd /rpi-kernel/linux/
make INSTALL_MOD_PATH=/raspios/mnt/disk modules_install
make INSTALL_DTBS_PATH=/raspios/mnt/boot dtbs_install
cd -
cp /rpi-kernel/linux/arch/arm64/boot/Image /raspios/mnt/boot/$KERNEL\_rt.img
cp /rpi-kernel/linux/arch/arm64/boot/dts/broadcom/*.dtb /raspios/mnt/boot/
cp /rpi-kernel/linux/arch/arm64/boot/dts/overlays/*.dtb* /raspios/mnt/boot/overlays/
cp /rpi-kernel/linux/arch/arm64/boot/dts/overlays/README /raspios/mnt/boot/overlays/
cp /raspios/config.txt /raspios/mnt/boot/
touch /raspios/mnt/boot/ssh
umount /raspios/mnt/disk
umount /raspios/mnt/boot
zip 2021-05-07-raspios-buster-armhf-lite.zip 2021-05-07-raspios-buster-armhf-lite.img
I got a custom .config file, which I copied from the container after running make menuconfig and enabling Fully PREEMPT RT. I had to disable KVM first in order for Full PREEMPT RT option to show up but that is ok for my use case. This is the only customization I do to the kernel, nothing else.
The outcome is a zipped sd card image that I then decompress and dd to an sdcard with command:
sudo dd if=2021-05-07-raspios-buster-armhf-lite.img of=/dev/mmcblk0 bs=1M status=progress
Am I missing something in the above process or where should I look to get a working custom Linux kernel image? I followed the guide on https://www.raspberrypi.org/documentation/computers/linux_kernel.html and it works fine if I set the vanilla kernel8.img in config.txt but my custom kernel with preempt rt enabled doesn't work.
Many thanks!
It almost worked. I was missing:
arm_64bit=1
in config.txt.

Singularity 3.6.2 Installation

I have problems with installation of singularity 3.6.2 in linux mint, I followed the instructions of https://sylabs.io/guides/3.0/user-guide/installation.html.
I installed the dependencies and Go.
Then I run the command for install the latest version:
export VERSION=3.6.2 && # adjust this as necessary \
mkdir -p $GOPATH/src/github.com/sylabs && \
cd $GOPATH/src/github.com/sylabs && \
wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \
tar -xzf singularity-${VERSION}.tar.gz && \
cd ./singularity && \
./mconfig
The error is:
Configuring for project `singularity' with languages: C, Golang
=> running pre-basechecks project specific checks ...
=> running base system checks ...
checking: host C compiler... cc
checking: host C++ compiler... c++
checking: host Go compiler (at least version 1.13)... not found!
mconfig: could not complete configuration
I have go (go version)
go version go1.15.2 linux/amd64
I don't know what happend!
Thanks so much!
I was struggling with the same error. All the suggestions say that probably you have an older version of Go and that's why. But turned out it's even more important to place Go and Singularity in the right locations.
I found these docs https://github.com/hpcng/singularity/blob/release-3.5/INSTALL.md are the most useful and correct about where to put what in terms of directories.
The key is to clone Singularity in a directory which is GOPATH:
You won't have this directory by default so create it first
$ mkdir -p ${GOPATH}/src/github.com/sylabs && \
cd ${GOPATH}/src/github.com/sylabs && \
git clone https://github.com/sylabs/singularity.git && \
cd singularity
Make sure your singularity is here: {GOPATH}/src/github.com/sylabs/singularity
To summarize:
The Go itself is located here /usr/local/go
GOPATH would be something like home/your_username/go and the singularity will be located inside in e.g. home/your_username/go/src/github.com/sylabs/singularity
The issue was reported in 5099.
# 5320 also mentions:
I deleted the PPO python 3.6 and this worked fine!
Make sure nothing is executed as root, which would have a $PATH different from your current user.
If someone faces this issue, follow this installation guide.
sudo apt-get update && \
sudo apt-get install -y build-essential \
libseccomp-dev pkg-config squashfs-tools cryptsetup
sudo rm -r /usr/local/go
export VERSION=1.13.15 OS=linux ARCH=amd64 # change this as you need
wget -O /tmp/go${VERSION}.${OS}-${ARCH}.tar.gz https://dl.google.com/go/go${VERSION}.${OS}-${ARCH}.tar.gz && \
sudo tar -C /usr/local -xzf /tmp/go${VERSION}.${OS}-${ARCH}.tar.gz
echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
source ~/.bashrc
curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh |
sh -s -- -b $(go env GOPATH)/bin v1.21.0
mkdir -p ${GOPATH}/src/github.com/sylabs && \
cd ${GOPATH}/src/github.com/sylabs && \
git clone https://github.com/sylabs/singularity.git && \
cd singularity
git checkout v3.6.3
cd ${GOPATH}/src/github.com/sylabs/singularity && \
./mconfig && \
cd ./builddir && \
make && \
sudo make install
singularity version

Dockerfile - ARG SHA and Curl

I am newbie to Docker. I can create a docker image for Java and Maven from https://github.com/carlossg/docker-maven/blob/master/jdk-13/Dockerfile . I can understand most of the commands there inside dockerfile, there are some that I could not find sufficient info on net. Can someone please help me ?
(1) What does below ARG SHA do. If I understand it right, SHA is immutable identifier that is associated with image, so I am downloading image with that identifier, I mean specific image with changes I want and stored with that SHA, is this right?
ARG SHA=c35a1803a6e70a126e80b2b3ae33eed961f83ed74d18fcd16909b2d44d7dada3203f1ffe726c17ef8dcca2dcaa9fca676987befeadc9b9f759967a8cb77181c0
(2) I know what RUN, echo does and how the variable works. But not sure what is happening below with curl command . No idea what below lines of code does for sure.
RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
&& curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
&& echo "${SHA} /tmp/apache-maven.tar.gz" | sha512sum -c - \
&& tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
&& rm -f /tmp/apache-maven.tar.gz \
&& ln -s /usr/share/maven/bin/mvn /usr/bin/mvn```
You have to read it like a shell script.
1.
SHA is SHA512 hash
function used in line 10 to
check if downloaded /tmp/apache-maven.tar.gz is what we expect. It
has nothing to do with Docker image ID, if you mean that. You can
reproduce the check locally on your system:
$ SHA=c35a1803a6e70a126e80b2b3ae33eed961f83ed74d18fcd16909b2d44d7dada3203f1ffe726c17ef8dcca2dcaa9fca676987befeadc9b9f759967a8cb77181c0
$ BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries
$ curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz
$ echo "${SHA} /tmp/apache-maven.tar.gz" | sha512sum -c -
/tmp/apache-maven.tar.gz: OK
(Notice that $ here is a command line
prompt
used to indicate start of a new line, not a part of the
command).
curl here downloads
https://apache.osuosl.org/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
to /tmp/apache-maven.tar.gz.
2.
Again, read it like a shell script. && is used for chaining commands and \ is used to concatenate lines.
RUN mkdir -p /usr/share/maven /usr/share/maven/ref
Create /usr/share/maven and /usr/share/maven/ref directories.
curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz
Download temporary apache-maven tarball to /tmp/apache-maven.tar.gz.
echo "${SHA} /tmp/apache-maven.tar.gz" | sha512sum -c -
Check if the downloaded tarball has the correct checksum.
tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1
Extract /tmp/apache-maven.tar.gz to /usr/share/maven.
rm -f /tmp/apache-maven.tar.gz
Remove temporary tarball after extracting it.
ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
Create /usr/bin/mvn that points to /usr/share/maven/bin/mvn. This
is done because /usr/bin directory is typically in $PATH so that
mvn can be run without providing a full path to it.

Why does "docker run" error with "no such file or directory"?

I am trying to run a container which runs an automated build. Here is the dockerfile:
FROM ubuntu:14.04
MAINTAINER pmandayam
# update dpkg repositories
RUN apt-get update
# install wget
RUN apt-get install -y wget
# get maven 3.2.2
RUN wget --no-verbose -O /tmp/apache-maven-3.2.2.tar.gz http://archive.apache.or
g/dist/maven/maven-3/3.2.2/binaries/apache-maven-3.2.2-bin.tar.gz
# verify checksum
RUN echo "87e5cc81bc4ab9b83986b3e77e6b3095 /tmp/apache-maven-3.2.2.tar.gz" | md5
sum -c
# install maven
RUN tar xzf /tmp/apache-maven-3.2.2.tar.gz -C /opt/
RUN ln -s /opt/apache-maven-3.2.2 /opt/maven
RUN ln -s /opt/maven/bin/mvn /usr/local/bin
RUN rm -f /tmp/apache-maven-3.2.2.tar.gz
ENV MAVEN_HOME /opt/maven
# remove download archive files
RUN apt-get clean
# set shell variables for java installation
ENV java_version 1.8.0_11
ENV filename jdk-8u11-linux-x64.tar.gz
ENV downloadlink http://download.oracle.com/otn-pub/java/jdk/8u11-b12/$filename
# download java, accepting the license agreement
RUN wget --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie
" -O /tmp/$filename $downloadlink
# unpack java
RUN mkdir /opt/java-oracle && tar -zxf /tmp/$filename -C /opt/java-oracle/
ENV JAVA_HOME /opt/java-oracle/jdk$java_version
ENV PATH $JAVA_HOME/bin:$PATH
# configure symbolic links for the java and javac executables
RUN update-alternatives --install /usr/bin/java java $JAVA_HOME/bin/java 20000 &
& update-alternatives --install /usr/bin/javac javac $JAVA_HOME/bin/javac 20000
# install mongodb
RUN echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen
' | sudo tee /etc/apt/sources.list.d/mongodb.list && \
apt-get update && \
apt-get --allow-unauthenticated install -y mongodb-org mongodb-org-s
erver mongodb-org-shell mongodb-org-mongos mongodb-org-tools && \
echo "mongodb-org hold" | dpkg --set-selections && \
echo "mongodb-org-server hold" | dpkg --set-selections && \
echo "mongodb-org-shell hold" | dpkg --set-selections &&
\
echo "mongodb-org-mongos hold" | dpkg --set-selectio
ns && \
echo "mongodb-org-tools hold" | dpkg --set-selec
tions
RUN mkdir -p /data/db
VOLUME /data/db
EXPOSE 27017
COPY build-script /build-script
CMD ["/build-script"]
I can build the image successfully but when I try to run the container I get this error:
$ docker run mybuild
no such file or directory
Error response from daemon: Cannot start container 3e8aa828909afcd8fb82b5a5ac894
97a537bef2b930b71a5d20a1b98d6cc1dd6: [8] System error: no such file or directory
what does it mean 'no such file or directory'?
Here is my simple script:
#!/bin/bash
sudo service mongod start
mvn clean verify
sudo service mongod stop
I copy it like this: COPY build-script /build-script
and run it like this: CMD ["/build-script"] not sure why its not working
Using service isn't going to fly - the Docker base images are minimal and don't support this. If you want to run multiple processes, you can use supervisor or runit etc.
In this case, it would be simplest just to start mongo manually in the script e.g. /usr/bin/mongod & or whatever the correct incantation is.
BTW the lines where you try to clean up don't have much effect:
RUN rm -f /tmp/apache-maven-3.2.2.tar.gz
...
# remove download archive files
RUN apt-get clean
These files have already been committed to a previous image layer, so doing this doesn't save any disk-space. Instead you have to delete the files in the same Dockerfile instruction in which they're added.
Also, I would consider changing the base image to a Java one, which would save a lot of work. However, you may have trouble finding one which bundles the official Oracle JDK rather than OpenJDK if that's a problem.

Make 2 volumes in docker gcc and java

I have a docker volume set up in a gcc container. I need to compile the code and give it some stdin through a text file. I'm able to do that using the following command.
docker run \
-v /home/usr/workspace/proj/WebContent/files:/mycode \
gcc:4.9 \
sh -c 'cd mycode; gcc -o myapp ./mycode.c; ./myapp < ./test.txt'
now my question is, I need to make a separate folder for each of my users with their username, but the text.txt stays in the same folder as above. How do I give them their own paths. because right now i get an error that test.txt is not found, and of course it wouldn't. I tried making a separate volume for the test.txt but I guess making two volumes in one container isn't possible or I'm doing it wrong.
What I've tried (please don't judge, i'm just learning :P)
docker run \
-v /home/usr/workspace/proj/WebContent/file/username:/mycode \
-v /home/usr/workspace/proj/WebContent/file/:/tst \
gcc:4.9 \
sh -c 'cd mycode; gcc -o myapp ./mycode.c; cd tst; ./myapp < ./test.txt'
Have a close look at this command:
sh -c 'cd mycode; gcc -o myapp ./mycode.c; cd tst; ./myapp < ./test.txt'
Because you first cd mycode and then cd tst inside the same shell, you're trying to cd mycode/tst, whereas your mount is at /tst.
Similarly, myapp is in /mycode, not in tst, so you can't run ./myapp inside the tst directory and expect it to work.
Instead:
sh -xc 'cd /mycode && gcc -o myapp ./mycode.c && cd /tst && /mycode/myapp < ./test.txt'

Resources