How to build raspbian with custom kernel 5.10 (PREEMPT RT) - linux-kernel

I am trying to enable PREEMPT RT (Fully preemptive model) in Linux kernel 5.10. However, I get a black screen when booting from the custom kernel image and I have no clue why that happens. Any help is greatly appreciated. Here are all my steps:
All below files are available at https://github.com/remusmp/rpi-rt-kernel. Just clone and run make.
I build with docker. Here is my Dockerfile:
FROM ubuntu:20.04
ENV TZ=Europe/Copenhagen
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt-get update
RUN apt-get install -y git make gcc bison flex libssl-dev bc ncurses-dev kmod
RUN apt-get install -y crossbuild-essential-arm64
RUN apt-get install -y wget zip unzip fdisk nano
WORKDIR /rpi-kernel
RUN git clone https://github.com/raspberrypi/linux.git -b rpi-5.10.y --depth=1
RUN wget https://mirrors.edge.kernel.org/pub/linux/kernel/projects/rt/5.10/patch-5.10.59-rt52.patch.gz
WORKDIR /rpi-kernel/linux/
ENV KERNEL=kernel8
ENV ARCH=arm64
ENV CROSS_COMPILE=aarch64-linux-gnu-
RUN gzip -cd ../patch-5.10.59-rt52.patch.gz | patch -p1 --verbose
RUN make bcm2711_defconfig
ADD .config ./
RUN make Image modules dtbs
WORKDIR /raspios
RUN apt -y install
RUN wget https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2021-05-28/2021-05-07-raspios-buster-armhf-lite.zip
RUN unzip 2021-05-07-raspios-buster-armhf-lite.zip && rm 2021-05-07-raspios-buster-armhf-lite.zip
RUN mkdir /raspios/mnt && mkdir /raspios/mnt/disk && mkdir /raspios/mnt/boot
ADD build.sh ./
ADD config.txt ./
Script build.sh:
#!/bin/sh
mount -t ext4 -o loop,offset=$((532480*512)) 2021-05-07-raspios-buster-armhf-lite.img /raspios/mnt/disk
mount -t vfat -o loop,offset=$((8192*512)),sizelimit=$((524288*512)) 2021-05-07-raspios-buster-armhf-lite.img /raspios/mnt/boot
cd /rpi-kernel/linux/
make INSTALL_MOD_PATH=/raspios/mnt/disk modules_install
make INSTALL_DTBS_PATH=/raspios/mnt/boot dtbs_install
cd -
cp /rpi-kernel/linux/arch/arm64/boot/Image /raspios/mnt/boot/$KERNEL\_rt.img
cp /rpi-kernel/linux/arch/arm64/boot/dts/broadcom/*.dtb /raspios/mnt/boot/
cp /rpi-kernel/linux/arch/arm64/boot/dts/overlays/*.dtb* /raspios/mnt/boot/overlays/
cp /rpi-kernel/linux/arch/arm64/boot/dts/overlays/README /raspios/mnt/boot/overlays/
cp /raspios/config.txt /raspios/mnt/boot/
touch /raspios/mnt/boot/ssh
umount /raspios/mnt/disk
umount /raspios/mnt/boot
zip 2021-05-07-raspios-buster-armhf-lite.zip 2021-05-07-raspios-buster-armhf-lite.img
I got a custom .config file, which I copied from the container after running make menuconfig and enabling Fully PREEMPT RT. I had to disable KVM first in order for Full PREEMPT RT option to show up but that is ok for my use case. This is the only customization I do to the kernel, nothing else.
The outcome is a zipped sd card image that I then decompress and dd to an sdcard with command:
sudo dd if=2021-05-07-raspios-buster-armhf-lite.img of=/dev/mmcblk0 bs=1M status=progress
Am I missing something in the above process or where should I look to get a working custom Linux kernel image? I followed the guide on https://www.raspberrypi.org/documentation/computers/linux_kernel.html and it works fine if I set the vanilla kernel8.img in config.txt but my custom kernel with preempt rt enabled doesn't work.
Many thanks!

It almost worked. I was missing:
arm_64bit=1
in config.txt.

Related

Docker Build Image -- cant cd into directory and run commands

Docker Version: 17.09.1-ce
I am beginner in docker and I am trying to build docker image on centos. The below is the snippet of docker file i am having
FROM centos
RUN yum -y install samba-common && \
yum -y install gcc perl mingw-binutils-generic mingw-filesystem-base mingw32-binutils mingw32-cpp mingw32-crt mingw32-filesystem mingw32-gcc mingw32-headers mingw64-binutils mingw64-cpp mingw64-crt mingw64-filesystem mingw64-gcc mingw64-headers libcom_err-devel popt-devel zlib-devel zlib-static glibc-devel glibc-static python-devel && \
yum -y install git gnutls-devel libacl1-dev libacl-devel libldap2-dev openldap-devel && \
yum -y remove libbsd-devel && \
WORKDIR /usr/src && \
git clone git://xxxxxxxx/p/winexe/winexe-waf winexe-winexe-wafgit && \
WORKDIR /usr/src/samba && \
WORKDIR /usr/src/winexe-winexe-wafgit/source && \
head -n -3 wscript_build > tmp.txt && cp -f tmp.txt wscript_build && \
echo -e '\t'"stlib='smb_static bsd z resolv rt'", >> wscript_build && \
echo -e '\t'"lib='dl gnutls'", >> wscript_build && \
echo -e '\t'")" >> wscript_build && \
rm -rf tmp.txt && \
./waf --samba-dir=../../samba configure build
I tried with the normal cd which not work. WORKDIR does not work. How I can set working directory in Dockerfile?
I am getting an error like below using the above Dockerfile
/bin/sh: WORKDIR: command not found
The command '/bin/sh -c yum -y install samba-common && yum -y install gcc perl mingw-binutils-generic mingw-filesystem-base mingw32-binutils mingw32-cpp mingw32-crt mingw32-filesystem mingw32-gcc mingw32-headers mingw64-binutils mingw64-cpp mingw64-crt mingw64-filesystem mingw64-gcc mingw64-headers libcom_err-devel popt-devel zlib-devel zlib-static glibc-devel glibc-static python-devel && yum -y install git gnutls-devel libacl1-dev libacl-devel libldap2-dev openldap-devel && yum -y remove libbsd-devel && WORKDIR /usr/src && git clone git://xxxxxxxx/p/winexe/winexe-waf winexe-winexe-wafgit && WORKDIR /usr/src/samba && git reset --hard a6bda1f2bc85779feb9680bc74821da5ccd401c5 && WORKDIR /usr/src/winexe-winexe-wafgit/source && head -n -3 wscript_build > tmp.txt && cp -f tmp.txt wscript_build && echo -e '\t'"stlib='smb_static bsd z resolv rt'", >> wscript_build && echo -e '\t'"lib='dl gnutls'", >> wscript_build && echo -e '\t'")" >> wscript_build && rm -rf tmp.txt && ./waf --samba-dir=../../samba configure build' returned a non-zero code: 127
When I tried with normal cd instead work WORKDIR then I got below error
/bin/sh: line 0: cd: /usr/src/samba: No such file or directory but with sudo i can go into it. Then I tried to include sudo cd directory in docker file then it said no sudo found
UPDATE 1:
This is how I started build
sudo docker build -t abwinexeimage -f ./abwinexeimage . The build got successfully but unfortunately when i list images i dont see any image with tag namme of abwinexeimage.
I dont understand what is that first entry with tag name as none. what it represents ? it shows size of 1.23 GB, Do I really need this image or can i safely delete ?
When I started build first line showed that Sending build context to Docker daemon 303.9MB that means in that image list repository named centos with tag name latest is one the right image which I built ? I assuming so as the size says 202 MB ?
Then I issued docker ps, but no container running, then issued docker ps -a to see stopped containers
Then I tried to run image as container..
Now tried to issue docker ps to check whether container is running
Now i can tell you why i am so concerned about multiple containers present. Actually I wanted to manually CD into cd /usr/src/samba this inside docker container to verify if changes done via docker file got updated correctly or not. Now since i have multiple containers, really not sure which container I need to look into. In that stunt, i tried to start all containers, then manually issue
docker exec -it CONTAINER_NAME [bash | sh] to verify if i am able to find that file system there. This is the reason why I asked whether I can have single container so that i can easily find the file system there,my understanding is since multiple RUN statements created different layers, then its difficult for me to find in which container my file system resides, so that I can CD into it.. sorry for big explanation... I am trying to understand concepts better. Your comments please..
You need to use WORKDIR as a Dockerfile instruction, instead of using it together with run instruction.
RUN has 2 forms:
RUN (shell form, the command is run in a shell, which by default is /bin/sh -c on Linux or cmd /S /C on Windows) RUN
["executable", "param1", "param2"] (exec form)
WORKDIR
WORKDIR /path/to/workdir The WORKDIR instruction sets the working
directory for any RUN, CMD, ENTRYPOINT, COPY and ADD instructions that
follow it in the Dockerfile
FROM centos
RUN yum -y install samba-common && \
yum -y install gcc perl mingw-binutils-generic mingw-filesystem-base mingw32-binutils mingw32-cpp mingw32-crt mingw32-filesystem mingw32-gcc mingw32-headers mingw64-binutils mingw64-cpp mingw64-crt mingw64-filesystem mingw64-gcc mingw64-headers libcom_err-devel popt-devel zlib-devel zlib-static glibc-devel glibc-static python-devel && \
yum -y install git gnutls-devel libacl1-dev libacl-devel libldap2-dev openldap-devel && \
yum -y remove libbsd-devel
WORKDIR /usr/src
#use git clone with RUN not with WORKDIR
RUN git clone git://xxxxxxxx/p/winexe/winexe-waf winexe-winexe-wafgit
#So start it with new line
WORKDIR /usr/src/samba
WORKDIR /usr/src/winexe-winexe-wafgit/source
#start RUN with new line
RUN head -n -3 wscript_build > tmp.txt && cp -f tmp.txt wscript_build && \
echo -e '\t'"stlib='smb_static bsd z resolv rt'", >> wscript_build && \
echo -e '\t'"lib='dl gnutls'", >> wscript_build && \
echo -e '\t'")" >> wscript_build && \
rm -rf tmp.txt && \
./waf --samba-dir=../../samba configure build

How can I use systemtap(stap) in the centos container in docker for mac

When I use stap, it tell me this error as the image,how can i fix it.
Really don't know why this question has been downvotted once. When i get concern, downvotter are one of the reason i always hesitate to ask thing on stackoverflow...
So, got the same concern with a container under ubuntu and it is how i fix it : try to launch an ubuntu bash on docker (not centos but the step should be similar)
docker run -it --cap-add sys_admin ubuntu bash
Then install some library and systemtap
apt-get update
apt-get --yes install build-essential cmake bc curl diffutils git kmod libcurl4-openssl-dev wget
apt-get --yes install systemtap systemtap-sdt-dev
Now install the kernel headers (will take time)
export KERNELVER=$(uname -r | cut -d '-' -f 1)
export KERNELDIR=/linux-$KERNELVER
cd /
curl -o linux-${KERNELVER}.tar.gz https://www.kernel.org/pub/linux/kernel/v4.x/linux-${KERNELVER}.tar.gz
tar zxf linux-${KERNELVER}.tar.gz
cd linux-${KERNELVER}
zcat /proc/1/root/proc/config.gz > .config
make all
make modules_prepare
make headers_install
make modules_install
Now run stap -l 'syscall.*'. It should work and display :
...
syscall.waitpid
syscall.write
syscall.writev
This is taken from : https://github.com/fdebonneval/mobydig/blob/master/resources/build

How do I add an updated ghostscript to AWS EC2 Instance with Amazon Linux AMI

How can I get 9.20 - the current version?
(I'm doing the research and will answer as I finish.)
This answer is an update to the now stale answer here, with a few extra details and a reference to the current repo location.
This may be obvious to the more experienced, but this is intended to help those less experienced with CLI/make/gcc who just need gs.
First set up gcc-c++ if you haven't already [Are all packages necessary? Last two seem to be not needed.]:
sudo yum install -y gcc gcc-c++ compat-gcc-32 compat-gcc-32-c++
Then download, make and install ghostscript:
wget https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs920/ghostscript-9.20.tar.gz
tar -zxvf ghostscript-9.20.tar.gz
cd ghostscript-9.20
./configure --prefix=/usr --enable-dynamic --disable-compile-inits --with-system-libtiff
make
make so
sudo make install
sudo chmod go+w /usr/include/ghostscript/
sudo make soinstall && install -v -m644 base/*.h /usr/include/ghostscript && sudo ln -v -s ghostscript /usr/include/ps
sudo ln -sfv ../ghostscript/9.20/doc /usr/share/doc/ghostscript-9.20
cd ..
wget http://sourceforge.net/projects/gs-fonts/files/latest/download?source=files --output-document=ghostscript-fonts-std-8.11.tar.gz
sudo tar -xvf ghostscript-fonts-std-8.11.tar.gz -C /usr/share/ghostscript
fc-cache -v /usr/share/ghostscript/fonts/
sudo mkdir /usr/include/ghostscript/
sudo chmod go-w /usr/include/ghostscript/
ghostscript -v
gs -v
You could put the source code on the instance and compile it.....
Met an error when ran
sudo chmod go+w /usr/include/ghostscript/
because the folder didn't exist yet.
So I did minor adjustment to the command order:
wget https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs920/ghostscript-9.20.tar.gz
tar -zxvf ghostscript-9.20.tar.gz
cd ghostscript-9.20
./configure --prefix=/usr --enable-dynamic --disable-compile-inits --with-system-libtiff
make
make so
sudo make install
sudo mkdir /usr/include/ghostscript/
sudo chmod go+w /usr/include/ghostscript/
sudo make soinstall && install -v -m644 base/*.h /usr/include/ghostscript && sudo ln -v -s ghostscript /usr/include/ps
sudo ln -sfv ../ghostscript/9.20/doc /usr/share/doc/ghostscript-9.20
cd ..
wget http://sourceforge.net/projects/gs-fonts/files/latest/download?source=files --output-document=ghostscript-fonts-std-8.11.tar.gz
sudo tar -xvf ghostscript-fonts-std-8.11.tar.gz -C /usr/share/ghostscript
fc-cache -v /usr/share/ghostscript/fonts/
sudo chmod go-w /usr/include/ghostscript/
ghostscript -v
gs -v

Why does "docker run" error with "no such file or directory"?

I am trying to run a container which runs an automated build. Here is the dockerfile:
FROM ubuntu:14.04
MAINTAINER pmandayam
# update dpkg repositories
RUN apt-get update
# install wget
RUN apt-get install -y wget
# get maven 3.2.2
RUN wget --no-verbose -O /tmp/apache-maven-3.2.2.tar.gz http://archive.apache.or
g/dist/maven/maven-3/3.2.2/binaries/apache-maven-3.2.2-bin.tar.gz
# verify checksum
RUN echo "87e5cc81bc4ab9b83986b3e77e6b3095 /tmp/apache-maven-3.2.2.tar.gz" | md5
sum -c
# install maven
RUN tar xzf /tmp/apache-maven-3.2.2.tar.gz -C /opt/
RUN ln -s /opt/apache-maven-3.2.2 /opt/maven
RUN ln -s /opt/maven/bin/mvn /usr/local/bin
RUN rm -f /tmp/apache-maven-3.2.2.tar.gz
ENV MAVEN_HOME /opt/maven
# remove download archive files
RUN apt-get clean
# set shell variables for java installation
ENV java_version 1.8.0_11
ENV filename jdk-8u11-linux-x64.tar.gz
ENV downloadlink http://download.oracle.com/otn-pub/java/jdk/8u11-b12/$filename
# download java, accepting the license agreement
RUN wget --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie
" -O /tmp/$filename $downloadlink
# unpack java
RUN mkdir /opt/java-oracle && tar -zxf /tmp/$filename -C /opt/java-oracle/
ENV JAVA_HOME /opt/java-oracle/jdk$java_version
ENV PATH $JAVA_HOME/bin:$PATH
# configure symbolic links for the java and javac executables
RUN update-alternatives --install /usr/bin/java java $JAVA_HOME/bin/java 20000 &
& update-alternatives --install /usr/bin/javac javac $JAVA_HOME/bin/javac 20000
# install mongodb
RUN echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen
' | sudo tee /etc/apt/sources.list.d/mongodb.list && \
apt-get update && \
apt-get --allow-unauthenticated install -y mongodb-org mongodb-org-s
erver mongodb-org-shell mongodb-org-mongos mongodb-org-tools && \
echo "mongodb-org hold" | dpkg --set-selections && \
echo "mongodb-org-server hold" | dpkg --set-selections && \
echo "mongodb-org-shell hold" | dpkg --set-selections &&
\
echo "mongodb-org-mongos hold" | dpkg --set-selectio
ns && \
echo "mongodb-org-tools hold" | dpkg --set-selec
tions
RUN mkdir -p /data/db
VOLUME /data/db
EXPOSE 27017
COPY build-script /build-script
CMD ["/build-script"]
I can build the image successfully but when I try to run the container I get this error:
$ docker run mybuild
no such file or directory
Error response from daemon: Cannot start container 3e8aa828909afcd8fb82b5a5ac894
97a537bef2b930b71a5d20a1b98d6cc1dd6: [8] System error: no such file or directory
what does it mean 'no such file or directory'?
Here is my simple script:
#!/bin/bash
sudo service mongod start
mvn clean verify
sudo service mongod stop
I copy it like this: COPY build-script /build-script
and run it like this: CMD ["/build-script"] not sure why its not working
Using service isn't going to fly - the Docker base images are minimal and don't support this. If you want to run multiple processes, you can use supervisor or runit etc.
In this case, it would be simplest just to start mongo manually in the script e.g. /usr/bin/mongod & or whatever the correct incantation is.
BTW the lines where you try to clean up don't have much effect:
RUN rm -f /tmp/apache-maven-3.2.2.tar.gz
...
# remove download archive files
RUN apt-get clean
These files have already been committed to a previous image layer, so doing this doesn't save any disk-space. Instead you have to delete the files in the same Dockerfile instruction in which they're added.
Also, I would consider changing the base image to a Java one, which would save a lot of work. However, you may have trouble finding one which bundles the official Oracle JDK rather than OpenJDK if that's a problem.

How to provision software using Vagrant without sudo

I'm trying to setup Vagrant virtual machines to support my learning through Seven Databases in Seven Weeks. I'm provisioning software using basic shell scripts which performs appropriate actions within a sudo environment. However, I'm using the vagrant user to run the tutorials, and would like the provisioning to install the appropriate node / NPM modules as Vagrant, rather than through sudo.
My current npm command is the last line in this provisioning script, but the module is unavailable when vagrant tried to execute node scripts.
apt-get update
apt-get -y install build-essential
apt-get -y install tcl8.5
wget http://redis.googlecode.com/files/redis-2.6.0-rc3.tar.gz
tar xzf redis-2.6.0-rc3.tar.gz
cd redis-2.6.0-rc3
make
make install
make test
mkdir /etc/redis
mv redis.conf /etc/redis/redis.conf
sed -i.bak 's/127.0.0.1/0.0.0.0/g' /etc/redis/redis.conf
sed -i.bak 's/daemonize no/daemonize yes/g' /etc/redis/redis.conf
sed -i.bak 's/dir .\//dir \/var\/lib\/redis/g' /etc/redis/redis.conf
cd src/
wget https://raw.github.com/gist/1053791/880a4a046e06028e160055406d02bdc7c57f3615/redis-server
mv redis-server.1 /etc/init.d/redis-server
mv redis-cli /etc/init.d/redis-cli
chmod +x /etc/init.d/redis-server
sed -i.bak 's/DAEMON=\/usr\/bin\/redis-server/DAEMON=\/usr\/local\/bin\/redis-server/g' /etc/init.d/redis-server
useradd redis
mkdir -p /var/lib/redis
mkdir -p /var/log/redis
chown redis.redis /var/lib/redis
chown redis.redis /var/log/redis
update-rc.d redis-server defaults
/etc/init.d/redis-server start
cd /etc/init.d/
echo ./redis-cli
echo http://blog.hemantthorat.com/install-redis-2-6-on-ubuntu/
apt-get -y install python-software-properties python g++ make
add-apt-repository -y ppa:chris-lea/node.js
apt-get update
apt-get -y install nodejs
npm install hiredis redis csv
Simply set privileged to false in your VagrantFile like this:
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
...
config.vm.provision :shell, privileged: false, path: "script.sh"
...
end
The shell provision runs as the root user. If you with to run as the vagrant user, you can do something like this:
sudo -u vagrant npm install hiredis redis
..or for multiple lines:
sudo -u vagrant << EOF
[...]
npm install hiredis
npm install redis
EOF
Maybe use npm install -g to install it globally in the vm?
sed -i 's/.*requiretty$/Defaults !requiretty/' /etc/sudoers

Resources