I have a makefile to install and compile an ocaml project. I need to install ocaml 4.07 to use some specific features of this version. After installing opam, I download the specific version, but in order to compile the project with this version I get a message from the terminal saying
The environment is not in sync with the current switch.
You should run: eval $(opam env)
I want to be able to run make install and compile the project in the specific version.
I tried creating an specific rule for the eval command to be called and tried also to call both eval $(opam env), eval 'opam config env'. But none of these seem to work.
Here is the makefile that I am trying to modify
#install package dependencies
install:
sudo apt install gcc
sudo add-apt-repository ppa:avsm/ppa
sudo apt update
sudo apt install opam
opam init --yes
sudo opam instal ocaml-base-compiler
#opam init --yes
sudo apt install ocaml-nox
#eval 'opam env'
#eval $(opam env)
#ev
#clear
#run
# Building the world
run: ev depend $(EXEC)
ev:
eval $(opam env)
# Clean up
clear:
rm -f *.cm[io] *.cmx *~ .*~ #*#
rm -f $(GENERATED)
rm -f $(EXEC)
How can I compile this project with the specific version I downloaded without having to call the eval command manually in the terminal, or are there other ways to configure this.
The advice to eval something is correct for an interactive shell. Make by default runs each command in a separate shell - it's sort of like you would shut down and restart your computer after each command; the result of eval is immediately lost.
There is a GNU make extension .ONESHELL which allows you to run all lines in a recipe in a single shell, or you can refactor your recipe to do this explicitly:
install:
sudo apt install gcc
# ... ett etc ...
opam init --yes \
&& sudo opam instal ocaml-base-compiler
opam init --yes \
&& sudo apt install ocaml-nox
eval $(opam env) \
&& ev \
&& run
(I had to guess some things about which things need to run in the same shell invocation. I am assuming ev and run are shell commands which need the eval, and that clear is the regular shell command which should probably never be used in a Makefile.)
You can use opam config exec ... to run your build command in the appropriate environment. The build command could be a recursive call to make.
But it seems a bit messy to be installing opam inside your Makefile. Are you sure you want to do that?
Related
Trying to build a docker image with the execution of a pre-requisites installation script inside the Dockerfile fails for fetching packages via apt-get from archive.ubuntu.com.
Using the apt-get command inside the Dockerfile works flawless, despite being behind a corporate proxy, which is setup via the ENV command in the Dockerfile.
Anyway, executing the apt-get command from a bash-script in a terminal inside the resulting docker container or as "postCreateCommand" in a devcontainer.json of Visual Studio Code does work as expected too. But it won't work in my case for the invocation of a bash script from inside a Dockerfile.
It simply will tell:
Starting installation of package iproute2
Reading package lists...
Building dependency tree...
The following additional packages will be installed:
libatm1 libcap2 libcap2-bin libmnl0 libpam-cap libxtables12
Suggested packages:
iproute2-doc
The following NEW packages will be installed:
iproute2 libatm1 libcap2 libcap2-bin libmnl0 libpam-cap libxtables12
0 upgraded, 7 newly installed, 0 to remove and 0 not upgraded.
Need to get 971 kB of archives.
After this operation, 3,287 kB of additional disk space will be used.
Err:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libcap2 amd64 1:2.32-1
Could not resolve 'archive.ubuntu.com'
... more output ...
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libc/libcap2/libcap2_2.32-1_amd64.deb Could not resolve 'archive.ubuntu.com'
... more output ...
Just for example a snippet of the Dockerfile looks like this:
FROM ubuntu:20.04 as builderImage
USER root
ARG HTTP_PROXY_HOST_IP='http://172.17.0.1'
ARG HTTP_PROXY_HOST_PORT='3128'
ARG HTTP_PROXY_HOST_ADDR=$HTTP_PROXY_HOST_IP':'$HTTP_PROXY_HOST_PORT
ENV http_proxy=$HTTP_PROXY_HOST_ADDR
ENV https_proxy=$http_proxy
ENV HTTP_PROXY=$http_proxy
ENV HTTPS_PROXY=$http_proxy
ENV ftp_proxy=$http_proxy
ENV FTP_PROXY=$http_proxy
# it is always helpful sorting packages alpha-numerically to keep the overview ;)
RUN apt-get update && \
apt-get -y upgrade && \
apt-get -y install --no-install-recommends apt-utils dialog 2>&1 \
&& \
apt-get -y install \
default-jdk \
git \
python3 python3-pip
SHELL ["/bin/bash", "-c"]
ADD ./env-setup.sh .
RUN chmod +x env-setup.sh && ./env-setup.sh
CMD ["bash"]
The minimal version of the environment script env-setup.sh, which is supposed to be invoked by the Dockerfile, would look like this:
#!/bin/bash
packageCommand="apt-get";
sudo $packageCommand update;
packageInstallCommand="$packageCommand install";
package="iproute2"
packageInstallCommand+=" -y";
sudo $packageInstallCommand $package;
Of course the usage of variables is down to making use of a list for the packages to be installed and other aspects.
Hopefully that has covered everything essential to the question:
Why is the execution of apt-get working with a RUN and as well running the bash script inside the container after creating, but not from the very same bash script while building the image from a Dockerfile?
I was hoping to find the answer with the help of an extensive web-search, but unfortunately I was only able to find anything but an answer to this case.
As pointed out in the comment section underneath the question:
using sudo to launch the command, wiping out all the current vars set in the current environment, more specifically your proxy settings
So that is the case.
The solution is either to remove sudo from the bash script and invoke the script as root inside the Dockerfile.
Or, using sudo will work with ENV variables, just apply sudo -E.
Right now, I'm trying to integrate a GitHub Action that checks if some code on a pull request compiles properly (VEX Robotics for anyone interested). However, when it gets to running the make command, I get this error:
Building Project
make: Entering directory '/github/workspace/V5'
Adding timestamp [OK]
Creating cold package with libpros,okapilib [ERRORS]
/usr/lib/gcc/arm-none-eabi/6.3.1/../../../arm-none-eabi/bin/ld: unrecognized option '--gc-keep-exported'
/usr/lib/gcc/arm-none-eabi/6.3.1/../../../arm-none-eabi/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
make: *** [bin/cold.package.elf] Error 1
common.mk:200: recipe for target 'bin/cold.package.elf' failed
I'm extremely confused as to why this is occurring? --gc-keep-exported is a real option, and this code compiles perfectly on my local machine. I've tried changing the ubuntu version and updating the VEX SDK to see if it helps, but I keep on getting the same error. What should I do?
Code:
Dockerfile:
FROM ubuntu:18.04
RUN apt-get update
# Install GCC & Clang
RUN apt-get install build-essential -y
RUN apt-get install clang -y
# Install needed ARM deps
RUN apt-get install gcc-arm-none-eabi -y
RUN apt-get install binutils-arm-none-eabi -y
# Install 7z & cURL
RUN apt-get install p7zip-full -y
RUN apt-get install curl -y
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
entrypoint.sh:
echo "Downloading VEX SDK"
# Get VEX SDK and put it in ~/sdk
curl -L https://content.vexrobotics.com/vexcode/v5code/VEXcodeProV5_2_0_1.dmg -o _vexcode_.dmg
7z x _vexcode_.dmg || :
7z x Payload~ ./VEXcode\ Pro\ V5.app/Contents/Resources/sdk -osdk_temp || :
mkdir ~/sdk
mv sdk_temp/VEXcode\ Pro\ V5.app/Contents/Resources/sdk/* ~/sdk
rm -fR _vex*_ _vex*_.dmg sdk_temp/ Payload~
ls ~/sdk # ls just for testing
echo "Building Project"
# Now make the makefile in the set path
make --directory=$1
The reason this was happening was because I used an old version of gcc-arm-none-eabi. The version on apt is super outdated (v6.3.1 vs v10.2.1).
I was able to use the new version by downloading the tarball available on their site and using the direct paths to compile my code.
There is a newer GCC version available for a newer Ubuntu version, you could browse for another one, or be lazy and enjoy some malpractice with me:
FROM ubuntu:latest
I am entirely new to the concept of dockers. I am creating the following Dockerfile as an exercise.
FROM ubuntu:latest
MAINTAINER kesarling
RUN apt update && apt upgrade -y
RUN apt install nginx curl zip unzip -y
RUN apt install openjdk-14-jdk python3 python3-doc clang golang-go gcc g++ -y
RUN curl -s "https://get.sdkman.io" | bash
RUN bash /root/.sdkman/bin/sdkman-init.sh
RUN sdk version
RUN yes | bash -c 'sdk install kotlin'
CMD [ "echo","The development environment has now been fully setup with C, C++, JAVA, Python3, Go and Kotlin" ]
I am using SDKMAN! to install Kotlin. The problem initially was that instead of using RUN bash /root/.sdkman/bin/sdkman-init.sh, I was using RUN source /root/.sdkman/bin/sdkman-init.sh. However, it gave the error saying source not found. So, I tried using RUN . /root/.sdkman/bin/sdkman-init.sh, and it did not work. However, RUN bash /root/.sdkman/bin/sdkman-init.sh seems to work, as in does not give any error and tries to run the next command. However, the docker then gives error saying sdk: not found
Where am I going wrong?
It should be noted that these steps worked like charm for my host distribution (The one on which I'm running docker) which is Pop!_OS 20.04
Actually the script /root/.sdkman/bin/sdkman-init.sh sources the sdk
source is a built-in to bash rather than a binary somewhere on the filesystem.
source command executes the file in the current shell.
Each RUN instruction will execute any commands in a new layer on top of the current image and commit the results.
The resulting committed image will be used for the next step in the Dockerfile.
Try this:
FROM ubuntu:latest
MAINTAINER kesarling
RUN apt update && apt upgrade -y
RUN apt install nginx curl zip unzip -y
RUN apt install openjdk-14-jdk python3 python3-doc clang golang-go gcc g++ -y
RUN curl -s "https://get.sdkman.io" | bash
RUN /bin/bash -c "source /root/.sdkman/bin/sdkman-init.sh; sdk version; sdk install kotlin"
CMD [ "echo","The development environment has now been fully setup with C, C++, JAVA, Python3, Go and Kotlin" ]
SDKMAN in Ubuntu Dockerfile
tl;dr
the sdk command is not a binary but a bash script loaded into memory
Shell sessions are a "process", which means environment variables and declared shell function only exist for the duration that shell session exists; which lasts only as long as the RUN command.
Manually tweak your PATH
RUN apt-get update && apt-get install curl bash unzip zip -y
RUN curl -s "https://get.sdkman.io" | bash
RUN source "$HOME/.sdkman/bin/sdkman-init.sh" \
&& sdk install java 8.0.275-amzn \
&& sdk install sbt 1.4.2 \
&& sdk install scala 2.12.12
ENV PATH=/root/.sdkman/candidates/java/current/bin:$PATH
ENV PATH=/root/.sdkman/candidates/scala/current/bin:$PATH
ENV PATH=/root/.sdkman/candidates/sbt/current/bin:$PATH
Full Version
Oh wow this was a journey to figure out. Below each line is commented as to why certain commands are run.
I learnt a lot about how unix works and how sdkman works and how docker works and why the intersection of the three give very unusual behaviour.
# I am using a multi-stage build so I am just copying the built artifacts
# from this stage to keep final image small.
FROM ubuntu:latest as ScalaBuild
# Switch from `sh -c` to `bash -c` as the shell behind a `RUN` command.
SHELL ["/bin/bash", "-c"]
# Usual updates
RUN apt-get update && apt-get upgrade -y
# Dependencies for sdkman installation
RUN apt-get install curl bash unzip zip -y
#Install sdkman
RUN curl -s "https://get.sdkman.io" | bash
# FUN FACTS:
# 1) the `sdk` command is not a binary but a bash script loaded into memory
# 2) Shell sessions are a "process", which means environment variables
# and declared shell function only exist for
# the duration that shell session exists
RUN source "$HOME/.sdkman/bin/sdkman-init.sh" \
&& sdk install java 8.0.275-amzn \
&& sdk install sbt 1.4.2 \
&& sdk install scala 2.12.12
# Once the real binaries exist these are
# the symlinked paths that need to exist on PATH
ENV PATH=/root/.sdkman/candidates/java/current/bin:$PATH
ENV PATH=/root/.sdkman/candidates/scala/current/bin:$PATH
ENV PATH=/root/.sdkman/candidates/sbt/current/bin:$PATH
# This is specific to running a minimal empty Scala project and packaging it
RUN touch build.sbt
RUN sbt compile
RUN sbt package
FROM alpine AS production
# setup production environment image here
COPY --from=ScalaBuild /root/target/scala-2.12/ $INSTALL_PATH
ENTRYPOINT ["java", "-cp", "$INSTALL_PATH", "your.main.classfile"]
Generally you want to avoid using "version manager" type tools in Docker; it's better to install a specific version of the compiler or runtime you need.
In the case of Kotlin, it's a JVM application distributed as a zip file so it should be fairly easy to install:
FROM openjdk:15-slim
ARG KOTLIN_VERSION=1.3.72
# Get OS-level updates:
RUN apt-get update \
&& apt-get install --no-install-recommends --assume-yes \
curl \
unzip
# and if you need C/Python dependencies, those too
# Download and unpack Kotlin
RUN cd /opt \
&& curl -LO https://github.com/JetBrains/kotlin/releases/download/v${KOTLIN_VERSION}/kotlin-compiler-${KOTLIN_VERSION}.zip \
&& unzip kotlin-compiler-${KOTLIN_VERSION}.zip \
&& rm kotlin-compiler-${KOTLIN_VERSION}.zip
# Add its directory to $PATH
ENV PATH=/opt/kotlinc/bin:$PATH
The real problem with version managers is that they heavily depend on the tool setting environment variables. As #JeevanRao notes in their answer, each Dockerfile RUN command runs in a separate shell in a separate container, and any environment variable settings within that command get lost for the next command.
# Does absolutely nothing: environment variables do not stay set
RUN . /root/.sdkman/bin/sdkman-init.sh
Since an image generally contains only one application and its runtime, you don't need the ability to change which version of the runtime or compiler you're using. My Dockerfile example passes it as an ARG, so you can change it in the Dockerfile or pass a docker build --build-arg KOTLIN_VERSION=... option to use a different version.
I would like to rebuild/recompile all Debian packages of a machine with specific flags.
How can I do that with less command as possible?
I have found that https://debian-administration.org/article/20/Rebuilding_Debian_packages but it does not explain how to do that for all the packages installed on a system.
You can write a script that does something like this:
for each $pkg in dpkg-query -W -f '${status} ${package}\n' | sed -n 's/^install ok installed //p':
run apt-get source $pkg
run apt-get build-dep $pkg
cd $pkg-version/
run DEB_CPPFLAGS_SET="-I/foo/bar/baz" DEB_CFLAGS_SET="-g -O3" DEB_LDFLAGS_SET="-L/fruzzel/frazzel/" dpkg-buildpackage
install package with dpkg -i deb-file
cd ..
This will go through all of your installed packages and generate .deb files for each of them. Probably there are some edge cases etc. that will have to be handled. You could also leave out packages that are not built from C code etc.
Info taken from these questions:
https://unix.stackexchange.com/questions/184812/how-to-update-all-debian-packages-from-source-code
How to override dpkg-buildflags CFLAGS?
Try this approach:
dpkg --get-selections > selections
sudo dpkg --clear-selections
sudo dpkg --set-selections < selections
sudo apt-get --reinstall dselect-upgrade
Source:
https://www.linuxquestions.org/questions/linux-software-2/force-apt-get-to-redownload-and-reinstall-dependencies-as-well-873038/
When I try to compile the newest version of Clisp on Ubuntu 8.04 I always get this error after running configure:
Configure findings:
FFI: no (user requested: default)
readline: yes (user requested: yes)
libsigsegv: no, consider installing GNU libsigsegv
./configure: libsigsegv was not detected, thus some features, such as
generational garbage collection and
stack overflow detection in interpreted Lisp code
cannot be provided.
Please do this:
mkdir tools; cd tools; prefix=`pwd`/i686-pc-linux-gnu
wget http://ftp.gnu.org/pub/gnu/libsigsegv/libsigsegv-2.5.tar.gz
tar xfz libsigsegv-2.5.tar.gz
cd libsigsegv-2.5
./configure --prefix=${prefix} && make && make check && make install
cd ../..
./configure --with-libsigsegv-prefix=${prefix} --with-readline --with-unicode --with-module=i18n --with-module=gdbm --with-module=pcre --with-module=readline --with-module=regexp
If you insist on building without libsigsegv, please pass
--ignore-absence-of-libsigsegv
to this script:
./configure --ignore-absence-of-libsigsegv --with-readline --with-unicode --with-module=i18n --with-module=gdbm --with-module=pcre --with-module=readline --with-module=regexp
I've tried doing as requested, but it didn't help: it seems to ignore the --with-libsigsegv-prefix option. I also tried putting installing libsigsegv in a standard location (/usr/local). Oh, and of course, Ubuntu tells me that libsigsegv and libsigsegv-dev are installed in the system.
I'd really like to be able to compile this version of Clips, as it introduces some serious improvements over the version shipped with Ubuntu (I'd also like to have PCRE).
Here are my notes from compiling CLISP on Ubuntu in the past, hope this helps:
sudo apt-get install libsigsegv-dev libreadline5-dev
# as of 7.10, Ubuntu's libffcall1-dev is broken and I had to get it from CVS
# and make sure CLISP didn't use Ubuntu's version.
sudo apt-get remove libffcall1-dev libffcall1
cvs -z3 -d:pserver:anonymous#cvs.sv.gnu.org:/sources/libffcall co -P ffcall
cd ffcall; ./configure; make
sudo make install
cvs -z3 -d:pserver:anonymous#clisp.cvs.sourceforge.net:/cvsroot/clisp co -P clisp
cd clisp
./configure --with-libffcall-prefix=/usr/local --prefix=/home/luis/Software
ulimit -s 16384
cd src; make install
If you look at 'config.log' it might tell you why configure is not finding libsigsegv