Expand environment variable inside container on Docker command line - bash

Suppose that I create a Dockerfile that just runs an echo command:
FROM alpine
ENTRYPOINT [ "echo" ]
and that I build it like this:
docker build -t my_echo .
If I run docker run --rm my_echo test it will output test as expected.
But how can I run the command to display an environment variable that is inside the container?
Example:
docker run --rm --env MYVAR=foo my_echo ???
How to access the $MYVAR variable that is in the container to display foo by replacing the ??? part of that command?
Note:
This is a simplified version of my real use case. My real use case is a WP-CLI Docker image that I built with a Dockerfile. It has the wp-cli command as the ENTRYPOINT.
I am trying to run a container based on this image to update a WordPress parameter with an environment variable. My command without Docker is wp-cli option update siteurl "http://example.com" where http://example.com would be in an environment variable.
This is the command I am trying to run (wp_cli is the name of my container):
docker run --rm --env WEBSITE_URL="http://example.com" wp_cli option update siteurl ???

It's possible to have the argument that immediately follows ["bash", "-c"] itself be a shell script that looks for sigils to replace. For example, consider the following script, which I'm going to call argEnvSubst:
#!/usr/bin/env bash
args=( "$#" ) # collect all arguments into a single array
for idx in "${!args[#]}"; do # iterate over the indices of that array...
arg=${args[$idx]} # ...and collect the associated values.
if [[ $arg =~ ^#ENV[.](.*)#$ ]]; then # if we have a value that matches a pattern...
varname=${BASH_REMATCH[1]} # extract the variable name from that pattern
args[$idx]=${!varname} # and replace the value with a lookup result
fi
done
exec "${args[#]}" # run our resulting array as a command.
Thus, argEnvSubst "echo" "#ENV.foobar#" will replace #ENV.foobar# with the value of the environment named foobar before it invokes echo.
While I would strongly suggest injecting this into your Dockerfile as a separate script and naming that script as your ENTRYPOINT, it's possible to do it in-line:
ENTRYPOINT [ "bash", "-c", "args=(\"$#\"); for idx in \"${!args[#]}\"; do arg=${args[$idx]}; if [[ $arg =~ ^#ENV[.](.*)#$ ]]; then varname=${BASH_REMATCH[1]}; args[$idx]=${!varname}; fi; done; \"${args[#]}\"", "_" ]
...such that you can then invoke:
docker run --rm --env WEBSITE_URL="http://example.com" \
wp_cli option update siteurl '#ENV.WEBSITE_URL#'
Note the use of bash -- this means alpine (providing only dash) isn't sufficient.

Related

source in every Dockerfile RUN call

I have a Dockerfile, where I am finding myself constantly needing to call source /opt/ros/noetic/setup.bash.
e.g.:
RUN source /opt/ros/noetic/setup.bash \
&& SOME_COMMAND
RUN source /opt/ros/noetic/setup.bash \
&& SOME_OTHER_COMMAND
Is there a method to have this initialised in every RUN call in a Dockerfile?
Have tried adding to ~/.bash_profile and Docker's ENV command with no luck.
TL;DR: what you want is feasible by copying your .sh script in /etc/profile.d/ and using the SHELL Dockerfile command to tweak the default shell.
Details:
To start with, consider the following sample script setup.sh:
#!/bin/sh
echo "# setup.sh"
ENV_VAR="some value"
some_fun() {
echo "## some_fun"
}
Then, it can be noted that bash provides the --login CLI option:
When Bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable. The --noprofile option may be used when the shell is started to inhibit this behavior.
When an interactive login shell exits, or a non-interactive login shell executes the exit builtin command, Bash reads and executes commands from the file ~/.bash_logout, if it exists.
− https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Bash-Startup-Files
Furthermore, instead of appending the setup.sh code in /etc/profile, you can take advantage of the /etc/profile.d folder that is read in the following way by most distributions:
$ docker run --rm -i debian:10 cat /etc/profile | tail -n 9
if [ -d /etc/profile.d ]; then
for i in /etc/profile.d/*.sh; do
if [ -r $i ]; then
. $i
fi
done
unset i
fi
Note in particular that the .sh extension is mandatory, hence the naming of the minimal-working-example above: setup.sh (not setup.bash).
Finally, it is possible to rely on the SHELL command to replace the default shell used by RUN (in place of ["/bin/sh", "-c"]) to incorporate the --login option of bash.
Concretely, you could phrase your Dockerfile like this:
FROM debian:10
# WORKDIR /opt/ros/noetic
# COPY setup.sh .
# RUN . /opt/ros/noetic/setup.sh && echo "ENV_VAR=$ENV_VAR"
# empty var here
RUN echo "ENV_VAR=$ENV_VAR"
# enable the extra shell init code
COPY setup.sh /etc/profile.d/
SHELL ["/bin/bash", "--login", "-c"]
# nonempty var and function
RUN echo "ENV_VAR=$ENV_VAR" && some_fun
# DISABLE the extra shell init code!
RUN rm /etc/profile.d/setup.sh
# empty var here
RUN echo "ENV_VAR=$ENV_VAR"
Outcome:
$ docker build -t test .
Sending build context to Docker daemon 6.144kB
Step 1/7 : FROM debian:10
---> ef05c61d5112
Step 2/7 : RUN echo "ENV_VAR=$ENV_VAR"
---> Running in 87b5c589ec60
ENV_VAR=
Removing intermediate container 87b5c589ec60
---> 6fdb70be76f9
Step 3/7 : COPY setup.sh /etc/profile.d/
---> e6aab4ebf9ef
Step 4/7 : SHELL ["/bin/bash", "--login", "-c"]
---> Running in d73b0d13df23
Removing intermediate container d73b0d13df23
---> ccbe789dc36d
Step 5/7 : RUN echo "ENV_VAR=$ENV_VAR" && some_fun
---> Running in 42fd1ae14c17
# setup.sh
ENV_VAR=some value
## some_fun
Removing intermediate container 42fd1ae14c17
---> de74831896a4
Step 6/7 : RUN rm /etc/profile.d/setup.sh
---> Running in bdd969a63def
# setup.sh
Removing intermediate container bdd969a63def
---> 5453be3271e5
Step 7/7 : RUN echo "ENV_VAR=$ENV_VAR"
---> Running in 0712cea427f1
ENV_VAR=
Removing intermediate container 0712cea427f1
---> 216a421f5659
Successfully built 216a421f5659
Successfully tagged test:latest

How to transform the output of a command in a list of arguments in BASH?

there
I need to pass multiple arguments to a docker command, but I wanted this string of arguments to be built interactively and not by hand. In my specific problem, I want to get all the environment variables in this machine and set each one of them as a build-arg argument.
For example, if the env variables are
ENV_VAR1=ENV_VALUE1
ENV_VAR2=ENV_VALUE2
ENV_VAR3=ENV_VALUE3
I want to build a string like this
docker build --build-arg ENV_VAR1=ENV_VALUE1 --build-arg ENV_VAR2=ENV_VALUE2 --build-arg ENV_VAR1=ENV_VALUE1 .`
I was wondering how to wrap this repetition in a bash script something like this (please this is just a pseudocode):
docker build $(FOREACH get_output_of_env() AS $env DO `--build-arg $env` END) .
Is that anyhow possible?
Thanks in advance
Use an array:
dockerargs=()
for i in ENV_VAR1 ENV_VAR2 ENV_VAR3; do
dockerargs+=(--build-arg "$i=${!i}")
done
docker build "${dockerargs[#]}" ...
If you really want to use how to iterate over env and do for i in $(compgen -e)
#!/bin/sh
cat > args < EOF
#!/bin/sh
EOF
env -0 >> args
sed -i s'#^#--build-arg #g' args
tr '\n' ' '
sed -i s'#^#docker build#' args
Then just execute "args" as another shell script. I admit that I haven't tested this, but I can't see too many reasons why it shouldn't work.

Bash - Check If given argument exits

I have a separate shell script that runs my docker-compose environment in attached mode or detached if I pass -d or --detach argument. It works fine when I pass this argument (./run-env.sh -d) but it doesn't when I run my script without any option ( ./run-env, just getting blank output and docker-compose doesn't run), where can be a problem?
#!/usr/bin/env bash
for arg in "$#"; do
if [ "$arg" = '-d' ] || [ "$arg" = '--detach' ]
then
docker-compose -f docker-compose.local-environment.yml up --build -V --detach
else
docker-compose -f docker-compose.local-environment.yml up --build -V --abort-on-container-exit
fi
done
When you don't give argument, you don't even enter the for loop, that's why nothing happens.
#!/usr/bin/env bash
# By default, use '--abort-on-container-exit' option
abort_or_detach="--abort-on-container-exit"
# Search for a parameter asking to run in detached mode
for arg in "$#"; do
if [ "$arg" = '-d' ] || [ "$arg" = '--detach' ]
then
abort_or_detach="--detach"
fi
done
# Run with the correct option
docker-compose -f docker-compose.local-environment.yml up --build -V $abort_or_detach
Here in this script, you call one time docker-compose, and you can manage easily the options with the for loop
Also, with your first try, you would launch docker-compose as many times as you have different parameters. Here, you treat them, and then do a single launch
for arg in "$#" iterates over the arguments. When you pass no arguments, it iterates zero times. Instead, try something like:
extra=--abort-on-container-exit
for arg; do
case "$arg" in
-d|--detach) extra=--detach
esac
done
docker-compose -f docker-compose.local-environment.yml up --build -V $extra
Note that this is one of those cases where you do not want to put quotes around $extra, because if extra is the empty string you don't want to pass anything to docker-compose. (Here, the default will ensure it is not empty, but this is a fairly common pattern and there are cases where it will be the empty string.)

In this simple Docker wrapper script example, how may one correctly pass a current working directory path which contains spaces?

My Docker wrapper script works as intended when the current working directory does not contain spaces, however there is a bug when it does.
I have simplified an example to make use of the smallest official Docker image I could find and a well known GNU core utility. Of course this example is not very useful. In my real world use case, a much more complicated environment is packaged.
Docker Wrapper Script:
#!/usr/bin/env bash
##
## Dockerized ls
##
set -eux
# Only allocate tty if one is detected
# See https://stackoverflow.com/questions/911168/how-to-detect-if-my-shell-script-is-running-through-a-pipe
if [[ -t 0 ]]; then
DOCKER_RUN_OPTIONS+="-i "
fi
if [[ -t 1 ]]; then
DOCKER_RUN_OPTIONS+="-t "
fi
WORK_DIR="$(realpath .)"
DOCKER_RUN_OPTIONS+="--rm --user=$(id -u $(logname)):$(id -g $(logname)) --workdir=${WORK_DIR} --mount type=bind,source=${WORK_DIR},target=${WORK_DIR}"
exec docker run ${DOCKER_RUN_OPTIONS} busybox:latest ls "$#"
You can save this somewhere as /tmp/docker_ls for example. Remember to chmod +x /tmp/docker_ls
Now you are able to use this Dockerized ls in any path which contains no spaces as follows:
/tmp/docker_ls -lah
/tmp/docker_ls -lah | grep 'r'
Note that /tmp/docker_ls -lah /path/to/something is not implemented. The wrapper script would have to be adapted to parse parameters and mount the path argument into the container.
Can you see why this would not work when current working directory path contains spaces? What can be done to rectify it?
Solution:
#david-maze's answer solved the problem. Please see: https://stackoverflow.com/a/55763212/1782641
Using his advice I refactored my script as follows:
#!/usr/bin/env bash
##
## Dockerized ls
##
set -eux
# Only allocate tty if one is detected. See - https://stackoverflow.com/questions/911168
if [[ -t 0 ]]; then IT+=(-i); fi
if [[ -t 1 ]]; then IT+=(-t); fi
USER="$(id -u $(logname)):$(id -g $(logname))"
WORKDIR="$(realpath .)"
MOUNT="type=bind,source=${WORKDIR},target=${WORKDIR}"
exec docker run --rm "${IT[#]}" --user "${USER}" --workdir "${WORKDIR}" --mount "${MOUNT}" busybox:latest ls "$#"
If your goal is to run a process on the current host directory as the current host user, you will find it vastly easier and safer to use a host process, and not an isolation layer like Docker that intentionally tries to hide these things from you. For what you’re showing I would just skip Docker and run
#!/bin/sh
ls "$#"
Most software is fairly straightforward to install without Docker, either using a package manager like APT or filesystem-level isolation like Python’s virtual environments and Node’s node_modules directory. If you’re writing this script then Docker is just getting in your way.
In a portable shell script there’s no way to make “a list of words” in a way that keeps their individual wordiness. If you know you’ll always want to pass some troublesome options then this is still fairly straightforward: include them directly in the docker run command and don’t try to create a variable of options.
#!/bin/sh
RM_IT="--rm"
if [[ -t 0 ]]; then RM_IT="$RM_IT -i"; fi
if [[ -t 1 ]]; then RM_IT="$RM_IT -t"; fi
UID=$(id -u $(logname))
GID=$(id -g $(logname))
# We want the --rm -it options to be expanded into separate
# words; we want the volume options to stay as a single word
docker run $RM_IT "-u$UID:$GID" "-w$PWD" "-v$PWD:$PWD" \
busybox \
ls "$#"
Some shells like ksh, bash, and zsh have array types, but these shells may not be present on every system or environment (your busybox image doesn’t have any of these for example). You also might consider picking a higher-level scripting language that can more explicitly pass words into an exec type call.
I'm taking a stab at this to give you something to try:
Change this:
DOCKER_RUN_OPTIONS+="--rm --user=$(id -u $(logname)):$(id -g $(logname)) --workdir=${WORK_DIR} --mount type=bind,source=${WORK_DIR},target=${WORK_DIR}"
To this:
DOCKER_RUN_OPTIONS+="--rm --user=$(id -u $(logname)):$(id -g $(logname)) --workdir=${WORK_DIR} --mount type=bind,source='${WORK_DIR}',target='${WORK_DIR}'"
Essentially, we are putting the ' in there to escape the space when the $DOCKER_RUN_OPTIONS variable is evaluated by bash on the 'exec docker' command.
I haven't tried this - it's just a hunch / first shot.

How to escape space in bash script from inline if?

I know that similar questions have been asked and answered before on stackoverflow (for example here and here) but so far I haven't been able to figure it out for my particular case.
I'm trying to create a script that adds the -v flag only if the variable something is equal to "true" (what I'm trying to do is to mount the current folder as a volume located at /src in the Docker container):
docker run --name image-name `if [ "${something}" == "true" ]; then echo "-v $PWD:/src"; fi` ....
The problem is that $PWD may contain spaces and if so my script won't work. I've also tried assigning "$PWD" to an intermediate variable but it still doesn't work:
temp="$PWD"
docker run --name image-name `if [ "${something}" == "true" ]; then echo "-v $temp:/src"; fi` ....
If I run:
docker run --name image-name -v "$PWD":/src ....
from plain bash (without using my script) then everything works.
Does anyone know how to solve this?
Use an array.
docker_args=()
if something; then
docker_args+=( -v "$PWD/src" )
fi
docker run --blah "${docker_args[#]}" …
Don't have arrays? Use set (in a function, so it doesn't affect outer scope).
Generally:
knacker() {
if something; then
set -- -v "$PWD:/src" "$#"
fi
crocker "$#"
}
knacker run --blah
But some commands (like docker, git, etc) need special treatment because of their two-part command structure.
slacker() {
local cmd="$1"
shift
if something; then
set -- -v "$PWD:/src" "$#"
fi
docker "$cmd" "$#"
}
slacker run --blah
Try this (using the array way):
declare -a cmd=()
cmd+=(docker run --name image-name)
if [ "${something}" = "true" ]
then
cmd+=(-v "$PWD:/src")
fi
"${cmd[#]}"

Resources