Often times, I encounter commands being executed with /bin/bash -c or /bin/sh -c instead of directly. For example, instead of cp /tmp/file1 /tmp/file2, it'd be /bin/bash -c "cp /tmp/file1 /tmp/file2".
What are some reasons for doing this instead of executing the command directly? In recent memory, I've seen this the most in Docker and K8s commands. The only thing I can really think of is because you specifically want to run the command with a specific shell, but this seems like a pretty rare/niche use-case?
Here is a specific example, the k8s deployment uses:
command: ["/bin/sh"]
args: ["-c", ". /config/dynamicenv.sh && /app/bin/docker-entrypoint server"]
Instead of what I would expect the default to be:
. /config/dynamicenv.sh && /app/bin/docker-entrypoint server
Without specific examples it's hard to tell, but a common reason for doing this is that you want to make use of shell i/o redirection, pipes, etc. For example, this fragment of a Kubernetes pod manifest would fail because it involves a pipe, which requires the shell to execute the command line:
containers:
image: docker.io/alpine:latest
command:
- echo hello world | sed s/world/container/
But this would work:
containers:
image: docker.io/alpine:latest
command:
- /bin/sh
- -c
- echo hello world | sed s/world/container/
This is one relatively common situation in which you'll see things explicitly execute with a shell. If you'd like to update your question with some specific examples, we can provide a more thorough answer.
Your example is very close to what I've already included here in my answer. The command . /config/dynamicenv.sh && /app/bin/docker-entrypoint server isn't a simple command; it's a shell script that makes use of both the . and the && operators.
If they were to write:
command: [". /config/dynamicenv.sh && /app/bin/docker-entrypoint server"]
It would fail with an error along the lines of:
exec: "[\". /config/dynamicenv.sh && /app/bin/docker-entrypoint server\"]": stat [". /config/dynamicenv.sh && /app/bin/docker-entrypoint server"]: no such file or directory: unknown.
The command needs to be wrapped with sh -c in order to execute correctly.
Related
I need to save multiline content about xdebug from makefile, but I cannot use file in repo, previosuly I had do this within:
docker exec -i $(CONTAINER) bash -c 'cat > /etc/php8/conf.d/50_xdebug.ini' < file
But now I need to use variable (multiline) or insert line by line.
xdebug: check-xdebug-env
ifeq ($(XDEBUG_STATUS),1)
#make xdebug-enable
else
#make xdebug-disable
endif
xdebug-disable:
#make restart
xdebug-enable:
#make restart
restart:
#docker exec $(CONTAINER) bash -c "kill -9 $(shell $(CMD) ps -a | grep '[p]hp-fpm' | awk '{print $$1}')"
Rule #1 for debugging make: remove ALL the # prefixes to your recipe lines. Rule #2: examine the recipe lines make prints carefully and make sure they're the same thing you would run if you were running it by hand. Anywhere they are different, that's where you have to concentrate on fixing something.
Tip for writing makefiles: if you're using the $(shell ...) make function inside a recipe, you're almost certainly doing something wrong.
I don't understand what you want to accomplish here and you've left out important parts in your question (what does $(CMD) expand to, for example) but remember that make functions, like $(shell ...), are all expanded by make before it invokes any command in the recipe. So this script is run on your host system, before make invokes the docker command. It's not run inside the container. That's probably not what you want.
restart:
docker exec $(CONTAINER) bash -c 'kill -9 $$($(CMD) ps -a | grep "[p]hp-fpm" | awk "{print \$$1}")'
I have a Makefile that looks like this:
build-docker:
DOCKER_BUILDKIT=1 docker build --ssh default=~/.ssh/id_rsa -t my-app .
If I run make build-docker I get the following error:
$ make build-docker
DOCKER_BUILDKIT=1 docker build --ssh default=~/.ssh/id_rsa -t my-app .
could not parse ssh: [default=~/.ssh/id_rsa]: stat ~/.ssh/id_rsa: no such file or directory
make: *** [Makefile:12: build-docker] Error 1
However, if I run the command directly in the shell it runs just fine:
$ DOCKER_BUILDKIT=1 docker build --ssh default=~/.ssh/id_rsa -t my-app .
[+] Building 65.5s (20/20) FINISHED
Why is this and how do I solve it?
You are running the same command, but in different shells. Your interactive shell is probably bash. But the shell make uses is /bin/sh which is a POSIX standard shell (often).
The special handling of ~ in an argument is a shell feature: it's not embedded in programs like docker or ssh. And, it's not defined in POSIX; it's an additional feature that some shells, like bash.
On my system:
bash$ echo foo=~
foo=/home/me
bash$ /bin/sh
$ echo foo=~
foo=~
To be portable you should use the full pathname or $HOME instead (remember that in a make recipe you have to double the $ to escape it from make: $$HOME).
I need to run below code as a single line in docker run -it image_name -c \bin\bash --script with --script below
(dir and dockerImageName being parameters)
'''cd ''' + dir+ ''' \
&& if make image ''' + dockerImageName''' 2>&1 | grep -m 1 "No rule to make target"; then
exit 1
fi'''
How can this be run as a single line?
You can abstract all of this logic into your higher-level application. If you can't do this, write a standard shell script and COPY it into your image.
The triple quotes look like Python syntax. You can break this up into three parts:
The cd $dir part specifies the working directory for the subprocess;
make ... is an actual command to run;
You're inspecting its output for some condition.
In Python, you can call subprocess.run() with an array of arguments and specify these various things at the application level. The array of arguments isn't reinterpreted by a shell and so protects you from this particular security issue. You might run:
completed = subprocess.run(['make', 'image', dockerImageName],
cwd=dir,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
if 'No rule to make target' in completed.stdout:
...
If you need to do this as a shell script, doing it as a proper shell script and making sure to quote your arguments again protects you.
#!/bin/sh
set -e
cd "$1"
if make image "$2" 2>&1 | grep -m 1 "No rule to make target"; then
exit 1
fi
You should never construct a command line by combining strings in the way you've shown. This makes you vulnerable to a shell injection attack. Especially if an attacker knows that the user has permissions to run docker commands, they can set
dir = '.; docker run --rm -v /:/host busybox cat /host/etc/shadow'
and get a file of encrypted passwords they can crack at their leisure. Pretty much anything else is possible once the attacker uses this technique to get unlimited root-level read/write access to the host filesystem.
My Docker wrapper script works as intended when the current working directory does not contain spaces, however there is a bug when it does.
I have simplified an example to make use of the smallest official Docker image I could find and a well known GNU core utility. Of course this example is not very useful. In my real world use case, a much more complicated environment is packaged.
Docker Wrapper Script:
#!/usr/bin/env bash
##
## Dockerized ls
##
set -eux
# Only allocate tty if one is detected
# See https://stackoverflow.com/questions/911168/how-to-detect-if-my-shell-script-is-running-through-a-pipe
if [[ -t 0 ]]; then
DOCKER_RUN_OPTIONS+="-i "
fi
if [[ -t 1 ]]; then
DOCKER_RUN_OPTIONS+="-t "
fi
WORK_DIR="$(realpath .)"
DOCKER_RUN_OPTIONS+="--rm --user=$(id -u $(logname)):$(id -g $(logname)) --workdir=${WORK_DIR} --mount type=bind,source=${WORK_DIR},target=${WORK_DIR}"
exec docker run ${DOCKER_RUN_OPTIONS} busybox:latest ls "$#"
You can save this somewhere as /tmp/docker_ls for example. Remember to chmod +x /tmp/docker_ls
Now you are able to use this Dockerized ls in any path which contains no spaces as follows:
/tmp/docker_ls -lah
/tmp/docker_ls -lah | grep 'r'
Note that /tmp/docker_ls -lah /path/to/something is not implemented. The wrapper script would have to be adapted to parse parameters and mount the path argument into the container.
Can you see why this would not work when current working directory path contains spaces? What can be done to rectify it?
Solution:
#david-maze's answer solved the problem. Please see: https://stackoverflow.com/a/55763212/1782641
Using his advice I refactored my script as follows:
#!/usr/bin/env bash
##
## Dockerized ls
##
set -eux
# Only allocate tty if one is detected. See - https://stackoverflow.com/questions/911168
if [[ -t 0 ]]; then IT+=(-i); fi
if [[ -t 1 ]]; then IT+=(-t); fi
USER="$(id -u $(logname)):$(id -g $(logname))"
WORKDIR="$(realpath .)"
MOUNT="type=bind,source=${WORKDIR},target=${WORKDIR}"
exec docker run --rm "${IT[#]}" --user "${USER}" --workdir "${WORKDIR}" --mount "${MOUNT}" busybox:latest ls "$#"
If your goal is to run a process on the current host directory as the current host user, you will find it vastly easier and safer to use a host process, and not an isolation layer like Docker that intentionally tries to hide these things from you. For what you’re showing I would just skip Docker and run
#!/bin/sh
ls "$#"
Most software is fairly straightforward to install without Docker, either using a package manager like APT or filesystem-level isolation like Python’s virtual environments and Node’s node_modules directory. If you’re writing this script then Docker is just getting in your way.
In a portable shell script there’s no way to make “a list of words” in a way that keeps their individual wordiness. If you know you’ll always want to pass some troublesome options then this is still fairly straightforward: include them directly in the docker run command and don’t try to create a variable of options.
#!/bin/sh
RM_IT="--rm"
if [[ -t 0 ]]; then RM_IT="$RM_IT -i"; fi
if [[ -t 1 ]]; then RM_IT="$RM_IT -t"; fi
UID=$(id -u $(logname))
GID=$(id -g $(logname))
# We want the --rm -it options to be expanded into separate
# words; we want the volume options to stay as a single word
docker run $RM_IT "-u$UID:$GID" "-w$PWD" "-v$PWD:$PWD" \
busybox \
ls "$#"
Some shells like ksh, bash, and zsh have array types, but these shells may not be present on every system or environment (your busybox image doesn’t have any of these for example). You also might consider picking a higher-level scripting language that can more explicitly pass words into an exec type call.
I'm taking a stab at this to give you something to try:
Change this:
DOCKER_RUN_OPTIONS+="--rm --user=$(id -u $(logname)):$(id -g $(logname)) --workdir=${WORK_DIR} --mount type=bind,source=${WORK_DIR},target=${WORK_DIR}"
To this:
DOCKER_RUN_OPTIONS+="--rm --user=$(id -u $(logname)):$(id -g $(logname)) --workdir=${WORK_DIR} --mount type=bind,source='${WORK_DIR}',target='${WORK_DIR}'"
Essentially, we are putting the ' in there to escape the space when the $DOCKER_RUN_OPTIONS variable is evaluated by bash on the 'exec docker' command.
I haven't tried this - it's just a hunch / first shot.
I am creating a Dockerfile which starts a Java application. This Java application is given a file path which contains the output of ls -l.
(Note that in my real Dockerfile I am not doing ls - l but rather complex commands. I altered that to ls - l to simplify the question.)
I tried the following:
FROM openjdk:8-jre
ARG JAR
COPY target/$JAR /app/app.jar
CMD java -jar /app/app.jar <( ls -l )
This bash <( ... ) construction should create a temporary file containing the output of ls -l.
When starting the Docker file, I get:
/bin/sh: 1: Syntax error: "(" unexpected
Now, sh does not support the <( ... ) construction, hence the error. How can I start the application safely via bash instead of sh? With safely I mean that the Java app still will receive all OS signals (SIGHUP, ...) and react appropriately.
Replace your command with a JSON list, for which the first two elements are bash -c, and the last element is the shell command you actually want to run.
CMD ["bash", "-c", "exec java -jar /app/app.jar <( ls -l )"]
To generate such an array for a more complex command, you might consider using jq to ensure that syntax is correct, even for input with backslashes, newlines or quotes:
jq -cnRs '["bash", "-c", input]' <<'EOF'
# you can put other shell commands here if you need to
exec java -jar /app/app.jar <( ls -l )
EOF
exec ensures that the java instance replaces bash, and thus is sent signals directly.
If you're doing complex things on startup it's often easier to write them into a script than try to build a very complicated command line. Once you're doing that, you can use the set of primitives that are available in the POSIX shell standard, even if they require multiple commands to do things that GNU bash could do inline.
For this I might write a script:
#!/bin/sh
ls -l >/ls-l.txt
exec java -jar /app/app.jar /ls-l.txt
and then copy it in as the default thing your image runs
FROM openjdk:8-jre
ARG JAR
COPY target/$JAR /app/app.jar
COPY launch-app.sh /usr/bin/app
# RUN chmod +x /usr/bin/app, if it's not already executable
CMD ["app"]