How does docker run interpret dynamically generated --env arguments - bash

I am trying to provide a dynamically generated list of --env VAR1 --env VAR2 --env-file env.list environment variables to docker run.
Unfortunately it is not working.
for --env mapped variables, the variables are not visible in the container.
for --env-file provided file, docker complains that it cannot find the file: docker: open "env.list": no such file or directory.
Details
Running:
# env_params contains either --env or --env-file arguments
MY_VAR=123
env_params='--env "MY_VAR"'
echo ${env_params}
docker run -it --rm \
${env_params} \
my_docker_image env | grep MY_VAR
will not output anything. MY_VAR is not visible inside the container. But:
MY_VAR=123
docker run -it --rm \
--env "MY_VAR" \
my_docker_image env | grep MY_VAR
will work and 123 will be printed.
In a similar way --env-file will not work when provided through env_params but will work when provided directly to the docker run command.
What am I doing wrong?

There are two issues here.
First, When you run, in your shell:
MY_VAR=123
You have not set an environment variable. You have set a local shell variable. When you use --env MY_VAR, you are telling Docker that you want to make the environment variable MY_VAR available inside the container, and since it doesn't exist you get nothing:
$ MY_VAR=123
$ docker run -it --rm -e MYVAR alpine env | grep MY_VAR
<crickets>
If you first export that to the environment:
$ export MY_VAR=123
$ docker run -it --rm -e MYVAR alpine env | grep MY_VAR
MY_VAR=123
Then it will work as you expect. Alternately, you can use the VARNAME=VARVALUE form of the --env option:
docker run -e "MY_VAR=${MY_VAR}" ...
The second issue has to do with how shell variable interpolation works. If you have:
env_params='--env "MY_VAR"'
docker run -it --rm \
${env_params} \
alpine env
Then the resulting command line is:
docker run -it --rm --env '"MY_VAR"' alpine env
That is, the argument you're passing to docker run includes literal double quotes. You can fix that through the use of the eval statement (keeping in mind that you'll need to modify your script to export MY_VAR):
eval docker run -it --rm \
${env_params} \
alpine env | grep MY_VAR
Alternately (and I would argue preferably) you can use your env_params variable as an array, as long as you're using bash:
env_params=(--env MY_VAR)
env_params+=(--env SOME_OTHER_VAR)
docker run -it --rm \
"${env_params[#]}" \
alpine env | grep MY_VAR
Which would result in the correct command line:
docker run -it --rm --env MY_VAR --env SOME_OTHER_VAR alpine env
I guess the summary here is that your issues ultimately have nothing to do with "how docker run interprets dynamically generated arguments", but have everything to do with "how shell variables and interpolation work".

Related

Save output of bash command from Dockerfile after Docker container was launched

I have a Dockerfile with ubuntu image as a base.
FROM ubuntu
ARG var_name
ENV env_var_name=$var_name
ENTRYPOINT ["/bin/bash", "-c", "echo $env_var_name"]
I expect from this
executing of a simple bash script, which will take an environment variable from user keyboard input and output this value after running docker container. It goes right.
(part where i have a problem) saving values of environment variables to file + after every running of docker run --rm -e env_var_name=%valueOfVar% IMAGE-NAME i can see a list of entered from keyboard values.
My idea about part 2 were like
docker run --rm -e env_var_name=%valueOfVar% IMAGE-NAME > /directory/tosave/values.txt. That works, but only one last value saves, not a list of values.
How can i change Dockerfile to save values to a file, which Docker will see and from which Docker after running will read and ouyput values? May be i shouldn`t use ENTRYPOINT?
Appreciate for any possible help. I`ve stuck.
Emphasizing that output and save of environment variables expected.
Like #lojza hinted at, > overwrites files whereas >> appends to them which is why your command is clobbering the file instead of adding to it. So you could fix it with this:
docker run --rm -e env_var_name=%valueOfVar% IMAGE-NAME >> /directory/tosave/values.txt
Or using tee(1):
docker run --rm -e env_var_name=%valueOfVar% IMAGE-NAME | tee -a /directory/tosave/values.txt
To clarify though, the docker container is not writing to values.txt, your shell is what is redirecting the output of the docker run command to the file. If you want the file to be written to by docker you should mount a file or directory into it the container using -v and redirect the output of the echo there. Here's an example:
FROM ubuntu
ARG var_name
ENV env_var_name=$var_name
ENTRYPOINT ["/bin/bash", "-c", "echo $env_var_name | tee -a /data/values.txt"]
And then run it like so:
$ docker run --rm -e env_var_name=test1 -v "$(pwd):/data:rw" IMAGE-NAME
test1
$ docker run --rm -e env_var_name=test2 -v "$(pwd):/data:rw" IMAGE-NAME
test2
$ ls -l values.txt
-rw-r--r-- 1 root root 12 May 3 15:11 values.txt
$ cat values.txt
test1
test2
One more thing worth mentioning. echo $env_var_name is printing the value of the environment variable whose name is literally env_var_name. For example if you run the container with -e env_var_name=PATH it would print the literal string PATH and not the value of your $PATH environment variable. This does seem to be the desired outcome, but I thought it was worth explicitly spelling this out.

How to use docker container's env variable while running docker exec command

I am running a docker exec -it ... command and I need to use an environment variable of my docker container. An example:
docker exec -it container_id command_here param_1 $param_2_as_env_variable
In the case above, it pulls param_2_as_env_variable from host machine, and not the docker container. Is it possible to use env variable from container itself while running docker exec ... command from another machine?
Update: I can use ouput of docker exec -it container_id printenv | grep .... But I couldn't separate value and key. How can I get only value here?
Something like this could work (it assumes, the container has a shell installed)
docker exec -it container_id sh -c 'command_here param_1 $param_2_as_env_variable'
For example the following works:
docker exec -it test sh -c 'echo $HOSTNAME'
to give the host name of the container.
You need to escape the variable to pass it to the docker exec command unresolved:
Try:
docker exec -it container_id command_here param_1 \$param_2_as_env_variable

Pass ENV to Docker container running single command

This prints blank:
docker run --rm --env HELLO="world" ubuntu:18.04 bash -c "echo $HELLO"
However this works:
docker run --rm -it --env HELLO="world" ubuntu:18.04 bash
# in the container
echo $HELLO
HELLO seems passed to the container though:
docker run --rm --env HELLO="world" ubuntu:18.04 env
Why is the first command not seeing HELLO? What am I missing?
Because of the double quotes, $HELLO will be evaluated by the docker host itself once the command got executed before going inside the container. So you need to either escape the dollar sign ($) using Backslash (\) which tells the bash that the $ is a part of the command itself and no need to be evaluated by the current shell (which is the docker host in our case) or use single quotes ('') like this:
Using single quotes
$ docker run --rm --env HELLO="world" ubuntu:18.04 bash -c 'echo $HELLO'
world
Using Backslash to escape
$ docker run --rm --env HELLO="world" ubuntu:18.04 bash -c "echo \$HELLO"
world
The reason you are not seeing what you expect is because things are being evaluated before you expect them to be evaluated.
When you run:
docker run --rm --env HELLO="world" ubuntu:18.04 bash -c "echo $HELLO"
The "echo $HELLO" really isn't any different to bash than:
echo "echo $HELLO"
The shell (bash) parses double quotes (") and things inside them. It sees "echo $HELLO" and replaces the variable $HOME with it's value. If $HOME is not defined, this evaluates to echo.
So,
echo "echo $HELLO"
is parsed and evaluted by your shell. Which then just runs this at the end:
echo "echo "
So the "echo $HELLO" in your docker command is evaluated to "echo " and that's what gets passed to the docker command.
What you want to do is to prevent your shell from evaluating the variable. You can do it a couple of ways:
You can use single quotes instead of double quotes. Your shell doesn't parse it; it will be passed to the bash inside the container as is:
docker run --rm --env HELLO="world" ubuntu:18.04 bash -c 'echo $HELLO'
You can escape the $ to avoid evaluating it in this shell and let the bash inside the docker container evaluate it:
docker run --rm --env HELLO="world" ubuntu:18.04 bash -c "echo \$HELLO"

How to pass environment variable spring.main.web-application-type=SERVLET to run docker image

When I use docker run command, the variable "SPRING_PROFILES_ACTIVE" is right, but "SPRING_MAIN_WEB-APPLICATION-TYPE" does not work, how to pass "SPRING_MAIN_WEB-APPLICATION-TYPE" to dokcer image?
sudo docker run -d -e SPRING_PROFILES_ACTIVE=product -e SPRING_MAIN_WEB-APPLICATION-TYPE=SERVLET -e SERVER_PORT=6789 --network mongo_network
As described in the documentation, the expected format for that environment variable is SPRING_MAIN_WEBAPPLICATIONTYPE.
I use docker ENV to get it worked, here is the Dockerfile:
FROM openjdk:8-jre
ENV TYPE NONE
COPY data.jar data.jar
CMD ["java","-jar","data.jar", "--spring.main.web-application-type=${TYPE}"]
then run the docker image:
sudo docker run -d -e SPRING_PROFILES_ACTIVE=product -e TYPE=SERVLET -e SERVER_PORT=6789
hope this helps.

Get Environment Variable from Docker Container

What's the simplest way to get an environment variable from a docker container that has not been declared in the Dockerfile?
For instance, an environment variable that has been set through some docker exec container /bin/bash session?
I can do docker exec container env | grep ENV_VAR, but I would prefer something that just returns the value.
I've tried using docker exec container echo "$ENV_VAR", but the substitution seems to happen outside of the container, so I don't get the env var from the container, but rather the env var from my own computer.
Thanks.
To view all env variables:
docker exec container env
To get one:
docker exec container env | grep VARIABLE | cut -d'=' -f2
The proper way to run echo "$ENV_VAR" inside the container so that the variable substitution happens in the container is:
docker exec <container_id> bash -c 'echo "$ENV_VAR"'
You can use printenv VARIABLE instead of /bin/bash -c 'echo $VARIABLE. It's much simpler and it does not perform substitution:
docker exec container printenv VARIABLE
The downside of using docker exec is that it requires a running container, so docker inspect -f might be handy if you're unsure a container is running.
Example #1. Output a list of space-separated environment variables in the specified container:
docker inspect -f \
'{{range $index, $value := .Config.Env}}{{$value}} {{end}}' container_name
the output will look like this:
ENV_VAR1=value1 ENV_VAR2=value2 ENV_VAR3=value3
Example #2. Output each env var on new line and grep the needed items, for example, the mysql container's settings could be retrieved like this:
docker inspect -f \
'{{range $index, $value := .Config.Env}}{{println $value}}{{end}}' \
container_name | grep MYSQL_
will output:
MYSQL_PASSWORD=secret
MYSQL_ROOT_PASSWORD=supersecret
MYSQL_USER=demo
MYSQL_DATABASE=demodb
MYSQL_MAJOR=5.5
MYSQL_VERSION=5.5.52
Example #3. Let's modify the example above to get a bash friendly output which can be directly used in your scripts:
docker inspect -f \
'{{range $index, $value := .Config.Env}}export {{$value}}{{println}}{{end}}' \
container_name | grep MYSQL
will output:
export MYSQL_PASSWORD=secret
export MYSQL_ROOT_PASSWORD=supersecret
export MYSQL_USER=demo
export MYSQL_DATABASE=demodb
export MYSQL_MAJOR=5.5
export MYSQL_VERSION=5.5.52
If you want to dive deeper, then go to Go’s text/template package documentation with all the details of the format.
Since we are dealing with JSON and unlike the accepted answer, we don't need to exec the container.
docker inspect <NAME|ID> | jq '.[] | .Config.Env'
Output sample
[
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"NGINX_VERSION=1.19.4",
"NJS_VERSION=0.4.4",
"PKG_RELEASE=1~buster"
]
To retrieve a specific variable
docker inspect <NAME|ID> | jq -r '.[].Config.Env[]|select(match("^<VAR_NAME>"))|.[index("=")+1:]'
See jq
None of the above answers show you how to extract a variable from a non-running container (if you use the echo approach with run, you won't get any output).
Simply run with printenv, like so:
docker run --rm <container> printenv <MY_VAR>
(Note that docker-compose instead of docker works too)
If by any chance you use VSCode and has installed the docker extension, just right+click on the docker you want to check (within the docker extension), click on Inspect, and there search for env, you will find all your env variables values
We can modify entrypoint of a non-running container with the docker run command.
Example show PATH environment variable:
using bash and echo: This answer claims that echo will not produce any output, which is incorrect.
docker run --rm --entrypoint bash <container> -c 'echo "$PATH"'
using printenv
docker run --rm --entrypoint printenv <container> PATH
#aisbaa's answer works if you don't care when the environment variable was declared. If you want the environment variable, even if it has been declared inside of an exec /bin/bash session, use something like:
IFS="=" read -a out <<< $(docker exec container /bin/bash -c "env | grep ENV_VAR" 2>&1)
It's not very pretty, but it gets the job done.
To then get the value, use:
echo ${out[1]}
This command inspects docker stack processes' environment in the host :
pidof dockerd containerd containerd-shim | tr ' ' '\n' \
| xargs -L1 -I{} -- sudo xargs -a '/proc/{}/environ' -L1 -0
The first way we use to find the ENV variables is docker inspect <container name>
The second way is docker exec <4 alphanumeric letter of CONTAINER id> bash -c 'echo "$ENV_VAR"'
There is a misconception in the question, that causes confusion:
you cannot access a "running session", so no bash session can change anything.
docker exec -ti container /bin/bash
starts a new console process in the container, so if you do export VAR=VALUE this will go away as soon as you leave the shell, and it won't exist anymore.
Perhaps a good example:
# assuming TESTVAR did not existed previously this is empty
docker exec container env | grep TESTVAR
# -> TESTVAR=a new value!
docker exec container /bin/bash -c 'TESTVAR="a new value!" env' | grep TESTVAR
# again empty
docker exec container env | grep TESTVAR
The variables from env come from the Dockerfile or command, docker itself and whatever the entrypoint sets.
The other answers here are good. But if you really need to get the environmental properties used when starting a program, then you can inspect the /proc/pid/environ contents in the container, where pid is the container process id of the running comand.
# environmental props
docker exec container cat /proc/pid/environ | tr '\0' '\n'
# you can check this is the correct pid by checking the ran command
docker exec container cat /proc/pid/cmdline | tr '\0' ' '

Resources