I am trying to develop a Dockerfile for my application that loads a large number of environment variables after initialisation. Somehow, these variables are not reachable when I later execute the following commands:
docker exec -it container_name bash
printenv
My environment variables are not visible. If I load the files manually however, they are:
docker exec -it container_name bash
source .env
printenv
... environment variables are shown ...
This is my dockerfile:
Dockerfile
FROM python:3.6
WORKDIR /usr/src/app
COPY requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt
COPY . /usr/src/app/
RUN chmod 755 load_env_variables.sh
ENTRYPOINT ["/bin/bash", "-c", "/usr/src/app/load_env_variables.sh"]
load_env_variables.sh
#!/bin/bash
source .env
python start_application
And my .env file contains lines als follows: 'export name=value'.
The reason for this behavior is that docker exec -it container_name bash starts a new bash. A new bash has only the standard environment variables plus the ones specified in the .bashrc or .bash_profile files.
A proper solution for your problem would be to use the option --env-file with the docker run command. Be aware that the env-file needs to look like this:
test1=test1
test2=test2
Related
I have a docker image inside which some installations require adding exports to .bashrc.
My export variables are inside /root/.bashrc on the image.
Here is the dockerfile
FROM ubuntu:20.04
RUN apt-get update && apt-get install -y python3
RUN echo "export PATH=/path/to/stuff:\$PATH" >> /root/.bashrc
CMD ["python3"]
The environment variables are present when using the following command
docker run -it image /bin/bash
When I run the following command, environment variables are not present.
docker run -it image
It is expected since /bin/sh is the default entry point of docker
But after the following change, the environment variable are not set either.
docker commit --change='ENTRYPOINT ["/bin/bash","-c"]' container image
I tried different combinations such as
docker commit --change='CMD ["/bin/bash","-c","python3 myProgram.py"]' container image
or
docker commit --change='ENTRYPOINT ["/bin/bash","-c"]' --change='CMD ["source /root/.bashrc && python3 myProgram.py"]' container image
But the environment variables are not present.
How do I run the CMD statement with the environment variable from .bashrc loaded ?
In order to see the path variable, I use echo $PATH when I run /bin/bash and import os followed by os.getenv("PATH") when I run python3 from CMD.
Edit:
The exports are part of the installation of a library. In order to use the library, the updated exports (such as PYTHONPATH and LD_LIBRARY_PATH) needs to be set.
If .bashrc is not intended to be launched, as mentioned in the comments. How can I make this library work in the docker environment ?
As mentioned by #itshosyn in the comments, the standard way to override environment variables such as PATH consists in using the ENV directive.
So you may try writing something like this:
FROM ubuntu:20.04
RUN apt-get update && apt-get install -y python3
ENV PATH="/path/to/stuff:$PATH"
CMD ["python3"]
Here is my dockerfile
FROM httpd:latest
ENV ENV_VARIABLE "http://localhost:8081"
# COPY BUILD AND CONFIGURATION FILES
COPY entrypoint.sh /
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
Here is the entrypoint.sh file
#!/bin/bash
sed -i 's,ENV_VARIABLE,'"$ENV_VARIABLE"',g' /path/to/config/file
exec "$#"
To run the container
docker run -e ENV_VARIABLE=some-value <image-name>
The sed command works perfectly fine and the value from environment variable gets reflected in config file. But whenever i start the container the container stops automatically.
I ran the command docker logs to check the logs but logs were empty.
The Dockerfile reference notes:
If CMD is defined from the base image, setting ENTRYPOINT will reset CMD to an empty value. In this scenario, CMD must be defined in the current image to have a value.
So you need to find the CMD from the base image and repeat it in your Dockerfile. Among other places, you can find that on the Docker hub listing of the image history
CMD ["httpd-foreground"]
docker inspect httpd or docker history httpd would also be able to tell you this.
I'm having trouble with Docker creating a container that does not have environment variables set that I know I set in the image definition.
I have created a Dockerfile that generates an image of OpenSuse 42.3. I need to have some environment variables set up in the image so that anyone that starts a container from the image can use a code that I've compiled and placed in the image.
I have created a shell file called "image_env_setup.sh" that contains the necessary environment variable definitions. I also manually added those environment variable definitions to the Dockerfile.
USER codeUser
COPY ./docker/image_env_setup.sh /opt/MyCode
ENV PATH="$PATH":"/opt/MyCode/bin:/usr/lib64/mpi/gcc/openmpi/bin"
ENV LD_LIBRARY_PATH="/usr/lib64:/opt/MyCode/lib:"
ENV PS1="[\u#docker: \w]\$ "
ENV TERM="xterm-256color"
ENV GREP_OPTIONS="--color=auto"
ENV EDITOR=/usr/bin/vim
USER root
RUN chmod +x /opt/MyCode/image_env_setup.sh
USER codeUser
RUN /opt/MyCode/image_env_setup.sh
RUN /bin/bash -c "source /opt/MyCode/image_env_setup.sh"
The command that I use to create the container is:
docker run -it -d --name ${containerName} -u $userID:$groupID \
-e USER=$USER --workdir="/home/codeUser" \
--volume="${home}:/home/codeUser" ${imageName} /bin/bash \
The only thing that works is to pass the shell file to be run again when the container starts up.
docker start $MyImageTag
docker exec -it $MyImageTag /bin/bash --rcfile /opt/MyCode/image_env_setup.sh
I didn't think it would be that difficult to just have the shell variables setup within the container so that any entry into it would provide a user with them already defined.
RUN entries cannot modify environment variables (I assume you want to set more variables in image_env_setup.sh). Only ENV entries in the Dockerfile (and docker options like --rcfile can change the environment).
You can also decide to source image_env_setup.sh from the .bashrc, of course.
For example, you could either pre-fabricate a .bashrc and pull it in with COPY, or do
RUN echo '. /opt/MyCode/image_env_setup.sh' >> ~/.bashrc
you can put /opt/MyCode/image_env_setup.sh in ~/.bash_profile or ~/.bashrc of the container so that everytime you get into the container you have the env's set
There seems to be a difference between how Docker runs commands in a Dockerfile versus running commands manually after starting a container. This seems to be due to the kind of shells you can start, a (I assume) non-interactive shell with a Dockerfile vs an interactive one when running something like docker run -it <some-img-id>.
How can I debug running commands in a Docker container so that it runs exactly like the commands are run from a Dockerfile? Would just adding /bin/bash --noprofile to the run cmd suffice? Or is there anything else different about the environment when started from a Dockerfile?
What you are experiencing is the behavior because of the shell. Most of us are used to using the bash shell. So generally we would attempt to run the commands in the below fashion
For new container
docker run -it <imageid> bash
For existing container
docker exec -it <containerid> bash
But when we specify some command using RUN directive inside a Dockerfile
RUN echo Testing
Then it is equivalent to running /bin/sh -c 'echo Testing'. So you can expect certain differences as both the shells are different.
In Docker 1.12 or higher you have a Dockerfile directive named SHELL this allows you to override the default SHELL
SHELL ["/bin/bash", "-c"]
RUN echo Testing
This would make the RUN command be executed as bash -c 'echo Testing'. You can learn more about the SHELL directive here
Short answer 1:
If Dockerfile don't use USER and SHELL commands, then this:
docker --entrypoint "/bin/sh -c" -u root <image> cmd
Short answer 2:
If you don't squash or compress image after the build, Docker creates images layers for each of the Dockerfile commands. You can see them in the output of docker build at the end of each step with --->:
Step 2/8 : WORKDIR /usr/src/app
---> 5a5964bed25d # <== THIS IS IMAGE ID OF STEP 2
Removing intermediate container b2bc9558e499
Step 3/8 : RUN something
---> f6e90f0a06e2 # <== THIS IS IMAGE ID OF STEP 3
Removing intermediate container b2bc9558e499
Look for the image id just before the RUN step you want to debug (for example you want to debug step 3 on above, take the step 2 image id). Then just run the command in that image:
docker run -it 5a5964bed25d cmd
Long answer 1:
When you run docker run [image] cmd Docker in fact starts the cmd in this way:
Executes the default entrypoint of the image with the cmd as its argument. Entrypoint is stored in the image on build by ENTRYPOINT command in Dockerfile. Ie if cmd is my-app and entrypoint is /bin/sh -c, it executes /bin/sh -c my-app.
Starts it with default user id of the image, which is defined by the last USER command in Dockerfile
Starts it with the environment variables from all ENV commands from image's Dockerfile commulative
When docker build runs the Dockerfile RUN, it does exatly the same, only with the values present at that time (line) of the Dockerfile.
So to be exact, you have to take the value of ENVs and last USER command before your RUN line, and use those in the docker run command.
Most common images have /bin/sh -c or /bin/bash -c as entrypoint and most likely the build operates with root user. Therefore docker --entrypoint "/bin/bash -c" -u root <image> cmd should be sufficient
If I include the following line in /root/.bashrc:
export $A = "AAA"
then when I run the docker container in interactive mode (docker run -i), the $A variable keeps its value. However if I run the container in detached mode I cannot access the variable. Even if I run the container explicitly sourcing the .bashrc like
docker run -d my_image /bin/bash -c "cd /root && source .bashrc && echo $A"
such line produces an empty output.
So, why is this happening? And how can I set the environment variables defined in the .bashrc file?
Any help would be very much appreciated!
The first problem is that the command you are running has $A being interpreted by your hosts shell (not the container shell). On your host, $A is likely black, so your effectively command becomes:
docker run -i my_image /bin/bash -c "cd /root && source .bashrc && echo "
Which does exactly as it says. We can escape the variable so it is sent to the container and properly evaluated there:
docker run -i my_image /bin/bash -c "echo \$A"
But this will also be blank because, although the container is, the shell is not in interactive mode. But we can force it to be:
docker run -i my_image /bin/bash -i -c "echo \$A"
Woohoo, we finally got our desired result. But with an added error from bash because there is no TTY. So, instead of interactive mode, we can just set a psuedo-TTY:
docker run -t my_image /bin/bash -i -c "echo \$A"
After running some tests, it appears that when running a container in detached mode, overidding the default environment variables doesnt always happen the way we want, depending on where you are in the Dockerfile.
As an exemple if, running a container in a detached container like so:
docker run **-d** --name image_name_container image_name
Whatever ENV variables you defined within the Dockerfile takes effect everywhere (read the rest and you will understand what the everywhere means).
example of a simple dockerfile (alpine is just a lighweight linux distribution):
FROM alpine:latest
#declaring a docker env variable and giving it a default value
ENV MY_ENV_VARIABLE dummy_value
#copying two dummy scripts into a place where i can execute them straight away
COPY ./start.sh /usr/sbin
COPY ./not_start.sh /usr/sbin
#in this script i could do: echo $MY_ENV_VARIABLE > /test1.txt
RUN not_start.sh
RUN echo $MY_ENV_VARIABLE > /test2.txt
#in this script i could do: echo $MY_ENV_VARIABLE > /test3.txt
ENTRYPOINT ["start.sh"]
Now if you want to run your container in detached and override some ENV variables, like so:
docker run **-d** -e MY_ENV_VARIABLE=new_value --name image_name_container image_name
Surprise! The var MY_ENV_VARIABLE is only overidden inside the script that is run in the ENTRYPOINT (and i checked, same thing happens if your replace ENTRYPOINT with CMD). It would also be overidden in a subscript that you could call from this start.sh script. But the MY_EV_VARIABLE variables that are called within a RUN dockerfile command or within the dockerfile itself do not get overidden.
In other words we would have $MY_ENV_VARIABLE being replaced by the value dummy_value and new_value depending on if you are in the ENTRYPOINT or not.