Environment variables not showing up on mac - bash

Environment variables set outside the shell, through ~/.bash_profile or ~/.bashrc do not appear to docker, or env, despite being accessible in the shell.
Bash_profile contains the line TEST_ENV_VAR=123, after restarting terminal, the variable can be accessed through $TEST_ENV_VAR, however docker and env cannot access this environment variable.
Henrys-MacBook-Pro:~ henry$ echo $TEST_ENV_VAR
123
Henrys-MacBook-Pro:~ henry$ docker run -it -e TEST_ENV_VAR ubuntu env | grep TEST_ENV_VAR
Henrys-MacBook-Pro:~ henry$ env | grep TEST_ENV_VAR
And yet the terminal can access it, and even pass it into docker:
Henrys-MacBook-Pro:~ henry$ docker run -it -e TEST_ENV_VAR=${TEST_ENV_VAR} ubuntu env | grep TEST_ENV_VAR
TEST_ENV_VAR=123
And the issue isn't an issue with environment variable in general, as variables set in the terminal work as expected:
Henrys-MacBook-Pro:~ henry$ export TEST_ENV_VAR=1234
Henrys-MacBook-Pro:~ henry$ docker run -it -e TEST_ENV_VAR ubuntu env | grep TEST_ENV_VAR
TEST_ENV_VAR=1234
I'm running macOS Mojave, 10.14.5, classic terminal, docker 19.03.4, the output of ls -al ~:
-rw-r--r-- 1 henry staff 455 Nov 12 11:50 .bash_profile

docker doesn't actually start a container. Instead, it sends a message to the Docker engine (running in a separate environment unrelated to the environment in which docker is executed) requesting that it start a container for you. As such, the new container won't inherit any of variables in your current shell environment.
With your TEST_ENV_VAR=${TEST_ENV_VAR} attempt, you are explicitly telling docker to create an environment variable named TEST_ENV_VAR, with the value produced by expanding TEST_ENV_VAR now, in the new container, rather than trying to inherit the variable from the appropriate environment.
Even ignoring that, you aren't actually creating an environment variable with TEST_ENV_VAR=123; you've only created an ordinary shell variable. For env to see it, you need to first export it.
$ TEST_ENV_VAR=123
$ env | grep TEST_ENV_VAR
$ export TEST_ENV_VAR
$ env | grep TEST_ENV_VAR
TEST_ENV_VAR=123

The .bash_profile is for your user. Docker is running in its own environment (as opposed to running in a subshell spawned by your user). Export only sends information down into child shells. You need to make the variable available at a higher level.
/etc/environment
Be careful in there.
Alternatively, you may look into asking Docker to make these changes itself.

Related

Copy Current Env Vars into `docker run`'s Scope

If I'm using a docker container with an entry point set, I can run that container via the following command
docker run -it my-container-tag
If the program in my container requires an environmental variable, I can pass that var via the -e flag
docker run -it -e FOO=bar my-container-tag
If I have a program that uses many environment variable, I get an unwieldy mess that becomes hard to type.
docker run -it -e FOO=bar -e BAZ=zip -e ZAP=zing -e ETC=omg-stop my-container-tag
Is there a way to tell docker run to inherit all the env variables currently set in my shell's scope? If not, are there common practices for working around needing to type in these variables again and again?
You cant inherit the envs, I usually use docker-compose to set my envs when there is too much, or build the container with the environments variables inside it if you dont need to change frequently.

Set environment variables in Docker

I'm having trouble with Docker creating a container that does not have environment variables set that I know I set in the image definition.
I have created a Dockerfile that generates an image of OpenSuse 42.3. I need to have some environment variables set up in the image so that anyone that starts a container from the image can use a code that I've compiled and placed in the image.
I have created a shell file called "image_env_setup.sh" that contains the necessary environment variable definitions. I also manually added those environment variable definitions to the Dockerfile.
USER codeUser
COPY ./docker/image_env_setup.sh /opt/MyCode
ENV PATH="$PATH":"/opt/MyCode/bin:/usr/lib64/mpi/gcc/openmpi/bin"
ENV LD_LIBRARY_PATH="/usr/lib64:/opt/MyCode/lib:"
ENV PS1="[\u#docker: \w]\$ "
ENV TERM="xterm-256color"
ENV GREP_OPTIONS="--color=auto"
ENV EDITOR=/usr/bin/vim
USER root
RUN chmod +x /opt/MyCode/image_env_setup.sh
USER codeUser
RUN /opt/MyCode/image_env_setup.sh
RUN /bin/bash -c "source /opt/MyCode/image_env_setup.sh"
The command that I use to create the container is:
docker run -it -d --name ${containerName} -u $userID:$groupID \
-e USER=$USER --workdir="/home/codeUser" \
--volume="${home}:/home/codeUser" ${imageName} /bin/bash \
The only thing that works is to pass the shell file to be run again when the container starts up.
docker start $MyImageTag
docker exec -it $MyImageTag /bin/bash --rcfile /opt/MyCode/image_env_setup.sh
I didn't think it would be that difficult to just have the shell variables setup within the container so that any entry into it would provide a user with them already defined.
RUN entries cannot modify environment variables (I assume you want to set more variables in image_env_setup.sh). Only ENV entries in the Dockerfile (and docker options like --rcfile can change the environment).
You can also decide to source image_env_setup.sh from the .bashrc, of course.
For example, you could either pre-fabricate a .bashrc and pull it in with COPY, or do
RUN echo '. /opt/MyCode/image_env_setup.sh' >> ~/.bashrc
you can put /opt/MyCode/image_env_setup.sh in ~/.bash_profile or ~/.bashrc of the container so that everytime you get into the container you have the env's set

Mac Docker Machine ZSH shell set env variable not applying

I am running Docker Machine on Mac.
docker-machine version 0.6.0, build e27fb87
In my shell I have done
export DOCKER_HOST=tcp://docker.local:2375
export DOCKER_TLS_VERIFY=0
Restarted the machine
When I do
docker-machine env
These do not seem to have been set. I am using the ZSH shell, could this be an issue?
docker-machine env
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.100:2376"
export DOCKER_CERT_PATH="/Users/ciaran/.docker/machine/machines/default"
export DOCKER_MACHINE_NAME="default"
When you run docker-machine env it will show the variables that are needed in order to connect to the default machine. It has nothing to do with the variables in the current shell, and will not set any variables in your shell.
To see the current docker variables in your shell, you can run
$ env | grep DOCKER
If you want to set your shell ENV variables to the ones in docker-machine env, you will need to either copy and paste the output of docker-machine env, or eval the output like this.
$ eval $(docker-machine env <machine name>)
That will set the variables in your shell. This command is actually given to you when you run docker-machine env look at the end of the output.
To confirm it worked, check the shell again.
$ env | grep DOCKER

Setting environment variables when running docker in detached mode

If I include the following line in /root/.bashrc:
export $A = "AAA"
then when I run the docker container in interactive mode (docker run -i), the $A variable keeps its value. However if I run the container in detached mode I cannot access the variable. Even if I run the container explicitly sourcing the .bashrc like
docker run -d my_image /bin/bash -c "cd /root && source .bashrc && echo $A"
such line produces an empty output.
So, why is this happening? And how can I set the environment variables defined in the .bashrc file?
Any help would be very much appreciated!
The first problem is that the command you are running has $A being interpreted by your hosts shell (not the container shell). On your host, $A is likely black, so your effectively command becomes:
docker run -i my_image /bin/bash -c "cd /root && source .bashrc && echo "
Which does exactly as it says. We can escape the variable so it is sent to the container and properly evaluated there:
docker run -i my_image /bin/bash -c "echo \$A"
But this will also be blank because, although the container is, the shell is not in interactive mode. But we can force it to be:
docker run -i my_image /bin/bash -i -c "echo \$A"
Woohoo, we finally got our desired result. But with an added error from bash because there is no TTY. So, instead of interactive mode, we can just set a psuedo-TTY:
docker run -t my_image /bin/bash -i -c "echo \$A"
After running some tests, it appears that when running a container in detached mode, overidding the default environment variables doesnt always happen the way we want, depending on where you are in the Dockerfile.
As an exemple if, running a container in a detached container like so:
docker run **-d** --name image_name_container image_name
Whatever ENV variables you defined within the Dockerfile takes effect everywhere (read the rest and you will understand what the everywhere means).
example of a simple dockerfile (alpine is just a lighweight linux distribution):
FROM alpine:latest
#declaring a docker env variable and giving it a default value
ENV MY_ENV_VARIABLE dummy_value
#copying two dummy scripts into a place where i can execute them straight away
COPY ./start.sh /usr/sbin
COPY ./not_start.sh /usr/sbin
#in this script i could do: echo $MY_ENV_VARIABLE > /test1.txt
RUN not_start.sh
RUN echo $MY_ENV_VARIABLE > /test2.txt
#in this script i could do: echo $MY_ENV_VARIABLE > /test3.txt
ENTRYPOINT ["start.sh"]
Now if you want to run your container in detached and override some ENV variables, like so:
docker run **-d** -e MY_ENV_VARIABLE=new_value --name image_name_container image_name
Surprise! The var MY_ENV_VARIABLE is only overidden inside the script that is run in the ENTRYPOINT (and i checked, same thing happens if your replace ENTRYPOINT with CMD). It would also be overidden in a subscript that you could call from this start.sh script. But the MY_EV_VARIABLE variables that are called within a RUN dockerfile command or within the dockerfile itself do not get overidden.
In other words we would have $MY_ENV_VARIABLE being replaced by the value dummy_value and new_value depending on if you are in the ENTRYPOINT or not.

How do I undo the command $ eval "$(docker-machine env blog)"

I think that command redirected $ docker commands to the docker machine. Now all my docker commands are giving me an error FATA[0000] Couldn't read ca cert... follwed by the path to the docker-machine I created. How can I fix my shell?
What you are looking for is:
eval "$(docker-machine env -u)"
It will unset the DOCKER_* variables.
For the record, here's the output of docker-machine env -u:
unset DOCKER_TLS_VERIFY
unset DOCKER_HOST
unset DOCKER_CERT_PATH
unset DOCKER_MACHINE_NAME
You could also restart your shell. This will drop the variables that minkube docker-env exports.
I can see that this is an old post but if someone else runs into this issue, who is new to docker like me this can help. By typing:
eval $(docker-machine env nameOfVm)
you are setting your current shell to use docker in that docker-machine. You can check if you type docker-machine ls
that under active tab, that status is changed from - to * for that machine. You can also check which machine is active by running docker-machine active.
If you want to undo eval, just run:
eval $(docker-machine env -u)
and it will unset variables for active machine (You don't have to specify the name of the machine). This is all under macOS but I think it should be same on linux as well.
You can read more about this here:
Docker documentation: docker-machine env
I had been searching for an answer to this for quite awhile. Shortly after posting the question on stackoverflow I realized typing in to the terminal the export commands docker gives on startup resolved my issue.
To connect the Docker client to the Docker daemon, please set:
export DOCKER_HOST=tcp:// some IP address
export DOCKER_CERT_PATH= some file path
export DOCKER_TLS_VERIFY=1
All you have to do is run
docker-machine env machine-name
Then, copy and run the last segment in the output to set or remove the env variables.
Which looks like this in Windows :
eval $("C:\Program Files\Docker Toolbox\docker-machine.exe" env machine-name)
If it's set already, docker adds a "-u" at the tail to make the task easy.
like this in Windows :
eval $("C:\Program Files\Docker Toolbox\docker-machine.exe" env -u)
That's all.

Resources