Example
Consider this example docker-compose file with custom .env file:
version: '3'
services:
service_example:
build:
dockerfile: Dockerfile
context: .
args:
AAA: ${AAA}
command: python3 src/service/run.py
env_file:
- custom_env.env
custom_env.env:
AAA=qqq
When I run docker-compose config I get the following output:
WARNING: The AAA variable is not set. Defaulting to a blank string.
services:
service_example:
build:
args:
AAA: '' <----------------------------- ??????
context: /Users/examples
dockerfile: Dockerfile
command: python3 src/service/run.py
environment:
AAA: qqq
version: '3'
Question
Why AAA is unset in build section?
What should I do to set it properly (to the value provided from custom file: AAA=qqq)?
I've also noticed that if I change the env file name to the default setting mv custom_env.env .env and remove env_file section from docker-compose.yml - everything will be just fine:
services:
service_example:
build:
args:
AAA: qqq
context: /Users/examples
dockerfile: Dockerfile
command: python3 src/service/run.py
version: '3'
Quick Answer
docker-compose --env-file custom_env.env config
Answers Explanation
Question-1: Why AAA is unset in build section?
Because the env file specified in env_file property custom_env.env is specific for the Container only, i.e. those variables are to be passed to container while running not during image build.
Question-2: What should I do to set it properly (to the value provided from custom file: AAA=qqq)?
To provide environment variables for build step in docker-compose file using custom env file, we need to specify the custom file path. Like
Syntax: docker-compose --env-file FILE_PATH config
Example: docker-compose --env-file custom_env.env config
Question-3: How .env works?
Because that is the default file for which docker-compose looks for.
Summary
So, In docker-compose for current scenario we can consider 2 stages for specifying env
Build Stage(Image)
Running Stage(Container)
For Build stage - we can use .env default file or we can use --env-file option to specify custom env file
For Running Stage - we can specify environment variables using environment: property or we can use env_file: property to specify a env file
References
https://docs.docker.com/compose/env-file/
https://docs.docker.com/compose/environment-variables/
https://docs.docker.com/compose/compose-file/#env_file
https://docs.docker.com/compose/compose-file/#environment
Related
I would like use docker-compose to build/run dockerfiles that have envars in their FROM keyword. The problem that I am getting now is that I seem to be unable to pass envars from my environment through docker-compose into the dockerfile.
docker-compose.yml
version: "3.2"
services:
api:
build: 'api/'
restart: on-failure
depends_on:
- mysql
networks:
- frontend
- backend
volumes:
- ./api/php/:/var/www/html/
Dockerfile in 'api/'
FROM ${DOCKER_IMAGE_API}
RUN apk update
RUN apk upgrade
RUN docker-php-ext-install mysqli
Why?
I want to do this so that I can run docker-compose from a bash script that detects the host architecture and changes the base image of the underlying dockerfiles in the host application.
FROM instructions support variables that are declared by any ARG instructions that occur before the first FROM. So what you can do is this:
ARG IMAGE
FROM $IMAGE
when you run the build command, you then pass the --build-arg as follows:
docker build -t test --build-arg IMAGE=alpine .
you can also choose to have a default value for the IMAGE variable, to be used if the --build-arg flag isn't used.
Alternatively, in case you were to use docker compose build and not docker build (and I think this is your case), you can specify the variable in the docker-compose build --build-arg:
version: "3.9"
services:
api:
build: .
and then
docker compose build --build-arg IMAGE=alpine
I define a variable in an .env file:
LOCAL_IP=127.0.0.1
When using this in the following docker-compose.yml I can see that the variable is expanded to 127.0.0.1 by running docker-compose config.
docker-compose.yml
services:
my-service:
environment:
# This gets expanded to http://127.0.0.1/services
SERVICE_ENDPOINT: http://${LOCAL_IP}/services
Now I'd like to reference the value of LOCAL_IP in another variable in the same file like:
LOCAL_IP=127.0.0.1
SOME_OTHER_VAR=$LOCAL_IP
Updated docker-compose.yml
services:
my-service:
environment:
SERVICE_ENDPOINT: http://${SOME_OTHER_VAR}/services
Running docker-compose config then gives me:
services:
my-service:
environment:
# I'd have expected http://127.0.0.1/services
SERVICE_ENDPOINT: http://$$LOCAL_IP:8090/services
I have this Dockerfile:
FROM php:5.6-apache
WORKDIR /var/www/html/
# ENV VARIABLES
ENV INI_FOLDER /usr/local/etc/php
ENV WWW_FOLDER /var/www
# ADD THE ENV CONFIGURATOR AND SET PERMISSIONS
ADD env.sh $WWW_FOLDER/
RUN chmod +x $WWW_FOLDER/env.sh
RUN /var/www/env.sh
The problems is that in env.sh I don't have access to the variables set on the docker-compose. Is there any workaround to fix this?
UPDATE: added docker-compose
version: '2.0'
services:
app:
env_file:
- app/mysql.env
- app/app.env
volumes:
- C:\Users\svirl\Documents\workspace\docker\my-app:/var/www/html/:rw
build: app
Below is the content of my docker-compose.yml file:
eureka-server:
image: controlsplm/eureka-server
environment:
HOST_IP: X.X.X.X
ACTIVE_PROFILE=docker-development-cloud
ports:
- "8761:8761"
restart: always
And below is the content of my docker file:
FROM java:8
VOLUME /tmp
ADD eureka-server-0.1.0-SNAPSHOT.jar app.jar
EXPOSE 8761
RUN bash -c 'touch /app.jar'
ENTRYPOINT ["java","-Denvironment=$HOST_IP","-Dspring.profiles.active=$ACTIVE_PROFILE","-jar","/app.jar"]
But when i run the docker container using compose, HOST_IP is picked up but not ACTIVE_PROFILE. AM i missing anything here? Kindly help...
Indeed, if you have Spring properties
my.spring.property.one=green
my.spring.property.two=blue
You can include them as follows in the docker-compose.yml:
environment:
- my_spring_property_one=green
- my_spring_property_two=blue
worked for my with
Spring Boot v1.5.10.RELEASE
Spring v4.3.14.RELEASE
docker version '3.6'
I see two problems:
the format of the ACTIVE_PROFILE is wrong, it would be ACTIVE_PROFILE: docker-development-cloud as mentioned by #andreas-jägle in the comments
The entrypoint uses json array notation, which means the command is exec'ed directly without a shell. The shell (usually bash) is what replaced the variables with their values, so you need to run in a shell to use those variables. You can either use the string form of ENTRYPOINT, or use:
ENTRYPOINT ["bash", "-c" ,"java - Dspring.profiles.active=$ACTIVE_PROFILE ..."]
I resolved the issue by adding below lines to my docker-compose.yml file:
eureka-server:
image: controlsplm/eureka-server:latest
environment:
HOST_IP: X.X.X.X
SPRING_PROFILES_ACTIVE: docker-development-cloud
ports:
- "8761:8761"
restart: always
and below lines in dockerfile:
FROM java:8
VOLUME /tmp
ADD eureka-server-0.1.0-SNAPSHOT.jar app.jar
EXPOSE 8761
RUN bash -c 'touch /app.jar'
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-Denvironment=$HOST_IP","-jar","/app.jar"]
I have a Dockerised application which I would like to run in both proxy and non-proxy host environments. I'm trying to resolve this problem by copying the normal environment variables, such as http_proxy, into the containers if and only if they exist in the host.
I can get 90% of the way there by running
set | grep -i _proxy=>proxies.env
in a top-level script, and then having, in my docker-compose.yml:
myserver:
build: ./myserver
env_file:
- proxies.env
This copies the host's environmental proxy variables, if any, into the server container, and it works in the sense that these variables are available at container run time, in other words by the stage that the Dockerfile CMD or ENTRYPOINT executes.
However I have one container which needs to run npm as a build step, ie from a RUN command in the Dockerfile, and these variables appear not to be present at this stage, so npm can't find the proxy and hangs. In other works, if I have
RUN set
in my Dockerfile, I can't see any variables from proxies.env, but if I do
docker exec -it myserver /bin/bash
and then run set, I can see everything from proxies.env.
Can anyone recommend a way to make these variables visible at container build time, without having to hard-code them, so that my docker-compose.yml and Dockerfile will still work both for hosts with proxies and hosts without proxies?
(Running with centos 7, docker-compose 1.3.1 and docker 1.7.0)
Update 2016, docker-compose 1.6.2, docker 1.10+, with a docker-compose.yml version 2:
You now have the args: sub-section of the build: section, which includes that very interesting possibility:
Build arguments with only a key are resolved to their environment value on the machine Compose is running on.
See PR 2653 (January 2016)
As a result, a way to introduce the proxy variables without hard-coding them in the docker-compose.yml file itself is with that precise syntax:
version: '2'
services:
myservice:
build:
context: .
args:
- http_proxy
- https_proxy
- no_proxy
Before calling docker-compose up, you need to make sure your proxy environment variables are set:
export http_proxy=http://username:password#proxy.com:port
export https_proxy=http://username:password#proxy.com:port
export no_proxy=localhost,127.0.0.1,company.com
docker-compose up
Then your Dockerfile built by the docker-compose process will pick up automatically the proxy variable values, even though the docker-compose.yml does not include any hard-coded specific values.
May be you the "environment" option solves your problem. In your docker compose file would looks like:
myserver:
build: ./myserver
environment:
- HTTP_PROXY=192.168.1.8
- VARIABLE=value
- ...
Maybe you can try this:
Before you call RUN, ADD the .env file into the image
ADD proxies.env proxies.env
then prefix your RUN statement:
RUN export `cat proxies.env` && echo "FOO is $FOO and BAR is $BAR"
This produces the following output:
root#armenubuntudev:~/Dockers/set-env# docker build -t ashimoon/envtest .
Sending build context to Docker daemon 3.584 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu
---> 91e54dfb1179
Step 1 : ADD proxies.env proxies.env
---> Using cache
---> 181d0e082e65
Step 2 : RUN export `cat proxies.env` && echo "FOO is $FOO and BAR is $BAR"
---> Running in 30426910a450
FOO is 1 and BAR is 2
---> 5d88fcac522c
Removing intermediate container 30426910a450
Successfully built 5d88fcac522c
docker-compose.yml
...
server:
build: .
args:
env: $ENV
...
Dockerfile
ARG env
ENV NODE_ENV $env
This example fixes YUM.
version: '2'
services:
example-service:
build:
context: .
args:
http_proxy: proxy.example.com:80