I would like use docker-compose to build/run dockerfiles that have envars in their FROM keyword. The problem that I am getting now is that I seem to be unable to pass envars from my environment through docker-compose into the dockerfile.
docker-compose.yml
version: "3.2"
services:
api:
build: 'api/'
restart: on-failure
depends_on:
- mysql
networks:
- frontend
- backend
volumes:
- ./api/php/:/var/www/html/
Dockerfile in 'api/'
FROM ${DOCKER_IMAGE_API}
RUN apk update
RUN apk upgrade
RUN docker-php-ext-install mysqli
Why?
I want to do this so that I can run docker-compose from a bash script that detects the host architecture and changes the base image of the underlying dockerfiles in the host application.
FROM instructions support variables that are declared by any ARG instructions that occur before the first FROM. So what you can do is this:
ARG IMAGE
FROM $IMAGE
when you run the build command, you then pass the --build-arg as follows:
docker build -t test --build-arg IMAGE=alpine .
you can also choose to have a default value for the IMAGE variable, to be used if the --build-arg flag isn't used.
Alternatively, in case you were to use docker compose build and not docker build (and I think this is your case), you can specify the variable in the docker-compose build --build-arg:
version: "3.9"
services:
api:
build: .
and then
docker compose build --build-arg IMAGE=alpine
Related
I am trying Teamcity on-premise solution using docker-compose.
I have a project which needs a docker compose runner type.
Here is the docker compose override yaml file:
version: '3'
services:
nextjs:
build:
context: ./frontend
dockerfile: ${NEXTJS_DOCKERFILE:-Dockerfile}
volumes:
- ./frontend:/app
- /app/node_modules
- /app/.next
nginx:
build: ./nginx
ports:
- 80:80
I have added the env variable using parameter on build configuration too:
The docker compose step gives the following error:
It seems like the variable substitution is not happening, I am not sure why.
Could anyone tell me what is happening?
Team city version : 2022.04.2
Thanks
I've set up two windows container for ASP.NET and MSSQL server. On the first docker-compose build everything works as expected. Then after I've made some changes to the custom dockerfile and run docker-compose build again it uses the old container again, not making any changes.
I assumed that when i did a build it created a new container. Am i misunderstanding how docker works?
This is the docker-compose.yml
version: '3'
services:
db:
image: microsoft/mssql-server-windows-developer
environment:
sa_password: "Password1234!"
ACCEPT_EULA: "Y"
ports:
- "8003:1433"
build:
context: .
dockerfile: mssql.dockerfile
web:
build:
context: .
dockerfile: web.dockerfile
image: mcr.microsoft.com/dotnet/framework/aspnet:4.8
#volumes:
# - .:C:/inetpub/wwwroot
ports:
- "8080:80"
- "8081:431"
This is the mssql.dockerfile
# escape=`
FROM microsoft/mssql-server-windows-developer
#set shell
SHELL ["powershell.exe", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
#make temp folder
RUN mkdir C:\temp
#copy script to temp folder
COPY DownloadDatabase.ps1 C:\temp
COPY RestoreDatabase.ps1 C:\temp
#run script to retrieve production database
WORKDIR C:\temp
RUN .\DownloadDatabase.ps1 -sourcefile <url> -destinationfile <target>
CMD .\RestoreDatabase.ps1
It is very easy to tell if the image has been re-used because the mkdir C:\temp errors out saying the directory already exists.
EDIT: I've already tried all the options on docker compose. no-cache, force-rm
docker-compose build
Only builds images but does not start containers.
That's why your changes in dockerfile are not applied. You have rebuilded the image but not the container. It's the reason why the container previoulsy launched is based on the older version of the image.
docker-compose up
From Docker documentation :
If there are existing containers for a service, and the service’s configuration or image was changed after the container’s creation, docker-compose up picks up the changes by stopping and recreating the containers (preserving mounted volumes). To prevent Compose from picking up changes, use the --no-recreate flag.
In order to make shure that both of your image and container are rebuilded you have to add this flags :
docker-compose up --force-recreate --build
That way your containers are based on the correct image version.
Explanation on flags from Docker documentation :
--build Build images before starting containers.
--force-recreate Recreate containers even if their configuration
and image haven't changed.
If you want to do this for a specific service just add the service name at the end of command line :
docker-compose up --force-recreate --build serviceName
Another flag useful if you want a clear output is the -d flag :
-d, --detach Detached mode: Run containers in the background,
print new container names. Incompatible with
It turns out i simply had to do docker-compose pull before docker-compose build to refresh the dockerfile! Now it builds a fresh image every time!
I'm having a problem installing laravel through a dockerfile. I'm using docker-compose that pulls a dockerfile where I basically have this:
FROM php:7.3-apache-stretch
*some apt-get and install composer*
WORKDIR /var/www
RUN composer create-project --prefer-dist laravel/laravel app
CMD apachectl -D FOREGROUND
but when I access the container and I will see the files that should have been created with the composer I see that it is empty even though I have seen the command executed in the container build.
The container is working perfectly and even I can access it ... only files that do not even appear.
If I run the composer command manually after the container is created the files appear.
In your Dockerfile, you used WORKDIR /var/www and then RUN composer create-project ... which makes composer create files under /var/www on the container file system.
In your docker-compose.yml file you used to start your container:
version: '3.7'
services:
app:
container_name: "app"
build:
context: ./docker
dockerfile: Dockerfile-app
ports:
- "80"
- "443"
restart: always
depends_on:
- db
volumes:
- ".:/var/www"
You are declaring a volume that will be mounted on that same location /var/www in your container.
What happens is that the volume content will take the place of whatever you had on /var/www in the container file system.
I suggest you read carefully the documentation regarding docker volumes, and more specifically the part titled Populate a volume using a container.
Now to move on, ask yourself why you needed that volume in the first place. Is it necessary to change files at run time ?
If not, just add your files at build time:
FROM php:7.3-apache-stretch
*some apt-get and install composer*
WORKDIR /var/www
RUN composer create-project --prefer-dist laravel/laravel app
COPY . /var/www
CMD apachectl -D FOREGROUND
and remove the volume for /var/www.
EDIT
Developing with the help of a Docker container
During development, you change php files on your docker host (assumed to be you development computer) and need to frequently test the result by testing your app served by the webserver from the docker container.
It would be cumbersome to have to rebuild a Docker image every time you need to test your app. The solution is to mount a volume so that the container can serve the files from your development computer:
FROM php:7.3-apache-stretch
*some apt-get and install composer*
WORKDIR /var/www
CMD apachectl -D FOREGROUND
and start it with:
version: '3.7'
services:
app:
container_name: "app"
build:
context: ./docker
dockerfile: Dockerfile-app
ports:
- "80"
- "443"
restart: always
depends_on:
- db
volumes:
- ".:/var/www"
...
When you need to run some commands within that container, just use docker exec:
docker-compose exec app composer create-project --prefer-dist laravel/laravel app
Producing project artifacts
Since what you will be deploying is not a zip/tar archive containing your source code and configurations but a docker image, you need to build the docker image you will use at deployment time.
Dockerfile for production
For production use, you want to have a Docker image which holds all required files and does not need any docker volume, excepted for holding data produced by users (uploaded files, database files, etc)
FROM php:7.3-apache-stretch
*some apt-get and install composer*
WORKDIR /var/www
COPY . /var/www
CMD apachectl -D FOREGROUND
Notice that there is no RUN composer create-project --prefer-dist laravel/laravel app in this Dockerfile. This is because this command is to initialise your project and this is a development time task, not a deployment time task.
You will also need a place to host your docker images (a Docker registry). You can deploy your own registry as a Docker container using the official registry image, or use the one provided by companies:
Gitlab.com - Gitlab Registry (free)
Docker.com - hub.docker.com (1 private image free)
Google.com - Google Container Registry
...
So you need to build a docker image, and then push that image on your registry. Best practice is to automate those tasks with the help of continuous integration tools such as Jenkins, Gitlab CI, Travis CI, Circle CI, Google Cloud Build ...
Your CI job will run the following commands:
git clone <url of you git repo> my_app
cd my_app
git checkout <some git tag>
docker build -t <registry>/<my_app>:<version>
docker login <registry> --user=<registry user> --password=<registry password>
docker push <registry>/<my_app>:<version>
Deploying your Docker image
Start you container with:
version: '3.7'
services:
app:
container_name: "app"
image: <registry>/<my_app>:<version>
ports:
- "80"
- "443"
restart: always
depends_on:
- db
...
Notice here that the docker-compose file does not build any image. For production it is a better practice to refer to an already built docker image (which has been deployed earlier on a staging environment for validation).
I am trying to run a golang app using docker-compose, below is my compose configuration.
version: '2'
services:
#Application container
go:
image: golang:1.8-alpine
ports:
- "80:8080"
links:
- mongodb
environment:
DEBUG: 'true'
PORT: '8080'
working_dir: /go/src/simple-golang-app
command: go run main.go
volumes:
- ./simple-golang-app:/go/src/simple-golang-app
mongodb:
image: mvertes/alpine-mongo:3.2.3
restart: unless-stopped
ports:
- "27017:27017"
On running the compose using command "docker-compose up" i get error "stat main.go: no such file or directory" even when main.go is available in working directory.
it works fine when your host dir layout is
oxo#thor ~/Dropbox/Documents/code/docker/golang_working_dir $ find .
.
./docker-compose.yaml
./simple-golang-app
./simple-golang-app/main.go
so here we
cd ~/Dropbox/Documents/code/docker/golang_working_dir
docker-compose up
for a more complex build involving dependancies I use a Dockerfile :
FROM golang:1.8-alpine
RUN mkdir -p /go/src/simple-golang-app/
COPY simple-golang-app/main.go /go/src/simple-golang-app
WORKDIR /go/src/simple-golang-app
RUN apk add --no-cache git mercurial && go get -v -t ./... && apk del git mercurial
RUN go install ./...
RUN go build
ENV PORT 9000
now update your docker-compose.yaml to use this new image :
old
image: golang:1.8-alpine
new
image: nirmal_golang_alpine:latest
so your commands are
docker build --tag nirmal_golang_alpine
docker-compose up
I want to test ELK.
It works fine
BUt when I want to do a
docker-compose up
behind a proxy
docker-compose up --no-recreate
Building kibana
Step 1 : FROM kibana:latest
---> 544887fbfa30
Step 2 : RUN apt-get update && apt-get install -y netcat
---> Running in 794342b9d807
It failed
W: Some index files failed to download. They have been ignored, or old ones used instead.
Is' OK with
docker build --build-arg http_proxy=http://proxy:3128 --build-arg https_proxy=http://proxy:3128 kibana
But when I redo a docker-compose up, il tries to re-build, and failed to pass through proxy
Any help ?
You will need docker-compose 1.6.0-rc1 in order to pass the proxy to your build through docker-compose.
See commit 47e53b4 from PR 2653 for issue 2163.
Move all build related configuration into a build: section in the service.
Example:
web:
build:
context: .
dockerfile: Dockerfile.name
args:
key: value
As mkjeldsen points out in the comments
If key should assume the value of an environment variable of the same name, value can be omitted (docker-compose ARGS):
Especially useful for https_proxy: if the envvar is unset or empty, the builder will not apply proxy, otherwise it will.
I ran into the same problem. What helped me was using the explicit version 2.2 and then build - args and - network as described in the documentation.
VonC is right, it works for me by adding args section under the build lines in docker-compose file:
original:
ssh:
build: ssh/.
container_name: ssh
ports:
- "3000:22"
networks:
vault_net:
ipv4_address: 172.16.238.20
Modified:
ssh:
build:
context: "ssh/."
args:
HTTP_PROXY: http://X.X.X.X:XXXX
HTTPS_PROXY: http://X.X.X.X:XXXX
NO_PROXY: .domain.ltd,127.0.0.1
container_name: ssh
ports:
- "3000:22"
networks:
vault_net:
ipv4_address: 172.16.238.20
Note that I have to add quotes for context since it needs to be formatted as string.
Thanks a lot.
did you try it on clean machine?
docker-machine stop default
docker-machine create -d virtualbox test
docker-machine start test
eval $(docker-machine env test)
docker-compose up