I have a rather simple architecture.
A frontend listen from the internet; it puts message on a message broker; the message are consume by a backend.
I have a docker-compose file for the frontend and the message broker, one for the message broker and the backend both of which works very well. However I need a complete docker-compose with all 3 of those services together.
The main problem is that either the frontend and the backend need to start using make.
As an example the frontend:
frontend:
build: .
command: make serve
depends_on:
- message_broker
message_broker:
image: message/broker
the backend is pretty much the same.
Now I tried to just clue the two together:
frontend:
build: frontend
command: make serve
depends_on:
- message_broker
backend:
build: backend
command: make serve
depends_on:
- message_broker
- frontend
message_broker:
image: message/broker
However when I try to run it with docker-compose up I get make: *** No rule to make target 'serve'. Stop. for either services, frontend and backend.
Of course there is the rule for serve inside each Makefile.
A single dockerfile is like this:
FROM golang:1.6.2
ENV PROJECT_PATH=/go/src/github.com/brocaar/lora-semtech-bridge
ENV PATH=$PATH:$PROJECT_PATH/build
# install tools
RUN go get github.com/golang/lint/golint
RUN go get github.com/kisielk/errcheck
# setup work directory
RUN mkdir -p $PROJECT_PATH
WORKDIR $PROJECT_PATH
# copy source code
COPY . $PROJECT_PATH
# build
RUN make build
CMD ["semtech-bridge"]
I have like the impression that I am running make inside the wrong directory, how do I fix it?
Your impression is correct. :p Set the WORKDIR in your Dockerfile.
Related
I have a weird problem, that sometimes a docker container cannot see a .jar file, while most of the time it does not have any problem with it.
Before i show you the docker image, a little bit of background. Normally i build a jar archive before running my container, a pretty simple container to run a spring boot application. However at some seemingly random point in the daily routine it does not boot up with the container reporting "Unable to access jarfile".
I thought it must be some weird permission stuff, so i took snapshot of my "target" directory when working and when it stopped working via ls -alR target and later comparing those snapshot with git diff. It does not show any difference. I am still pretty convinced it must be related to file-permissions, locking or something of that sort but i do not know where to start.
I am on Mac 12.0.1 btw. Any ideas appreciated.
The docker file
FROM openjdk:8-oraclelinux8
RUN mkdir /app
WORKDIR /app
CMD "java" "-jar" "app.war"
And docker-compose.yml
version: "3.9"
services:
app:
build: .
depends_on:
- sql1
volumes:
- ./target:/app
ports:
- "8080:8080"
links:
- "sql1:sqlserver"
...
I'm not sure if this helps, but I don't see your Dockerfile as robust enough to produce consistent results regardless of the state of your localhost workspace. I may ask, are you building your war file manually and then creating your Docker container?
Please try to follow this approach if it fits your needs :
make sure you delete jar/war files before building the container.
Have a multistage Dockerfile with a "build" phase for your spring boot app where you generate the jar/war file from a builder image (ant, gradle, maven), and then have a second stage where the jar/war file gets copied over to it's final location and the application gets executed, this way you ensure consistency and that the file will be there at all times :
This is an example for my spring boot templates that I use very often, it's quite generic (as I handle the renaming of the jar file without having to worry about how pom.xml is configured individually) and I guess could be implemented in a variety of scenarios
FROM maven:3.8.6-openjdk-18 as builder
WORKDIR /usr/app/
COPY . /usr/app
RUN mvn package -Dmaven.test.skip
RUN JAR_FILE="target/*.jar"; cp ${JAR_FILE} /app.jar
FROM openjdk:18
WORKDIR /usr/app
COPY --from=builder /app.jar /usr/app
EXPOSE 8080
CMD ["java","-jar","app.jar"]
docker compose :
services:
app:
build: .
depends_on:
- sql1
ports:
- 8080:8080
networks:
- spring-boot-api-network
volumes:
- ./target:/app
...
NOTE : I would also remove the "links" option as it is a legacy feature you should avoid using and use networks instead :
You can try this network implementation added at the bottom of your compose file, just make sure you don't forget to add the network: to the sql1 portion as well
networks:
spring-boot-api-network:
driver: bridge
ipam:
driver: default
config:
- subnet: 182.16.0.1/24
gateway: 182.16.0.1
name: spring-boot-api-network
So I know that there are a lot of tutorials on the topics, both docker and maven, but I'm having some confusion in combining them alltogether.
I created a multi-module Maven project with 2 modules, 2 spring applications, let's call them application 1 and application 2.
Starting each other via IntelliJ IDEA green "run" button works fine, now I'd like to automate things and run via docker.
I have Dockerfiles that looks the same in both cases:
(in both modules it's the same, only JAR name's different)
FROM adoptopenjdk:11-jre-hotspot
MAINTAINER *my name here lol*
ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.9.0/wait /wait
RUN chmod +x /wait
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} application1-0.0.1-SNAPSHOT-jar-with-dependencies.jar
ENTRYPOINT ["java","-jar","/application1-0.0.1-SNAPSHOT-jar-with-dependencies.jar"]
CMD /wait && /*.jar
I also have docker-compose:
version: '2.1'
services:
application1:
container_name: app1
build:
context: ../app1
image: docker.io/myname/app1:latest
hostname: app1
ports:
- "8080:8080"
networks:
- spring-cloud-network-app1
application2:
container_name: app2
build:
context: ../app2
depends_on:
application1:
condition: service_started
links:
- application1
image: docker.io/myname/app2:latest
environment:
WAIT_HOSTS: application1:8080
ports:
- "8070:8070"
networks:
- spring-cloud-network-app2
networks:
spring-cloud-network-app1:
driver: bridge
spring-cloud-network-app2:
driver: bridge
What I do currently is:
I run maven package for each module and receive files like "application1(-2)-0.0.1-SNAPSHOT-jar-with-dependencies.jar" in both target folders.
"docker build -t springio/app1 ."
"docker-compose up --build"
And it works, but I feel I do some extra steps.
How can I do the project so that I ONLY have to run docker compose?
(after each time I change things in the code)
Again, I know it's a quite simple thing but I kinda lost the logic.
Thanks!
P.S
Ah, and about the "...docker-compose-wait/releases/download/2.9.0/wait /wait"
It's important that app start one after another, tried different solutions, unfortunately, doesn't really work as good as I would like to. But I guess I'll leave it as is.
So, again, if anyone ever wonders how to do the things I asked, here's the answer: you need multi-stage build Dockerfile.
It'll look like this:
#
# Build stage
#
FROM maven:3.6.0-jdk-11-slim AS build
COPY src /home/app/src
COPY pom.xml /home/app
RUN mvn -f /home/app/pom.xml clean package
#
# Package stage
#
FROM openjdk:11-jre-slim
COPY --from=build /home/app/target/demo-0.0.1-SNAPSHOT.jar /usr/local/lib/demo.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/usr/local/lib/demo.jar"]
What it does is it basically first creates a jar file, copies it into package stage and eventually runs.
That's allow you to run your app in docker by running only docker compose.
I'm new to Elixir and Phoenix, and having to work in CI/CD environment I'm trying to figure out how to use Phoenix with Docker.
I've tried various tutorials and videos out there, many of them doesn't work, but those who do work, they have the same result.
Phoenix server doesn't seems to find some resources (the assets folder?).
But inside my Dockerfile I'm copying the entire app folder, and I can confirm that /assets is inside the container by attaching to it.
Dockerfile:
FROM elixir:alpine
RUN apk add --no-cache build-base git
WORKDIR /app
RUN mix local.hex --force && \
mix local.rebar --force
COPY . .
RUN mix do deps.get, deps.compile
CMD ["mix", "phx.server"]
Docker-compose
version: '3.6'
services:
db:
environment:
PGDATA: /var/lib/postgresql/data/pgdata
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
POSTGRES_HOST_AUTH_METHOD: trust
image: 'postgres:11-alpine'
restart: always
volumes:
- 'pgdata:/var/lib/postgresql/data'
web:
build: .
depends_on:
- db
environment:
MIX_ENV: dev
env_file:
- .env
ports:
- '4000:4000'
volumes:
- .:/app
volumes:
pgdata:
Steps I'm doing to create the containers and running the server:
docker-compose build
docker-compose run web mix ecto.create
docker-compose up
The database is created successfully in the db container.
What can be happening here?
Sorry if it's straightforward, I don't use Docker for a while and I still didn't understood Phoenix boilerplate completely.
If you know some good resources about Docker and CI/CD pipelines with Phoenix, I also appreciate so I can study it.
You also need to build the assets. npm install --prefix assets This needs to be done after after mix deps.get but can be done after the mix deps.compile which isn't really needed. You can start the server after mix deps.get and it will compile the deps and your app automatically.
Given a spring boot app that writes files to /var/lib/app/files.
I create an docker image with the gradle task:
./gradlew bootBuildImage --imageName=app:latest
Then, I want to use it in docker-compose:
version: '3.5'
services:
app:
image: app:latest
volumes:
- app-storage:/var/lib/app/files
// ...ports etc
volumes:
app-storage:
This will fail, because the folder is created during docker-compose up and is owned by root and the app, hence, has no write access to the folder.
The quick fix is to run the image as root by specifying user: root:
version: '3.5'
services:
app:
image: app:latest
user: root # <------------ required
volumes:
- app-storage:/var/lib/app/files
// ...ports etc
volumes:
app-storage:
This works fine, but I do not want to run it as root. I wonder how to achieve it? I normally could create a Dockerfile that creates the desired folder with correct ownership and write permissions. But as far as I know build packs do not use a custom Dockerfile and hence bootBuildImage would not use it - correct? How can we create writable volumes then?
By inspecting the image I found that the buildpack uses /cnb/lifecycle/launcher to launch the application. Hence I was able to customize the docker command and fix the owner of the specific folder before launch:
version: '3.5'
services:
app:
image: app:latest
# enable the app to write to the storage folder (docker will create it as root by default)
user: root
command: "/bin/sh -c 'chown 1000:1000 /var/lib/app/files && /cnb/lifecycle/launcher'"
volumes:
- app-storage:/var/lib/app/files
// ...ports etc
volumes:
app-storage:
Still, this is not very nice, because it is not straight forward (and hence my future self will need to spent time on understand it again) and also it is very limited in its extensibility.
Update 30.10.2020 - Spring Boot 2.3
We ended up creating another Dockerfile/layer so that we do not need to hassle with this in the docker-compose file:
# The base_image should hold a reference to the image created by ./gradlew bootBuildImage
ARG base_image
FROM ${base_image}
ENV APP_STORAGE_LOCAL_FOLDER_PATH /var/lib/app/files
USER root
RUN mkdir -p ${APP_STORAGE_LOCAL_FOLDER_PATH}
RUN chown ${CNB_USER_ID}:${CNB_GROUP_ID} ${APP_STORAGE_LOCAL_FOLDER_PATH}
USER ${CNB_USER_ID}:${CNB_GROUP_ID}
ENTRYPOINT /cnb/lifecycle/launcher
Update 25.11.2020 - Spring Boot 2.4
Note that the above Dockerfile will result in this error:
ERROR: failed to launch: determine start command: when there is no default process a command is required
The reason is that the default entrypoint by the paketo builder changed. Changing the entrypoint from /cnb/lifecycle/launcher to the new one fixes it:
ENTRYPOINT /cnb/process/web
See also this question: ERROR: failed to launch: determine start command: when there is no default process a command is required
I'm having a problem installing laravel through a dockerfile. I'm using docker-compose that pulls a dockerfile where I basically have this:
FROM php:7.3-apache-stretch
*some apt-get and install composer*
WORKDIR /var/www
RUN composer create-project --prefer-dist laravel/laravel app
CMD apachectl -D FOREGROUND
but when I access the container and I will see the files that should have been created with the composer I see that it is empty even though I have seen the command executed in the container build.
The container is working perfectly and even I can access it ... only files that do not even appear.
If I run the composer command manually after the container is created the files appear.
In your Dockerfile, you used WORKDIR /var/www and then RUN composer create-project ... which makes composer create files under /var/www on the container file system.
In your docker-compose.yml file you used to start your container:
version: '3.7'
services:
app:
container_name: "app"
build:
context: ./docker
dockerfile: Dockerfile-app
ports:
- "80"
- "443"
restart: always
depends_on:
- db
volumes:
- ".:/var/www"
You are declaring a volume that will be mounted on that same location /var/www in your container.
What happens is that the volume content will take the place of whatever you had on /var/www in the container file system.
I suggest you read carefully the documentation regarding docker volumes, and more specifically the part titled Populate a volume using a container.
Now to move on, ask yourself why you needed that volume in the first place. Is it necessary to change files at run time ?
If not, just add your files at build time:
FROM php:7.3-apache-stretch
*some apt-get and install composer*
WORKDIR /var/www
RUN composer create-project --prefer-dist laravel/laravel app
COPY . /var/www
CMD apachectl -D FOREGROUND
and remove the volume for /var/www.
EDIT
Developing with the help of a Docker container
During development, you change php files on your docker host (assumed to be you development computer) and need to frequently test the result by testing your app served by the webserver from the docker container.
It would be cumbersome to have to rebuild a Docker image every time you need to test your app. The solution is to mount a volume so that the container can serve the files from your development computer:
FROM php:7.3-apache-stretch
*some apt-get and install composer*
WORKDIR /var/www
CMD apachectl -D FOREGROUND
and start it with:
version: '3.7'
services:
app:
container_name: "app"
build:
context: ./docker
dockerfile: Dockerfile-app
ports:
- "80"
- "443"
restart: always
depends_on:
- db
volumes:
- ".:/var/www"
...
When you need to run some commands within that container, just use docker exec:
docker-compose exec app composer create-project --prefer-dist laravel/laravel app
Producing project artifacts
Since what you will be deploying is not a zip/tar archive containing your source code and configurations but a docker image, you need to build the docker image you will use at deployment time.
Dockerfile for production
For production use, you want to have a Docker image which holds all required files and does not need any docker volume, excepted for holding data produced by users (uploaded files, database files, etc)
FROM php:7.3-apache-stretch
*some apt-get and install composer*
WORKDIR /var/www
COPY . /var/www
CMD apachectl -D FOREGROUND
Notice that there is no RUN composer create-project --prefer-dist laravel/laravel app in this Dockerfile. This is because this command is to initialise your project and this is a development time task, not a deployment time task.
You will also need a place to host your docker images (a Docker registry). You can deploy your own registry as a Docker container using the official registry image, or use the one provided by companies:
Gitlab.com - Gitlab Registry (free)
Docker.com - hub.docker.com (1 private image free)
Google.com - Google Container Registry
...
So you need to build a docker image, and then push that image on your registry. Best practice is to automate those tasks with the help of continuous integration tools such as Jenkins, Gitlab CI, Travis CI, Circle CI, Google Cloud Build ...
Your CI job will run the following commands:
git clone <url of you git repo> my_app
cd my_app
git checkout <some git tag>
docker build -t <registry>/<my_app>:<version>
docker login <registry> --user=<registry user> --password=<registry password>
docker push <registry>/<my_app>:<version>
Deploying your Docker image
Start you container with:
version: '3.7'
services:
app:
container_name: "app"
image: <registry>/<my_app>:<version>
ports:
- "80"
- "443"
restart: always
depends_on:
- db
...
Notice here that the docker-compose file does not build any image. For production it is a better practice to refer to an already built docker image (which has been deployed earlier on a staging environment for validation).