I am a Java backend developer and i want to dockerize the backend part in order for my front end college to easily run in on his local machine. I will describe my problem based on demo project. So the demo project consists out of mysql and a spring boot web app. The app is works fine when i am using docker-compose, all the dependecies are download before the start and the endpoint "test" is returning "Some message". The problem occurse when i change the code and docker-compose down and docker-compose up the app, the code isnt changing. What i can do to solve this problem is to package the app on local machine and then deliver it to docker, but then the local machine has to has maven in order to compile it, and i dont want that, i want my college just to use docker and dont care about how all this thing works. Another solution could be not to cache, but then the dependencies have to be downloaded every time we restart the app, that is also not acceptable. So my question is "Is there a way to cache the dependecies (to download it once) but not the code, so that when i am starting the app, a fresh app ,with updated code, would compile?" (Sorry for my english). Here is the demo project that i am testing on:
The controller:
#RestController
#RequestMapping("message")
#RequiredArgsConstructor
public class Controller {
private final MessageRepository messageRepository;
#GetMapping("/{id}")
public Message getById(#PathVariable Long id) {
return messageRepository.findById(id).get();
}
#PostMapping
public Message save(#RequestBody Message message){
return messageRepository.save(message);
}
#GetMapping("/test")
public String ss(){
return "Some message";
}
}
The model:
#Getter
#Setter
#Entity
#NoArgsConstructor
#AllArgsConstructor
#Table(name = "messages")
public class Message {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "id", nullable = false)
private Long id;
#Column(name = "text")
private String text;
}
The repository:
#Repository
public interface MessageRepository extends JpaRepository<Message,Long> {}
property file:
spring:
datasource:
url: jdbc:mysql://localhost:3306/test?useUnicode=yes&characterEncoding=UTF-8&useSSL=false&serverTimezone=UTC&allowPublicKeyRetrieval=true
username: root
password: 123456789
driver-class-name: com.mysql.cj.jdbc.Driver
sql-script-encoding: UTF-8
jpa:
database-platform: org.hibernate.dialect.MySQL8Dialect
hibernate:
ddl-auto: create-drop
Docker file that is in root of the spring project:
FROM maven:3.8.4-jdk-11
WORKDIR /demo
COPY . .
RUN mvn clean install package -Dmaven.test.skip
CMD mvn spring-boot:run
Docker compose file:
version: "3.8"
services:
mysqldb:
image: mysql:8.0.27
restart: unless-stopped
environment:
- MYSQL_DATABASE=db
- MYSQL_ROOT_PASSWORD=123456789
ports:
- 3307:3306
volumes:
- db:/var/lib/mysql
app:
depends_on:
- mysqldb
build: ./demo
ports:
- 8080:8080
environment:
SPRING_APPLICATION_JSON: '{
"spring.datasource.url" : "jdbc:mysql://mysqldb:3306/db?useUnicode=yes&characterEncoding=UTF-8&useSSL=false&serverTimezone=UTC&allowPublicKeyRetrieval=true",
"spring.datasource.username" : "root",
"spring.datasource.password" : "123456789",
"spring.jpa.database-platform" : "org.hibernate.dialect.MySQL8Dialect",
"spring.jpa.hibernate.ddl-auto" : "update"
}'
volumes:
- .:/root
stdin_open: true
tty: true
volumes:
db:
The image for your app is built and it reuses the same image for every next docker-compose up.
If you want to force the rebuild you could use docker-compose up --build. Alternatively, you could remove the image by hand. But first, you should name your image, by using this property:
services:
#...
app:
#...
build: ./demo
image: my_app
then by using docker image rm my_app you could remove it. After that, the docker-compose up should rebuild your app.
So you could give your frontend colleague following one-liner:
docker image rm my_app && docker-compose up -d
Note that -d runs the docker-compose detached from the terminal.
1.In your project create a jar. since i use eclipse i just click maven install or maven build to create jar.
In your docker file you must specified the jar file.
In your docker compose specify also the jar, where is the jar, jar file name, the port and etc . And this will create container for you application.
4.Inorder to run with updated code you must use this comman for docker docker-compose -f dockercomposeYourFile.yaml up --build.
If there is other container in you docker compose just use depends_on then if the conatiner need to wait and cause to stop you need restart: always or anything about restart
I found the solution. In order to not download the dependencies, i need to cache them in upper layer and code in lower layer so that when i change the code, i dont have to redownload the dependencies. The docker file looks like this.
FROM maven:3.8.4-jdk-11-slim
WORKDIR /opt/app
COPY pom.xml .
RUN mvn dependency:go-offline
COPY src/ src/
RUN mvn clean install -Dmaven.test.skip=true
ENTRYPOINT ["java", "-jar", "/opt/app/target/demo-0.0.1-SNAPSHOT.jar"]
And as #valerij-dobler said i need to remove my container in order for my code changes to take effect
docker-compose down && docker image rm -f myapp && docker-compose up -d
Related
I have a weird problem, that sometimes a docker container cannot see a .jar file, while most of the time it does not have any problem with it.
Before i show you the docker image, a little bit of background. Normally i build a jar archive before running my container, a pretty simple container to run a spring boot application. However at some seemingly random point in the daily routine it does not boot up with the container reporting "Unable to access jarfile".
I thought it must be some weird permission stuff, so i took snapshot of my "target" directory when working and when it stopped working via ls -alR target and later comparing those snapshot with git diff. It does not show any difference. I am still pretty convinced it must be related to file-permissions, locking or something of that sort but i do not know where to start.
I am on Mac 12.0.1 btw. Any ideas appreciated.
The docker file
FROM openjdk:8-oraclelinux8
RUN mkdir /app
WORKDIR /app
CMD "java" "-jar" "app.war"
And docker-compose.yml
version: "3.9"
services:
app:
build: .
depends_on:
- sql1
volumes:
- ./target:/app
ports:
- "8080:8080"
links:
- "sql1:sqlserver"
...
I'm not sure if this helps, but I don't see your Dockerfile as robust enough to produce consistent results regardless of the state of your localhost workspace. I may ask, are you building your war file manually and then creating your Docker container?
Please try to follow this approach if it fits your needs :
make sure you delete jar/war files before building the container.
Have a multistage Dockerfile with a "build" phase for your spring boot app where you generate the jar/war file from a builder image (ant, gradle, maven), and then have a second stage where the jar/war file gets copied over to it's final location and the application gets executed, this way you ensure consistency and that the file will be there at all times :
This is an example for my spring boot templates that I use very often, it's quite generic (as I handle the renaming of the jar file without having to worry about how pom.xml is configured individually) and I guess could be implemented in a variety of scenarios
FROM maven:3.8.6-openjdk-18 as builder
WORKDIR /usr/app/
COPY . /usr/app
RUN mvn package -Dmaven.test.skip
RUN JAR_FILE="target/*.jar"; cp ${JAR_FILE} /app.jar
FROM openjdk:18
WORKDIR /usr/app
COPY --from=builder /app.jar /usr/app
EXPOSE 8080
CMD ["java","-jar","app.jar"]
docker compose :
services:
app:
build: .
depends_on:
- sql1
ports:
- 8080:8080
networks:
- spring-boot-api-network
volumes:
- ./target:/app
...
NOTE : I would also remove the "links" option as it is a legacy feature you should avoid using and use networks instead :
You can try this network implementation added at the bottom of your compose file, just make sure you don't forget to add the network: to the sql1 portion as well
networks:
spring-boot-api-network:
driver: bridge
ipam:
driver: default
config:
- subnet: 182.16.0.1/24
gateway: 182.16.0.1
name: spring-boot-api-network
So I know that there are a lot of tutorials on the topics, both docker and maven, but I'm having some confusion in combining them alltogether.
I created a multi-module Maven project with 2 modules, 2 spring applications, let's call them application 1 and application 2.
Starting each other via IntelliJ IDEA green "run" button works fine, now I'd like to automate things and run via docker.
I have Dockerfiles that looks the same in both cases:
(in both modules it's the same, only JAR name's different)
FROM adoptopenjdk:11-jre-hotspot
MAINTAINER *my name here lol*
ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.9.0/wait /wait
RUN chmod +x /wait
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} application1-0.0.1-SNAPSHOT-jar-with-dependencies.jar
ENTRYPOINT ["java","-jar","/application1-0.0.1-SNAPSHOT-jar-with-dependencies.jar"]
CMD /wait && /*.jar
I also have docker-compose:
version: '2.1'
services:
application1:
container_name: app1
build:
context: ../app1
image: docker.io/myname/app1:latest
hostname: app1
ports:
- "8080:8080"
networks:
- spring-cloud-network-app1
application2:
container_name: app2
build:
context: ../app2
depends_on:
application1:
condition: service_started
links:
- application1
image: docker.io/myname/app2:latest
environment:
WAIT_HOSTS: application1:8080
ports:
- "8070:8070"
networks:
- spring-cloud-network-app2
networks:
spring-cloud-network-app1:
driver: bridge
spring-cloud-network-app2:
driver: bridge
What I do currently is:
I run maven package for each module and receive files like "application1(-2)-0.0.1-SNAPSHOT-jar-with-dependencies.jar" in both target folders.
"docker build -t springio/app1 ."
"docker-compose up --build"
And it works, but I feel I do some extra steps.
How can I do the project so that I ONLY have to run docker compose?
(after each time I change things in the code)
Again, I know it's a quite simple thing but I kinda lost the logic.
Thanks!
P.S
Ah, and about the "...docker-compose-wait/releases/download/2.9.0/wait /wait"
It's important that app start one after another, tried different solutions, unfortunately, doesn't really work as good as I would like to. But I guess I'll leave it as is.
So, again, if anyone ever wonders how to do the things I asked, here's the answer: you need multi-stage build Dockerfile.
It'll look like this:
#
# Build stage
#
FROM maven:3.6.0-jdk-11-slim AS build
COPY src /home/app/src
COPY pom.xml /home/app
RUN mvn -f /home/app/pom.xml clean package
#
# Package stage
#
FROM openjdk:11-jre-slim
COPY --from=build /home/app/target/demo-0.0.1-SNAPSHOT.jar /usr/local/lib/demo.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/usr/local/lib/demo.jar"]
What it does is it basically first creates a jar file, copies it into package stage and eventually runs.
That's allow you to run your app in docker by running only docker compose.
Given a spring boot app that writes files to /var/lib/app/files.
I create an docker image with the gradle task:
./gradlew bootBuildImage --imageName=app:latest
Then, I want to use it in docker-compose:
version: '3.5'
services:
app:
image: app:latest
volumes:
- app-storage:/var/lib/app/files
// ...ports etc
volumes:
app-storage:
This will fail, because the folder is created during docker-compose up and is owned by root and the app, hence, has no write access to the folder.
The quick fix is to run the image as root by specifying user: root:
version: '3.5'
services:
app:
image: app:latest
user: root # <------------ required
volumes:
- app-storage:/var/lib/app/files
// ...ports etc
volumes:
app-storage:
This works fine, but I do not want to run it as root. I wonder how to achieve it? I normally could create a Dockerfile that creates the desired folder with correct ownership and write permissions. But as far as I know build packs do not use a custom Dockerfile and hence bootBuildImage would not use it - correct? How can we create writable volumes then?
By inspecting the image I found that the buildpack uses /cnb/lifecycle/launcher to launch the application. Hence I was able to customize the docker command and fix the owner of the specific folder before launch:
version: '3.5'
services:
app:
image: app:latest
# enable the app to write to the storage folder (docker will create it as root by default)
user: root
command: "/bin/sh -c 'chown 1000:1000 /var/lib/app/files && /cnb/lifecycle/launcher'"
volumes:
- app-storage:/var/lib/app/files
// ...ports etc
volumes:
app-storage:
Still, this is not very nice, because it is not straight forward (and hence my future self will need to spent time on understand it again) and also it is very limited in its extensibility.
Update 30.10.2020 - Spring Boot 2.3
We ended up creating another Dockerfile/layer so that we do not need to hassle with this in the docker-compose file:
# The base_image should hold a reference to the image created by ./gradlew bootBuildImage
ARG base_image
FROM ${base_image}
ENV APP_STORAGE_LOCAL_FOLDER_PATH /var/lib/app/files
USER root
RUN mkdir -p ${APP_STORAGE_LOCAL_FOLDER_PATH}
RUN chown ${CNB_USER_ID}:${CNB_GROUP_ID} ${APP_STORAGE_LOCAL_FOLDER_PATH}
USER ${CNB_USER_ID}:${CNB_GROUP_ID}
ENTRYPOINT /cnb/lifecycle/launcher
Update 25.11.2020 - Spring Boot 2.4
Note that the above Dockerfile will result in this error:
ERROR: failed to launch: determine start command: when there is no default process a command is required
The reason is that the default entrypoint by the paketo builder changed. Changing the entrypoint from /cnb/lifecycle/launcher to the new one fixes it:
ENTRYPOINT /cnb/process/web
See also this question: ERROR: failed to launch: determine start command: when there is no default process a command is required
I am new baby to the Docker. I am using docker within the spring boot application. Below is my file .
Application.properties file:
spring.datasource.url=jdbc:mysql://${DATABASE_HOST}:3306/${DATABASE_NAME}?allowPublicKeyRetrieval=true&useSSL=true
spring.datasource.username = root
spring.datasource.password = root
server.port=8443
DockerFile:
FROM openjdk:latest
ADD target/app.jar app.jar
EXPOSE 8443
ENTRYPOINT ["java","-jar","app.jar", "--spring.profiles.active=docker"]
Docker Compose :
version: '3.1'
services:
mysqlDb:
image: mysql:latest
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=springdemo
- MYSQL_USER=root
- MYSQL_PASSWORD=root
- DATABASE_HOST= 192.168.0.1
volumes:
- /data/mysql
ports:
- "3306:3306"
container_name: mysql-db
springbootdocker:
image: xyz/signup
restart: always
ports:
- "8080:8080"
depends_on:
- mysqlDb
links:
- mysqlDb
environment:
- DATABASE_HOST= mysqlDb
- DATABASE_USER=root
- DATABASE_PASSWORD=root
- DATABASE_NAME=springdemo
- DATABASE_PORT=3306
healthcheck:
test: ["CMD","curl","-f","http://localhost:8080/health"]
interval: 1m30s
timeout: 10s
retries: 3
1st part of Issue is Resolved :
when i run the App , i get the following error :
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
Caused by: java.net.UnknownHostException: ${DATABASE_HOST}
1) What i am trying to Achieve -
Want to create a jar for my application.
After creating a jar , i want to create the image from the jar
After creating the image of the application , i want to push into the docker hub .
From other server , I will run docker-compose.yml file . It will fetch the mysql and image from the docker hub and run the application in the other port as mentioned in the docker-compose.
Hope I am clear ,what i want to acheive .
Here comes the problem :
when i am trying to run the application locally , i am able to create the jar only when ,i write this line in application.properties class.
spring.datasource.url=jdbc:mysql://127.0.0.1:3306/springdemo?allowPublicKeyRetrieval=true&useSSL=true
I am able to create the jar and after that image to -> upload that image to docker hub , when i am running the docker-compose able to pull that image but can't able to connect to the mysql workbench.
So,I thought, I am running the docker-compose in diffrent System/Server. There ip address is different.
So, I changed the url to this :
spring.datasource.url=jdbc:mysql://192.168.0.1:3306/springdemo?allowPublicKeyRetrieval=true&useSSL=true
where
192.168.0.1 - this ip is for the diffrent server, where i want to run the docker-compose file
But when i am running the spring boot application , I am unable to run the application because it's saying the almost the same error as mentioned above i.e Connection refuse . So i am unable to create the jar so , unable to create the image . So my idea fails here also.
Then Again, One idea again come into my mind , can't we put the dynamic ip address in application.properties file,so i tried this line :
spring.datasource.url=jdbc:mysql://${DATABASE_HOST}:3306/${DATABASE_NAME}?allowPublicKeyRetrieval=true&useSSL=true
DATABASE_HOST- you can find it in docker-compose file
I thought may be this one : ${DATABASE_HOST} , will pick the value of the DATABASE_HOST from the docker-compose .
But my idea failed , I am getting the error as mentioned in top while running the application. can't able to create jar ,so unable to create the image .
2nd Part - Issue is still Open
when i am able to run the docker-compose in my local terminal , and able to create the container . I can also see mysql is connected as well as my schema .
But when i am trying to access my API using postman or web browser then , it's saying not connected
http://localhost:8080/dockerex
Inside my Controller Class:
#RestController
#RequestMapping(value = "/dockerex")
public class SpringBootDockerController {
#GetMapping
public String check() {
return "checking";
}
}
when i inspect my container , ip address is coming blank : snapshot of conatiner :
[
{
"Id": "7664bab5ed452e3e2243e1802010240ab5a3eb817af6164e985b10e729ff4c8f",
"Created": "2020-02-04T17:30:08.5351511Z",
"Path": "java",
"Args": [
"-jar",
"app.jar",
"--spring.profiles.active=docker"
],
"State": {
"Status": "running"
.......................
"PortBindings": {
"8080/tcp": [
{
"HostIp": "",
"HostPort": "8080"
}
]
},
Good it worked but here you see you were using DB_HOST earlier, now you are using DATABASE_HOST= mysqlDb , which is currently pointing to your mysql service name.
this service name can be discovered within the single network created by docker compose.
You were hardcoding DB_HOST=192.168.0.1 and trying to connect using this IP , which is invalid as docker compose create its network.
You have to mention service name here as in your case db service name is mysqlDb. Below is the updated application.properties
spring.datasource.url=jdbc:mysql://$(DB_HOST:mysqlDb):3306/springdemo?allowPublicKeyRetrieval=true&useSSL=true
spring.datasource.username = root
spring.datasource.password = root
server.port=8443
Try this:
Replace DB_HOST= 192.168.0.7 to DB_HOST= mysqldb in your docker-compose file.
And update your application.properties file:
spring.datasource.url=jdbc:mysql://$(DB_HOST:mysqlDb):3306/springdemo?allowPublicKeyRetrieval=true&useSSL=true
to this:
spring.datasource.url=jdbc:mysql://$(mysqlDb):3306/springdemo?allowPublicKeyRetrieval=true&useSSL=true
Thanks everyone : I got the solution by hit and trail :P
spring.datasource.url=jdbc:mysql://${DATABASE_HOST:localhost}:${DATABASE_PORT:3306}/springdemo?allowPublicKeyRetrieval=true&useSSL=true
I need to provide a POC as argument for the migration of workflows in my current job. Currently we do this:
People code on Netbeans
People click on build on netbeans
Deploy locally
Apply code changes
Netbeans rebuilds and redeploy the code.
Things to know:
It seems tomcat detects when a new WAR is put in the directory and hot-deploys it;
What I aim to automate is not the hot-deploy(since this is already a tomcat feature), but the build process;
We are using Maven to build the project.
I'm using docker-compose to get everything up in one single specification.
So far I was able to containerize the Postgres database, the PGAdmin we use and the initial build of the application using a multi-stage Dockerfile.
Tomcat app Dockerfile
FROM maven AS buildserver
ADD . /usr/src/mymaven/
WORKDIR /usr/src/mymaven
# build the project
RUN mvn -f pom.xml clean package -DskipTests
FROM tomcat:latest
COPY conf-tomcat/tomcat-users.xml /usr/local/tomcat/conf/
COPY conf-tomcat/server.xml /usr/local/tomcat/conf/
COPY conf-tomcat/context.xml /usr/local/tomcat/webapps/manager/META-INF/
# Copy the built war file into webapps folder of tomcat container
COPY --from=buildserver /usr/src/mymaven/target/*.war /usr/local/tomcat/webapps
What I am having trouble with is triggering the rebuild when there's code changes (imitating what netbeans does). I can't find in either maven's or netbeans documentation how that detection and triggering works.
I am using volumes to map the app source directory to the container in hopes that it would just work, but I was wrong.
My docker-compose.yml is as follows:
version: '3'
services:
pgadmin:
container_name: pgadmin
image: dpage/pgadmin4
env_file:
- ../db-postgres/pgadmin/pgadmin.env
depends_on:
- pg-dev
networks:
- dev-network
volumes:
- pgadmin-data:/var/lib/pgadmin
ports:
- "88:80"
pg-dev:
container_name: pg-dev
image: pg-dev:latest
env_file:
- ../db-postgres/db-dev/pg-dev.env
volumes:
- pg-data:/var/lib/postgresql/data
networks:
- dev-network
ports:
- "5433:5432"
app:
container_name: app
build: .
volumes:
- app-src:/usr/src/mymaven
- artifacts:/usr/src/mymaven/target
- maven-repo:/root/.m2
networks:
- dev-network
ports:
- "8888:8080"
depends_on:
- pg-dev
volumes:
maven-repo:
driver: local
driver_opts:
type: bind
device: $HOME/.m2
o: bind
app-src:
driver: local
driver_opts:
type: bind
device: .
o: bind
artifacts:
driver: local
driver_opts:
type: bind
device: target/
o: bind
pg-data:
pgadmin-data:
networks:
dev-network:
Any help in coming up with a solution for this is appreciated, as well as any general advice in how to make this workflow/build improve.
UPDATE
I came up with somewhat of a work around, but now I am having problem testing it.
I defined a maven container to work as a build server:
FROM maven
ADD . /usr/src/mymaven/
WORKDIR /usr/src/mymaven
RUN apt update && apt install entr -y
# build the project
RUN mvn -f pom.xml clean package -DskipTests
and now I am defining the entrypoint on the docker-compose.yml:
...
buildserver:
container_name: buildserver
build:
context: .
dockerfile: maven-builder.Dockerfile
volumes:
- app-src:/usr/src/mymaven
- maven-repo:/root/.m2
- artifacts:/usr/src/mymaven/target
networks:
- dev-network
entrypoint: sh -c 'find src/ | entr mvn -f pom.xml clean package -DskipTests --batch-mode'
...
But now I am getting an error message when this container gets up:
find: ‘src/’: No such file or directory
entr: No regular files to watch
Which is weird to me as I successfully build the project in the first run, but the entry-point seems to be failing.
Clarification: What I am being asked is come up with a workflow that removes the need to use the deploy from Netbeans (they want everything automatic). I looked around for a Jenkins workflow, but could not really find a way to achieve the desired results.
According to the Netbeans docs, you can bind Maven goals to IDE actions (http://wiki.netbeans.org/MavenBestPractices section Binding Maven goals to IDE actions):
It's possible to customize the default Maven goal to IDE Action binding from the project's customizer. Right click on the project node and select "Properties" or use the File/Project Properties main menu item to invoke the Project properties dialog. On the left hand side, select the panel named "Actions". The panel lists all available default project actions that can be mapped. When selecting one from the list the textfields in the bottom allow to change the values.
It looks to me that you should bind the Build Project Netbeans action to a specific Maven goal. From this point, it is up to you to come up with a creative solution. You could explore the Maven Exec plugin capabilities and run custom commands during the build proccess (check I want to execute shell commands from Maven's pom.xml). For instance, it should be possible to copy the .war file from target folder to wherever you want on the filesystem, or even execute scripts inside the running container.
PS: It looks like you are trying to do something quite odd, but I'll assume here it makes sense to you solving this somehow.