Spring boot java Docker connecting mongoDB in K8s - spring-boot

I am struggling to connect mongodb using spring boot docket application, Below are the further details.
a) Created a mongoDB cluster in minikube.
b) Exposed an external service which connects to the mongodb. The service exposed was through Nodeport. Below is the code snippet.
mongo-service
---------------
`apiVersion: v1
kind: Service
metadata:
name: mongodb-external-service
spec:
selector:
app: mongodb
type: NodePort
ports:
- port: 27017
protocol: TCP
targetPort: 27017
nodePort: 32000
c) Minikube exposed the service with IP 192.168.49.2
d) I was able to connect mongodb using 192.168.49.2:2000 from my localhost using mongodb compass.
e) Created simple spring boot application which was able to connect the mongodb service using Ip and port.
So far no problems things were moving as expected. Next my objective was to containerize my spring boot application and execute in in the Minikube. Before i execute it as pod in minikube i wanted to test the container using the docker-compose. Below is my docker-compose and Dockerfile code snippets
Dockerfile
FROM adoptopenjdk:11-jre-hotspot as builder
ARG JAR_FILE=target/\*.jar
COPY ${JAR_FILE} tradeprocessor.jar
RUN java -Djarmode=layertools -jar tradeprocessor.jar extract
FROM adoptopenjdk:11-jre-hotspot
COPY --from=builder dependencies/ ./
COPY --from=builder snapshot-dependencies/ ./
COPY --from=builder spring-boot-loader/ ./
COPY --from=builder application/ ./
ENTRYPOINT \["java", "org.springframework.boot.loader.JarLauncher"\]
docker-compose
---------------
`version: '3.7'
services:
trdprocess:
build:
context: .
dockerfile: Dockerfile
image: tradeprocess:latest
ports:
- 8088:8088
extra_hosts:
- dockerhost:192.168.49.2
environment:
- spring_data_mongodb_authentication_database=admin
- spring_data_mongodb_username=mongouser
- spring_data_mongodb_password=mongopassword
- spring_data_mongodb_database=transactions
- spring_data_mongodb_port=32000
- spring_data_mongodb_host=192.168.49.2`
however Upon executing the above step, application is spinning up but it is throwing SocketTimeoutException.
Exception:
`Started TradeprocessingApplication in 2.09 seconds (JVM running for 2.973)`
2023-01-04 20:16:13.529 INFO 1 --- [.168.49.2:32000] o.m.d.cluster : Exception in monitor thread while connecting to server 192.168.49.2:32000
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:70) ~[mongodb-driver-core-4.6.1.jar:?]
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:180) ~[mongodb-driver-core-4.6.1.jar:?]
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:193) ~[mongodb-driver-core-4.6.1.jar:?]
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:157) ~[mongodb-driver-core-4.6.1.jar:?]
at java.lang.Thread.run(Unknown Source) ~[?:?]
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) ~[?:?]
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) ~[?:?]
at java.net.AbstractPlainSocketImpl.connect(Unknown Source) ~[?:?]
at java.net.SocksSocketImpl.connect(Unknown Source) ~[?:?]
at java.net.Socket.connect(Unknown Source) ~[?:?]
at com.mongodb.internal.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:107) ~[mongodb-driver-core-4.6.1.jar:?]
It looks like it has to do with network setting but not sure how to correct it. Any help is highly appreciated
Tried to change few option with docker-compose such as extra_hosts port mappings etc but no luck

I finally was able to solve my own problem. A bit of learning curve. Hear are the details.
My entire setup is running on Ubuntu workstation with minikube cluster.
Minikube started with command minikube start --driver=docker. Which means minikube itself is running as container within docker runtime environment.
docker network ls
docker network ls clearly shows that both my custom build application and the minikube are running as containers. Therefore i had to link these containers network. A small change in the docker-compose did the trick. Below is the code:
networks:
default :
external :
name : minikube

Related

Docker Compose Volumes - Device Option - Windows - JHipster

I'm trying to configure a JHipster web application (Java Spring Boot + Angular) to use an embedded H2 DB in production environment.
What I want to achieve is to save the H2 generated DB files on the host machine so that if I need to backup those file for some reasons I have easy access.
The application will be deployed using docker compose.
DEV ENVIRONMENT
I'm using a Windows machine to develop this web application.
I have installed Docker Desktop on my machine and is running on WSL 2 engine.
JHIPSTER DOCKER IMAGE
The docker image I am using to deploy the application is the standard used by the JHipster project. This means that the web application inside docker runs with UID = 1000
The User UID = 1000 doesn't have root access (which is of course correct for security reasons)
DOCKER COMPOSE AND VOLUME CONFIGURATIONS
The first docker compose configuration I tried was the following one:
version: '3.8'
services:
app:
image: scriba_webapp
environment:
- ...
ports:
- ...
volumes:
- scriba-db:/tmp/h2
volumes:
scriba-db:
but when the web applications starts I receive a "permission denied" error. Looking at the logs the problem seems to be when the application tries to initialize the DB (which means creating and writing a file to the /tmp/h2 container directory).
Here the stacktrace:
org.h2.message.DbException: Log file error: "/tmp/h2/db.trace.db", cause: "java.io.FileNotFoundException: /tmp/h2/db.trace.db (Permission denied)" [90034-200]
at org.h2.message.DbException.get(DbException.java:194)
at org.h2.message.TraceSystem.logWritingError(TraceSystem.java:294)
at org.h2.message.TraceSystem.openWriter(TraceSystem.java:315)
at org.h2.message.TraceSystem.writeFile(TraceSystem.java:263)
at org.h2.message.TraceSystem.write(TraceSystem.java:247)
at org.h2.message.Trace.error(Trace.java:180)
at org.h2.engine.Database.setBackgroundException(Database.java:2230)
at org.h2.mvstore.db.MVTableEngine$1.uncaughtException(MVTableEngine.java:93)
at org.h2.mvstore.MVStore.handleException(MVStore.java:2877)
at org.h2.mvstore.MVStore.panic(MVStore.java:481)
at org.h2.mvstore.MVStore.<init>(MVStore.java:402)
at org.h2.mvstore.MVStore$Builder.open(MVStore.java:3579)
at org.h2.mvstore.db.MVTableEngine$Store.open(MVTableEngine.java:170)
at org.h2.mvstore.db.MVTableEngine.init(MVTableEngine.java:103)
at org.h2.engine.Database.getPageStore(Database.java:2659)
at org.h2.engine.Database.open(Database.java:675)
at org.h2.engine.Database.openDatabase(Database.java:307)
at org.h2.engine.Database.<init>(Database.java:301)
at org.h2.engine.Engine.openSession(Engine.java:74)
at org.h2.engine.Engine.openSession(Engine.java:192)
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:171)
at org.h2.engine.Engine.createSession(Engine.java:166)
at org.h2.engine.Engine.createSession(Engine.java:29)
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:340)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:173)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:152)
at org.h2.Driver.connect(Driver.java:69)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:266)
at org.springframework.boot.autoconfigure.liquibase.DataSourceClosingSpringLiquibase.afterPropertiesSet(DataSourceClosingSpringLiquibase.java:46)
at tech.jhipster.config.liquibase.AsyncSpringLiquibase.initDb(AsyncSpringLiquibase.java:118)
at tech.jhipster.config.liquibase.AsyncSpringLiquibase.afterPropertiesSet(AsyncSpringLiquibase.java:103)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1845)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1782)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:602)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:524)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1154)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:908)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:434)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:338)
at it.brainylabs.scriba.ScribaApp.main(ScribaApp.java:69)
Caused by: org.h2.jdbc.JdbcSQLNonTransientException: Log file error: "/tmp/h2/db.trace.db", cause: "java.io.FileNotFoundException: /tmp/h2/db.trace.db (Permission denied)" [90034-200]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:505)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:429)
... 56 more
Caused by: java.io.FileNotFoundException: /tmp/h2/db.trace.db (Permission denied)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(Unknown Source)
at java.base/java.io.FileOutputStream.<init>(Unknown Source)
at java.base/java.io.FileOutputStream.<init>(Unknown Source)
at org.h2.store.fs.FilePathDisk.newOutputStream(FilePathDisk.java:306)
at org.h2.store.fs.FileUtils.newOutputStream(FileUtils.java:239)
at org.h2.message.TraceSystem.openWriter(TraceSystem.java:311)
... 53 more
Exception in thread "Logback shutdown hook [default]" java.util.ConcurrentModificationException
at java.base/java.util.ArrayList$Itr.checkForComodification(Unknown Source)
at java.base/java.util.ArrayList$Itr.next(Unknown Source)
at ch.qos.logback.classic.LoggerContext.fireOnReset(LoggerContext.java:323)
at ch.qos.logback.classic.LoggerContext.reset(LoggerContext.java:226)
at ch.qos.logback.classic.LoggerContext.stop(LoggerContext.java:348)
at ch.qos.logback.core.hook.ShutdownHookBase.stop(ShutdownHookBase.java:39)
at ch.qos.logback.core.hook.DelayingShutdownHook.run(DelayingShutdownHook.java:57)
at java.base/java.lang.Thread.run(Unknown Source)
The next configuration I tried was using the "old" bind mounts instead of the new docker volumes:
version: '3.8'
services:
app:
image: scriba_webapp
environment:
- ...
ports:
- ...
volumes:
- type: bind
source: D:\...\h2
target: /tmp/h2
With this everything works fine, the web applications startup correctly and I can see the DB files in the source directory. So without other changes to the project a bind mount works correctly while a docker volume doesn't.
The final test I did was to configure the docker volume like this:
version: '3.8'
services:
app:
image: ...
environment:
- ...
ports:
- ...
volumes:
- scriba-db
volumes:
scriba-db:
driver_opts:
type: "volume"
o: "uid=1000,rw"
device: **???**
Since the original error was a permission denied I figured I should try and give that user read/write access to the docker volume but I didn't manage to do so since it's not clear to me how the volume option device should be configured for this case.
If I configure the driver_opts like this:
scriba-db:
driver_opts:
type: "tmpfs"
o: "uid=1000,rw"
device: "tmpfs"
The web application starts correctly but I can't see the DB files in the docker volume since tmpfp (If I understood this correctly) means that the FS is just temporary and will be delete once the containers stops.
THE QUESTIONS
How come a bind mount works just like that and a docker volume instead has problems?
Let's say I want to use a docker volume and save those DB files on the host machine, how should I configure the driver_opts on a Windows machine?

Jooq unable to find database when triggered via a gradle image on Dockerfile

I have a Dockerfile that uses gradle to build my Spring boot application just before coping it into the container and trigger it. This is how it looks:
FROM gradle:5.4.1-jdk8-alpine AS build
COPY --chown=gradle:gradle . /home/gradle/src
WORKDIR /home/gradle/src
RUN gradle build --no-daemon
FROM openjdk:8-jdk-alpine
EXPOSE 8080
RUN mkdir /app
COPY --from=build /home/gradle/src/build/libs/*.jar yurlapp.jar
ENTRYPOINT ["java","-jar" , "/yurlapp.jar"]
It's quite simple, it will execute Gradle and the jar will be added into the docker container. So far so good. This Dockerfile will be used inside a docker-compose.yml once the database is provided(the DB is required for the app to work). This is how that docker-compose.yml looks like:
version: '2.1'
services:
yurldb:
image: postgres:11
container_name: yurldb
networks:
- yunet
ports:
- 5432:5432
environment:
- POSTGRES_DB=yurldb
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=somepassword
volumes:
- ./init.sql:/docker-entrypoint-initdb.d/
- yudata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 10s
retries: 5
flyway:
image: boxfuse/flyway
command: -url=jdbc:postgresql://yurldb:5432/yurldb -schemas=public -user=postgres -password=somepassword migrate
networks:
- yunet
volumes:
- .:/flyway/sql
depends_on:
yurldb:
condition: service_healthy
yurlapp:
container_name: yurlapp
image: javing/yurlapp:0.0.1
build:
context: .
dockerfile: Dockerfile
networks:
- yunet
ports:
- 8080:8080
environment:
- DB_HOST=yurldb #the name of the DB container is used as url
depends_on:
yurldb:
condition: service_healthy
flyway:
condition: service_healthy
networks:
yunet:
volumes:
yudata:
As you can see the app image be built last, once the database image is ready and also after flyway performs the migrations. The problem that I have happens during the building of the app.
So once gradle starts it will do all it has to do. But at one point it requires Jooq, to autogenerate some classes based on the existing tables on the database. And in here something goes wrong. The build blows up and tells me this:
Task :generateSampleJooqSchemaSource FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '/home/gradle/src/build.gradle' line: 81
* What went wrong:
Execution failed for task ':generateSampleJooqSchemaSource'.
> jOOQ source code generation failed:
Jun 01, 2020 7:37:39 PM org.jooq.tools.JooqLogger info
INFO: Initialising properties : /home/gradle/src/build/tmp/generateSampleJooqSchemaSource/config.xml
Jun 01, 2020 7:37:39 PM org.jooq.tools.JooqLogger error
SEVERE: Cannot read /home/gradle/src/build/tmp/generateSampleJooqSchemaSource/config.xml. Error : Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:285)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:211)
at org.postgresql.Driver.makeConnection(Driver.java:459)
at org.postgresql.Driver.connect(Driver.java:261)
at org.jooq.codegen.GenerationTool.run0(GenerationTool.java:342)
at org.jooq.codegen.GenerationTool.run(GenerationTool.java:221)
at org.jooq.codegen.GenerationTool.generate(GenerationTool.java:216)
at org.jooq.codegen.GenerationTool.main(GenerationTool.java:188)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.postgresql.core.PGStream.<init>(PGStream.java:81)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:93)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:197)
... 8 more
It says that it can't connect to the database. Why is that? This is the Jooq configuration I have in my build.gradle file
jooq {
sample(sourceSets.main) {
jdbc {
driver = 'org.postgresql.Driver'
url = 'jdbc:postgresql://localhost:5432/yurldb'
user = 'postgres'
password = 'somepassword'
}
generator {
database() {
name = 'org.jooq.meta.postgres.PostgresDatabase'
inputSchema = 'public'
includes = '.*'
}
target {
packageName = 'com.javing.yurl'
directory = 'build/generated/java'
}
}
}
}
tasks.generateSampleJooqSchemaSource.with {
def out = new ByteArrayOutputStream()
javaExecSpec = { JavaExecSpec s ->
s.standardOutput = out
s.errorOutput = out
s.ignoreExitValue = true
s.jvmArgs '-Xmx512M'
}
execResultHandler = { ExecResult r ->
if (r.exitValue != 0) {
throw new RuntimeException('jOOQ source code generation failed:\n\n' + out.toString())
}
}
}
I am very confused, I am not sure I understand what is happening. If I was to use docker-compose to just boot the database image just by itself and then I run gradle manually from my terminal(not the gradle from docker), those classes will be generated without problem. So I don't know when the gradle is used from the Dockerfile Jooq fails to find the DB. Any idea?
Do i also need to trigger Jooq maybe from the Dockerfile somehow, in the same way that I triggered gradle? Can someone help me with this please, I am a bit stuck.
Update
I tried changing the url in the jooq config to use yurldb instead of localhost since maybe when networking inside the container the url resloves to the name of the container, but that didn't work. What happens is this:
FAILURE: Build failed with an exception.
* Where:
Build file '/home/gradle/src/build.gradle' line: 81
* What went wrong:
Execution failed for task ':generateSampleJooqSchemaSource'.
> jOOQ source code generation failed:
Jun 01, 2020 8:29:57 PM org.jooq.tools.JooqLogger info
INFO: Initialising properties : /home/gradle/src/build/tmp/generateSampleJooqSchemaSource/config.xml
Jun 01, 2020 8:30:08 PM org.jooq.tools.JooqLogger error
SEVERE: Cannot read /home/gradle/src/build/tmp/generateSampleJooqSchemaSource/config.xml. Error : The connection attempt failed.
org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:297)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:211)
at org.postgresql.Driver.makeConnection(Driver.java:459)
at org.postgresql.Driver.connect(Driver.java:261)
at org.jooq.codegen.GenerationTool.run0(GenerationTool.java:342)
at org.jooq.codegen.GenerationTool.run(GenerationTool.java:221)
at org.jooq.codegen.GenerationTool.generate(GenerationTool.java:216)
at org.jooq.codegen.GenerationTool.main(GenerationTool.java:188)
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.postgresql.core.PGStream.<init>(PGStream.java:81)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:93)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:197)
... 8 more
The problem is that you have localhost as the database host in the configuration of your jooq plugin in build.gradle.
You have correctly set environment variables for DB_HOST in other places so I assume you understand that name of the service is the DNS name that you can use to refer to other services when they are connected to a common network. You should change
url = 'jdbc:postgresql://localhost:5432/yurldb'
to
url = 'jdbc:postgresql://yurldb:5432/yurldb'
or you can get the host of db in environment variable for application service and then read it from build.gradle :
def db = System.getenv('DB_HOST')
url = "jdbc:postgresql://${db}:5432/yurldb"

Docker compose error on mysql db connect for spring boot app Caused by: java.net.UnknownHostException: [duplicate]

This question already has answers here:
Communications link failure , Spring Boot + MySql +Docker + Hibernate
(6 answers)
Closed 3 years ago.
I have spring boot app
My Dockerfile is
FROM openjdk:8-jdk-alpine
EXPOSE 8080
ARG JAR_FILE=target/demo-0.0.1-SNAPSHOT.jar
ADD ${JAR_FILE} demo.jar
ENTRYPOINT ["java","-jar","/demo.jar"]
My docker compose file
# Docker Compose file Reference (https://docs.docker.com/compose/compose-file/)
version: '3.7'
# Define services
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: . # Use an image built from the specified dockerfile in the `polling-app-server` directory.
dockerfile: ./Dockerfile
container_name: empserver
ports:
- "3000:3000" # Forward the exposed port 8080 on the container to port 8080 on the host machine
restart: always
depends_on:
- db # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://db:3306/employee_entries?useSSL=false&serverTimezone=UTC&useLegacyDatetimeCode=false
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: root
# Database Service (Mysql)
db:
image: mysql:5.7
ports:
- "3306:3306"
restart: always
environment:
MYSQL_DATABASE: employee_entries
MYSQL_USER: root
MYSQL_PASSWORD: root
MYSQL_ROOT_PASSWORD: root
My docker net works
NETWORK ID NAME DRIVER SCOPE
b95e3d99b266 Default Switch ics local
7fff4f9713f8 demo_default nat local
fe8883b77d1d emp-mysql ics local
f464aab9064a nat nat local
a5bd5e8efe61 none null local
App is successfully running using java -jar target\demo-0.0.1-SNAPSHOT.jar
but when I am doing docker-compose up
I got below error
app-server_1 | Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
app-server_1 |
app-server_1 | The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
app-server_1 | at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_212]
app-server_1 | at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_212]
app-server_1 | at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_212]
app-server_1 | at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_212]
app-server_1 | at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:91) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.NativeSession.connect(NativeSession.java:144) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:956) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:826) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | ... 56 common frames omitted
app-server_1 | Caused by: java.net.UnknownHostException: db
app-server_1 | at java.net.InetAddress.getAllByName0(InetAddress.java:1281) ~[na:1.8.0_212]
app-server_1 | at java.net.InetAddress.getAllByName(InetAddress.java:1193) ~[na:1.8.0_212]
app-server_1 | at java.net.InetAddress.getAllByName(InetAddress.java:1127) ~[na:1.8.0_212]
app-server_1 | at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:132) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | ... 59 common frames omitted
I am able to access mysql database and tables but from docker compose it was not
any suggestion would really be helpful
You need to provide the container names to the services and use them when referring them from each other. In your environment section for app-server, the url for database points to 127.0.0.1 but the database is not running on same container as app-server so this will fail.
To make this work, provide container names to services for eg : my_mysql and my_app-server and use it in environment url as jdbc:mysql://my_mysql:3306.
Please see the modified file below:
# Docker Compose file Reference (https://docs.docker.com/compose/compose-file/)
version: '3.7'
# Define services
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: . # Use an image built from the specified dockerfile in the `polling-app-server` directory.
dockerfile: ./Dockerfile
container_name: my_app-server
ports:
- "3000:3000" # Forward the exposed port 8080 on the container to port 8080 on the host machine
restart: always
depends_on:
- db # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://my_mysql:3306/employee_entries?useSSL=false&serverTimezone=UTC&useLegacyDatetimeCode=false
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: root
# Database Service (Mysql)
db:
image: mysql:5.7
container_name: my_mysql
ports:
- "3306:3306"
restart: always
environment:
MYSQL_DATABASE: employees
MYSQL_USER: root
MYSQL_PASSWORD: root
MYSQL_ROOT_PASSWORD: root
network:
my-network:

Spring Cloud Config in Docker container is not accessible

I have a relatively simple setup. A Spring Could Config server, and a Spring Cloud Eureka server. (I refer Eureka server as Discover everywhere in my config...) The config server uses a Git repository for finding the appropriate config.
Everything is in Docker containers. I use docker-compose to build my services. The Gateway server should wait for config server to come up. I'll provide the source of the file further down.
So, when I build and start the containers, the discovery (Eureka) server doesn't want to come up. It exits with an exception. It is really long, so I just add the most important part here:
019-01-27 23:52:17,494 [INFO ] o.s.c.c.c.ConfigServicePropertySourceLocator.getRemoteEnvironment [main] – Fetching config from server at : http://tao-elszamolas-config:9001
2019-01-27 23:52:17,898 [INFO ] o.s.c.c.c.ConfigServicePropertySourceLocator.getRemoteEnvironment [main] – Connect Timeout Exception on Url - http://tao-elszamolas-config:9001. Will be trying the next url if available
2019-01-27 23:52:17,902 [ERROR ] o.s.b.SpringApplication.reportFailure [main] – Application run failed
java.lang.IllegalStateException: Could not locate PropertySource and the fail fast property is set, failing
... Something...Something
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://tao-elszamolas-config:9001/discovery-server/prod": Connection refused (Connection refused); nested exception is java.net.ConnectException: Connection refused (Connection refused)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:743) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:669) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:578) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.getRemoteEnvironment(ConfigServicePropertySourceLocator.java:218) ~[spring-cloud-config-client-2.1.0.RC3.jar!/:2.1.0.RC3]
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.locate(ConfigServicePropertySourceLocator.java:96) ~[spring-cloud-config-client-2.1.0.RC3.jar!/:2.1.0.RC3]
... 15 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[?:?]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[?:?]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[?:?]
at java.net.Socket.connect(Socket.java:591) ~[?:?]
at java.net.Socket.connect(Socket.java:540) ~[?:?]
at sun.net.NetworkClient.doConnect(NetworkClient.java:182) ~[?:?]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:474) ~[?:?]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:569) ~[?:?]
at sun.net.www.http.HttpClient.<init>(HttpClient.java:242) ~[?:?]
at sun.net.www.http.HttpClient.New(HttpClient.java:341) ~[?:?]
at sun.net.www.http.HttpClient.New(HttpClient.java:362) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1242) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1181) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1075) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1009) ~[?:?]
at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:76) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:734) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:669) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:578) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.getRemoteEnvironment(ConfigServicePropertySourceLocator.java:218) ~[spring-cloud-config-client-2.1.0.RC3.jar!/:2.1.0.RC3]
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.locate(ConfigServicePropertySourceLocator.java:96) ~[spring-cloud-config-client-2.1.0.RC3.jar!/:2.1.0.RC3]
... 15 more
The only exception what I found in the Config server log is the one complaining about the connection to Discovery (Eureka) server. Yes, I try to register my config service inEureka for looking it up at startup of further microservices.
The connection is refused which means, the config server is not running at the time when Eureka server is trying to get its config or it is not listening on the given port, 9001.
How is it possible? I used the depends_on in my compose file, so it should wait for that.
Or why it finishes working? If I do the same thing on local without docker then it keeps trying to connect to Config, until the config server starts up. I'd like achieve the same behaviour using Docker.
Any advice would be appreciated!
And finally here are my config files:
docker-compose.yml:
version: "3"
services:
tao-elszamolas-config:
build: ./tao-elszamolas-config
container_name: tao-elszamolas-config
ports:
- "9001:9001"
volumes:
- "/tao-elszamolas/logs:/var/log/tao-elszamolas"
networks:
- taonetwork
tao-elszamolas-discovery:
build: ./tao-elszamolas-discovery
container_name: tao-elszamolas-discovery
depends_on:
- tao-elszamolas-config
ports:
- "9002:9002"
volumes:
- "/tao-elszamolas/logs:/var/log/tao-elszamolas"
networks:
- taonetwork
networks:
taonetwork:
driver: bridge
Dockerfile of Config Server:
FROM openjdk:11-jdk-slim
MAINTAINER BeszterceKK
COPY ./tao-elszamolas-config.jar /usr/src/taoelszamolas/tao-elszamolas-config.jar
WORKDIR /var/log/tao-elszamolas
WORKDIR /usr/src/taoelszamolas
ENV SPRING_PROFILES_ACTIVE prod
EXPOSE 9001
ENTRYPOINT ["java", "-DlogFileLocation=/var/log/tao-elszamolas", "-jar", "tao-elszamolas-config.jar"]
application.yml for Config server:
server:
port: 9001
spring:
application:
name: config-server
cloud:
config:
server:
git:
uri: https://github.com/something/something
username: something
password: something
skip-ssl-validation: true
security:
user:
name: something
password: something
eureka:
client:
serviceUrl:
defaultZone: http://something:something#tao-elszamolas-discovery:9002/eureka/
info:
artifact: tao-elszamolas-config
name: TAO Elszamolas Config Application
description: Tao elszamolas microservice elosztott szolgaltatas konfiguracio
version: 1.0.0
Dockerfile of Discovery (Eureka) Server:
FROM openjdk:11-jdk-slim
MAINTAINER BeszterceKK
COPY ./tao-elszamolas-discovery.jar /usr/src/taoelszamolas/tao-elszamolas-discovery.jar
WORKDIR /var/log/tao-elszamolas
WORKDIR /usr/src/taoelszamolas
ENV SPRING_PROFILES_ACTIVE prod
EXPOSE 9002
ENTRYPOINT ["java", "-DlogFileLocation=/var/log/tao-elszamolas", "-jar", "tao-elszamolas-discovery.jar"]
bootstrap.yml for Discovery (Eureka) server:
spring:
application:
name: discovery-server
cloud:
config:
fail-fast: true
name: discovery-server
uri: http://tao-elszamolas-config:9001
username: something
password: something
application.yml for Discovery (Eureka) server:
server:
port: 9002
eureka:
instance:
hostname: tao-elszamolas-discovery
client:
registerWithEureka: false
fetchRegistry: false
serviceUrl:
defaultZone: http://tao-elszamolas-discovery:9002/eureka
info:
artifact: tao-elszamolas-discovery
name: TAO Elszamolas Eureka Szolgaltatas monitor
description: Tao elszamolas automatikus szolgaltatas regisztracio
version: 1.0.0
And finally, this is how I start my containers. It is simply:
docker-compose up -d
The whole application stack works on local without Docker. So I guess the problem is with my Docker configuration.
One more thing to mention, generally, I don't want to publish the ports to host, only expose them on my custom network between the containers. This is why I tried to replace the PORTS with EXPOSE, but it didn't really help.
for startup Compose does not wait until a container is “ready” (whatever that means for your particular application) - only until it’s running
So, you should refer : https://docs.docker.com/compose/startup-order/
for wait until for config server container is ready first (already listened on port 9001), then starting Eureka server container.
You registry service try to start before configuration service is up. You have to force him to wait.
You can do it with condition: service_started, like below. But I think it is not working with version 3
version: '2.1'
# ...
tao-elszamolas-discovery:
build: ./tao-elszamolas-discovery
container_name: tao-elszamolas-discovery
depends_on:
- tao-elszamolas-config:
condition: service_started
ports:
- "9002:9002"
volumes:
- "/tao-elszamolas/logs:/var/log/tao-elszamolas"
networks:
- taonetwork
A solution for version 3 is to use healthcheck and restart: on-failure
Take a look at this post
Another solution is to use entrypoint section to launch a batch file at start which is pinging configuration service container while it is not responding.

Connect Spring with Elasticsearch in Docker

I have to connect my Spring client with Elasticsearch whose image have been taken by the official ES 2.4.6 image that Elastic has in Docker Hub but when I try to run the containers, the Docker console reports me this error:
[Glitch] failed to connect to node
[{#transport#-1}{localhost}{127.0.0.1:9300}], removed from nodes list
The application.properties of my Spring project is:
spring.data.elasticsearch.cluster-nodes=localhost:9300
index.v = default
server.port = 8443
And the docker-compose.yml is:
version: "2.2"
services:
elk:
image: cvazquezlos/elk:2.4.6
ports:
- 5000:5000
- 5601:5601
- 9200:9200
- 9300:9300
volumes:
- elk-data:/var/lib/elasticsearch
testloganalyzer:
image: cvazquezlos/testloganalyzer
ports:
- 8443:8443
volumes:
elk-data:
If I run the backend without Docker it works as expected, but when I run the backend with Docker, reports me the above error. The complete error is:
failed to connect to node [{#transport#-1}{localhost}{127.0.0.1:9300}], removed from nodes list
org.elasticsearch.transport.ConnectTransportException: [][127.0.0.1:9300] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(NettyTransport.java:967) ~[elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:933) ~[elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906) ~[elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267) ~[elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler.doSample(TransportClientNodesService.java:390) ~[elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.client.transport.TransportClientNodesService$NodeSampler.sample(TransportClientNodesService.java:336) [elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.client.transport.TransportClientNodesService$ScheduledNodeSampler.run(TransportClientNodesService.java:369) [elasticsearch-2.4.6.jar!/:2.4.6]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]
Caused by: java.net.ConnectException: Connection refused: localhost/127.0.0.1:9300
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_151]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[na:1.8.0_151]
at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[netty-3.10.6.Final.jar!/:na]
... 3 common frames omitted
When the SpringBoot service works in docker it cannot resolve the localhost as a host of the ES. In this case you can also use the links property like following:
In testloganalyzer section add the links option:
testloganalyzer:
image: cvazquezlos/testloganalyzer
ports:
- 8443:8443
links:
- elk:elk
First is the service and second is an alias.
Next change it reference in the application.properties:
spring.data.elasticsearch.cluster-nodes=elk:9300
I think you're missing the networks section in your docker-compose configuration, try this:
version: "2.2"
services:
elk:
image: cvazquezlos/elk:2.4.6
ports:
- 5000:5000
- 5601:5601
- 9200:9200
- 9300:9300
networks:
- elk-network
volumes:
- elk-data:/var/lib/elasticsearch
testloganalyzer:
image: cvazquezlos/testloganalyzer
ports:
- 8443:8443
networks:
- elk-network
volumes:
elk-data:
networks:
elk-network:
driver: bridge

Resources