Docker Compose Volumes - Device Option - Windows - JHipster - spring-boot

I'm trying to configure a JHipster web application (Java Spring Boot + Angular) to use an embedded H2 DB in production environment.
What I want to achieve is to save the H2 generated DB files on the host machine so that if I need to backup those file for some reasons I have easy access.
The application will be deployed using docker compose.
DEV ENVIRONMENT
I'm using a Windows machine to develop this web application.
I have installed Docker Desktop on my machine and is running on WSL 2 engine.
JHIPSTER DOCKER IMAGE
The docker image I am using to deploy the application is the standard used by the JHipster project. This means that the web application inside docker runs with UID = 1000
The User UID = 1000 doesn't have root access (which is of course correct for security reasons)
DOCKER COMPOSE AND VOLUME CONFIGURATIONS
The first docker compose configuration I tried was the following one:
version: '3.8'
services:
app:
image: scriba_webapp
environment:
- ...
ports:
- ...
volumes:
- scriba-db:/tmp/h2
volumes:
scriba-db:
but when the web applications starts I receive a "permission denied" error. Looking at the logs the problem seems to be when the application tries to initialize the DB (which means creating and writing a file to the /tmp/h2 container directory).
Here the stacktrace:
org.h2.message.DbException: Log file error: "/tmp/h2/db.trace.db", cause: "java.io.FileNotFoundException: /tmp/h2/db.trace.db (Permission denied)" [90034-200]
at org.h2.message.DbException.get(DbException.java:194)
at org.h2.message.TraceSystem.logWritingError(TraceSystem.java:294)
at org.h2.message.TraceSystem.openWriter(TraceSystem.java:315)
at org.h2.message.TraceSystem.writeFile(TraceSystem.java:263)
at org.h2.message.TraceSystem.write(TraceSystem.java:247)
at org.h2.message.Trace.error(Trace.java:180)
at org.h2.engine.Database.setBackgroundException(Database.java:2230)
at org.h2.mvstore.db.MVTableEngine$1.uncaughtException(MVTableEngine.java:93)
at org.h2.mvstore.MVStore.handleException(MVStore.java:2877)
at org.h2.mvstore.MVStore.panic(MVStore.java:481)
at org.h2.mvstore.MVStore.<init>(MVStore.java:402)
at org.h2.mvstore.MVStore$Builder.open(MVStore.java:3579)
at org.h2.mvstore.db.MVTableEngine$Store.open(MVTableEngine.java:170)
at org.h2.mvstore.db.MVTableEngine.init(MVTableEngine.java:103)
at org.h2.engine.Database.getPageStore(Database.java:2659)
at org.h2.engine.Database.open(Database.java:675)
at org.h2.engine.Database.openDatabase(Database.java:307)
at org.h2.engine.Database.<init>(Database.java:301)
at org.h2.engine.Engine.openSession(Engine.java:74)
at org.h2.engine.Engine.openSession(Engine.java:192)
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:171)
at org.h2.engine.Engine.createSession(Engine.java:166)
at org.h2.engine.Engine.createSession(Engine.java:29)
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:340)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:173)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:152)
at org.h2.Driver.connect(Driver.java:69)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:266)
at org.springframework.boot.autoconfigure.liquibase.DataSourceClosingSpringLiquibase.afterPropertiesSet(DataSourceClosingSpringLiquibase.java:46)
at tech.jhipster.config.liquibase.AsyncSpringLiquibase.initDb(AsyncSpringLiquibase.java:118)
at tech.jhipster.config.liquibase.AsyncSpringLiquibase.afterPropertiesSet(AsyncSpringLiquibase.java:103)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1845)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1782)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:602)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:524)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1154)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:908)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:434)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:338)
at it.brainylabs.scriba.ScribaApp.main(ScribaApp.java:69)
Caused by: org.h2.jdbc.JdbcSQLNonTransientException: Log file error: "/tmp/h2/db.trace.db", cause: "java.io.FileNotFoundException: /tmp/h2/db.trace.db (Permission denied)" [90034-200]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:505)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:429)
... 56 more
Caused by: java.io.FileNotFoundException: /tmp/h2/db.trace.db (Permission denied)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(Unknown Source)
at java.base/java.io.FileOutputStream.<init>(Unknown Source)
at java.base/java.io.FileOutputStream.<init>(Unknown Source)
at org.h2.store.fs.FilePathDisk.newOutputStream(FilePathDisk.java:306)
at org.h2.store.fs.FileUtils.newOutputStream(FileUtils.java:239)
at org.h2.message.TraceSystem.openWriter(TraceSystem.java:311)
... 53 more
Exception in thread "Logback shutdown hook [default]" java.util.ConcurrentModificationException
at java.base/java.util.ArrayList$Itr.checkForComodification(Unknown Source)
at java.base/java.util.ArrayList$Itr.next(Unknown Source)
at ch.qos.logback.classic.LoggerContext.fireOnReset(LoggerContext.java:323)
at ch.qos.logback.classic.LoggerContext.reset(LoggerContext.java:226)
at ch.qos.logback.classic.LoggerContext.stop(LoggerContext.java:348)
at ch.qos.logback.core.hook.ShutdownHookBase.stop(ShutdownHookBase.java:39)
at ch.qos.logback.core.hook.DelayingShutdownHook.run(DelayingShutdownHook.java:57)
at java.base/java.lang.Thread.run(Unknown Source)
The next configuration I tried was using the "old" bind mounts instead of the new docker volumes:
version: '3.8'
services:
app:
image: scriba_webapp
environment:
- ...
ports:
- ...
volumes:
- type: bind
source: D:\...\h2
target: /tmp/h2
With this everything works fine, the web applications startup correctly and I can see the DB files in the source directory. So without other changes to the project a bind mount works correctly while a docker volume doesn't.
The final test I did was to configure the docker volume like this:
version: '3.8'
services:
app:
image: ...
environment:
- ...
ports:
- ...
volumes:
- scriba-db
volumes:
scriba-db:
driver_opts:
type: "volume"
o: "uid=1000,rw"
device: **???**
Since the original error was a permission denied I figured I should try and give that user read/write access to the docker volume but I didn't manage to do so since it's not clear to me how the volume option device should be configured for this case.
If I configure the driver_opts like this:
scriba-db:
driver_opts:
type: "tmpfs"
o: "uid=1000,rw"
device: "tmpfs"
The web application starts correctly but I can't see the DB files in the docker volume since tmpfp (If I understood this correctly) means that the FS is just temporary and will be delete once the containers stops.
THE QUESTIONS
How come a bind mount works just like that and a docker volume instead has problems?
Let's say I want to use a docker volume and save those DB files on the host machine, how should I configure the driver_opts on a Windows machine?

Related

Spring boot java Docker connecting mongoDB in K8s

I am struggling to connect mongodb using spring boot docket application, Below are the further details.
a) Created a mongoDB cluster in minikube.
b) Exposed an external service which connects to the mongodb. The service exposed was through Nodeport. Below is the code snippet.
mongo-service
---------------
`apiVersion: v1
kind: Service
metadata:
name: mongodb-external-service
spec:
selector:
app: mongodb
type: NodePort
ports:
- port: 27017
protocol: TCP
targetPort: 27017
nodePort: 32000
c) Minikube exposed the service with IP 192.168.49.2
d) I was able to connect mongodb using 192.168.49.2:2000 from my localhost using mongodb compass.
e) Created simple spring boot application which was able to connect the mongodb service using Ip and port.
So far no problems things were moving as expected. Next my objective was to containerize my spring boot application and execute in in the Minikube. Before i execute it as pod in minikube i wanted to test the container using the docker-compose. Below is my docker-compose and Dockerfile code snippets
Dockerfile
FROM adoptopenjdk:11-jre-hotspot as builder
ARG JAR_FILE=target/\*.jar
COPY ${JAR_FILE} tradeprocessor.jar
RUN java -Djarmode=layertools -jar tradeprocessor.jar extract
FROM adoptopenjdk:11-jre-hotspot
COPY --from=builder dependencies/ ./
COPY --from=builder snapshot-dependencies/ ./
COPY --from=builder spring-boot-loader/ ./
COPY --from=builder application/ ./
ENTRYPOINT \["java", "org.springframework.boot.loader.JarLauncher"\]
docker-compose
---------------
`version: '3.7'
services:
trdprocess:
build:
context: .
dockerfile: Dockerfile
image: tradeprocess:latest
ports:
- 8088:8088
extra_hosts:
- dockerhost:192.168.49.2
environment:
- spring_data_mongodb_authentication_database=admin
- spring_data_mongodb_username=mongouser
- spring_data_mongodb_password=mongopassword
- spring_data_mongodb_database=transactions
- spring_data_mongodb_port=32000
- spring_data_mongodb_host=192.168.49.2`
however Upon executing the above step, application is spinning up but it is throwing SocketTimeoutException.
Exception:
`Started TradeprocessingApplication in 2.09 seconds (JVM running for 2.973)`
2023-01-04 20:16:13.529 INFO 1 --- [.168.49.2:32000] o.m.d.cluster : Exception in monitor thread while connecting to server 192.168.49.2:32000
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:70) ~[mongodb-driver-core-4.6.1.jar:?]
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:180) ~[mongodb-driver-core-4.6.1.jar:?]
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:193) ~[mongodb-driver-core-4.6.1.jar:?]
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:157) ~[mongodb-driver-core-4.6.1.jar:?]
at java.lang.Thread.run(Unknown Source) ~[?:?]
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) ~[?:?]
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) ~[?:?]
at java.net.AbstractPlainSocketImpl.connect(Unknown Source) ~[?:?]
at java.net.SocksSocketImpl.connect(Unknown Source) ~[?:?]
at java.net.Socket.connect(Unknown Source) ~[?:?]
at com.mongodb.internal.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:107) ~[mongodb-driver-core-4.6.1.jar:?]
It looks like it has to do with network setting but not sure how to correct it. Any help is highly appreciated
Tried to change few option with docker-compose such as extra_hosts port mappings etc but no luck
I finally was able to solve my own problem. A bit of learning curve. Hear are the details.
My entire setup is running on Ubuntu workstation with minikube cluster.
Minikube started with command minikube start --driver=docker. Which means minikube itself is running as container within docker runtime environment.
docker network ls
docker network ls clearly shows that both my custom build application and the minikube are running as containers. Therefore i had to link these containers network. A small change in the docker-compose did the trick. Below is the code:
networks:
default :
external :
name : minikube

Java.net.ConnectException: Keycloak's Client gets Connection refused on Docker while Auth routing

If I have a Keycloak server running in local docker desktop and a client application running locally (non dockerized scenario), It works fine.
If I have Keycloak server and a client application running in local docker desktop, It throws
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:609)
at com.mysql.jdbc#8.0.22//com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:155)
at com.mysql.jdbc#8.0.22//com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:63)
... 64 more
If I run my client application in a local (non-dockerized) environment and pointing to a dockerized keycloak server of the cloud instance, I get the following error.
2022-02-18 12:59:55.039 WARN 1192 --- [nio-8080-exec-7] o.keycloak.adapters.KeycloakDeployment : Failed to load URLs from http://XXXX.CLOUD.XXXX:8180/auth/realms/SampleKeycloakApp/.well-known/openid-configuration
java.lang.Exception: Forbidden
at org.keycloak.adapters.KeycloakDeployment.getOidcConfiguration(KeycloakDeployment.java:233) ~[keycloak-adapter-core-16.1.1.jar!/:16.1.1]
at org.keycloak.adapters.KeycloakDeployment.resolveUrls(KeycloakDeployment.java:182) ~[keycloak-adapter-core-16.1.1.jar!/:16.1.1]
at org.keycloak.adapters.KeycloakDeployment.getAuthUrl(KeycloakDeployment.java:251) ~[keycloak-adapter-core-16.1.1.jar!/:16.1.1]
at org.keycloak.adapters.OAuthRequestAuthenticator.getRedirectUri(OAuthRequestAuthenticator.java:175) ~[keycloak-adapter-core-16.1.1.jar!/:16.1.1]
at org.keycloak.adapters.OAuthRequestAuthenticator.loginRedirect(OAuthRequestAuthenticator.java:213) ~[keycloak-adapter-core-16.1.1.jar!/:16.1.1]
at org.keycloak.adapters.OAuthRequestAuthenticator.authenticate(OAuthRequestAuthenticator.java:275) ~[keycloak-adapter-core-16.1.1.jar!/:16.1.1]
at org.keycloak.adapters.RequestAuthenticator.authenticate(RequestAuthenticator.java:138) ~[keycloak-adapter-core-16.1.1.jar!/:16.1.1]
at org.keycloak.adapters.springsecurity.filter.KeycloakAuthenticationProcessingFilter.attemptAuthentication(KeycloakAuthenticationProcessingFilter.java:154) ~[keycloak-spring-security-adapter-16.1.1.jar!/:16.1.1]
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212) ~[spring-security-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) ~[spring-security-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
at org.keycloak.adapters.springsecurity.filter.KeycloakPreAuthActionsFilter.doFilter(KeycloakPreAuthActionsFilter.java:96) ~[keycloak-spring-security-adapter-16.1.1.jar!/:16.1.1]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) ~[spring-security-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
at org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:100) ~[spring-security-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.1.10.RELEASE.jar!/:5.1.10.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) ~[spring-security-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:74) ~[spring-security-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.1.10.RELEASE.jar!/:5.1.10.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) ~[spring-security-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105) ~[spring-security-web-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
Sharing the docker-compose file for reference here when I run the both services in docker.
version: "3"
services: springboot:
build: .
container_name: springboot
ports:
- 8081:8081
restart: always
depends_on:
- db
- keycloak
environment:
SPRING_DATASOURCE_URL: jdbc:mysql://db:3306/todolist
SPRING_DATASOURCE_USERNAME: admin
SPRING_DATASOURCE_PASSWORD: admin
KEYCLOAK_URI: https://keycloak:8443/auth
REALM: SpringBootKeycloakApp
networks:
- common-network keycloak:
image: jboss/keycloak:14.0.0
container_name: keycloak
ports:
- "8180:8180"
- "8443:8443"
command: ["-Djboss.socket.binding.port-offset=100"]
environment:
DB_VENDOR: POSTGRES
DB_ADDR: postgres
DB_DATABASE: keycloak
DB_USER: keycloak
DB_PASSWORD: password
DB_SCHEMA: public
KEYCLOAK_USER: admin
KEYCLOAK_PASSWORD: admin
depends_on:
- postgres
networks:
- common-network db:
image: mysql:5.7
ports:
- "3307:3306"
restart: always
environment:
MYSQL_DATABASE: todolist
MYSQL_USER: admin
MYSQL_PASSWORD: admin
MYSQL_ROOT_PASSWORD: root
volumes:
- db-data:/var/lib/mysql
networks:
- common-network postgres:
image: postgres
volumes:
- postgres_data:/var/lib/postgresql/data
environment:
POSTGRES_DB: keycloak
POSTGRES_USER: keycloak
POSTGRES_PASSWORD: password
networks:
- common-network networks: common-network:
driver: bridge
volumes:
db-data:
driver: local
postgres_data:
driver: local
and when I run the client app separately in non-docker env, I run it with java -jar ClientApp.jar
and when I run the client app separately in docker env, I use the following Dockerfile
FROM adoptopenjdk/openjdk11:latest
ARG JAR_FILE=./build/libs/*.jar
COPY ${JAR_FILE} ClientApp.jar
EXPOSE 8081
ENTRYPOINT ["java", "-jar", "ClientApp.jar"]
I would be happy to understand the real root cause behind this and run the keycloak server and the client application in a dockerized cloud instance without any issues. Thx.
If you use both keycloak server and the client in Cloud, use ssl-required=none
keycloak.ssl-required=none
If you use both keycloak server and the client in localhost, use ssl-required=external
keycloak.ssl-required=external

Docker compose error on mysql db connect for spring boot app Caused by: java.net.UnknownHostException: [duplicate]

This question already has answers here:
Communications link failure , Spring Boot + MySql +Docker + Hibernate
(6 answers)
Closed 3 years ago.
I have spring boot app
My Dockerfile is
FROM openjdk:8-jdk-alpine
EXPOSE 8080
ARG JAR_FILE=target/demo-0.0.1-SNAPSHOT.jar
ADD ${JAR_FILE} demo.jar
ENTRYPOINT ["java","-jar","/demo.jar"]
My docker compose file
# Docker Compose file Reference (https://docs.docker.com/compose/compose-file/)
version: '3.7'
# Define services
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: . # Use an image built from the specified dockerfile in the `polling-app-server` directory.
dockerfile: ./Dockerfile
container_name: empserver
ports:
- "3000:3000" # Forward the exposed port 8080 on the container to port 8080 on the host machine
restart: always
depends_on:
- db # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://db:3306/employee_entries?useSSL=false&serverTimezone=UTC&useLegacyDatetimeCode=false
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: root
# Database Service (Mysql)
db:
image: mysql:5.7
ports:
- "3306:3306"
restart: always
environment:
MYSQL_DATABASE: employee_entries
MYSQL_USER: root
MYSQL_PASSWORD: root
MYSQL_ROOT_PASSWORD: root
My docker net works
NETWORK ID NAME DRIVER SCOPE
b95e3d99b266 Default Switch ics local
7fff4f9713f8 demo_default nat local
fe8883b77d1d emp-mysql ics local
f464aab9064a nat nat local
a5bd5e8efe61 none null local
App is successfully running using java -jar target\demo-0.0.1-SNAPSHOT.jar
but when I am doing docker-compose up
I got below error
app-server_1 | Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
app-server_1 |
app-server_1 | The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
app-server_1 | at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_212]
app-server_1 | at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_212]
app-server_1 | at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_212]
app-server_1 | at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_212]
app-server_1 | at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:91) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.NativeSession.connect(NativeSession.java:144) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:956) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:826) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | ... 56 common frames omitted
app-server_1 | Caused by: java.net.UnknownHostException: db
app-server_1 | at java.net.InetAddress.getAllByName0(InetAddress.java:1281) ~[na:1.8.0_212]
app-server_1 | at java.net.InetAddress.getAllByName(InetAddress.java:1193) ~[na:1.8.0_212]
app-server_1 | at java.net.InetAddress.getAllByName(InetAddress.java:1127) ~[na:1.8.0_212]
app-server_1 | at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:132) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65) ~[mysql-connector-java-8.0.19.jar!/:8.0.19]
app-server_1 | ... 59 common frames omitted
I am able to access mysql database and tables but from docker compose it was not
any suggestion would really be helpful
You need to provide the container names to the services and use them when referring them from each other. In your environment section for app-server, the url for database points to 127.0.0.1 but the database is not running on same container as app-server so this will fail.
To make this work, provide container names to services for eg : my_mysql and my_app-server and use it in environment url as jdbc:mysql://my_mysql:3306.
Please see the modified file below:
# Docker Compose file Reference (https://docs.docker.com/compose/compose-file/)
version: '3.7'
# Define services
services:
# App backend service
app-server:
# Configuration for building the docker image for the backend service
build:
context: . # Use an image built from the specified dockerfile in the `polling-app-server` directory.
dockerfile: ./Dockerfile
container_name: my_app-server
ports:
- "3000:3000" # Forward the exposed port 8080 on the container to port 8080 on the host machine
restart: always
depends_on:
- db # This service depends on mysql. Start that first.
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:mysql://my_mysql:3306/employee_entries?useSSL=false&serverTimezone=UTC&useLegacyDatetimeCode=false
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: root
# Database Service (Mysql)
db:
image: mysql:5.7
container_name: my_mysql
ports:
- "3306:3306"
restart: always
environment:
MYSQL_DATABASE: employees
MYSQL_USER: root
MYSQL_PASSWORD: root
MYSQL_ROOT_PASSWORD: root
network:
my-network:

Spring Cloud Config in Docker container is not accessible

I have a relatively simple setup. A Spring Could Config server, and a Spring Cloud Eureka server. (I refer Eureka server as Discover everywhere in my config...) The config server uses a Git repository for finding the appropriate config.
Everything is in Docker containers. I use docker-compose to build my services. The Gateway server should wait for config server to come up. I'll provide the source of the file further down.
So, when I build and start the containers, the discovery (Eureka) server doesn't want to come up. It exits with an exception. It is really long, so I just add the most important part here:
019-01-27 23:52:17,494 [INFO ] o.s.c.c.c.ConfigServicePropertySourceLocator.getRemoteEnvironment [main] – Fetching config from server at : http://tao-elszamolas-config:9001
2019-01-27 23:52:17,898 [INFO ] o.s.c.c.c.ConfigServicePropertySourceLocator.getRemoteEnvironment [main] – Connect Timeout Exception on Url - http://tao-elszamolas-config:9001. Will be trying the next url if available
2019-01-27 23:52:17,902 [ERROR ] o.s.b.SpringApplication.reportFailure [main] – Application run failed
java.lang.IllegalStateException: Could not locate PropertySource and the fail fast property is set, failing
... Something...Something
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://tao-elszamolas-config:9001/discovery-server/prod": Connection refused (Connection refused); nested exception is java.net.ConnectException: Connection refused (Connection refused)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:743) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:669) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:578) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.getRemoteEnvironment(ConfigServicePropertySourceLocator.java:218) ~[spring-cloud-config-client-2.1.0.RC3.jar!/:2.1.0.RC3]
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.locate(ConfigServicePropertySourceLocator.java:96) ~[spring-cloud-config-client-2.1.0.RC3.jar!/:2.1.0.RC3]
... 15 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[?:?]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[?:?]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[?:?]
at java.net.Socket.connect(Socket.java:591) ~[?:?]
at java.net.Socket.connect(Socket.java:540) ~[?:?]
at sun.net.NetworkClient.doConnect(NetworkClient.java:182) ~[?:?]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:474) ~[?:?]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:569) ~[?:?]
at sun.net.www.http.HttpClient.<init>(HttpClient.java:242) ~[?:?]
at sun.net.www.http.HttpClient.New(HttpClient.java:341) ~[?:?]
at sun.net.www.http.HttpClient.New(HttpClient.java:362) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1242) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1181) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1075) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1009) ~[?:?]
at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:76) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:734) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:669) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:578) ~[spring-web-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.getRemoteEnvironment(ConfigServicePropertySourceLocator.java:218) ~[spring-cloud-config-client-2.1.0.RC3.jar!/:2.1.0.RC3]
at org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.locate(ConfigServicePropertySourceLocator.java:96) ~[spring-cloud-config-client-2.1.0.RC3.jar!/:2.1.0.RC3]
... 15 more
The only exception what I found in the Config server log is the one complaining about the connection to Discovery (Eureka) server. Yes, I try to register my config service inEureka for looking it up at startup of further microservices.
The connection is refused which means, the config server is not running at the time when Eureka server is trying to get its config or it is not listening on the given port, 9001.
How is it possible? I used the depends_on in my compose file, so it should wait for that.
Or why it finishes working? If I do the same thing on local without docker then it keeps trying to connect to Config, until the config server starts up. I'd like achieve the same behaviour using Docker.
Any advice would be appreciated!
And finally here are my config files:
docker-compose.yml:
version: "3"
services:
tao-elszamolas-config:
build: ./tao-elszamolas-config
container_name: tao-elszamolas-config
ports:
- "9001:9001"
volumes:
- "/tao-elszamolas/logs:/var/log/tao-elszamolas"
networks:
- taonetwork
tao-elszamolas-discovery:
build: ./tao-elszamolas-discovery
container_name: tao-elszamolas-discovery
depends_on:
- tao-elszamolas-config
ports:
- "9002:9002"
volumes:
- "/tao-elszamolas/logs:/var/log/tao-elszamolas"
networks:
- taonetwork
networks:
taonetwork:
driver: bridge
Dockerfile of Config Server:
FROM openjdk:11-jdk-slim
MAINTAINER BeszterceKK
COPY ./tao-elszamolas-config.jar /usr/src/taoelszamolas/tao-elszamolas-config.jar
WORKDIR /var/log/tao-elszamolas
WORKDIR /usr/src/taoelszamolas
ENV SPRING_PROFILES_ACTIVE prod
EXPOSE 9001
ENTRYPOINT ["java", "-DlogFileLocation=/var/log/tao-elszamolas", "-jar", "tao-elszamolas-config.jar"]
application.yml for Config server:
server:
port: 9001
spring:
application:
name: config-server
cloud:
config:
server:
git:
uri: https://github.com/something/something
username: something
password: something
skip-ssl-validation: true
security:
user:
name: something
password: something
eureka:
client:
serviceUrl:
defaultZone: http://something:something#tao-elszamolas-discovery:9002/eureka/
info:
artifact: tao-elszamolas-config
name: TAO Elszamolas Config Application
description: Tao elszamolas microservice elosztott szolgaltatas konfiguracio
version: 1.0.0
Dockerfile of Discovery (Eureka) Server:
FROM openjdk:11-jdk-slim
MAINTAINER BeszterceKK
COPY ./tao-elszamolas-discovery.jar /usr/src/taoelszamolas/tao-elszamolas-discovery.jar
WORKDIR /var/log/tao-elszamolas
WORKDIR /usr/src/taoelszamolas
ENV SPRING_PROFILES_ACTIVE prod
EXPOSE 9002
ENTRYPOINT ["java", "-DlogFileLocation=/var/log/tao-elszamolas", "-jar", "tao-elszamolas-discovery.jar"]
bootstrap.yml for Discovery (Eureka) server:
spring:
application:
name: discovery-server
cloud:
config:
fail-fast: true
name: discovery-server
uri: http://tao-elszamolas-config:9001
username: something
password: something
application.yml for Discovery (Eureka) server:
server:
port: 9002
eureka:
instance:
hostname: tao-elszamolas-discovery
client:
registerWithEureka: false
fetchRegistry: false
serviceUrl:
defaultZone: http://tao-elszamolas-discovery:9002/eureka
info:
artifact: tao-elszamolas-discovery
name: TAO Elszamolas Eureka Szolgaltatas monitor
description: Tao elszamolas automatikus szolgaltatas regisztracio
version: 1.0.0
And finally, this is how I start my containers. It is simply:
docker-compose up -d
The whole application stack works on local without Docker. So I guess the problem is with my Docker configuration.
One more thing to mention, generally, I don't want to publish the ports to host, only expose them on my custom network between the containers. This is why I tried to replace the PORTS with EXPOSE, but it didn't really help.
for startup Compose does not wait until a container is “ready” (whatever that means for your particular application) - only until it’s running
So, you should refer : https://docs.docker.com/compose/startup-order/
for wait until for config server container is ready first (already listened on port 9001), then starting Eureka server container.
You registry service try to start before configuration service is up. You have to force him to wait.
You can do it with condition: service_started, like below. But I think it is not working with version 3
version: '2.1'
# ...
tao-elszamolas-discovery:
build: ./tao-elszamolas-discovery
container_name: tao-elszamolas-discovery
depends_on:
- tao-elszamolas-config:
condition: service_started
ports:
- "9002:9002"
volumes:
- "/tao-elszamolas/logs:/var/log/tao-elszamolas"
networks:
- taonetwork
A solution for version 3 is to use healthcheck and restart: on-failure
Take a look at this post
Another solution is to use entrypoint section to launch a batch file at start which is pinging configuration service container while it is not responding.

Connect Spring with Elasticsearch in Docker

I have to connect my Spring client with Elasticsearch whose image have been taken by the official ES 2.4.6 image that Elastic has in Docker Hub but when I try to run the containers, the Docker console reports me this error:
[Glitch] failed to connect to node
[{#transport#-1}{localhost}{127.0.0.1:9300}], removed from nodes list
The application.properties of my Spring project is:
spring.data.elasticsearch.cluster-nodes=localhost:9300
index.v = default
server.port = 8443
And the docker-compose.yml is:
version: "2.2"
services:
elk:
image: cvazquezlos/elk:2.4.6
ports:
- 5000:5000
- 5601:5601
- 9200:9200
- 9300:9300
volumes:
- elk-data:/var/lib/elasticsearch
testloganalyzer:
image: cvazquezlos/testloganalyzer
ports:
- 8443:8443
volumes:
elk-data:
If I run the backend without Docker it works as expected, but when I run the backend with Docker, reports me the above error. The complete error is:
failed to connect to node [{#transport#-1}{localhost}{127.0.0.1:9300}], removed from nodes list
org.elasticsearch.transport.ConnectTransportException: [][127.0.0.1:9300] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(NettyTransport.java:967) ~[elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:933) ~[elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:906) ~[elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:267) ~[elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler.doSample(TransportClientNodesService.java:390) ~[elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.client.transport.TransportClientNodesService$NodeSampler.sample(TransportClientNodesService.java:336) [elasticsearch-2.4.6.jar!/:2.4.6]
at org.elasticsearch.client.transport.TransportClientNodesService$ScheduledNodeSampler.run(TransportClientNodesService.java:369) [elasticsearch-2.4.6.jar!/:2.4.6]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]
Caused by: java.net.ConnectException: Connection refused: localhost/127.0.0.1:9300
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_151]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[na:1.8.0_151]
at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[netty-3.10.6.Final.jar!/:na]
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[netty-3.10.6.Final.jar!/:na]
... 3 common frames omitted
When the SpringBoot service works in docker it cannot resolve the localhost as a host of the ES. In this case you can also use the links property like following:
In testloganalyzer section add the links option:
testloganalyzer:
image: cvazquezlos/testloganalyzer
ports:
- 8443:8443
links:
- elk:elk
First is the service and second is an alias.
Next change it reference in the application.properties:
spring.data.elasticsearch.cluster-nodes=elk:9300
I think you're missing the networks section in your docker-compose configuration, try this:
version: "2.2"
services:
elk:
image: cvazquezlos/elk:2.4.6
ports:
- 5000:5000
- 5601:5601
- 9200:9200
- 9300:9300
networks:
- elk-network
volumes:
- elk-data:/var/lib/elasticsearch
testloganalyzer:
image: cvazquezlos/testloganalyzer
ports:
- 8443:8443
networks:
- elk-network
volumes:
elk-data:
networks:
elk-network:
driver: bridge

Resources