Traefik throwing error 502 with Kotlin Spring Boot application - spring-boot

Ok so I'm facing an annoying issue with Traefik in Docker: it constantly throws an error 502: Bad Gateway when I'm trying to reach my HTTP service.
All I'm trying to do is add it as a simple load balancer to build 3 replicas of my Spring Boot app. The app works well when I remove anything concerning Traefik.
My Spring Boot Controller maps /api/v1 endpoint, which is working well without load balancing. I'm trying to map this same endpoint to Traefik, which is using port :80.
Dashboard at :8080 works just fine, I have no error whatsoever about my configuration mentioned in it; only problems when I'm reaching anything under :80.
I have to mention that this exact same configuration (docker-compose, traefik conf, postgres db, and spring boot app) DOES WORK fine with the exact same docker-compose.yml AND traefik.yml, which makes my problem even weirder. Maybe the issue comes from the fact that my Spring Boot app is made of Kotlin and not the one where it's working?. Anyway, here are my files:
# Dockerfile (default one generated)
FROM maven:latest
WORKDIR /usr/src/app
COPY . /usr/src/app
RUN mvn package
ENV PORT 5000
EXPOSE $PORT
CMD [ "sh", "-c", "mvn -Dserver.port=${PORT} spring-boot:run" ]
# docker-compose.yml (works as-is in my other app)
version: "3.8"
services:
project:
build: .
deploy:
replicas: 3
labels:
- "traefik.http.routers.project.rule=PathPrefix(`/api/v1`)" # same endpoint as in my Spring controller
- "traefik.http.middlewares.project.stripprefix.forceSlash=false"
- "traefik.http.routers.project.middlewares=project#docker"
traefik:
image: traefik:latest
restart: unless-stopped
ports:
- "80:80"
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik.yml:/etc/traefik/traefik.yml
postgres:
image: postgres:latest
container_name: postgres
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: example
# rabbitmq:
# image: rabbitmq:management-alpine
# container_name: rabbitmq
# ports:
# - "5672:5672" # AMQP
# - "15672:15672" # HTTP (Management)
# traefik.yml
global:
checkNewVersion: true
sendAnonymousUsage: false
providers:
docker: {}
log:
level: DEBUG # temporarily
api:
insecure: true
entryPoints:
insecure:
address: ":80"
secure:
address: ":443"
# src/main/resources/application.yml
spring:
sql.init.mode: always
datasource:
url: jdbc:postgresql://localhost:5432,postgres:5432/postgres
username: postgres
password: example
jpa:
open-in-view: false
<!-- pom.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>project</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.6.4</version>
<relativePath/>
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>17</java.version>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
<kotlin.version>1.6.10</kotlin.version>
</properties>
<dependencies>
<!-- Kotlin -->
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-reflect</artifactId>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-stdlib-jdk8</artifactId>
</dependency>
<!-- Spring Boot -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<!-- PostgreSQL -->
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
</dependency>
</dependencies>
<build>
<sourceDirectory>${project.basedir}/src/main/kotlin</sourceDirectory>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-plugin</artifactId>
<configuration>
<args>
<arg>-Xjsr305=strict</arg>
</args>
<compilerPlugins>
<plugin>spring</plugin>
</compilerPlugins>
</configuration>
<dependencies>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-allopen</artifactId>
<version>${kotlin.version}</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</project>
I'm using JDK 17, and docker-compose up --build to start everything. Thanks!

Related

Skaffold dev doesn't do automatic file sync for spring boot app

I have a spring boot app that generates a jar artifact.I want to configure skaffold to make use of automatic file sync. Below are my config files :
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.13</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>rest-service-complete</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>rest-service-complete</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>${project.parent.version}</version>
</plugin>
</plugins>
</build>
</project>
Dockerfile
VOLUME /tmp
ARG DEPENDENCY=target/dependency
COPY ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY ${DEPENDENCY}/META-INF /app/META-INF
COPY ${DEPENDENCY}/BOOT-INF/classes /app
ENTRYPOINT ["java","-cp","app:app/lib/*","com.example.restservice.RestServiceApplication"]
deployment.yaml
kind: Deployment
metadata:
labels:
app: demo
name: demo
spec:
replicas: 1
selector:
matchLabels:
app: demo
strategy: {}
template:
metadata:
labels:
app: demo
spec:
containers:
- image: mariatamas89/spring-boot-hw
name: spring-boot-hw
imagePullPolicy: Always
resources: {}
imagePullSecrets:
- name: myregistrykey
status: {}
---
apiVersion: v1
kind: Service
metadata:
labels:
app: demo
name: demo
spec:
ports:
- name: 8080-8080
port: 8080
protocol: TCP
targetPort: 8080
selector:
app: demo
type: LoadBalancer
status:
loadBalancer: {}
skaffold.yaml
kind: Config
metadata:
name: complete
build:
tagPolicy:
sha256: { }
local:
push: true
artifacts:
- image: mariatamas89/spring-boot-hw
sync:
infer:
- '/target/dependency/app/com/example/*.java'
docker:
dockerfile: Dockerfile
noCache: true
deploy:
kubectl:
manifests:
- deployment.yaml
I run skaffold dev , app starts correctly , but when I do a change to a java file , nothing happens in the skaffold cli, the change is not picked up. Any ideas how to make skaffold file sync work?Thank you.

Spring Gateway deployment on external tomcat

I am developing a micro-service solution with spring boot. I have 3 micro services:
authentication services (manages users, roles, permissions)
project service (services used to do CRUD actions on Project entity)
gateway (used as entry point of the solution, also used to validate token and route to services)
I want to deploy these on a tomcat server as war placed in the webapps (knowing that they have an embedded tomcat).
I am able to deploy the authentication and project services and they are reachable over the URI ** localhost:8080/authentication/* ** and ** /project/* *, but I'm unable to reach the gateway on ** localhost:8080/gateway/ **, it returns a 404
I am sure that the gateway is running cause it created the tables in the DB after deployment.
**Knowing that in eclipse it works fine (since i run each service on a port specified in the application.yml) i don't understand why when deploying the gateway on tomcat it is unreachable and always return a 404. I can see it in the tomcat manager as deployed.
Here is the pom.xml of the gateway :
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.4.1</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>com.myProject</groupId>
<artifactId>gateway</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>gateway</name>
<description>Gateway</description>
<profiles>
<profile>
<id>dev</id>
<properties>
<activatedProperties>dev</activatedProperties>
</properties>
</profile>
</profiles>
<properties>
<java.version>11</java.version>
<springfox-swagger.version>3.0.0</springfox-swagger.version>
</properties>
<dependencies>
<dependency>
<groupId>com.nimbusds</groupId>
<artifactId>nimbus-jose-jwt</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc8</artifactId>
<version>21.1.0.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
<version>3.0.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
</plugins>
</build>
</project>
and here is my application.yml file :
spring:
cloud:
gateway:
routes:
- id: incident
uri: http://localhost:8080/project
predicates:
- Path=/gateway/v1/project/**
filters:
- RewritePath=/gateway/(?<RID>.*), /$\{RID}
- JwtFilter
- RemoveRequestHeader=Authorization
- id: authentication
uri: http://localhost:8080/auth
predicates:
- Path=/gateway/auth/**
filters:
- RewritePath=/gateway/(?<RID>.*), /$\{RID}
- id: userManagement
uri: http://localhost:8080/auth
predicates:
- Path=/gateway/v1/user/**,/gateway/v1/permission/**,/gateway/v1/role/**,/gateway/v1/role-permission/**,/gateway/v1/user-role/**
filters:
- RewritePath=/gateway/(?<RID>.*), /$\{RID}
- JwtFilter
- RemoveRequestHeader=Authorization
datasource:
url: jdbc:oracle:thin:#//xxx.xxx.xxx.xxx:1521/ORCLCDB.localdomain
username: xxx
password: xxx
jpa:
hibernate:
ddl-auto: update
id:
new_generator_mappings=true
properties:
hibernate:
temp:
use_jdbc_metadata_defaults: false
dialect: org.hibernate.dialect.Oracle12cDialect
physical_naming_strategy: org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
show-sql: true
logging:
level:
org.springframework: TRACE
org.hibernate.type: TRACE
org.apache.tomcat: INFO
org.apache.catalina: INFO
pattern:
console: "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
file: "%d %p %c{1.} [%t] %m%n"
file:
name: ./tomcat-logs/gateway
path: ./tomcat-logs/gateway
Spring Cloud Gateway is itself a Spring Boot WebFlux application. It's support only embedded server. Please check spring documentation
https://docs.spring.io/spring-boot/docs/current/reference/html/howto.html#howto-traditional-deployment

Springboot and Mongo containers are not connecting

I'm running 2 containers on my local machine using docker compose. The first one is mongo container and the second is spring boot application.
This is my spring boot's Dockerfile
VOLUME /tmp
ADD target/app.jar app.jar
EXPOSE 8080
RUN bash -c "touch /app.jar"
ENTRYPOINT ["java","-Dspring.data.mongodb.uri=mongodb://mongo/test", "-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
This is my spring boot's application properties
# mongodb configuration0
dockerspring.data.mongodb.uri= mongodb://mongo:27000/test
This is my docker-compose.yml
version: '3.1'
services:
level-2-springboot:
image: level-2:latest
restart: always
container_name: level_2_microservice
ports:
- 8081:8080
working_dir: /tmp
depends_on:
- mongo
mongo:
image: mongo
container_name: mongo
ports:
- 27000:27017
restart: always
This is my pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.6.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.yash</groupId>
<artifactId>spring-sync-ms</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>spring-sync</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
<maven-jar-plugin.version>3.1.1</maven-jar-plugin.version>
</properties>
<dependencies>
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongo-java-driver</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
<finalName>app</finalName>
</build>
</project>
I execute this to create image for my spring boot app
docker build -t level-2 .
After I see my image, I execute docker-compose up and they work just fine no error
Here is the containers running
But when i do a POST or GET request to localhost:8081/api/employees/ or localhost:8080/api/employees/ I'd get this message from Postman
Could not get any response
Below is my project structure
If I run the app on eclipse and mongo manually and do POST/GET request, I'd get 201 OK, but when I do it using docker in containers, they dont talk to each other. Anyone know whats causing this?
I've consulted the internet and i think i got what you need.
The question that follows is “how does our service container talk to
the Mongo container?” For that, we get into container linking.
When you run a container, you can pass an optional –link parameter,
specifying the running container name that the new container needs to
be able to communicate with.
So with our command
docker run -P -d --name employee --link mongodb microservicedemo/employee
we fire up our new container image, exposing its ports (-P) in the background
(-d), naming it employee (–name), and telling it to link to the
container named “mongodb” (–link). Linking these does the following:
Creates a hosts file entry in the employee container pointing at the
running location of the MongoDB container Inserts a number of
environment variables in the employee container to assist with any
other programmatic access needed.
To see them run: docker exec employee bash -c 'env |grep MONGODB'
Allows containers to communicate directly over ports exposed, so there is no need to
worry about the hose machine part mapping.
see:
https://www.3pillarglobal.com/insights/building-a-microservice-architecture-with-spring-boot-and-docker-part-iii
https://www.3pillarglobal.com/insights/building-a-microservice-architecture-with-spring-boot-and-docker-part-ii
This:
# mongodb configuration0
dockerspring.data.mongodb.uri= mongodb://mongo:27000/test
Should be:
# mongodb configuration0
dockerspring.data.mongodb.uri= mongodb://mongo:27017/test
The two services in your docker-compose.yml expose Ports 8081 and 27000 to the host machine.
But inside the network created by docker-compose they expose Ports 8080 and 27017.
As your Mongo-Container runs in the docker-compose network, it has to use Port 27017 to connect to the database.

Spring Cloud Data Flow Streams Error: Unable to access jarfile

I launch Spring cloud data flow with docker-compose base on this website.
https://dataflow.spring.io/docs/installation/local/docker/
I created 3 apps, Source, Processor & Sink.
I registered them on the Spring Data Flow Cloud Dashboard.
Then I created a stream with the source connecting to processor connecting to sink.
When I deployed the apps, and opened http://localhost:9393/streams/logs/{name-of-stream},
I get the following error,
{
logs: {
testdir-stream.sender1-v3: "Error: Unable to access jarfile /testdir/data-sender-0.0.1-SNAPSHOT.jar ",
testdir-stream.sink1-v3: "Error: Unable to access jarfile /testdir/data-sink-0.0.1-SNAPSHOT.jar ",
testdir-stream.processor1-v3: "Error: Unable to access jarfile /testdir/data-processor-0.0.1-SNAPSHOT.jar "
}
}
Apparently, the Spring Cloud Data Flow (SCDF) server cannot access(or executing it or something?) the jarfile. I have chmod +x the entire testdir folder in the docker container such that the permissions is 'drwxr-xrwx'.
However, SCDF is still unable to access the jar. How can I solve this problem? Thanks.
EDIT
I am running docker-compose -f ./docker-compose.yml -f ./docker-compose-prometheus.yml up
docker-compose.yml
version: '3'
services:
mysql:
image: mysql:5.7.25
container_name: dataflow-mysql
environment:
MYSQL_DATABASE: dataflow
MYSQL_USER: root
MYSQL_ROOT_PASSWORD: rootpw
expose:
- 3306
kafka-0:
image: confluentinc/cp-kafka:5.2.1
container_name: dataflow-kafka-0
expose:
- "9092"
environment:
- KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka-0:9092
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_ADVERTISED_HOST_NAME=kafka-0
- KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
depends_on:
- zookeeper
kafka-1:
image: confluentinc/cp-kafka:5.2.1
container_name: dataflow-kafka-1
expose:
- "9093"
environment:
- KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka-1:9093
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_ADVERTISED_HOST_NAME=kafka-1
- KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
depends_on:
- zookeeper
zookeeper:
image: confluentinc/cp-zookeeper:5.2.1
container_name: dataflow-kafka-zookeeper
expose:
- "2181"
environment:
- ZOOKEEPER_CLIENT_PORT=2181
dataflow-server:
image: springcloud/spring-cloud-dataflow-server:${DATAFLOW_VERSION:?DATAFLOW_VERSION is not set!}
container_name: dataflow-server
volumes:
- 'C:\Users\Desmond\Desktop\Desmond\Project\springcloud:/root/apps'
ports:
- "9393:9393"
environment:
- spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.brokers=PLAINTEXT://kafka-0:9092,PLAINTEXT://kafka-1:9093
- spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.streams.binder.brokers=PLAINTEXT://kafka-0:9092,PLAINTEXT://kafka-1:9093
- spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.zkNodes=zookeeper:2181
- spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.streams.binder.zkNodes=zookeeper:2181
- spring.cloud.skipper.client.serverUri=http://skipper-server:7577/api
- SPRING_DATASOURCE_URL=jdbc:mysql://mysql:3306/dataflow
- SPRING_DATASOURCE_USERNAME=root
- SPRING_DATASOURCE_PASSWORD=rootpw
- SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.mariadb.jdbc.Driver
depends_on:
- kafka-0
- kafka-1
entrypoint: "./wait-for-it.sh mysql:3306 -- java -jar /maven/spring-cloud-dataflow-server.jar"
app-import:
image: springcloud/openjdk:2.0.0.RELEASE
container_name: dataflow-app-import
depends_on:
- dataflow-server
command: >
/bin/sh -c "
./wait-for-it.sh -t 180 dataflow-server:9393;
wget -qO- 'http://dataflow-server:9393/apps' --post-data='uri=${STREAM_APPS_URI:-https://dataflow.spring.io/kafka-maven-latest&force=true}';
echo 'Stream apps imported'
wget -qO- 'http://dataflow-server:9393/apps' --post-data='uri=${TASK_APPS_URI:-https://dataflow.spring.io/task-maven-latest&force=true}';
echo 'Task apps imported'"
skipper-server:
image: springcloud/spring-cloud-skipper-server:${SKIPPER_VERSION:?SKIPPER_VERSION is not set!}
container_name: skipper
ports:
- "7577:7577"
- "9000-9010:9000-9010"
- "20000-20105:20000-20105"
environment:
- SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_LOCAL_ACCOUNTS_DEFAULT_PORTRANGE_LOW=20000
- SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_LOCAL_ACCOUNTS_DEFAULT_PORTRANGE_HIGH=20100
- SPRING_DATASOURCE_URL=jdbc:mysql://mysql:3306/dataflow
- SPRING_DATASOURCE_USERNAME=root
- SPRING_DATASOURCE_PASSWORD=rootpw
- SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.mariadb.jdbc.Driver
entrypoint: "./wait-for-it.sh mysql:3306 -- java -Djava.security.egd=file:/dev/./urandom -jar /spring-cloud-skipper-server.jar"
docker-compose-prometheus.yml
version: '3'
# Extends the default docker-compose.yml with Prometheus/Grafana monitoring configuration
# Usage: docker-compose -f ./docker-compose.yml -f ./docker-compose-prometheus.yml up
services:
dataflow-server:
environment:
- |
SPRING_APPLICATION_JSON=
{
"spring.cloud.dataflow.applicationProperties":{
"task.management.metrics.export.prometheus":{
"enabled":true,
"rsocket.enabled":true,
"rsocket.host":"prometheus-rsocket-proxy",
"rsocket.port":7001
},
"stream.management.metrics.export.prometheus":{
"enabled":true,
"rsocket.enabled":true,
"rsocket.host":"prometheus-rsocket-proxy",
"rsocket.port":7001
}
},
"spring.cloud.dataflow.grafana-info.url":"http://localhost:3000"
}
prometheus-rsocket-proxy:
image: micrometermetrics/prometheus-rsocket-proxy:0.8.0
container_name: prometheus-rsocket-proxy
expose:
- '9096'
- '7001'
- '8081'
ports:
- '9096:9096'
- '7001:7001'
- '8081:8081'
environment:
- server.port=9096
grafana:
image: springcloud/spring-cloud-dataflow-grafana-prometheus:${DATAFLOW_VERSION:?DATAFLOW_VERSION is not set! Use 'export DATAFLOW_VERSION=local-server-image-tag'}
container_name: grafana
ports:
- '3000:3000'
prometheus:
image: springcloud/spring-cloud-dataflow-prometheus-local:${DATAFLOW_VERSION:?DATAFLOW_VERSION is not set! Use 'export DATAFLOW_VERSION=local-server-image-tag'}
container_name: prometheus
ports:
- '9090:9090'
Pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.7.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>data-sender</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>data-sender</name>
<description>Test Proj</description>
<properties>
<java.version>1.8</java.version>
<spring-cloud.version>Greenwich.SR2</spring-cloud.version>
<mockito-junit-jupiter.version>2.19.0</mockito-junit-jupiter.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-test-support</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.4</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>${junit-jupiter.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<version>${mockito-junit-jupiter.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/com.googlecode.json-simple/json-simple -->
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>net.lingala.zip4j</groupId>
<artifactId>zip4j</artifactId>
<version>1.3.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/commons-io/commons-io -->
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.6</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.9</version>
</dependency>
<!-- prometheus-rsocket monitoring -->
<dependency>
<groupId>org.springframework.cloud.stream.app</groupId>
<artifactId>app-starters-micrometer-common</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer.prometheus</groupId>
<artifactId>prometheus-rsocket-spring</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-cloudfoundry-connector</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
For accessing custom applications from your local machine, you need to configure both the dataflow-server and the skipper-skipper Docker containers to access to your local file system. You have already mounted the source folder to dataflow-server but you need to do the same with the skipper-server definition like this:
skipper-server:
image: springcloud/spring-cloud-skipper-server:${SKIPPER_VERSION:?SKIPPER_VERSION is not set!}
container_name: skipper
volumes:
- 'C:\Users\Desmond\Desktop\Desmond\Project\springcloud:/root/apps'
...
We will clarify this in our documentation #128
Two additional points regarding prometheus-rsocket monitoring:
Since you are using the new prometheus-rsocket configuration you will need to import the Einstein-BUILD-SNAPSHOT-stream-applications-kafka-maven app starters instead of kafka-maven-latest. You can do this by setting the following property before starting docker-compose:
export STREAM_APPS_URI=https://dataflow.spring.io/Einstein-BUILD-SNAPSHOT-stream-applications-kafka-maven
The SR4 app starter release is expected very soon, but until then you need to use the BUILD-SNAPSHOT to leverage the prometheus metrics.
Can you please share how you have implemented your custom apps? Do they contain the following dependencies?
<dependencyManagement>
<dependencies>
...
<dependency>
<groupId>org.springframework.cloud.stream.app</groupId>
<artifactId>app-starters-core-dependencies</artifactId>
<version>2.1.2.BUILD-SNAPSHOT</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
....
<dependency>
<groupId>org.springframework.cloud.stream.app</groupId>
<artifactId>app-starters-micrometer-common</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer.prometheus</groupId>
<artifactId>prometheus-rsocket-spring</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-cloudfoundry-connector</artifactId>
</dependency>
</dependencies>
<repositories>
<repository>
<snapshots>
<enabled>true</enabled>
</snapshots>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
</repository>
...
</repositories>
If your apps's parent POM is org.springframework.cloud.stream.app:app-starters-build:2.1.2.BUILD-SNAPSHOT above dependencies are already injected. Otherwise you have to add them manually.
UPDATE -- adding a simple pom configuration to enable prometheus-rsocket metrics:
<properties>
<spring-cloud.version>Greenwich.SR2</spring-cloud.version>-->
<prometheus-rsocket-spring.version>0.8.0</prometheus-rsocket-spring.version>
</properties>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer.prometheus</groupId>
<artifactId>prometheus-rsocket-spring</artifactId>
<version>${prometheus-rsocket-spring.version}</version>
</dependency>
<dependency>
<groupId>io.micrometer.prometheus</groupId>
<artifactId>prometheus-rsocket-client</artifactId>
<version>${prometheus-rsocket-spring.version}</version>
</dependency>
I believe container configs are wrong for skipper server as that's the one running those containers if local setup is used. It should work if same volume is used in skipper as it's now done with dataflow server.
For getting those logs, dataflow simply requests those from a skipper and error originates from there.

Access log not created

I'm trying to enable SpringBoot's embedded Tomcat access logs. When the application starts I'm not seeing the log file at all. When I make requests to the application from browser I still get nothing. What am I missing?
This should be really straight forward. I'm wondering if there's stuff in my project that's interfering with Tomcat logging to the access_log.log file. Within the project I've done the following:
enable ssl and changed port
Via #Configuration, I created a #Bean EmbeddedServletContainerCustomizer that adds a TomcatConnectorCustomizer.
Added Spring Security to request authentication for /secure/** URL patterns.
From what I've read none of that should require anything special regarding Tomcat access logs. With my set up I expect the logging to be in my-tomcat/access_log.log at the same location I ran the java -jar command. Correct me if that's wrong.
Using...
SpringBoot 1.5.9.RELEASE
Win 7 Enterprise
application.yml
server:
port: 8443
ssl:
# 6.2 Ensure SSLEnabled is set to True for Sensitive Connectors (Not Scored)
enabled: true
key-store: classpath:keystore.p12
key-store-password: password
key-store-type: PKCS12
key-alias: tomcat
# 6.5 Ensure SSL Protocol is set to TLS for Secure Connectors (Scored)
protocol: TLS
tomcat:
basedir: my-tomcat
accesslog:
enabled: true
pattern: '%t %a "%r" %s (%D ms)'
security:
require-ssl: true
# 7.1 Application specific logging (Scored)
logging:
level.com.esd.springbootdemo: DEBUG
# 7.2 Specify file handler in logging.properties files (Scored)
file: logs/springbootdemo.log
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.esd.springboot</groupId>
<artifactId>springboot-demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>springboot-demo</name>
<description>Spring Boot security and hardening POC</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.9.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
Your tomcat YAML block is currently a child of the ssl YAML block - so you need to fix the indentation

Resources