Prometheus endpoint not exposed when running spring boot application inside docker container - spring-boot

I have a simple spring boot application with a single endpoint which returns hello.
Dependencies in pom.xml as follows
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
with application properties as below
spring:
application:
name: app1
server:
port: 9091
management:
endpoints:
web:
base-path: /actuator
exposure:
include: health,prometheus,info,metrics
endpoint:
health:
show-details: always
metrics:
enabled: true
prometheus:
enabled: true
logging:
level:
org.springframework.web.filter.CommonsRequestLoggingFilter: DEBUG
when run locally, the actuator endpoint exposed the prometheus metrics correctly
However when i run the same application inside a docker container, the prometheus metrics are not exposed
Following is my docker file
FROM eclipse-temurin:17-jdk-jammy as builder
WORKDIR /opt/app
COPY .mvn/ .mvn
COPY mvnw pom.xml ./
RUN ./mvnw dependency:go-offline
COPY ./src ./src
RUN ./mvnw clean install
FROM eclipse-temurin:17-jre-jammy
WORKDIR /opt/app
EXPOSE 9091
COPY --from=builder /opt/app/target/*.jar /opt/app/*.jar
ENTRYPOINT ["java", "-jar", "/opt/app/*.jar" ]
run command docker run -p 9091:9091 app1:latest produces
why are the prometheus sub-endpoints not exposed on the actuator endpoint when running inside docker? Can someple please help me with what am I missing

it turns out the actuator profile needs to be set explicitly in spring boot for dockr to expose them

Related

Azure AppConfiguration with Spring Boot 2.5.x

Question
How to integrate an Azure AppConfiguration with SpringBoot 2.5.x or higher?
Info
Im trying to use an Azure AppConfiguration resource with a Spring Boot 2.5.4 project. Unfortunately I cant get it to read a setting from the AppConfiguration or even connect to it as far as I can tell.
The project is newly created with the Spring Initializr where I only added
Spring Boot Starter Web
Spring Boot Starter Security
Spring Boot Starter WebSocket
Afterwards I tried following the Microsoft Quickstart documentation with no success. The documentation mentions that its using Spring 2.4.x so I assume some changes broke it.
I also tried to identify the issue by looking through some Azure Spring Boot Code Samples.
All the examples so far use the bootstrap.properties file which I learned during my search so far is deprecated. Moving the settings to the application.yml or enabling use-legacy-processing: true did not work either. Any ideas?
pom.xml
...
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.4</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
...
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-websocket</artifactId>
</dependency>
<dependency>
<groupId>com.azure.spring</groupId>
<artifactId>azure-spring-cloud-appconfiguration-config</artifactId>
<version>2.0.0</version>
</dependency>
...
application.yml
spring:
config:
use-legacy-processing: true
profiles:
active: "develop"
application:
name: "MySampleService"
cloud:
azure:
appconfiguration:
stores:
- connection-string: "SomeAzureAppConfigurationResourceConnectionString"
label: ${spring.profiles.active}
#mysampleservice:
# message: "this is a message from file"
AppConfiguration Resource
Im not entirely sure about the format for the setting name. I tried to build the format based on this documentation.
The configuration classes should be fine since commenting in the mysampleservice causes the value of message beeing used.
Any hints are appreciated!
Some more info to elaborate on the accepted answer
The documentation linked in the answer refers to two different packages. The one linked right at the start in the maven repository is spring-cloud-azure-appconfiguration-config while the one used further down is azure-spring-cloud-appconfiguration-config. The second one works with the bootstrap.properties file.
Working pom.xml and bootstrap.properties:
...
<dependencies>
<!-- Dependency to load configuration from azure app configuration resource. Note that additional settings are required in bootstrap.properties
Documentation of settings: https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/appconfiguration/azure-spring-cloud-starter-appconfiguration-config
-->
<dependency>
<groupId>com.azure.spring</groupId>
<artifactId>azure-spring-cloud-appconfiguration-config-web</artifactId>
<version>2.1.0</version>
</dependency>
...
# Use this to enable or disable the cloud config, disabling it results in application.yaml beeing used.
spring.cloud.azure.appconfiguration.enabled=true
# Connection string to azure app configuration resource
spring.cloud.azure.appconfiguration.stores[0].connection-string= Endpoint=https://myofficeconfiguration.azconfig.io;Id=zUcT-l9-s0:PFYfW7WM0/Pz7WZOnH3v;Secret=JTB9myJqGekDAJ5m8Z1vjmkJZrPd88JbOEE3EqoqJYs=
# Configured filters for settings in the previous defined app configuration resource
spring.cloud.azure.appconfiguration.stores[0].selects[0].key-filter = /mysampleservice/
spring.cloud.azure.appconfiguration.stores[0].selects[0].label-filter = Sample
spring.cloud.azure.appconfiguration.stores[0].selects[1].key-filter = /notificationservice/
spring.cloud.azure.appconfiguration.stores[0].selects[1].label-filter = Sample2
bootstrap.yml/bootstrap.properties can still be used, they are no longer part of the base Spring packages.
Also, you want to use this doc for 2.0.0 and newer https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/appconfiguration/azure-spring-cloud-starter-appconfiguration-config.

spring eureka - Error resolving template [eureka/status], template might not exist or might not be accessible

I am following this tutorial to implement a microservice architecture within my project.
First I added the following dependencies to my project:
</dependencies>
...
<dependency>
<!-- Spring Cloud starter -->
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter</artifactId>
</dependency>
<dependency>
<!-- Eureka service registration - CHANGED -->
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Finchley.RELEASE</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
After that I added a registrationServer class to the project (as described in the tutorial) and set up my config.
The config of my registration server remain very basic:
# Ignore JDBC Dependency
# This demo puts 3 applicatons in the same project so they all pick up the
# JDBC Depdendency, but this application doesn't need it.
spring.autoconfigure.exclude: org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration
# Configure this Discovery Server
eureka:
instance:
hostname: localhost
client: # Not a client, don't register with yourself
registerWithEureka: false
fetchRegistry: false
server:
port: 8761 # HTTP (Tomcat) port
Now as I understood it, at this point I should be able to access http://localhost:8761 and see my setup as tracked by the registration server.
Instead I get a Whitelabel Error Page containing the following error message:
Error resolving template [eureka/status], template might not exist or might not be accessible by any of the configured Template Resolvers
Note: Before adding Eureka, my project consisted of a REST application which I want now to transform into a microservice. The REST service contained a frontend which is organized in the project directories like this:
src
- main
- resources
- templates
- index.html
- static
- built
- bundle.js
Note 2: Also I tried to disable the thymeleaf template which led to an 404 error when trying to access http://localhost:8761.
# Discovery Server Dashboard uses FreeMarker. Don't want Thymeleaf templates
spring:
thymeleaf:
enabled: false # Disable Thymeleaf
datasource:
type: org.springframework.jdbc.datasource.SimpleDriverDataSource
As stated in this github issue:
If your project already uses Thymeleaf as its template engine, the Freemarker templates of the Eureka server may not be loaded correctly. In this case it is necessary to configure the template loader manually:
application.yml
spring:
freemarker:
template-loader-path: classpath:/templates/
prefer-file-system-access: false

How to collect turbine stream from a Standalone Turbine Stream application

Similar with this post, I created another turbine stream based application to collect hystrix stream from other applications.
Spring Boot: 2.0.4.RELEASE, Spring Cloud: Finchley.SR1
The application class:
#SpringBootApplication
#EnableDiscoveryClient
#EnableTurbineStream
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
}
The maven dependencies:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-turbine-stream</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-stream-rabbit</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
The application config file is:
server:
port: 8989
spring:
application:
name: hystrix-turbine-stream
management:
endpoints:
web.exposure.include: '*'
When I explore the source codes of #EanbleTurbineStream, and found TurbineController expose a Flux to the root '/' endpiont.
But when I tried to explore http://localhost:8989 in the Hystrix Dashboard, found it did not work as expected.
Update: When I tried to access the turbine stream application, and got:
curl http://localhost:8989
data:{"type":"ping"}
data:{"type":"ping"}
data:{"type":"ping"}
data:{"type":"ping"}
In the application console(logging), there is some log shown as:
: Registering MessageChannel turbineStreamInput
But I can not see the there are some message sent to this channel in my client app.
Here is the sample codes of my turbine stream application
Update2: Got it worked, I used a outdated spring-cloud-starter-netflix-hystrix-stream (which is existed in v2.0.0M2, but not existed in the final RELEASE version) in my client app, when I used spring-cloud-starter-netflix-hystrix and spring-cloud-netflix-hystrix-stream combination in the client app, it worked well.
#Hantsy We would need more details regarding what your failures are here. I have a running Spring Boot: 2.0.4.RELEASE, Spring Cloud: Finchley.SR1 turbine stream app so I can help if you need further clarification.
For #EnableTurbineStream to work properly,
you'll need to add the below dependencies in your app according to documentation here
https://cloud.spring.io/spring-cloud-netflix/single/spring-cloud-netflix.html#_turbine_stream
spring-cloud-starter-netflix-turbine-stream
spring-boot-starter-webflux
spring-cloud-stream-binder-rabbit
(any spring-cloud-stream-* would do, raabitmq worked for me)
On the client, add a dependency to spring-cloud-netflix-hystrix-stream and the spring-cloud-starter-stream-* of your choice.
Add the rabbitmq (in my case) configuration on your application.properties/application.yml file of your client and the turbine-stream app:
spring:
rabbitmq:
host: localhost
port: 5672
username: guest
password: guest
Hope this helps.
You should set properties on client app project: spring.cloud.stream.bindings.hystrixStreamOutput.destination=springCloudHystrixStream

Prometheus UI, Grafana settings for spring boot application

I am using spring boot 2.0 and added the below dependencies in POM
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
My application.yml looks like this
management.endpoints.web.base-path = /manage
management.endpoints.web.exposure.include = "*"
endpoints.prometheus.enabled = true
when I access Prometheus at
localhost/manage/prometheus
I am able to see all the metrics.
Next, my target is to see the above metrics in Prometheus UI. For this, I added the below dependencies in my POM
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_spring_boot</artifactId>
<version>${prometheus.version}</version>
</dependency>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_hotspot</artifactId>
<version>${prometheus.version}</version>
</dependency>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_servlet</artifactId>
<version>${prometheus.version}</version>
</dependency>
What are the next steps for me to see metrics in Prometheus UI, final target is to integrate Prometheus to Grafana.
PS: I did a search on google and tried adding prometheus.yml and adding annotations like #EnablePrometheusEndpoint nothing worked as all the articles are old.
Edit : Also how to configure prometheus.yml ( metrics_path, targets)if the spring boot jar is hosted in different host (Azure/AWS) and prometheus server is in different host.
If you are using Spring Boot 2 and micrometer you don't need to add the extra dependencies, they are imported when you added micrometer-registry-prometheus. if you are able to see the metrics on localhost/manage/prometheus your configuration on the spring-boot side is fine. there is no need to configure anything more.
To see the metrics in Prometheus you need:
install Prometheus Server https://prometheus.io/docs/prometheus/latest/installation/
configure Prometheus to scrape (pull) the metrics from your server. for that you will need to modify the prometheus.yml file by adding a new job (don't forget to restart Prometheus after changing the yml file)
scrape_configs:
- job_name: 'mySpringBoot'
metrics_path: '/manage/prometheus'
static_configs:
- targets: ['springBootHost:springBootPort']
once this is configured, go to the Prometheus UI, check that the target is UP - http://localhost:9090/targets (assuming Prometheus runs on localhost)
if you don't see your target or you see it as DOWN there is a configuration or network problem.
the following steps are straight forward, with lot of documentation elsewhere:
next step is to install Grafana
now configure Prometheus as a data-source in Grafana
start plotting your metrics.
Spring boot api application configure with prometheus and grafana on windows.
Create spring boot application and add dependency in pom.xml -
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-core</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
if its running go to your browser and type ‘http://localhost:8080/actuator/prometheus’ and you will get all metrics.
Download and install Prometheus Server https://prometheus.io/download/
and extract the zip and run prometheus exe.
for that you will need to modify the prometheus.yml file by adding a new job (don't forget to restart Prometheus after changing the yml file)
Note : configure spring application with prometheus.
scrape_configs:
- job_name: 'SpringBootApplicationName
metrics_path: ‘actuator-prometheus'
static_configs:
- targets: [‘IPADDRESS:springBootApplicationPort]
once this is configured, go to the Prometheus UI, check that the target is UP - http://localhost:9090/targets (assuming Prometheus runs on localhost)
go to ‘http://localhost:8080/actuator/prometheus’ and select one metrics and paste in - http://localhost:9090/graph and click on execute. example you can select one CPU and HTTP metric like - ‘http_server_request_seconds_max’
the following steps are straight forward, with lot of documentation elsewhere:
next step is to install Grafana from - https://grafana.com/grafana/download?platform=windows
and extract the zip and run the grafana-server.exe and check on your browser ‘http://localhost:3000’
now login as a default username ‘’admin and password ‘admin’ and go to setting select the deshboard and create a datasource and select prometheus and type url of prometheus like - ‘’http://ipaddress:pord of prometheus.
and go to graph and click edit and select datasource and type your metric query like ‘http_server_request_seconds_max’
9.start plotting your metrics.

Cannot include Prometheus metrics in spring boot 2 (version 2.0.0.M7)

Cannot include Prometheus metrics in spring boot 2 (version 2.0.0.M7) project.
According micrometer docs added spring-boot-starter-actuator dependency and in application.yaml added management.endpoints.web.expose: prometheus but when calling /actuator/prometheus get
{
"timestamp": 1518159066052,
"path": "/actuator/prometheus",
"message": "Response status 404 with reason \"No matching handler\"",
"status": 404,
"error": "Not Found"
}
Tell me please why I wasn't getting prometheus metrics?
I had trouble initiating micrometer with Springboot 2. x.
These changes in my project helped me to expose the metrics atactuator/prometheus endpoint
These are the changes in my application.properties file
management.endpoints.web.exposure.include=*
management.endpoint.metrics.enabled=true
My build.gradle file included
compile('org.springframework.boot:spring-boot-starter-web')
compile('org.springframework.boot:spring-boot-starter-actuator')
compile('io.micrometer:micrometer-registry-prometheus')
Edit: Since I gave this answer a lot has changed. It was valid for 2.0.0.RC1. Please read the documentation https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-metrics.html
If above solution doesn't work for someone try this:
I had the same issue with Spring Boot 2.0.0.RC1, spring-boot-starter-web and of course spring-boot-starter-actuator.
My application.properties file read:
management.endpoints.web.expose=prometheus,metrics,info,health
In my pom file I had additionally:
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient</artifactId>
<version>0.2.0</version>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<version>0.12.0.RELEASE</version>
</dependency>
Prometheus metrics under /actuator/prometheus where only shown after I had switched to the newest version of micrometer-registry-prometheus:
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<version>1.0.0-rc.9</version>
</dependency>
Did you add micrometer-registry-prometheus to your dependecies?
Micrometer has a pluggable architecture where you need to define (by plugging dependencies) what monitoring system you'd like to work with. (You can even add multiple, not just one.)
Btw, you should be switching to Spring Boot 2.0.0.RC1. That's the current one as of this writing.
Recap:
pom.xml
<parent>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.5</version>
</parent>
... ... ...
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
application.properties.yml (aka. application.yml)
management.endpoints.web.exposure.include=health,info,prometheus
management.endpoint.metrics.enabled=true
management.endpoint.prometheus.enabled=true
management.endpoint.health.show-details=always
management.endpoint.health.show-components=always
management.endpoint.health.probes.enabled=true
management.metrics.tags.application=TheApplicationName
After applying all the above, health & info endpoints were displayed in the http://localhost:8080/actuator -list, but not the prometheus endpoint.
Of course, rerun the Maven lifecycle - still failing prometheus to be displayed ...
Finally my issue disappeared after invalidate caches and restart of my IntelliJ IDE

Resources