I need to query for collected metrics in my application and the only way I found to do it is via Prometheus rest api as described here: https://prometheus.io/docs/prometheus/latest/querying/api/ Does any Spring or java api for fetching metrics from Prometheus exist, so that I wouldn't have to implement it from scratch?
If you want to collect metrics from your spring application using Prometheus, this is what you have to do:
In your application's pom.xml add this dependency:
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<version>1.7.0</version>
</dependency>
In application.properties you need to add:
management.endpoints.web.exposure.include=*
management.endpoint.health.show-details=always
And this should be all, now when you start your Spring application you can check the collected metrics at /actuator/prometheus, you'll have all the metrics grouped here.
Good luck!
Related
I am using Spring Boot 2.5.4 and Actuator with Micrometer and Prometheus support.
When I open the /actuator/prometheus endpoint, I see metrics like these:
# HELP jvm_threads_live_threads The current number of live threads including both daemon and non-daemon threads
# TYPE jvm_threads_live_threads gauge
jvm_threads_live_threads 28.0
Note that there is no timestamp attached to that metric. However, according to OpenMetrics / Prometheus spec, it is possible to add a timestamp to the metrics output.
My question: How can you tell Spring Boot Actuator to generate and add a timestamp to the metrics it creates? I have not found any documentation on it. Thanks!
References
My pom.xml dependencies look like this:
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-core</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
My application.yaml like this:
# Application name. Shows up in metrics etc.
spring:
application:
name: "some_app_name"
# Expose application on port 8080
server:
port: ${SERVER_PORT:8080}
# Expose all Actuator endpoints (don't do this in production!)
management:
endpoints:
web:
exposure:
include:
- "*"
Micrometer doesn't support this at the moment. The team's recommendation is to use the Prometheus Java Client directly for metrics where the timestamp is important to you.
I am using springboot gateway 2.0.4.RELEASE.
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
After adding this dependency, it is exposing all metrics to prometheus endpoint.
I want to disable some of the metrics from exposing through /actuator/prometheus endpoint.
Let's say I want to disable
gateway_requests_seconds_count
gateway_requests_seconds_sum
gateway_requests_seconds_max
Does anyone know how can I achieve this ?
Thanks
Alpesh
Is it possible to expose all the Spring metrics in an rest endpoint?
In our environment Micrometer cannot forward directly the metrics to our monitoring system, then my idea was to have all the metrics exposed in a rest endpoint (or on a file, json format but the endpoint is preferable) and then having a script to fetch all the metrics at once, manipulate the payload and forward the metrics to our customized monitoring api.
If micrometer is not the right choice, what is it?
EDIT
I tried to use the actuator/prometheus endpoint. I can see the endpoint but as soon as I hit the endpoint, it hang and does not return anything:
I am using spring boot 2.2.2 and I added this dependency
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<version>1.3.1</version>
</dependency>
application.yml
management:
endpoints:
web:
exposure:
include: health, metrics, prometheus
You could use Spring Boot's built-in prometheus endpoint. It exposes all of the Micrometer-based metrics. It's primarily intended for periodic fetching by a Prometheus server, but you could do similar in a script of your own that transforms the data to the necessary format and forwards it to your custom monitoring API.
At the level of idea you did everything right, and Micrometer is indeed the way to go here.
At the level of pom.xml definitions, you don't need to set the version:
If the version or micrometer/micrometer-prometheus module that your spring boot comes with clashes with your definitions it might not work properly.
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
Other than that, if you work only with core-micrometer (without even prometheus integration) you have metrics endpoint that also shows all the metrics in the "common" format. Prometheus shows the metrics in a format that prometheus can scrape. For your use case it seems like both ways can work but you'll have to be familiar with the formats in either way you chose.
Another thought: you say that the application can't forward to monitoring system, is it because some security constraints or because you use some kind of custom solution or something that micrometer can't integrate with?
If its not a security thing, you can consider to write your own micrometer registry that will act as a bridge between micrometer and the monitoring system.
One more thing I would like to answer on: you say that the endpoint doesn't finish.
I can only speculate, because basically there can be many reasons (one of which is clash of versions that I've referred above) - but all are bugs, it should not work like that.
Another reason reason is that you might have defined a Gauge in micrometer that in order to be calculated performs some "expensive" calculations: goes to the database that for some reason takes ages to access, calls some expensive http endpoint, etc.
When the rest endpoint gets invoked, it takes "snapshots" (the current values) of all the metering primitives, for gauges it actually invokes your custom code that can do anything, so I would have checked that.
Yet another reason might be connectivity / security configurations (the request doesn't reach actuator at all) but given the information you've presented I can't tell more than that...
I have got custom starter application (library) which has actuator and prometheus dependency
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<version>1.5.2</version>
</dependency>
in application.properties
management.endpoints.web.exposure.include=health, info, metrics, prometheus
I am using this starter in my other applications and I want to pass also this exposition of the endpoints from the starter.
Actuator and Prometheus dependecies works in applications but not show selected endpoints in starter. Ofc I can add also line with management.endpoints.web.exposure.include=health, info, metrics, prometheus to my apps but with several apps using this starter I want to pass this once for all and change endpoints only in starter if needed.
Do you have an idea how to expose those endpoints in my app which are set in starter?
Spring Boot v2.3.2 /
Maven 3.6.3
You can pass this as command line args or as ENV vars to your jar, when you start your application.
This way you can pass it to required application, as and when needed w/o updating application.properties.
It will also safeguard against exposing the actuator endpoints when not required, as actuator endpoints reveal sensitive info about the application.
eg. java -Dmanagement.endpoints.web.exposure.include=health, info, metrics, prometheus -jar myapp.jar
I have a spring cloud config server configured over spring cloud bus over kafka.
I am using Edgware.RELEASE with boot 1.5.9.
When I send a POST request to the endpoint /bus/refresh with destination=clientId:dev:** in the body via POSTMAN to config server, all the clients get their beans refreshed even though their clientId doesn't match the value in the destination field.
Here are additional configuration details:
spring cloud config server
pom.xml
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-monitor</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bus-kafka</artifactId>
</dependency>
application.properties
server.port=8888
management.security.enabled=false
spring.cloud.bus.enabled=true
spring.kafka.bootstrap-servers=localhost:9092
I have two config clients with id - config-client1 and config-client2
After changing an application property in the repository for config-client2, I submit a POST request to /bus/refresh endpoint on the config server, with destination=config-client2:dev:** in the body of the request. I was hoping this would refresh/re-initialize the beans only in config-client2 application. But I noticed beans in config-client application are also refreshed/re-initialized. I also noticed config-client application receiving refresh event along with config-client2.
I was hoping only config-client2 application receives the event and its beans are the only ones that are refreshed as a result of it.
Wondering if I am missing any configuration setting to enable granular refresh at specific client level.
I did go through the document posted at - http://cloud.spring.io/spring-cloud-static/Edgware.RELEASE/single/spring-cloud.html, but the results are not as explained.
Any help with a working example is appreciated.
You can use:
curl -X POST http://localhost:8001/actuator/bus-refresh/config-client2
You also need this in your application.properties or .yml.
spring.cloud.config.uri=http://localhost:8001
spring.cloud.config.uri should point to your Spring Config Server