My spring boot application pushes spring_data_repository_invocations_percentile metrics automatically. The application has grown very big over time and has lots of repository classes and this is causing issues with the Prometheus DB. Is there a way I can disable specific metrics like spring_data_repository_invocations_percentile or http_client_requests_percentile itself?
Sample metrics being pushed:
ad_manager_api_spring_data_repository_invocations_percentile{BusinessUnit="ads", Project="ads", TechTeam="ads", application="ad-manager-api", businessunit="ads", environment="staging", exception="None", exported_application="ad-manager-api", exported_environment="staging", instance="103.18.2.19:9273", job="ads-nodes-mumbai", method="findAllAppEvents", metric_type="gauge", phi="0.95", repository="MoEventRepository", state="SUCCESS", statistic="value", techteam="ads"}
Related
I have configured the following in my application but i cant seem to get some metrics from http.servlet.request published to datadog.
Spring 2.7.1
implementation("com.datadoghq:dd-trace-ot:0.98.1")
implementation("com.datadoghq:dd-trace-api:0.98.1")
implementation("io.micrometer:micrometer-core:1.9.2")
implementation("io.micrometer:micrometer-registry-statsd:1.9.2")
My props is as follows
management.endpoint.health.probes.enabled=true
management.metrics.export.datadog.step=1m
management.metrics.distribution.percentiles-histogram.http.server.requests=true
management.metrics.export.datadog.descriptions=true
management.metrics.web.server.request.autotime.enabled=true
management.endpoint.metrics.enabled=true
management.metrics.enable.logback=true
management.metrics.tags.application=my-app
management.metrics.export.statsd.enabled=true
management.metrics.export.statsd.host=${DD_HOST}
management.metrics.export.statsd.flavor=datadog
management.metrics.export.statsd.port=8125
So i can see http.servlet.request, but i cant seem to get http.servlet.request.max or http.servlet.request.count.. i really wanted latency is that even possible?
im using tomcat with spring as per standard, Anyone got any ideas why this would be the case why im not seeing
I have a spring batch jobs in which I want to push metrics to Prometheus as suggested here
I have a requirement to push only default metrics given by Spring batch via micrometer.There is a easy way in which I dont have to anything and just add following dependency:
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_pushgateway</artifactId>
</dependency>
as mentioned here
But the second way is setting up PushGateway server and then writing Configuration Class
Is there any difference in these two approaches?
In second approach how to write it when all we need is default metrics?Pushgatewayurl is fine ,I understood, but if I have dozen of jobs, what to set in jobname and grouping key.?
Is there any difference in these two approaches?
No, there is no difference. Please note that not all Spring Batch users are Spring Boot users. That's why the sample in Spring Batch's repository shows how to configure a task that pushes metrics to the gateway. Now if you use Spring Boot, you don't have to write that class since an equivalent is configured by Spring Boot automatically.
In second approach how to write it when all we need is default metrics?Pushgatewayurl is fine ,I understood, but if I have dozen of jobs, what to set in jobname and grouping key.?
Spring Batch metrics are tagged by job name, step name, etc, see the Tags column in the Built-in Metrics table. So even if you have multiple jobs, the metrics will be distinct. You can do the filtering on Prometheus side with the corresponding tag.
I have a springboot 2 application, and I want to display metrics in AWS Cloudwatch.
I have included micrometer cloudwatch dependency in pom.
Here setting is documented for various metric systems but not cloudwatch.
Whar else configurations I need to do for cloudwatch?
First of all you may have to add some additional dependecies. I needed the following:
org.springframework.boot - spring-boot-starter-actuator
org.springframework.cloud - spring-cloud-starter-aws
io.micrometer - micrometer-core
io.micrometer - micrometer-registry-cloudwatch
Boot was not able to manage the versions for these dependencies except for actuator in my case, so you might have to figure out the right versions for you.
Firthermore some application properties have to be set:
# disable unwanted features to prevent autoconfigure which will produce errors and abort of application startup eventually
# alternatively you can try to configure those features correctly if you intend to use them
cloud.aws.stack.auto=false
# enable micrometer for cloudwatch (only where there is actually access to it)
management.metrics.export.cloudwatch.enabled=true
# set the namespace that will contain the metrics for this application
management.metrics.export.cloudwatch.namespace=test
# set max batch size to the actual maximum (if not a bug in certain versions of micrometer for cloudwatch will send
# batches that are too big)
management.metrics.export.cloudwatch.batchSize=20
The next step will be in AWS. The role associated with your EC2-instance (or whatever you are using) needs to have the permission CloudWatch:PutMetricData.
Using this configuration should enable CloudWatch-Monitoring for your Spring-Boot-Application.
One of the sources I encountered stated that you should use:
cloud.aws.credentials.instanceProfile=false
This will prevent Spring Boot from automatically obtaining credentials that are necessary to push metrics to CloudWatch. You could also provide own credentials another way, but I didn't try that.
After some researching, I have discovered that there is a spring actuator project which is used to monitor and log information about the application. Here is the link that suggests how to put log in to my own custom repository for post processing. [http://www.sedooe.com/2017/08/tracing-requests-and-responses-with-spring-boot-actuator/ ]
However, the total in/out byte counts are not included. I know that I can try to wrap HttpServeletRequest and HttpServeletResponse (in the filter, pass them to the filter chain so that it can count in/out throughput whenever they are consumed or written out) which will return custom SerlvetInputStream and ServletOutputStream that can count the number of bytes.
This seems to be a common problem in business application and there should already be a solution so I wonder if there is any easier way to achieve this.
From my understanding the Spring-Boot 1.5 actuator implementation doesn't provide such a functionality. (Assuming you are talking about Spring-Boot 1.5.)
What you could do is having a look at the Micrometer project. It's the new actuator metrics "backend" for Spring-Boot 2.0. In case you are still using Spring-Boot 1.5 you can use the micrometer-spring-legacy library in your app.
Key thing to understand here is that for Spring-Boot 1.5 this is a metric backend which is living next to the "old" actuator.
What the project provides is instrumentation for embedded Jetty and Tomcat server. Allthough only the latter does provide in and out traffic metrics.
Assuming Tomcat, you'll get tomcat_global_received_bytes_total and tomcat_global_sent_bytes_total metrics over which you can reason about in your favorite monitoring system.
I want to deploy an application with a web interface. I want to use Spring YARN for this because that eases all the basic setup, and I can start the application with java -jar.
What steps do I have to do to:
have my application expose a web interface
have the tracking URI I get when submitting it proxy to that web interface
Unfortunately, I cannot find anything about this on the net, there is npthing on that particular issue in the Spring documentation and Google searches do not get me the correct results either.
Easiest way to do this is simply use Spring YARN Boot application model and framework is then trying to do the heavy lifting on your behalf. I actually showed a demo of this during my session at SpringOne 2GX 2014. You can find my session recording from youtube https://www.youtube.com/watch?v=qlvX7_r9aUA.
Interesting stuff for this particular feature is at the end (starting from 1:16:22) and you can see how web server address is registered into YARN resource manager and how I query it using a Spring YARN Boot CLI (around 1:32:13). Spring YARN will actually see that there is an embedded servlet context and registers it automatically. In this demo property "server.port=0" makes tomcat to choose random port which is then registered.
Code for this particular UI demo can be found from github https://github.com/SpringOne2GX-2014/JanneValkealahti-SpringYarn/tree/master/gs-yarn-rabbit. Demo was around RabbitMQ just to have some real UI functionality and not just a dummy hello world page.
There's also more up-to-date sample in https://github.com/spring-projects/spring-hadoop-samples/tree/master/boot/yarn-store-groups which doesn't have a real UI(just Boot management endpoints). Thought it's relatively easy to add Spring MVC magic there just by following normal Boot functionality(i.e. following https://spring.io/guides/gs/rest-service).
Lemmy know if this helps!