I have developed a couple of microservices using SpringBoot, has exposed some custom metrics using Micrometer which are available at /actuator/prometheus endpoint. I tested the application locally with docker prometheus container with scrape config pointing to the endpoint and its showing the metrics.
Now I have deployed it into kubernetes in GKE (version 1.14.7-gke.23) with Istio (version 1.4.0) enabled using the demo profile. But the prometheus that comes with istio is not scraping the SpringBoot /actuator/prometheus endpoint. I took queues from this link "https://preliminary.istio.io/faq/metrics-and-logs/#prometheus-application-metrics" and added the annotations to the Deployments but to no avail.
Help requested on how to get this working... Thanks
Related
Spring boot apps with Sleuth are deployed in pods with istio sidecar injected alongside.
istio virtual service ingresses traffic into the mesh and proxies the request to pods correctly.
Both logs are seen in kibana - one for istio-proxy (sidecar) and another for the spring app using logback + sleuth.
However I would like to see same traceId (x-request-id) header for both requests. Is this a good practice or is rather what is the right way to join/tie these request logs ?
I did one test and found out that istio-proxy is sending 'x-request-id' which corresponds to the value of traceId. However spring creates a new traceId and wondering if there is a way to make them same ?
I am going to create a kubernetes cluster to deploy all my spring boot microservices. Currently I have a spring cloud gateway server, a eureka discovery server and my back-end and front-end applications.
Do we need gateway server and discovery server if deployed on kubernetes, as these services are provided as part of cluster?
Thanks.
You are not required to use the Spring Cloud Gateway or Eureka discovery server. Kubernetes provides all basic building blocks (like Service) implemented without additional software requirements (for example with plain DNS).
If you need a more advanced setup you can integrate kubernetes API in Spring applications or use other ways to integrate. For the 90% case there is no need to do so.
I am trying to implement prometheus monitoring for spring boot apps but my spring boot apps are registered with spring cloud eureka server so I don't have an option to configure static ports for the apps in prometheus config, I can only configure scraping for eureka server like below but not for the apps registered with eureka. Can anyone please suggest?
static_configs:
- targets: ["http://mycompany.eurekaserver.net:8761"]
Came here to add new answer that, Eureka SD is supported now in Prometheus from 2.21 version onwards.
Documentation available at https://prometheus.io/docs/prometheus/latest/configuration/configuration/#eureka_sd_config
Sample configuration is available at Sample Eureka SD config
Currently eureka server has no direct integration with prometheus, so we have to use SD adapter, I would recommend below, which is working as expected without much effort.
https://github.com/twinformatics/eureka-consul-adapter
How to add/integrate rest api end points in prometheus. For eg in splunk we are monitoring micro services transactions and logs.
Lets say below spring boot:-
/abc/v1/something
/abce/v1/something2
In one line :-how can we add above micro services end points to prometheus for motioning
When using Spring Boot 2, you can easily expose your application metrics for Prometheus. See here. You'd add spring-boot-starter-actuator and the Prometheus support with micrometer-registry-prometheus to your dependencies.
This will expose the metrics under /actuator/prometheus in the Prometheus exposition format to be scraped by your Prometheus instance.
Assumung you are using Spring WebMvc, then you'll get out-of-the-box metrics for your HTTP endpoints. (Jersey support is also provided with an extra module/dependency.)
I'm using Spring Boot 1.5.4 and Spring Cloud Dalston SR4 to stand up a Turbine server on Cloud Foundry and aggregate my application Hystrix streams. In addition I want to add Spring Boot Actuator monitoring and management to the Turbine server. I realize there is ample documentation on how to do this in a local environment and I do have it working locally. However it is a different matter when deploying to Cloud Foundry where I cannot use port numbers in a Url binding.
The issue is that the Turbine stream is provided by an RxNetty server on one port and the Actuator endpoints are provided by via Tomcat on another port. In Cloud Foundry I can only bind my Url to RxNetty endpoint or the Tomcat endpoint, not both.
No combination of management.port and turbine.stream.port allows me to access the turbine stream and the actuator endpoints from one host binding. The following is an example of what I would expect to be able to do:-
https://myapp.mydomain.com/info (to report actuator info details)
https://myapp.mydomain.com/turbine.stream (to stream turbine metrics)
Note: There are no port numbers in these Urls.
Requests to your app on Cloud Foundry go through the Cloud Foundry Go Router, which uses the http host header to direct traffic to all the container instances running your app. The http based gorouter expecter only one port to be opened by the app to forward http traffic to. However, the gorouter also support tcp routing which should allow you to have multiple ports open. see the docsfor an explanation of tcp vs. http routes on cloud foundry.
If you are running on Pivotal Cloud Foundry you can use the Circuit Breaker Dashboard provided by Spring Cloud Serviecs for PCF then you won't need to setup the turbine stream. The Spring Cloud Services Dashboard uses RabbitMQ instead of SSE events see SCS docs for details
Just getting back to this now. As noted by spencergibb, moving to springboot 2.0 and cloud Finchley works.