I am working on a metric collector which uses Micrometer and we've decided to use it as a serverless function due to the nature of the metrics it is collecting.
We are using Kotlin with Spring Cloud Functions and the AWS Adapter.
We have a very simple function using the the Bean method from the docs. In Micrometer, the metrics are usually sent on a schedule based on a configured step (1m, 30s etc).
However, because this is a serverless function we want to send them as the Lambda is invoked obviously - I've attempted to do this by listening on the ContextClosedEvent from Spring where I manually close the Micrometer registry which sends the metrics to our backend.
When doing this I expected that there would be a new/different context for each lambda invocation but it looks like after the initial cold-start, the warm-start invocations look like they share some context, or that context isn't being re-created/instantiated on the invocations?
Can you offer an insight into if this is the case and expected outcome and perhaps a potential more reliable way to close the micrometer registry as this current pattern can cause metrics to be dropped as the context doesn't always exist and therefore the Micrometer registry is closed.
Thanks!
MeterRegistry has a close method that you can implement. Also, depending on which registry you extend, you can find a stop method too (close should call stop).
Related
I am working on the micro-service developed using Spring Boot . I have implemented following layers:
Controller layer: Invoked when user sends API request
Service layer: Processes the request. Either sends request to third-part service or sends request to database
Repository layer: Used to interact with the
database
.
Methods in all of above layers returns the CompletableFuture. I have following questions related to this setup:
Is it good practice to return Completable future from all methods across all layers?
Is it always recommended to use #Async annotation when using CompletableFuture? what happens when I use default fork-join pool to process the requests?
How can I configure the threads for above methods? Will it be a good idea to configure the thread pool per layer? what are other configurations I can consider here?
Which metrics I should focus while optimizing performance for this micro-service?
If the work your application is doing can be done on the request thread without too much latency, I would recommend it. You can always move to an async model if you find that your web server is running out of worker threads.
The #Async annotation is basically helping with scheduling. If you can, use it - it can keep the code free of the references to the thread pool on which the work will be scheduled. As for what thread actually does your async work, that's really up to you. If you can, use your own pool. That will make sure you can add instrumentation and expose configuration options that you may need once your service is running.
Technically you will have two pools in play. One that Spring will use to consume the result of your future, and another that you will use to do the async work. If I recall correctly, Spring Boot will configure its pool if you don't already have one, and will log a warning if you didn't explicitly configure one. As for your worker threads, start simple. Consider using Spring's ThreadPoolTaskExecutor.
Regarding which metrics to monitor, start first by choosing how you will monitor. Using something like Spring Sleuth coupled with Spring Actuator will give you a lot of information out of the box. There are a lot of services that can collect all the metrics actuator generates into time-based databases that you can then use to analyze performance and get some ideas on what to tweak.
One final recommendation is that Spring's Web Flux is designed from the start to be async. It has a learning curve for sure since reactive code is very different from the usual MVC stuff. However, that framework is also thinking about all the questions you are asking so it might be better suited for your application, specially if you want to make everything async by default.
We have a Spring Boot microservice which should get some data from old / legacy system. This microservice exposes external modern REST API. Sometimes we have to issue 7-10 requests to the legacy system in order to get all the data we need for single API call. Unfortunately we can't use Reactor / WebClient and have to stick with WebServiceTemplate to issue those "legacy" calls. We can't also use Reactive Spring WebClient - Making a SOAP call
What is the best way to scale such a miroservice in Kubernetes? We have very big concerns that Thread Pool used for parallel WebServiceTemplate invocation will be depleted very fast, but I'm not sure that creating and exposing custom metric based on active threads count / thread pool size is a good idea.
Any advice will be helpful.
Enable Prometheus exporter in Spring
Make sure metrics are scraped. You're going to watch for a threadpool_size metric. Refer your k8s/prometheus distro docs to get prometheus service discovery working for you.
Write a horizontal pod autoscaler (HPA) based on a Prometheus metric:
Setup Prometheus-Adapter and follow the HPA walkthrough.
Or follow this guide https://github.com/stefanprodan/k8s-prom-hpa
Depending on what k8s distro you are using, you might have different ways to get the Prometheus and prometheus discovery:
(example platform built-in) https://cloud.google.com/stackdriver/docs/solutions/gke/prometheus
(example product) https://docs.datadoghq.com/integrations/prometheus/
(example opensource) https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack
any other prometheus solution
I'm trying to get micrometer metrics data to Splunk. Each metric endpoint gives the current value of a metric, so I would need Splunk to send a http request to my application periodically, or I can write the metric values to a log file periodically.
So how do I get my application to write the metric values to logs?
If you are in spring boot 2.x and Micrometer is of version 1.1.0+ you can create a bean of
periodic (1 minute) special logging registry see (https://github.com/micrometer-metrics/micrometer/issues/605)
#Bean
LoggingMeterRegistry loggingMeterRegistry() {
return new LoggingMeterRegistry();
}
This is by far the easiest way to log everything via logging system.
Another alternative is creating a scheduled job that will run some method on a bean with injected metering registry that will iterate over all the metrics (with possibly filtering out the metrics that you won't need) and preparing the log of your format.
If you think about this, this is exactly what the metrics endpoint of spring boot actuator does, except returning the data via http instead of writing to log.
Here is an up-to-date implementation of the relevant endpoint from the spring boot actuator source
I've seen this post here: https://dzone.com/articles/making-spring-boot-application-run-serverless-with which gives an example of how to use Spring in a Serverless scenario, but I believe that this still involves creating the Spring context, an expensive thing to do every time a request comes in. And I am wondering if Spring, but also the traditional web application frameworks are even truely compatible with the severless model, as they all tend to assume the server is only going to initialise on start, and then not again till the server is restarted, as opposed to being immediately ready to handle a request and not needing to initialize a Spring context for instance. So then these frameworks tend to do allot of stuff in the start up phase, which is not good I believe when you don't have a server per-say, and you effectively need to start up every time your would call what would be a lambda in AWS.
So my question is are these traditional web frameworks, such as Spring, which perform allot of compute when starting up still applicable in the Serverless model, for instance: AWS lambda.
Spring can indeed be applicable with the Serverless model, but as you suggest, IMHO it is not suitable for all use cases.
For the reasons that you mention (comparatively long start up times for a "cold" Lambda), I would advise against using Spring when implementing a web app that is deployed to an AWS Lambda function behind an API Gateway as the response times will suffer.
However, there are scenarios when the long start up time of a JVM based function handler implementation in a cold AWS Lambda function is less of a headache and where you may consider this option. One example is as a consumer of a Kinesis stream. The cold start will still be as bad as in the previous case, but if you have a steady stream of events the cold start will only occur once per shard. Another difference is that when using Kinesis you have already chosen an asynchronous application flow. In other words, the event producer can continue its work as soon as the event has been put on the stream without waiting for the event to be processed.
There are some Spring sub-projects that try to deal with this scenario, like Spring Cloud Function:
https://spring.io/blog/2017/07/05/introducing-spring-cloud-function
The deployment profiles even extend into the realm of Serverless (a.k.a. Functions-as-a-Service) providers, such as AWS Lambda and Apache OpenWhisk (as well as Azure Functions and Google Cloud Functions once they provide support for Java)
However, context initialization is still needed, so I guess is up to the developer to make it as small as possible to guarantee a quick startup.
EDIT: Today, I was on a talk given by Dave Syer in the Spring I/O Conference, and he presented some solutions to make Spring Boot more suitable for serveless computing:
Spring Boot Mini Applications: They are SB application but with reduced contexts:
https://github.com/dsyer/spring-boot-thin-launcher
Spring Boot thin launcher:
https://github.com/dsyer/spring-boot-thin-launcher
Some benchmarks on how long does it take to launch several configurations:
https://github.com/dsyer/spring-boot-startup-bench
The case for event-driven microservices such as Spring Cloud Stream is their asynchronous nature, which I do agree it makes them more scalable
But I have an issue regarding how to code it in a way where I don't lose certain key features that I have access to using synchronous services
In a servlet-based MS, I make full use of servlet context variables and servlet-based Spring autowiring functions
For e.g., I leverage heavily on HTTP headers to carry metadata between microservices without having to impact the payload. But in Spring Cloud Stream using Kafka, Kafka doesn't support message headers of any kind! I lose that immediately if I use SCS. Putting them into the payload causes all sort of changes in my model classes if I define the attributes clearly. Yes, I can use a simple Hashmap to simulate the HTTP header object but it really seems like reinventing the wheel to me.
On the auto-wiring side: I maintain an audit log record per request, which I implement by declaring a request-scoped Hashmap bean and autowiring it into any methods in the Servlet's call stack that needs to append data to the audit log. Basically it's just a global variable to hold some data within a single request. But in SCS, again, I lose that cos bean scopes that leverage on servlets are not available.
So far, there seems to be a lot of trade-offs that I have to make just to make Spring Cloud Stream work for me.
I thought about an alternative approach where I use SCS just to create an entry point but the Source method would just get the event, use a Processor to construct a HTTP request and send the request along to a HTTP endpoint. But, why go through all that trouble then?
Hoping that some more experienced devs would be able to shed some light on how they leverage on SCS.
#feicipet Thanks for the detailed question. let me try to address some of your concerns in the order you have listed them:
+1
+1
I am not sure why you are referring to it as servlet-based instead of Spring-based? Those are features provided by Spring, but read on. . .
Spring Cloud Stream doesn't use Kafka, the end user does while Spring Cloud Stream provides Kafka binder allowing Spring Cloud Stream to integrate with Kafka. Further more, while Kafka indeed did not support headers prior to version 0.11, Spring Cloud Stream always supported and will continue support headers even with Kafka pre-0.11, embedding them in the Message and then extracting them in the consumer side into the proper Message headers completely transparent to the end user. In other words one would assume that Kafka did support headers by simply using Spring Cloud Stream. With Kafka 0.11+ headers are supported natively and we have adjusted to that with the same level of transparency.
So, you don't need to put anything in the payload. Just create an appropriate Message<payload, headers> and SCSt will take care of the rest regardless of the broker (Kafka, Rabbit, Foo etc.).
Yes you do simply due to the fact that as you eluded earlier SCSt promotes an asynchronous and stateless architecture. However, I do not agree that what you are trying to accomplish is un-accomplishable. Rather it is accomplishable the way you are describing, but there are other way to maintain context and I would be more then glad to discuss it as a separate topic.
I would not call them trade-offs, rather difference in the architecture, that has its benefits, but it is a not one-size-fits-all architecture and therefore its viability should be discussed within the context of a concrete use case.
+1. You don't have to separate it as Source and Processor. You can simply create a custom Source app with exposed REST endpoint and custom processing logic. However we are currently working on enhancements i the framework to ensure that you could do the same with the existing starter apps.
Obviously we have touched on many points here and some of them would probably need to be debated further, but I hope this clears up some of your concerns.
Cheers