I'm working on a large Spring Boot service with around 60k lines of code. It's calling around 10 dependencies for each incoming request to its single endpoint. There are circuit breakers, timeouts and metrics in place.
The service is not great at managing struggling dependencies. As soon as their responses take longer, the service needs more CPU and its latency goes up. This is bad because we have a latency SLO.
We have made experiments with WebFlux and the prototype looks very promising. Now we want to migrate.
One way to tackle this big project is by migrating dependencies one after the other. We could rewrite them as Mono<> and then call them using block(). The project can be deployed again immediately. After all dependencies are migrated like this, switch the engine from MVC to WebFlux, then rewrite the RestController and all code in between. This would work, but ideally we would like to immediately see performance benefits after migrating the first dependency.
Would it be possible to instead add a WebFlux event loop to the project, run it in a separate thread and migrate dependencies one by one into it? How would that look like? Currently we call dependencies with #Async and with a custom thread-pool.
The event loop is not something you start up by yourself, or write yourself. In webflux the event loop is created by the underlying webserver (netty) that runs a couple of event loops depending on how many cores the host has.
I can't see any way of running 2 different webserver implementations underneath at the same time on a single application. Im not sure and someone from the spring team needs to answer the more specifics here.
Tbh, if this is a big and important project, i'd keep the original server, and then by using a load balancer infront, start out by duplicating the requests and send them to both services and implement an endpoint, run it in parallell for a while to se that it works good, and then switch off the parallell running. And do this for each/a couple of endpoint at the time.
There are also specific shadowing tools for this purpose like goreplay.
req ---------> LB ------> original
\
\------> webflux
req ---------> LB
\
\------> webflux
// Or for instance goreplay that runs on a host and also
// shadows requests forward to another service
req ---------> original
\
\------> webflux
There is never any smooth way of migrating something, ever.
Related
Spring lists SO as the only place to ask questions on their community page, which is why I ask this rather generic question here. It may not be the best fit for SO, but, according to Spring's community overview page, there's no other adequate place to ask such questions.
I have a spring boot application built on spring cloud gateway (version 2) which also uses an embedded hazelcast cluster. It runs in multiple instances, which communicate via hazelcast. Everything works fine, except under heavy load. If one instance fails, restarting it is no longer possible.
When the instance is restarted while the cluster of instances is under heavy load, it will start creating and wiring beans, up to some point, after which it will not do anything spring-related anymore. Hazelcast-generated messages are visible in the log (with root log level DEBUG), past that point, but nothing generated by spring or the application itself.
In order to restart that one instance that failed, I need to stop the load generation, wait some 10-15 minutes, then restart the failed instance. Then the new/restarted instance starts up rather quickly, with no problems at all.
The load consists of http requests which get proxied to another application, and is of such nature that it generates a lot of read accesses to hazelcast's distributed storage, but very few writes.
My problem: I have no idea how to debug this. Since the http endpoint never becomes available, there's no way I can query metrics or other actuator information.
So my question is: what tools or mechanisms can I employ to debug this problem? I.e. how can I find out exactly how the boot sequence under heavy load of the other instances of the hazelcast cluster differs from the boot sequence when there is no load at all in the cluster? Once I have this information, the problem is narrowed down enough for me to investigate it further on my own.
I didn't find a way to debug the problem, but had an idea of what might cause it, tried it, and it was a fix.
My application was running as a Kubernetes deployment. A few beans inside the application were relying on a usable CP subsystem during their initialization. Spring's bean initialization process is by necessity sequential and blocking, to account for inter-bean dependencies.
I hypothesized that under heavy load, for whatever reason, the initialization of those beans was blocking forever. As a first experiment, I made that initialization code async, so that Spring can finish bean wiring, even if, until that async part finished too, the instance was unable to perform usable work, to see if that was the problem, at least.
To my surprise, that fully fixed the problem. This way, Spring finished bean wiring, the HZ-dependant initialization also finished rather quickly, when executed async, even under high load, and the instance became usable soon after being started.
I didn't have the time to dig deeper to find out what the precise failure mechanism was. What I believe might have been the problem is the interaction between HZ and K8s. K8s-based discovery works using a K8S service. A pod/instance isn't added to the service until it becomes healthy. If a bean inside the application prevents initialization, the instance is never added to the service. As such, discovery never finds the new/restarted instance. I don't know what effect this might have on the HZ cluster's inner workings.
I am very new to reactive programming, and currently working on microservice where in spring mvc is used and Spring Data MongoDb for database connectivity.
As I am going through spring data mongo db docs, there is support for reactive repositories, reative template api etc.
So Is there going to be any downside if I choose to use reactive templates and repository with blocking nature ?
Ex.
reactiveMongoTemplate.add(entity).block()
reactiveMongoTemplate.update(id, entity).block()
Also is there any significant difference with using calls like above than using blocking repository and template api itself ?
The answer depends on which stack you use: Spring WebFlux or Spring Web MVC.
In case of Spring WebFlux the choice is obvious: you have to use ReactiveMongoTemplate and never call block on it, instead return Mono/Flux as you get it from the template.
In case of Spring Web MVC you are free to use both the regular blocking MongoTemplate and ReactiveMongoTemplate with block. Although, in most cases you should go with the good old MongoTemplate for sake of simplicity and performance. ReactiveMongoTemplate has some overhead compared to the blocking MongoTemplate because the reactive types Mono/Flux put some additional pressure on memory.
I can imagine one use case where ReactiveMongoTemplate can provide some advantage even in Spring MVC: when during one HTTP request you have to execute multiple Mono operations concurrently. If you used blocking MongoTemplate then you would need to set up a thread pool and delegate the query execution there. However, with ReactiveMongoTemplate you could use the many operators of Mono and Flux to accomplish this task without worrying about threads, thread pools and scaling issues.
In the traditional programming you usually own the thread that you're running on, in the reactive programming its not the case. This "underlying unit of execution" (cpu resources consumer if you wish) is not yours, but rather a "global" thing that currently happens to execute your task, but can switch to do other things really soon.
So when you block, you say to this "global unit of execution" like "hey, stop doing anything else, wait for me". In the traditional approach, its kind of ok, because you have a thread associated with the current request, other requests (or flows if your system is not web based) are supposed to be executed with other threads taken from a fairly large thread pool. In the reactive system however, its not the case since you're trying to utilize a small amount of these "global units of execution".
Ok, so if you block, the events all over the place will stop emitting and will get start to buffer. And this may lead to the whole system becoming unusable.
Let's imagine a scenario where there's an old app written using Servlet stack and you decide to migrate it to a Spring Webflux. Let's also say that first thing to migrate is RestTemplate to WebClient.
What would be preferred way of handling Mono or Flux that WebClient returns? Calling block seems okay solution since app uses RestTemplate which blocks anyway. subscribe should be a bit nicer since it has a callback, but maybe you have to wait to be able to continue. Also, with subscribe, do we need to handle a Disposable and, if yes, how, because I'm not sure where is the best place to call dispose on it?
And one more question regarding JDBC. How dangerous is to use Schedulers.boundedElastic()? From my understanding, having a separate thread pool should help especially if Netty is used. It's not ideal, but could it be a temporary solution until R2DBC drivers reach 1.0.0? What if app uses Tomcat? Is the situation better since Tomcat by default has more threads?
What would be preferred way of handling Mono or Flux that WebClient returns?
Given you're aiming to use Webflux, the ultimate preferred way is to deal with these publishers the entire way up your reactive chain. This means you won't end up calling subscribe on anything directly, because your controller will itself return a Mono (and everything, including the WebClient calls, will just be incorporated into that Mono via a series of reactive opreations in the chain.)
However, as you point out, you can't realistically migrate everything at once.
If it's only the WebClient that you're migrating for now, then just call block() to treat it the same way as your RestController. There's no advantage with calling subscribe() and then waiting for the subscription to complete - that's just a less obvious and more long-winded way of blocking.
Once you migrate more and more of your stack to reactive, you can do more and more in a reactive fashion, and then start to "move" your block() calls further up the chain as a result (until the block is at the controller level, and then you can just switch over to returning the Mono.)
Also, with subscribe, do we need to handle a Disposable
Only if you need to potentially cancel the subscriber (and it supports it.) This is quite a rare occurance in my experience.
And one more question regarding JDBC. How dangerous is to use Schedulers.boundedElastic()?
boundedElastic() is designed to be a wrapper for blocking IO, so there's nothing inherently wrong with using that on the reactor side. The only "danger" would be on the JDBC / app side as to if it fits your requirements:
It's capped at 10x the number of CPU cores
The number of "queued" tasks is capped at 100K
If you've got enough long running DB queries that this isn't enough, then you might consider using newBoundedElastic() instead and specifying the threadCap and queuedTaskCap manually.
What if app uses Tomcat? Is the situation better since Tomcat by default has more threads?
I wouldn't advise relying on it. Tomcat still uses one thread per request (even with Webflux), so the danger there is you end up blocking on a thread you shouldn't, and end up not realising and then relying on that behaviour (which would then all come tumbling down when you switched to Netty.)
We are developing test cases for a micro service using Spring Boot. One of the requirement is that for each Junit test case we need to:
start the project
test a unit case and
then stop the project .
I feel this is an anti pattern, but this is the requirement.
I looked around internet but couldn't find a solution for the same. I was able to start a web server but it provided no response and this might be because the project is not assigned to the server.
Does anyone have any idea on how this can be achieved?
PS: We don't want to use Mockito
Before hand i want to make clear that this a very bad practice and should be avoided. This approach does not implement unit tests concept correctly because you are testing an entire system up, so JUnit wouldn't be the correct tool.
I pocked around and i don't seem to find a Runner that may be able to do this (does not surprise me although), the most similar Runner may be SpringJUnit4ClassRunner which provides you a complete Spring context in your test space, but won't go live with the application.
An approach i'd suggest if you really want to go with this is to use tools like REST Assured to do End-to-End API layer tests against the live application, but this implies that you have to find another way to start the app, and then point the REST Assured tests to that started app. Maybe a shell script that starts the app and then starts the REST Assured tests suits, then when the suit ends put down the server.
I highly suggest you to chat with your product/management teams to avoid this kind of stuff since the tests will take FOREVER to run and you will be polluting your local or remote DBs if you are persisting data or other systems through REST or SOAP calls.
Am a total beginner with spring framework and trying to know if it even fits my use case, before investing time learning it.
I'm responsible for a stand alone java project(used as a jar by a server) which basically serves requests from a server, and in turn makes service calls to various internal services.
This standalone java project, currently has all of its service calls hard coded. I want to use Spring to inject dependencies so I can make this stuff testable.
I have no idea how spring works. Does it even hold for standalone jars or is it only for 'running applications'?
If I make my standalone project 'spring enabled', when the server uses my jar, will it automagically work by creating beans or is there some requirement from the server side?
In short, yes, you can use Spring in a standalone jar-application ("console application", if you will), we do it all the time at work. You just need to create the ApplicationContext yourself when your application starts, see for example here: http://www.devdaily.com/blog/post/java/load-spring-application-context-file-java-swing-application
This is just one example I pulled straight out of Google, there are probably numerous others. Still, you really need to read at least the basics from the Spring documentation to get started, otherwise you'll probably hit a wall pretty soon.