I have a Play 2.1 Java webservice, that calls another webservice.
My service serves JSON made with transforming the XML response of the other webservice.
Initially , I was using a Jersey-Client lib to call my second service.
As this is a blocking call, I've modified the setting to allow 500 threads in parrallel.
I had good results with it , I made a gatling test and it was quite fast (32 seconds to handle 10 000 requests on a distant server).
Then I'ved decided to use only asynchronous calls to optimize the I/O.
Now I use play.libs.WS client instead of the jersey blocking client with the default thread pool parameters.
I use promises to get my results but the rest of my code is exactly the same as before.
Strangely, the performances with my gatling test (just a loop that sends some http GET requests) has dropped and is more than 10x slower...
I've tried to increase the number of threads for the "default-dispatcher" but it's not better.
Could it be because a single thread processes the XML and Json transformations after the WS call ?
The app is structured as following :
//controller :
Promise<Response> = myService.remoteEntities();
return async(promise.map(new Function<Response, Result>() {
public Result apply(Response response) {
List<Entity> entities = // ... XPATH and DOM parsing
return ok(jsonp(callback, toJson(entities)));
}
});
myService.remoteEntities is defined like that :
public Promise<List<Entity>> myServiceMethod {
return WS.url(url.toString())
}
Note : myService is a spring service singleton, and my routes/controllers use spring with the '#'notation
I 've put this in a separated class to be able to use mocks for my tests
Thanks,
Loïc
Note: I've initially posted my problem here : https://groups.google.com/forum/?fromgroups=#!topic/play-framework/SDpPA6UPyFQ
Related
I have a legacy Spring Boot REST app that interacts with downstream services that block. I'm new to reactive programming, and am unsure how to handle these blocking requests. Most Webflux examples I've seen are pretty trivial. Here's the flow-of-control of my app:
User queries MyApp at http://myapp.com
MyApp then queries partner REST API, which is BLOCKING.
Depending on account type, data from the blocking app needs to be queried to make another call to another blocking REST application.
All data is enriched and rendered by MyApp to the browser.
Where to start? I'm using WebClient currently, so that part's done. I know I should perform the blocking steps on a different scheduler (parallel or boundedElastic?) Should I use a Flux or Mono, since the partner APIs return the data all at once?
Both apps return thousands of rows of data, and the user just waits... Steps 1-2 take about 4 secs; add in step 3, and we're looking at over 30 seconds due to the inefficiency of the API. Can Flux help my users' wait time at all?
EDIT Below is a (long) example of what my application is doing. Notice that I block my first call to the API to get a count of what's being returned, then I fetch the rest in batches of TASK_QUERY_LIMIT.
#Bean
public WebClient authWebClient(WebClient.Builder builder) {
MultiValueMap<String, String> map = new LinkedMultiValueMap<>();
map.set(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
final int size = 48 * 1024 * 1024;
final ExchangeStrategies strategies = ExchangeStrategies.builder()
.codecs(codecs -> codecs.defaultCodecs().maxInMemorySize(size))
.build();
return builder.baseUrl(configProperties.getUrl())
.exchangeStrategies(strategies)
.defaultHeaders(httpHeaders -> httpHeaders.addAll(map))
.filters(exchangeFilterFunctions -> {
exchangeFilterFunctions.add(logResponseStatus());
exchangeFilterFunctions.add(logRequest());
})
.build();
}
public Mono<Task> getTasksMono() {
return getAuthWebClient()
.baseUrl("http://MyApp.com")
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(HttpStatus::isError, this::onHttpStatusError)
.bodyToMono(new ParameterizedTypeReference<Response<Task>>() {}));
}
// Service method
public List<Task> getTasksMono() {
Mono<Response<Task>> monoTasks = getTasksMono();
Task tasks = monoTasks.block();
int taskCount = tasks.getCount();
List<Task> returnTasks = new ArrayList<>(tasks.getData());
List<Mono<<Task>> tasksMonoList = new ArrayList<>();
// query API-ONE for all remaining tasks
if (taskCount > TASK_QUERY_LIMIT) {
retrieveAdditionalTasks(key, taskCount, tasksMonoList);
}
// Send out all of the calls at once, and subscribe to their results.
Flux.mergeSequential(tasksMonoList)
.map(Response::getData)
.doOnNext(returnTasks::addAll)
.blockLast();
return returnTasks.stream()
.map(this::transform) // This method performs business logic on the data before returning to user
.collect(Collectors.toList());
}
private void retrieveAdditionalTasks(String key, int taskCount,
List<Mono<Response<Task>>> tasksMonoList) {
int offset = TASK_QUERY_LIMIT;
int numRequests = (taskCount - offset) / TASK_QUERY_LIMIT + 1;
for (int i = 0; i < numRequests; i++) {
tasksMonoList.add(getTasksMono(processDefinitionKey, encryptedIacToken,
TASK_QUERY_LIMIT, offset));
offset += TASK_QUERY_LIMIT;
}
}
There are multiple questions here. Will try to highlight main points
1. Does it make sense refactoring to Reactive API?
From the first look your application is IO bound and typically reactive applications are much more efficient because all IO operations are async and non-blocking. Reactive application will not be faster but you will need less resources to The only caveat is that in order to get all benefits from the reactive API, your app should be reactive end-to-end (reactive drivers for DB, reactive WebClient, …). All reactive logic is executed on Schedulers.parallel() and you need small number of threads (by default, number of CPU cores) to execute non-blocking logic. It’s still possible use blocking API by “offloading” them to Schedulers.boundedElastic() but it should be an exception (not the rule) to make your app efficient. For more details, check Flight of the Flux 3 - Hopping Threads and Schedulers.
2. Blocking vs non-blocking.
It looks like there is some misunderstanding of the blocking API. It’s not about response time but about underlining API. By default, Spring WebFlux uses Reactor Netty as underlying Http Client library which itself is a reactive implementation of Netty client that uses Event Loop instead of Thread Per Request model. Even if request takes 30-60 sec to get response, thread will not be blocked because all IO operations are async. For such API reactive applications will behave much better because for non-reactive (thread per request) you would need large number of threads and as result much more memory to handle the same workload.
To quantify efficiency we could apply Little's Law to calculate required number of threads in a ”traditional” thread per request model
workers >= throughput x latency, where workers - number of threads
For example, to handle 100 QPS with 30 sec latency we would need 100 x 30 = 3000 threads. In reactive app the same workload could be handled by several threads only and, as result, much less memory. For scalability it means that for IO bound reactive apps you would typically scale by CPU usage and for “traditional” most probably by memory.
Sometimes it's not obvious what code is blocking. One very useful tool while testing reactive code is BlockHound that you could integrate into unit tests.
3. How to refactor?
I would migrate layer by layer but block only once. Moving remote calls to WebClient could be a first step to refactor app to reactive API. I would create all request/response logic using reactive API and then block (if required) at the very top level (e.g. in controller). Do’s and Don’ts: Avoiding First-Time Reactive Programmer Mines is a great overview of the common pitfalls and possible migration strategy.
4. Flux vs Mono.
Flux will not help you to improve performance. It’s more about downstream logic. If you process record-by-record - use Flux<T> but if you process data in batches - use Mono<List<T>>.
Your current code is not really reactive and very hard to understand mixing reactive API, stream API and blocking multiple times. As a first step try to rewrite it as a single flow using reactive API and block only once.
Not really sure about your internal types but here is some skeleton that could give you an idea about the flow.
// Service method
public Flux<Task> getTasks() {
return getTasksMono()
.flatMapMany(response -> {
List<Mono<Response<Task>>> taskRequests = new ArrayList<>();
taskRequests.add(Mono.just(response));
if (response.getCount() > TASK_QUERY_LIMIT) {
retrieveAdditionalTasks(key, response.getCount(), taskRequests);
}
return Flux.mergeSequential(taskRequests);
})
.flatMapIterable(Response::getData)
.map(this::transform); // use flatMap in case transform is async
}
As I mentioned before, try to keep internal API reactive returning Mono or Flux and block only once in the upper layer.
this is more of a best practice question.
in my current system (monolith), a single incoming http api request might need to gather similarly structured data from to several backend sources, aggregate it and only then return the data to the client in the reponse of the API.
in the current implementation I simply use a threadpool to send all requests to the backend sources in parallel and a countdown latch of sorts to know all requests returned.
i am trying to figure out the best practice for transforming the described above using reactice stacks like vert.x/quarkus. i want to keep the reactiveness of the service that accepts this api call, calls multiple (similar) backend source via http, aggregates the data.
I can roughly guess I can use things like rest-easy reactive for the incoming request and maybe MP HTTP client for the backend requests (not sure its its reactive) but I am not sure what can replace my thread pool to execute things in parallel and whats the best way to aggregate the data that returns.
I assume that using a http reactive client I can invoke all the backend sources in a loop and because its reactive it will 'feel' like parralel work. and maybe the returned data should be aggragated via the stream API (to join streams of data)? but TBH I am not sure.
I know its a long long question but some pointers would be great.
thanks!
You can drop the thread pool, you don't need it to invoke your backend services in parallel.
Yes, the MP RestClient is reactive. Let's say you have this service which invokes a backend to get a comic villain:
#RegisterRestClient(configKey = "villain-service")
public interface VillainService {
#GET
#Path("/")
#NonBlocking
#CircuitBreaker
Uni<Villain> getVillain();
}
And a similar one for heroes, HeroService. You can inject them in your endpoint class, retrieve a villain and a hero, and then compute the fight:
#Path("/api")
public class Api {
#RestClient
VillainService villains;
#RestClient
HeroService heroes;
#Inject
FightService fights;
#GET
public Uni<Fight> fight() {
Uni<Villain> villain = villains.getVillain();
Uni<Hero> hero = heroes.getRandomHero();
return Uni.combine().all().unis(hero, villain).asTuple()
.chain(tuple -> {
Hero h = tuple.getItem1();
Villain v = tuple.getItem2();
return fights.computeResult(h, v);
});
}
}
I have used below as request body in my post method of Controller (#MicroserviceController).This Microservice is running as 2 instances in server (cloud based) to be used by consumer systems to update data.
#MicroserviceMethod
#RequestMapping(path = "/trial", method = RequestMethod.POST)
public MicroserviceResponse doPost(final MicroserviceResponse msResponse) {
}
Is there a possibility that when the consumer of this api hits this REST API post call then the request body of one is used by another api hit ? That is lets say consumer hits this api twice at exactly same time (upto milli second precision) and server processes at the same time. These are unique request body coming in msResponse, so will one will be used for another ?
Any guidance will help.
Thank you
I am working on a Spring Webflux project,
I want to do something like, When client make API call, I want to send success message to client and perform large file operation in background.
So client does not have to wait till my entire file is process.
For try out I made sample code as below
REST controller
#GetMapping(value = "/{jobId}/process")
#ApiOperation("Start import job")
public Mono<Integer> process(#PathVariable("jobId") long jobId) {
return service.process(jobId);
}
File processing Service
public Mono<Integer> process(Integer jobId) {
return repository
.findById(jobId)
.map(
job -> {
File file = new File("read.csv");
return processFile(file);
});
}
Following is my stack
Spring Webflux 2.2.2.RELEASE
I try to make this call using WebClient, but till entire file is not processed I am not getting response.
As one of the options, you can run processing in a different thread.
For example:
Create an Event Listener Link
Enable #Async and #EnableAsync Link
Or use deferent types of Executors from Java concurrency package
Or manually run the thread
Also for Kotlin you can use Coroutines
You can use the subscribe method and start a job with its own scope in background.
Mono.delay(Duration.ofSeconds(10)).subscribeOn(Schedulers.newElastic("myBackgroundTask")).subscribe(System.out::println);
As long as you do not tie this to your response publisher using one of the zip/merge or similar operators your job will be run on background on its own scheduler pool.
subscribe() method returns a Disposable instance which can later be used cancel the background job by calling dispose() method.
I have an Enterprise Service Bus (ESB) that posts Data to Microservices (MCS) via Rest. I use Spring to do this. The main Problem is that i have 6 Microservices, that run one after one. So it looks like this: MCS1 -> ESB -> MCS2 -> ESB -> ... -> MCS6
So my Problem looks like this: (ESB)
#RequestMapping(value = "/rawdataservice/container", method = RequestMethod.POST)
#Produces(MediaType.APPLICATION_JSON)
public void rawContainer(#RequestBody Container c)
{
// Here i want to do something to directly send a response and afterwards execute the
// heavy code
// In the heavy code is a postForObject to the next Microservice
}
And the Service does something like this:
#RequestMapping(value = "/container", method = RequestMethod.POST)
public void addDomain(#RequestBody Container container)
{
heavyCode();
RestTemplate rt = new RestTemplate();
rt.postForObject("http://134.61.64.201:8080/rest/rawdataservice/container",container, Container.class);
}
But i dont know how to do this. I looked up the post for Location method, but i dont think it would solve the Problem.
EDIT:
I have a chain of Microservices. The first Microservice waits for a Response of the ESB. In the response the ESB posts to another Microservice and waits for a response and the next one does the same as the first one. So the Problem is that the first Microservice is blocked as long as the complete Microservice Route is completed.
ESB Route
Maybe a picture could help. 1.rawdataService 2.metadataservice 3.syntaxservice 4.semantik
// Here i want to do something to directly send a response and afterwards execute the
// heavy code
The usual spelling of that is to use the data from the http request to create a Runnable that knows how to do the work, and dispatch that runnable to an executor service for later processing. Much the same, you copy the data you need into a queue, which is polled by other threads ready to complete the work.
The http request handler then returns as soon as the executor service/queue has accepted the pending work. The most common implementation is to return a "202 Accepted" response, including in the Location header the url for a resource that will allow the client to monitor the work in progress, if desired.
In Spring, it might be ResponseEntity that manages the codes for you. For instance
ResponseEntity.accepted()....
See also:
How to respond with HTTP 400 error in a Spring MVC #ResponseBody method returning String?
REST - Returning Created Object with Spring MVC
From the caller's point of view, it would invoke RestTemplate.postForLocation, receive a URI, and throw away that URI because the microservice only needs to know that the work as been accepted
Side note: in the long term, you are probably going to want to be able to correlate the activities of the different micro services, especially when you are troubleshooting. So make sure you understand what Gregor Hohpe has to say about correlation identifiers.