I am new to reactive programming and trying to simulate the below use case using project reactor but I see little bit difficult to pass the response from one service call to another dependent service. Any suggestions or references will be highly appreciated.
Response getDetails(Request inputRequest){
//Call two external services parallel based on the incoming request
Response1 = callExternalService1(inputRequest)
Response2 = callExternalService2(inputRequest)
//Call one external service based on the Response1 and Response2
Response3 = callExternalService3(Response1,Response2);
//Call three external service parallel based on the Response1, Response2 and Response3
Response4 = callExternalService4(Response1,Response2,Response3);
Response5 = callExternalService5(Response1,Response2,Response3);
Response6 = callExternalService6(Response1,Response2,Response3);
//Last service call
Response finalResponse= callLastExternalService(Response4,Response5,Response6);
return finalResponse;
}
I tried the below sample and it's working for one service call but not able to pass the response to other dependent service calls.
Updated answer:
Mono<Response> getDetails(Request inputRequest){
return Mono.just(inputRequest.getId())
.flatMap(id->{
DbResponse res = getDBCall(id).block();
if(res == null){
return Mono.error(new DBException(....));
}
return Mono.zip(callExternalService1(res),callExternalService2(inputRequest));
}).flatMap(response->{
Response extser1 = response.getT1();
Response extser2 = response.getT2();
//any exceptions?
return Mono.zip(Mono.just(extser1),Mono.just(extser2),callExternalService3();
}).flatMap(response->callExternalService4(response.getT1(),response.getT2(),response.getT3())
});
}
private Mono<DbResponse> getDBCall(String id) {
return Mono.fromCallable(()->dbservice.get(id))
.subscribeOn(Schedulers.boundedElastic());
}
Questions:
How to convert Mono<DbResponse> to DbResponse without using block
operation?
If any of the external service failed, how to build the
failure response inside the flatmap and return back?
If you have n calls and you want to move in steplock (that is, move ahead iff you have the response from all the calls), use zip. For instance:
Mono.zip(call1, call2)
.flatMap(tuple2 -> {
ResponseEntity<?> r1 = tuple2.getT1(); //response from call1
ResponseEntity<?> r2 = tuple2.getT2(); //response from call2
return Mono.zip(Mono.just(r1), Mono.just(r2), call3);
})
.flatMap(tuple3 -> {
//at this point, you have r1, r2, r3. tuple3.getT1() response from call 1
return Mono.zip(call4, call5, call6); //tuple3.getT2() response from call 2, tuple3.getT3() response from call3
})
.flatMap(tuple3 -> callLastService);
Note: If is more of a pseudo-code, won't compile right away
You can extend the above to answer your own question. Note that since call1 and call2 are independent, you can run them parallely using subscribeOn(Schedulers.boundedElastic())
Edit: Answering the two follow-up questions:
No need to subscribe using block() as flatMap subscribes to your inner streams eagerly. You can do something like:
Mono.just(inputRequest.getId())
.flatMap(a -> getDBCall(a).switchIfEmpty(Mono.defer(() -> Mono.error(..))))
Note: Mono.callable(..) returns an empty stream if the callable returns empty. That's why switchIfEmpty
You can use operators like onErrorResume to provide a fallback stream. See: The difference between onErrorResume and doOnError
if your services return Mono of Response (otherwise you have to transform them), you can make parllel calls using zip :
Mono.zip( callExternalService1( inputRequest ),
callExternalService2( inputRequest ) )
.flatMap( resp1AndResp2 -> this.callExternalService3( resp1AndResp2.getT1(),
resp1AndResp2.getT2() )
.flatMap( response3 -> Mono.zip( callExternalService4( resp1AndResp2.getT1(),
resp1AndResp2.getT2(),
response3 ),
callExternalService5( resp1AndResp2.getT1(),
resp1AndResp2.getT2(),
response3 ),
callExternalService6( resp1AndResp2.getT1(),
resp1AndResp2.getT2(),
response3 ) )
.flatMap( resp4AndResp5AndResp6 -> callLastExternalService( resp4AndResp5AndResp6.getT1(),
resp4AndResp5AndResp6.getT2(),
resp4AndResp5AndResp6.getT3() ) ) ) );
Related
I want to be able to extract the List<Payload> from the Mono<List<Payload>> to pass it to a downstream service for processing (or maybe return from the read(RequestParams params) method, instead of it returning void):
#PostMapping("/subset")
public void read(#RequestBody RequestParams params){
Mono<List<Payload>> result = reader.read(params.getDate(), params.getAssetClasses(), params.getFirmAccounts(), params.getUserId(), params.getPassword());
....
}
where reader.read(...) is a method on an autowired Spring service utilizing a webClient to get the data from external web service API:
public Mono<List<Payload>> read(String date, String assetClasses, String firmAccounts, String id, String password) {
Flux<Payload> nodes = client
.get()
.uri(uriBuilder -> uriBuilder
.path("/api/subset")
.queryParam("payloads", true)
.queryParam("date", date)
.queryParam("assetClasses", assetClasses)
.queryParam("firmAccounts", firmAccounts)
.build())
.headers(header -> header.setBasicAuth("abc123", "XXXXXXX"))
.retrieve()
.onStatus(HttpStatus::is4xxClientError, response -> {
System.out.println("4xx error");
return Mono.error(new RuntimeException("4xx"));
})
.onStatus(HttpStatus::is5xxServerError, response -> {
System.out.println("5xx error");
return Mono.error(new RuntimeException("5xx"));
})
.bodyToFlux(Payload.class);
Mono<List<Payload>> records = nodes
.collectList();
return records;
}
Doing a blocking result.block() is not allowed in WebFlux and throws an exception:
new IllegalStateException("block()/blockFirst()/blockLast() are blocking, which is not supported in thread ..." ;
What is a proper way to extract the contents of a Mono in WebFlux?
Is it some kind of a subscribe()? What would be the syntax?
Thank you in advance.
There is no "proper way" and that is the entire point. To get the value you need to block, and blocking is bad in webflux for many reasons (that I won't go into right now).
What you should do is to return the publisher all the way out to the calling client.
One of the things that many usually have a hard time understanding is that webflux works with a producer (Mono or Flux) and a subscriber.
Your entire service is also a producer, and the calling client can be seen as the subscriber.
Think of it as a long chain, that starts at the datasource, and ends up in the client showing the data.
A simple rule of thumb is that whomever is the final consumer of the data is the subscriber, everyone else is a producer.
So in your case, you just return the Mono<List<T> out to the calling client.
#PostMapping("/subset")
public Mono<List<Payload>> read(#RequestBody RequestParams params){
Mono<List<Payload>> result = reader.read(params.getDate(), params.getAssetClasses(), params.getFirmAccounts(), params.getUserId(), params.getPassword());
return result;
}
While the following does return the value of the Mono observable in the logs:
#PostMapping("/subset")
#ResponseBody
public Mono<ResponseEntity<List<Payload>>> read1(#RequestBody RequestParams params){
Mono<List<Payload>> result = reader.read(params.getDate(), params.getAssetClasses(), params.getFirmAccounts(), params.getUserId(), params.getPassword());
return result
.map(e -> new ResponseEntity<List<PayloadByStandardBasis>>(e, HttpStatus.OK));
}
the understanding I was seeking was a proper way to compose a chain of calls, with WebFlux, whereby a response from one of its operators/legs (materialized as as a result from a webclient call, producing a set of records, as above) could be passed downstream to another operator/leg to facilitate a side effect of saving those records in a DB, or something to that effect.
It would probably be a good idea to model each of those steps as a separate REST endpoint, and then have another endpoint for a composition operation which internally calls each independent endpoint in the right order, or would other design choices be more preferred?
That is ultimately the understanding I was looking for, so if anyone wants to share an example code as well as opinions to better implement the set of steps described above, I'm willing to accept the most comprehensive answer.
Thank you.
I am currently trying to consume a whole request coming back via webClient. I need to be able to read the body and the cookies that come with it. However I am currently having issues doing both together in one call. Below I have two ways of grabbing the data I want. I am pretty new at using lambdas and webClient. I do realize that exchangeToMono() is the latest method call to return a Mono or Flux because of the possible memory leak found in exchange().
I want to say subscribing() to a return monoBody might help me but so far I have had no luck extracting the data out that way as well. Thank you all for your input.
Mono<String> monoBody = webBuilder.build()
.get()
.uri(baseURI + "?Level=&pageSize=50&pageNo=&name=&id=&authToken=" + sessionToken)
.accept(MediaType.ALL)
.header("authToken", sessionToken)
.header("x-Token", sessionToken)
.header("X-token-X", sessionToken)
.header("Ref", ref)
.exchangeToMono(clientResponse -> clientResponse.bodyToMono(String.class));
Mono<MultiValueMap<String,ResponseCookie>> monoMap = webBuilder.build()
.get()
.uri(baseURI + "?Level=&pageSize=50&pageNo=&unitName=&id=&authToken=" + sessionToken)
.accept(MediaType.ALL)
.header("authToken", sessionToken)
.header("x-Token", sessionToken)
.header("X-token-X", sessionToken)
.header("Ref", ref)
.exchangeToMono(clientResponse -> Mono.fromCallable(()-> clientResponse.cookies() ));
The key thing to remember about the Reactor API (used by WebClient) is that you are always composing operations into a 'plan' of what to do and then at some point (subscription) you execute the entire plan.
In your case you've found the two methods to get the information you want but with a Reactive stream you can only ever have one type at any point in the stream, so if you need two types of data, you can always compose them into a composite object.
record DataWithCookies(MultiValueMap<String,ResponseCookie> cookies, String body){}
...
Mono<DataWithCookies> dataWithCookies =
webBuilder.build()
.get()
.uri(baseURI + "?Level=&pageSize=50&pageNo=&unitName=&id=&authToken=" + sessionToken)
.accept(MediaType.ALL)
.header("authToken", sessionToken)
.header("x-Token", sessionToken)
.header("X-token-X", sessionToken)
.header("Ref", ref)
.exchangeToMono(response ->
response.bodyToMono(String.class)
.map(stringBody -> new DataWithCookies(
stringBody,
response.cookies())
)
);
I used the previewed JDK 16 feature "Records" for brevity, but you could use a simple class for DataWithCookies.
You now have a Mono that will yield the body along with the cookies. You might want to even extract that lambda into a private method for readability.
...
.exchangeToMono(this::extractBodyAndCookies);
...
private DataWithCookies extractBodyAndCookies(final ClientResponse response) {
return response.bodyToMono(String.class)
.map(stringBody -> new DataWithCookies(stringBody,
response.cookies())
);
}
Another option is to use Tuples.of(stringBody, response.cookies()) which avoids creating another type while still composing the two data.
Staring with the tutorial code at benwilcock/spring-rsocket-demo I am trying to write a server that replicates messages to a second server before responding to a client.
To try to debug my issues I am only attempting a trivial ping-pong exchange between servers. Only when the second server responds to the pong message should the first server reply to the client:
#MessageMapping("request-response")
Mono<Message> requestResponse(final Message request) {
// register a mono that will be completed when replication to another server has happened
String uuid = UUID.randomUUID().toString();
Mono<Message> deferred = Mono.create(sink -> replicationNexus.registerRequest(uuid, sink));
// FIXME attempt to send a nested request-response message that will complete the outer message later
this.requesterMono.flatMap(requester -> requester.route("pong")
.data(uuid)
.retrieveMono(String.class))
.subscribeOn(Schedulers.elastic())
.subscribe( uuid2 -> replicationNexus.complete(uuid2, new Message(SERVER, RESPONSE)));
// return the deferred work that will be completed by the pong response
return deferred;
}
That logic is trying to use this answer to create a connection to the second server that will reconnect:
this.requesterMono = builder.rsocketConnector(connector -> connector
.reconnect(Retry.fixedDelay(Integer.MAX_VALUE, Duration.ofSeconds(1))))
.connectTcp("localhost", otherPort).cache();
To complete the picture here is the trivial ping-pong logic:
#MessageMapping("pong")
public Mono<String> pong(String m) {
return Mono.just(m);
}
and here is the logic that holds the state of the outer client response that is completed when the other server responds:
public class ReplicationNexus<T> {
final Map<String, MonoSink<T>> requests = new ConcurrentHashMap<>();
public void registerRequest(String v, MonoSink<T> sink) {
requests.put(v, sink);
}
public boolean complete(String uuid, T message) {
Optional<MonoSink<T>> sink = Optional.of(requests.get(uuid));
if( sink.isPresent() ){
sink.get().success(message);
}
return sink.isPresent();
}
}
Debugging the second server it never runs the pong method. It seems that the first server does not actually send the inner request message.
What is the correct way to run an inner request-response exchange that completes an outer message exchange with automated reconnection logic?
Not sure if I'm missing some of the complexity of your question, but if the middle server is just activing like a proxy I'd start with the simplest case of chaining through the calls. I feel like I'm missing some nuance of the question, so let's work through that next.
#MessageMapping("runCommand")
suspend fun runCommandX(
request: CommandRequest,
): Mono<String> {
val uuid = UUID.randomUUID().toString()
return requesterMono
.flatMap { requester: RSocketRequester ->
requester.route("pong")
.data("TEST")
.retrieveMono(String::class.java)
}
.doOnSubscribe {
// register request with uuid
}
.doOnSuccess {
// register completion
}
.doOnError {
// register failure
}
}
Generally if you can avoid calling subscribe yourself in typical spring/reactive/rsocket code. You want the framework to do this for you.
I have a list of Object Signers. For each of this signers I need to make a ReST request to get their signing URL. I am trying to do it with completable futures so all ReST requests can be send in parallel, then I need to set that URL in each of the signers, so this operation will not return new signers just update the ones that I am already iterating.
I have this code that is already working, but I think that could be improved.
List<Signer> signers=......
List<CompletableFuture> futures = signers.stream()
.map(signer -> CompletableFuture.completedFuture(signer))
.map(future -> future.thenCombine( CompletableFuture.supplyAsync(()-> signatureService.getSigningUrl(future.join().getSignerId())),
(signer, url) -> {
signer.setUrl(url);
return url;
}
)).collect(toList());
futures.stream()
.map(CompletableFuture::join)
.collect(toList());
Could I replace this
futures.stream()
.map(CompletableFuture::join)
.collect(toList());
With this?
futures.stream().forEach(CompletableFuture::join)
I don't like to return that because it was already used setting it in the signer. and don't like the second collect(toList()) because I am not trying to collect anything at that time.
What other implementation would you use?
No. futures.stream().forEach(CompletableFuture::join) returns void whereas futures.stream().map(CompletableFuture::join).collect(toList()); returns CompletableFuture<List<?>>.
They both are meant for different purposes. But both does one thing common(i.e, blocks the main thread till all the completablefutures are finished).
I would write your same code bit differently using CompletableFuture.allOf.
Stream<CompletableFuture<String>> streamFutures = signers.stream()
.map(signer -> CompletableFuture.completedFuture(signer))
.map(future -> future.thenCombine(CompletableFuture.supplyAsync(() -> signatureService.getSigningUrl(future.join().getSignerId())),
(signer, url) -> {
signer.setUrl(url);
return url;
}
));
CompletableFuture<String> [] futureArr = streamFutures.toArray(size -> new CompletableFuture[size]);
List<CompletableFuture<String>> futures = Arrays.asList(futureArr);
CompletableFuture<Void> allFuturesVoid = CompletableFuture.allOf(futureArr);
allFuturesVoid.join();
CompletableFuture<List<?>> allCompletableFuture = allFuturesVoid.thenApply(future -> futures.stream().map(completableFuture -> completableFuture.join()).collect(Collectors.toList()));
There is good tutorial here https://m-hewedy.blogspot.com/2017/02/completablefutureallof-that-doenst.html and here https://medium.com/#senanayake.kalpa/fantastic-completablefuture-allof-and-how-to-handle-errors-27e8a97144a0.
I do not know or ask this question except here, if it's not the place I apologize.
I am currently working on an application using spring webflux and I have a problem with the use of Mono and Flux.
Here I have a REST request that comes with a simple bean that contains attributes including a list. This list is iterated to use a responsive mongo call that returns a Mono (findOne).
But I do not think I've found the right way to do it:
#PostMapping
#RequestMapping("/check")
public Mono<ContactCheckResponse> check(#RequestBody List<ContactCheckRequest> list) {
final ContactCheckResponse response = new ContactCheckResponse();
response.setRsnCode("00");
response.setRspnCode("0000");
LOG.debug("o--> person - check {} items", list.size());
final List<ContactCheckResponse.Contact> contacts = new ArrayList<>();
response.setContacts(contacts);
return Mono.fromCallable(() -> {
list.stream().forEach( c -> {
Boolean exists = contactRespository.findOneByThumbprint(c.getIdentifiant()).block() != null;
ContactCheckResponse.Contact responseContact = new ContactCheckResponse.Contact();
responseContact.setExist(exists);
responseContact.setIdentifiant(c.getIdentifiant());
responseContact.setRsnCode("00");
responseContact.setRspnCode("0000");
response.getContacts().add(responseContact);
});
return response;
});
}
the fact of having to make a block does not seem to me in the idea "reactive" but I did not find how to do otherwise.
Could someone help me find the best way to do this task?
Thank you
Something along these lines:
return Flux.fromIterable(list)
.flatMap(c -> contactRespository.findOneByThumbprint(c.getIdentifiant())
.map(r -> r != null)
.map(exists -> {
ContactCheckResponse.Contact responseContact = new ContactCheckResponse.Contact();
...
return responseContact;
})
)
.reduce(response, (r,c) -> {
response.getContacts().add(responseContact);
return response;
});
Create a Flux from the list, create a contact for each entry and reduce everything to a Mono.