How to execute an Array of CompletableFuture and combine their results [duplicate] - java-8

I have 3 CompletableFutures all 3 returning different data types.
I am looking to create a result object that is a composition of the result returned by all the 3 futures.
So my current working code looks like this:
public ClassD getResultClassD() {
ClassD resultClass = new ClassD();
CompletableFuture<ClassA> classAFuture = CompletableFuture.supplyAsync(() -> service.getClassA() );
CompletableFuture<ClassB> classBFuture = CompletableFuture.supplyAsync(() -> service.getClassB() );
CompletableFuture<ClassC> classCFuture = CompletableFuture.supplyAsync(() -> service.getClassC() );
CompletableFuture.allOf(classAFuture, classBFuture, classCFuture)
.thenAcceptAsync(it -> {
ClassA classA = classAFuture.join();
if (classA != null) {
resultClass.setClassA(classA);
}
ClassB classB = classBFuture.join();
if (classB != null) {
resultClass.setClassB(classB);
}
ClassC classC = classCFuture.join();
if (classC != null) {
resultClass.setClassC(classC);
}
});
return resultClass;
}
My questions are:
My assumption here is that since I am using allOf and thenAcceptAsync this call will be non blocking. Is my understanding right ?
Is this the right way to deal with multiple futures returning different result types ?
Is it right to construct ClassD object within thenAcceptAsync ?
Is it appropriate to use the join or getNow method in the thenAcceptAsync lambda ?

Your attempt is going into the right direction, but not correct. Your method getResultClassD() returns an already instantiated object of type ClassD on which an arbitrary thread will call modifying methods, without the caller of getResultClassD() noticing. This can cause race conditions, if the modifying methods are not thread safe on their own, further, the caller will never know, when the ClassD instance is actually ready for use.
A correct solution would be:
public CompletableFuture<ClassD> getResultClassD() {
CompletableFuture<ClassA> classAFuture
= CompletableFuture.supplyAsync(() -> service.getClassA() );
CompletableFuture<ClassB> classBFuture
= CompletableFuture.supplyAsync(() -> service.getClassB() );
CompletableFuture<ClassC> classCFuture
= CompletableFuture.supplyAsync(() -> service.getClassC() );
return CompletableFuture.allOf(classAFuture, classBFuture, classCFuture)
.thenApplyAsync(dummy -> {
ClassD resultClass = new ClassD();
ClassA classA = classAFuture.join();
if (classA != null) {
resultClass.setClassA(classA);
}
ClassB classB = classBFuture.join();
if (classB != null) {
resultClass.setClassB(classB);
}
ClassC classC = classCFuture.join();
if (classC != null) {
resultClass.setClassC(classC);
}
return resultClass;
});
}
Now, the caller of getResultClassD() can use the returned CompletableFuture to query the progress state or chain dependent actions or use join() to retrieve the result, once the operation is completed.
To address the other questions, yes, this operation is asynchronous and the use of join() within the lambda expressions is appropriate. join was exactly created because Future.get(), which is declared to throw checked exceptions, makes the use within these lambda expressions unnecessarily hard.
Note that the null tests are only useful, if these service.getClassX() can actually return null. If one of the service calls fails with an exception, the entire operation (represented by CompletableFuture<ClassD>) will complete exceptionally.

I was going down a similar route to what #Holger was doing in his answer, but wrapping the Service Calls in an Optional, which leads to cleaner code in the thenApplyAsync stage
CompletableFuture<Optional<ClassA>> classAFuture
= CompletableFuture.supplyAsync(() -> Optional.ofNullable(service.getClassA())));
CompletableFuture<Optional<ClassB>> classBFuture
= CompletableFuture.supplyAsync(() -> Optional.ofNullable(service.getClassB()));
CompletableFuture<Optional<ClassC>> classCFuture
= CompletableFuture.supplyAsync(() -> Optional.ofNullable(service.getClassC()));
return CompletableFuture.allOf(classAFuture, classBFuture, classCFuture)
.thenApplyAsync(dummy -> {
ClassD resultClass = new ClassD();
classAFuture.join().ifPresent(resultClass::setClassA)
classBFuture.join().ifPresent(resultClass::setClassB)
classCFuture.join().ifPresent(resultClass::setClassC)
return resultClass;
});

I ran into something similar before and created a short demo to show how I solved this issue.
Similar concept to #Holger except I used a function to combine each individual future.
https://github.com/te21wals/CompletableFuturesDemo
Essentially:
public class CombindFunctionImpl implement CombindFunction {
public ABCData combind (ClassA a, ClassB b, ClassC c) {
return new ABCData(a, b, c);
}
}
...
public class FutureProvider {
public CompletableFuture<ClassA> retrieveClassA() {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(1000L);
} catch (InterruptedException e) {
e.printStackTrace();
}
return new ClassA();
});
}
public CompletableFuture<ClassB> retrieveClassB() {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(2000L);
} catch (InterruptedException e) {
e.printStackTrace();
}
return new ClassB();
});
}
public CompletableFuture<ClassC> retrieveClassC() {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(3000L);
} catch (InterruptedException e) {
e.printStackTrace();
}
return new ClassC();
});
}
}
......
public static void main (String[] args){
CompletableFuture<ClassA> classAfuture = futureProvider.retrieveClassA();
CompletableFuture<ClassB> classBfuture = futureProvider.retrieveClassB();
CompletableFuture<ClassC> classCfuture = futureProvider.retrieveClassC();
System.out.println("starting completable futures ...");
long startTime = System.nanoTime();
ABCData ABCData = CompletableFuture.allOf(classAfuture, classBfuture, classCfuture)
.thenApplyAsync(ignored ->
combineFunction.combind(
classAfuture.join(),
classBfuture.join(),
classCfuture.join())
).join();
long endTime = System.nanoTime();
long duration = (endTime - startTime);
System.out.println("completable futures are complete...");
System.out.println("duration:\t" + Duration.ofNanos(duration).toString());
System.out.println("result:\t" + ABCData);
}

Another way to handle this if you don't want to declare as many variables is to use thenCombine or thenCombineAsync to chain your futures together.
public CompletableFuture<ClassD> getResultClassD()
{
return CompletableFuture.supplyAsync(ClassD::new)
.thenCombine(CompletableFuture.supplyAsync(service::getClassA), (d, a) -> {
d.setClassA(a);
return d;
})
.thenCombine(CompletableFuture.supplyAsync(service::getClassB), (d, b) -> {
d.setClassB(b);
return d;
})
.thenCombine(CompletableFuture.supplyAsync(service::getClassC), (d, c) -> {
d.setClassC(c);
return d;
});
}
The getters will still be fired off asynchronously and the results executed in order. It's basically another syntax option to get the same result.

Related

Insert in DB for multiple records not working in webflux R2DBC

I am trying to insert some records in DB in one go, also tried for loop but it never happens, If I save a single record, it works
#RequiredArgsConstructor
#Service
public class UserServiceImpl {
private final UserRepository userRepo;
private final FamilyRepository familyRepo;
public Mono<ServerResponse> insertUserData(ServerRequest serverRequest) {
return serverRequest.bodyToMono(UserAndFamilyRequest.class)
// .map(userAndFamilyRequest -> {
// List<FamilyMember> list = userAndFamilyRequest.getFamilyMemberList();
// list.stream().forEach((familyMember) ->
// {
// System.out.println(familyMember.getName());
// FamilyMemberEntity familyMemberEntity = new FamilyMemberEntity();
// familyMemberEntity.setAge(familyMember.getAge());
// familyMemberEntity.setName(familyMember.getName());
// familyRepo.save(familyMemberEntity);//doesn't work either
// try{
// Thread.sleep(2000);
// }catch(Exception ex){
//
// }
//
// });
// return userAndFamilyRequest;
// })
.map(userAndFamilyRequest -> {
List<FamilyMember> list = userAndFamilyRequest.getFamilyMemberList();
var entityList = list.stream().map(familyMember -> {
FamilyMemberEntity familyMemberEntity = new FamilyMemberEntity();
familyMemberEntity.setName(familyMember.getName());
familyMemberEntity.setAge(familyMember.getAge());
return familyMemberEntity;
}).collect(Collectors.toList());
familyRepo.saveAll(entityList);//doesn't work
return userAndFamilyRequest;
})
.flatMap(userAndFamilyRequest -> {
UserEntity userEntity = new UserEntity();
User user = userAndFamilyRequest.getUser();
userEntity.setSeats(userAndFamilyRequest.getFamilyMemberList().size());
userEntity.setAge(user.getAge());
userEntity.setName(user.getName());
return userRepo.save(userEntity);
})
// .flatMap(userAndFamilyRequest -> {
// FamilyMember familyMember = userAndFamilyRequest.getFamilyMemberList().get(0);
// FamilyMemberEntity familyMemberEntity = new FamilyMemberEntity();
// familyMemberEntity.setAge(familyMember.getAge());
// familyMemberEntity.setName(familyMember.getName());
// return familyRepo.save(familyMemberEntity);//single save works
// })
.flatMap(userEntity -> ServerResponse.created(URI.create("users"+userEntity.getId()))
.contentType(MediaType.APPLICATION_JSON)
.bodyValue(userEntity));
}
}
no error in the console
add try {} catch in function save data, so you can know what error you face it
Well I found the answer by myself, the familyRepo.save inside map will not work by itself because it returns a Flux which will execute only when subscribed, replacing it with flatMap(which automatically subscribes) resolved the issue.

How to correctly chain Mono/Flux calls

I'm having trouble with understanding how to achieve my goal with reactive approach.
Let's assume that I have a Controller, that will return Flux:
#PostMapping(value = "/mutation/stream/{domainId}", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Mutation> getMutationReactive(#RequestBody List<MutationRequest> mutationRequests, #PathVariable Integer domainId) {
return mutationService.getMutations(mutationRequests, domainId);
}
In service, currently with .subscribeOn(Schedulers.boundedElastic()), because it calls for a blocking code that is wrapped into a Callable.
public Flux<Mutation> getMutations(List<MutationRequest> mutationRequests, int domainId) {
return Flux.fromIterable(mutationRequests)
.subscribeOn(Schedulers.boundedElastic())
.flatMap(mutationRequest -> getMutation(mutationRequest.getGameId(), mutationRequest.getTypeId(), domainId));
}
getMutation() with blocking calls, currently wrapped into a Callable:
private Mono<Mutation> getMutation(int gameId, int typeId, int domainId) {
return Mono.fromCallable(() -> {
Mutation mutation = mutationProvider.findByGameIdAndTypeId(gameId, typeId).block(); // mutationProvider.findByGameIdAndTypeId() returns Mono<Mutation>
if (mutation == null) {
throw new RuntimeException("Mutation was not found by gameId and typeId");
}
State state = stateService.getStateByIds(mutation.getId()), domainId).blockFirst(); //stateService.getStateByIds() returns Mono<State>
if (state == null || state.getValue() == null) {
log.info("Requested mutation with gameId[%s] typeId[%s] domainId[%s] is disabled. Value is null.".formatted(gameId, typeId, domainId));
return null;
}
mutation.setTemplateId(state.getTemplateId());
return (mutation);
});
}
How do I approach the getMutation() function to use reactive streams, instead of using .block() methods inside a Callable?
Basically, I first need to retrieve Mutation from DB -> then using ID of mutation, get its state from other service -> then if state and its value are not null, set templateId of state to mutation and return, or return null.
I've tried something like this:
private Mono<Mutation> getMutation(int gameId, int typeId, int domainId) {
return mutationProvider.findByGameIdAndTypeId(gameId, typeId)
.flatMap(mutation -> {
stateService.getStatesByIds(mutation.getId(), domainId).flatMap(state -> {
if (state != null && state.getValue() != null) {
mutation.setTemplateId(state.getTemplateId());
}
//TODO if state/value is null -> need to propagate further to return null instead of mutation...
return Mono.justOrEmpty(state);
});
return Mono.just(mutation);
});
}
But it's obviously incorrect, nothing is subscribed to stateService.getStatesByIds(mutation.getId()), domainId)
AND
I would like to return a null if the retrieved state of mutation or its value are null.
You are ignoring the value of the inner flatMap hence the warning.
Without trying you need something like this
private Mono<Mutation> getMutation(int gameId, int typeId, int domainId) {
return mutationProvider.findByGameIdAndTypeId(gameId, typeId)
.flatMap(mutation -> {
return stateService.getStatesByIds(mutation.getId(), domainId).flatMap(state -> {
if (state != null && state.getValue() != null) {
mutation.setTemplateId(state.getTemplateId());
return Mono.just(mutation);
}
return Mono.empty();
});
});
}
Although not sure if you could rewrite the outer flatMap not to a regular map instead and you might want to use filter and defaultIfEmpty with that as well
private Mono<Mutation> getMutation(int gameId, int typeId, int domainId) {
return mutationProvider.findByGameIdAndTypeId(gameId, typeId)
.flatMap(mutation -> {
return stateService.getStatesByIds(mutation.getId(), domainId)
.filter(state -> state != null && state.getValue() != null)
.flatMap(state -> {
mutation.setTemplateId(state.getTemplateId());
return Mono.just(mutation);})
.defaultIfEmpty(Mono.empty());
}
This is just from the top of my head and I have no idea what some of the return types are here (Flux or Mono) for your own APIs.

Skip chain of CompletableFuture based on a specific condition

There are several CompletionFutures methods that I'd like to chain. The problem is that I want to skip other chain and go to last method of the pipeline if some condition is met. Mentioned as comment in the below code. For example:
public CompletableFuture<Void> process() {
CompletableFuture<Set<String>> service1 = new CompletableFuture();
return service1
.thenCompose(data -> {
if (!data.isEmpty()) {
SomeObject obj = prepareSomeObject();
obj.addAll(data.stream().collect(Collectors.toSet()));
//Here want to check some condition and if it meets, then go to last method of the pipeline
//Otherwise continue with the current flow
CompletableFuture<String> someFuture = new CompletableFuture<>();
CompletableFuture<String> someOtherFuture= new CompletableFuture<>();
return CompletableFuture.allOf(someFuture, someOtherFuture)
.thenApply(aVoid -> {
String user = someFuture.join();
String unit = someOtherFuture.join();
obj.setUnit(unit);
obj.setUser(user)
return obj;
})
.thenCompose(this::someMethod)
.thenCompose(this::someOtherMethod); // last method in the pipeline
}
return completedFuture(null);
});
}
I have tried something like this, Please critique on the right way to do it.
public CompletableFuture<Void> process() {
CompletableFuture<Set<String>> service1 = new CompletableFuture();
return service1
.thenCompose(data -> {
if (!data.isEmpty()) {
SomeObject obj = prepareSomeObject();
obj.addAll(data.stream().collect(Collectors.toSet()));
if(condition not match) { //condition match
CompletableFuture<String> someFuture = new CompletableFuture<>();
CompletableFuture<String> someOtherFuture= new CompletableFuture<>();
return CompletableFuture.allOf(someFuture, someOtherFuture)
.thenApply(aVoid -> {
String user = someFuture.join();
String unit = someOtherFuture.join();
obj.setUnit(unit);
obj.setUser(user)
})
.thenCompose(this::someMethod)
.thenCompose(this::someOtherMethod); // last method in the pipeline
} else return someOtherMethod(obj);
}
return completedFuture(null);
});
}

Error in forEach when use a consumer to avoid try-catch in forEach in Java8

I have a log() method to avoid try catch statement in forEach() below which was working in other code.
public <T> Consumer<T> log(LogConsumer<T, Throwable> logConsumer)
{
return i -> {
try
{
logConsumer.accept(i);
}
catch (Throwable e)
{
log("e = " + e);
}
};
}
#FunctionalInterface
public interface LogConsumer<T, E extends Throwable> {
void accept(T t) throws E;
}
Now I just want to use log in forEach below but I have the red rippled line in LINE such that
new Task.runJob(job, type))
I have red rippled line under job, type in
"runJob(Job, JobType) in Task cannot be applied to (java.lang.Object, < lambda parameter>)"
Now sure how to fix it to use log in forEach just to avoid
try-catch inside of it.
execute() {
Map<Job, JobType> map = getJobMap();
map.forEach( log((job, type)-> new Taks().runJob(job,type)) ); // LINE: error here
}
class Task {
public String runJob(Job job, JobType type) throws Exception
{
...
return result;
}
}
It happens because you cannot execute functions that throw exceptions using lambda expressions. You have to handle the exception using try-catch block. However, in order for your code to look more readable, create a function, that will handle the exception and return the desired result.
class Task {
public String runJob(Job job, JobType type)
{
try {
...
return result;
} catch (Exception e) {
log.error(e.getMessage());
}
return null;
}
}
In case if you care what will be the result, map it and filter for the result of your function is not null, otherwise, ignore it, but watch logs for any errors.
And then call it like shown below.
Notice: both ways work below, but the second way is more robust because you can handle the scenario when not all jobs were executed without exception.
execute() {
Map<Job, JobType> map = getJobMap();
// First way
map.forEach( log((job, type) -> new Taks().runJob(job,type)) );
// Another way
List<Object> batchResult = map.entrySet().stream()
.map((job, type) -> new Task().runJob(jon, type))
.filter(Objects::nonNull)
.collect(Collectors.toList());
if (batchResult.size() == map.size()) {
// everythings is ok (all operations resulted in non-null result
} else {
// Have to study logs and figure out what went wrong
}
}

Java 8 Completable Futures allOf different data types

I have 3 CompletableFutures all 3 returning different data types.
I am looking to create a result object that is a composition of the result returned by all the 3 futures.
So my current working code looks like this:
public ClassD getResultClassD() {
ClassD resultClass = new ClassD();
CompletableFuture<ClassA> classAFuture = CompletableFuture.supplyAsync(() -> service.getClassA() );
CompletableFuture<ClassB> classBFuture = CompletableFuture.supplyAsync(() -> service.getClassB() );
CompletableFuture<ClassC> classCFuture = CompletableFuture.supplyAsync(() -> service.getClassC() );
CompletableFuture.allOf(classAFuture, classBFuture, classCFuture)
.thenAcceptAsync(it -> {
ClassA classA = classAFuture.join();
if (classA != null) {
resultClass.setClassA(classA);
}
ClassB classB = classBFuture.join();
if (classB != null) {
resultClass.setClassB(classB);
}
ClassC classC = classCFuture.join();
if (classC != null) {
resultClass.setClassC(classC);
}
});
return resultClass;
}
My questions are:
My assumption here is that since I am using allOf and thenAcceptAsync this call will be non blocking. Is my understanding right ?
Is this the right way to deal with multiple futures returning different result types ?
Is it right to construct ClassD object within thenAcceptAsync ?
Is it appropriate to use the join or getNow method in the thenAcceptAsync lambda ?
Your attempt is going into the right direction, but not correct. Your method getResultClassD() returns an already instantiated object of type ClassD on which an arbitrary thread will call modifying methods, without the caller of getResultClassD() noticing. This can cause race conditions, if the modifying methods are not thread safe on their own, further, the caller will never know, when the ClassD instance is actually ready for use.
A correct solution would be:
public CompletableFuture<ClassD> getResultClassD() {
CompletableFuture<ClassA> classAFuture
= CompletableFuture.supplyAsync(() -> service.getClassA() );
CompletableFuture<ClassB> classBFuture
= CompletableFuture.supplyAsync(() -> service.getClassB() );
CompletableFuture<ClassC> classCFuture
= CompletableFuture.supplyAsync(() -> service.getClassC() );
return CompletableFuture.allOf(classAFuture, classBFuture, classCFuture)
.thenApplyAsync(dummy -> {
ClassD resultClass = new ClassD();
ClassA classA = classAFuture.join();
if (classA != null) {
resultClass.setClassA(classA);
}
ClassB classB = classBFuture.join();
if (classB != null) {
resultClass.setClassB(classB);
}
ClassC classC = classCFuture.join();
if (classC != null) {
resultClass.setClassC(classC);
}
return resultClass;
});
}
Now, the caller of getResultClassD() can use the returned CompletableFuture to query the progress state or chain dependent actions or use join() to retrieve the result, once the operation is completed.
To address the other questions, yes, this operation is asynchronous and the use of join() within the lambda expressions is appropriate. join was exactly created because Future.get(), which is declared to throw checked exceptions, makes the use within these lambda expressions unnecessarily hard.
Note that the null tests are only useful, if these service.getClassX() can actually return null. If one of the service calls fails with an exception, the entire operation (represented by CompletableFuture<ClassD>) will complete exceptionally.
I was going down a similar route to what #Holger was doing in his answer, but wrapping the Service Calls in an Optional, which leads to cleaner code in the thenApplyAsync stage
CompletableFuture<Optional<ClassA>> classAFuture
= CompletableFuture.supplyAsync(() -> Optional.ofNullable(service.getClassA())));
CompletableFuture<Optional<ClassB>> classBFuture
= CompletableFuture.supplyAsync(() -> Optional.ofNullable(service.getClassB()));
CompletableFuture<Optional<ClassC>> classCFuture
= CompletableFuture.supplyAsync(() -> Optional.ofNullable(service.getClassC()));
return CompletableFuture.allOf(classAFuture, classBFuture, classCFuture)
.thenApplyAsync(dummy -> {
ClassD resultClass = new ClassD();
classAFuture.join().ifPresent(resultClass::setClassA)
classBFuture.join().ifPresent(resultClass::setClassB)
classCFuture.join().ifPresent(resultClass::setClassC)
return resultClass;
});
I ran into something similar before and created a short demo to show how I solved this issue.
Similar concept to #Holger except I used a function to combine each individual future.
https://github.com/te21wals/CompletableFuturesDemo
Essentially:
public class CombindFunctionImpl implement CombindFunction {
public ABCData combind (ClassA a, ClassB b, ClassC c) {
return new ABCData(a, b, c);
}
}
...
public class FutureProvider {
public CompletableFuture<ClassA> retrieveClassA() {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(1000L);
} catch (InterruptedException e) {
e.printStackTrace();
}
return new ClassA();
});
}
public CompletableFuture<ClassB> retrieveClassB() {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(2000L);
} catch (InterruptedException e) {
e.printStackTrace();
}
return new ClassB();
});
}
public CompletableFuture<ClassC> retrieveClassC() {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(3000L);
} catch (InterruptedException e) {
e.printStackTrace();
}
return new ClassC();
});
}
}
......
public static void main (String[] args){
CompletableFuture<ClassA> classAfuture = futureProvider.retrieveClassA();
CompletableFuture<ClassB> classBfuture = futureProvider.retrieveClassB();
CompletableFuture<ClassC> classCfuture = futureProvider.retrieveClassC();
System.out.println("starting completable futures ...");
long startTime = System.nanoTime();
ABCData ABCData = CompletableFuture.allOf(classAfuture, classBfuture, classCfuture)
.thenApplyAsync(ignored ->
combineFunction.combind(
classAfuture.join(),
classBfuture.join(),
classCfuture.join())
).join();
long endTime = System.nanoTime();
long duration = (endTime - startTime);
System.out.println("completable futures are complete...");
System.out.println("duration:\t" + Duration.ofNanos(duration).toString());
System.out.println("result:\t" + ABCData);
}
Another way to handle this if you don't want to declare as many variables is to use thenCombine or thenCombineAsync to chain your futures together.
public CompletableFuture<ClassD> getResultClassD()
{
return CompletableFuture.supplyAsync(ClassD::new)
.thenCombine(CompletableFuture.supplyAsync(service::getClassA), (d, a) -> {
d.setClassA(a);
return d;
})
.thenCombine(CompletableFuture.supplyAsync(service::getClassB), (d, b) -> {
d.setClassB(b);
return d;
})
.thenCombine(CompletableFuture.supplyAsync(service::getClassC), (d, c) -> {
d.setClassC(c);
return d;
});
}
The getters will still be fired off asynchronously and the results executed in order. It's basically another syntax option to get the same result.

Resources