Spring reactive : mixing RestTemplate & WebClient - spring-boot

Question

I have two endpoints : /parent and /child/{parentId}
I need to return list of all Child
public class Parent {
private long id;
private Child child;
}
public class Child {
private long childId;
private String someAttribute;
}
However, call to /child/{parentId} is quite slow, so Im trying to do this:
Call /parent to get 100 parent data, using asynchronous RestTemplate
For each parent data, call /child/{parentId} to get detail
Add the result call to /child/{parentId} into resultList
When 100 calls to /child/{parentId} is done, return resultList
I use wrapper class since most endpoints returns JSON in format :
{
"next": "String",
"data": [
// parent goes here
]
}
So I wrap it in this
public class ResponseWrapper<T> {
private List<T> data;
private String next;
}
I wrote this code, but the resultList always return empty elements.
What is the correct way to achieve this?
public List<Child> getAllParents() {
var endpointParent = StringUtils.join(HOST, "/parent");
var resultList = new ArrayList<Child>();
var responseParent = restTemplate.exchange(endpointParent, HttpMethod.GET, httpEntity,
new ParameterizedTypeReference<ResponseWrapper<Parent>>() {
});
responseParent.getBody().getData().stream().forEach(parent -> {
var endpointChild = StringUtils.join(HOST, "/child/", parent.getId());
// async call due to slow endpoint child
webClient.get().uri(endpointChild).retrieve()
.bodyToMono(new ParameterizedTypeReference<ResponseWrapper<Child>>() {
}).map(wrapper -> wrapper.getData()).subscribe(children -> {
children.stream().forEach(child -> resultList.add(child));
});
});
return resultList;
}

Answer 1

Calling subscribe on a reactive type starts the processing but returns immediately; you have no guarantee at that point that the processing is done. So by the time your snippet is calling return resultList, the WebClient is probably is still busy fetching things.
You're better off discarding the async resttemplate (which is now deprecated in favour of WebClient) and build a single pipeline like:
public List<Child> getAllParents() {
var endpointParent = StringUtils.join(HOST, "/parent");
var resultList = new ArrayList<Child>();
Flux<Parent> parents = webClient.get().uri(endpointParent)
.retrieve().bodyToMono(ResponseWrapper.class)
.flatMapMany(wrapper -> Flux.fromIterable(wrapper.data));
return parents.flatMap(parent -> {
var endpointChild = StringUtils.join(HOST, "/child/", parent.getId());
return webClient.get().uri(endpointChild).retrieve()
.bodyToMono(new ParameterizedTypeReference<ResponseWrapper<Child>>() {
}).flatMapMany(wrapper -> Flux.fromIterable(wrapper.getData()));
}).collectList().block();
}
By default, the parents.flatMap operator will process elements with some concurrency (16 by default I believe). You can choose a different value by calling another variant of the Flux.flatMap operator with a chosen concurrency value.