I'm presently experiencing a really strange and frustrating issue at the moment.
I have some code that is being tested that runs through a reactive call chain containing a series of filtering operations.
As the test runs through the code and a 'false' value is returned, the code still passes through to the next call in the chain instead of just returning.
Since I'm still a 'reactive newbie' I'm figuring I'm probably not doing something incorrectly here in the reactive code chain.
Here is the code:
private Mono<GetCardNumberServiceResponseData> updateCardNumberIfLastFourValidAndShaIsNull(Card card, GetCardNumberServiceResponseData responseData) {
return Mono.just(responseData)
.filter(response -> isValidLastFour(card, response))
.defaultIfEmpty(responseData)
.filter(response -> shaIsNull(card))
.defaultIfEmpty(responseData)
.flatMap(response -> updateCardNumber(card, response));
}
This is the portion that's not evaluating correctly:
.filter(response -> isValidLastFour(card, response))
This is what 'isValidLastFour' currently looks like:
private boolean isValidLastFour(Card card, GetCardNumberServiceResponseData responseData) {
// String cardNumberFromResponse = responseData.getCardNumber();
// String lastFourFromResponse =
// cardNumberFromResponse.substring(cardNumberFromResponse.length() - 4);
// return card.getLastFour().equals(lastFourFromResponse);
return false;
}
So presently I just have it hard-coded to return 'false', but as I step through the test with the debugger, the execution just passes right through as if 'true' is being returned, so I'm really just at a loss at what might be causing this behavior.
As always, any and all help is always greatly appreciated!
If you want responseData to be the default value, in case there is an empty Mono, you have to put defaultIfEmpty at the end of the chain:
return Mono.just(responseData)
.filter(response -> isValidLastFour(card, response))
.filter(response -> shaIsNull(card))
.flatMap(response -> updateCardNumber(card, response))
.defaultIfEmpty(responseData);
Even better, you can merge those filters:
return Mono.just(responseData)
.filter(response -> isValidLastFour(card, response) && shaIsNull(card))
.flatMap(response -> updateCardNumber(card, response))
.defaultIfEmpty(responseData);
Related
i currently have this Reactor code where im not sure im doing this the idiomatic way.
My requirements are that for a list of accountIds, I make 2 requests which are done one after the other. One to delete the account data, the other is to trigger an event afterwards. The second request is only made if the first one succeeds.
At the end, i would like to know if all of the sets of requests were successful. I have achieved this with the code below.
Flux.fromIterable(List.of("accountId", "someOtherAccountId"))
.flatMap(accountId -> someWebclient.deleteAccountData(accountId)
.doOnSuccess(response -> log.info("Delete account data success"))
.onErrorResume(e -> {
log.info("Delete account data failure");
return Mono.empty();
})
.flatMap(deleteAccountDataResponse -> {
return eventServiceClient.triggerEvent("deleteAccountEvent")
.doOnSuccess(response -> log.info("Delete account event success"))
.onErrorResume(e -> {
log.info("Delete account event failure");
return Mono.empty();
});
}))
.count()
.subscribe(items -> {
if (items.intValue() == accountIdsToForget.size()) {
log.info("All accountIds deleted and events triggered successfully");
} else {
log.info("Not all accoundIds deleted and events triggered successfully");
}
});
Is there a better way to achieve this?
As the webclients can return errors for 4xx and 5xx, i am having to swallow that up with onErrorResume in order to prevent the error from bubbling up. Similarly, the only way i have been able to capture if all of the accountIds have been processed is by checking the size of the Flux against the size of the List which it was started with
Disclaimer: it is a little subjective how to provide a better solution. In this answer, I will provide my personal choice of error handling, that, in my opinion, provides best extensibility and readability.
I would model a result/report object (kind like Either in functional paradigm), so that each success or error is sent as a "next signal" downstream.
It requires a little more code/boilerplate, but the benefit is that we end up with a flow of successes and failures produced on the fly. It allows to detect errors early, and ease both error recovery and pipeline extensibility (for example, it is then very easy to switch between fail-fast and error silencing strategies, or to build complex reports from upstream results, etc.).
Let's try to apply this to your example. For simplicity, I will mock deletion and notification service with two methods that return an empty result on success:
static Mono<Void> delete(String account) {
if (account.isBlank()) return Mono.error(new IllegalArgumentException("EMPTY ACCOUNT !"));
else return Mono.empty();
}
static Mono<Void> notify(String event) {
if (event.isBlank()) return Mono.error(new IllegalArgumentException("UNKNOWN EVENT !"));
return Mono.empty();
}
I would make this steps:
Create result model:
sealed interface Result { String accountId(); }
sealed interface Error extends Result { Throwable cause(); }
record DeletionError(String accountId, Throwable cause) implements Error {}
record NotifyError(String accountId, Throwable cause) implements Error {}
record Success(String accountId) implements Result {}
Then, we can prepare our pipeline that will wrap our delete and notify operations to make them produce result objects:
static Flux<Result> deleteAndNotify(Flux<String> accounts) {
Function<String, Mono<Result>> safeDelete = account
-> delete(account)
.<Result>thenReturn(new Success(account))
.onErrorResume(err -> Mono.just(new DeletionError(account, err)));
Function<Result, Mono<Result>> safeNotify = deletionResult -> deletionResult instanceof Success
? notify("deleteAccountEvent")
.thenReturn(deletionResult)
.onErrorResume(err -> Mono.just(new NotifyError(deletionResult.accountId(), err)))
: Mono.just(deletionResult);
return accounts.flatMap(safeDelete)
.flatMap(safeNotify);
}
With the code above, you can already receive errors as they arrive. A simple program:
var results = deleteAndNotify(Flux.just("a1", "a2", " ", "a3"));
results.subscribe(System.out::println);
prints:
Success[accountId=a1]
Success[accountId=a2]
DeletionError[accountId= , cause=java.lang.IllegalArgumentException: EMPTY ACCOUNT !]
Success[accountId=a3]
Now, it becomes very simple to adapt your flow of control:
if we want to keep track of errors only, we just have to chain a simple filter: results.filter(it -> it instanceof Error)
To fail-fast, just map error result to a real error: results.flatMap(result -> result instanceof Error err ? Mono.error(err.cause()) : Mono.just(result))
You want to get an idea of the flow throughput ? Just time it: results.timed()
etc.
And if you want to count, you can now directly count errors and successes on the fly. It provides a few advantages:
You are not forced to know the number of accounts to delete in advance to verify if any error happened
You can have a live monitoring of the failed/succeeded operations
We can program counting like that:
record Count(long success, long deleteFailed, long notifyFailed) {
Count() { this(0, 0, 0); }
Count newSuccess() { return new Count(success + 1, deleteFailed, notifyFailed); }
Count newDeletionFailure() { return new Count(success, deleteFailed + 1, notifyFailed); }
Count newNotifyFailure() { return new Count(success, deleteFailed, notifyFailed + 1); }
}
var counting = results.scanWith(Count::new, (count, result) -> switch (result) {
case Success s -> count.newSuccess();
case DeletionError de -> count.newDeletionFailure();
case NotifyError ne -> count.newNotifyFailure();
});
Subscribing to this counting flow using the same input accounts as above would produce that kind of input:
Count[success=0, deleteFailed=0, notifyFailed=0]
Count[success=1, deleteFailed=0, notifyFailed=0]
Count[success=2, deleteFailed=0, notifyFailed=0]
Count[success=2, deleteFailed=1, notifyFailed=0]
Count[success=3, deleteFailed=1, notifyFailed=0]
If you want only a total count, then either use counting.last() or replace scanWith by reduceWith operator.
I hope this answer is of any help to you to better model pipelines/DAG/flows of operations.
I am trying to make requests using WebClient in parallel, but I have no clue how to go about that,
because no matter what I do, the code is not waiting for requests to finish. If I execute just one request though (Commented fragment), everything works fine. Can someone help me with that?
#RequestMapping(method = [RequestMethod.POST], path = ["/upload/{batchId}"])
fun uploadFile(#RequestPart("file") file: Mono<FilePart>,
#PathVariable("batchId") batchId:String,
#RequestHeader("FILE-SIZE") fileSize:Int): Mono<ServiceResponse> {
val webClient = WebClient.create(commandEndpoint)
// return webClient.put().uri(seriesPath).retrieve().bodyToMono(String::class.java).map { ServiceResponse(it,0) }
return file.map{it.transferTo(Paths.get(storagePath,"excel"))}
.map{excelWorkbookToMetadata(WorkbookFactory.create(Paths.get(storagePath,"excel").toFile()))}
.flatMapMany{Flux.fromIterable(it)}
.flatMap {
it.transactionId = batchId
when (it) {
is SeriesMetadata -> webClient.put().uri(seriesPath,it.id)
.body(BodyInserters.fromObject(it))
.retrieve()
.onStatus({ it == HttpStatus.BAD_REQUEST },{
println("ERROR")
Mono.error(RuntimeException("blah")) }).toMono()
else -> Mono.error(NotImplementedError(""))
}
}
.collectList()
.map {ServiceResponse(batchId, it.size*2) }
}
So it seems, that collectList() filters out empty mono that are returned in case the body of the response is empty. The solution is basically, either to use Mono.defaultIfEmpty() method, or change retrieve() to exchange() which always returns something. At least that's what helped me.
I'm trying to implement the Reactor Subscriber Context (http://projectreactor.io/docs/core/release/reference/#context) so I can pass values from my SLF4J MDC into a Flux where I then can use the values for logging.
I use the subscriberContext() method to set the value like:
someFlux().subscriberContext(Context.of(MDC_ATTRIBUTE, MDC.get(MDC_ATTRIBUTE)));
I also can access the context in the chain. For example with a flatMap:
.flatMap(r -> Mono.subscriberContext().map(ctx -> {
String name = ctx.getOrDefault(MDC_ATTRIBUTE_NAME, "NO CTX");
return r;
}))
Also doOnEach() works:
.doOnEach(signal -> {
Context ctx = signal.getContext();
if (signal.isOnNext()) {
try (MDC.MDCCloseable closeable = MDC.putCloseable(MDC_ATTRIBUTE_NAME, ctx.getOrDefault(MDC_ATTRIBUTE_NAME, "MAAAAN"))) {
log.debug("FINISHED: {}", requestName);
}
}
})
There is just one problem with it. I want to log something in doOnSubscribe, doOnError and in doOnSuccess. While I could use doOnEach to check for signal.isOnNext() or signal.isOnComplete(), I found out that signal.isOnSubscribe() is never called.
So the question is: How can I get the context in doOnSubscribe() or is this simply not possible?
It is possible, not in 100% use cases and with a bit of a trick:
Flux.just("foo")
.doOnSubscribe(sub -> {
Scannable actual = Scannable.from(sub).scan(Scannable.Attr.ACTUAL);
if (actual instanceof CoreSubscriber) {
Context context = ((CoreSubscriber) actual).currentContext();
System.out.println(context);
}
})
.map(v -> "value: " + v) //below or above doOnSubscribe is fine
.subscriberContext(Context.of("foo", "bar")) //MUST be below doOnSubscribe
.blockLast();
This works:
public Long getMaxSalary(List<CompletableFuture<EmployeeData>> futures) {
CompletableFuture<Void> allDoneFuture = CompletableFuture.allOf(futures.toArray(new CompletableFuture[futures.size()]));
CompletableFuture<List<EmployeeData>> employeeDataList = allDoneFuture.thenApply(v ->
futures.stream()
.map(f -> f.join())
.collect(Collectors.toList()));
List<EmployeeData> rc = employeeDataList.get();
OptionalLong op = rc.stream().mapToLong(r -> r.salary()).max();
return op.getAsLong();
}
trying to make this concise throwing compiler errors in IDE. I cannot figure out what the error is. I am trying to combine it in one stream.
public Long getMaxSalary(List<CompletableFuture<EmployeeData>> futures) {
CompletableFuture<Void> allDoneFuture = CompletableFuture.allOf(futures.toArray(new CompletableFuture[futures.size()]));
return allDoneFuture.thenApply(v ->
futures.stream()
.map(f -> f.join())
.mapToLong(r -> r.salary())
.max()).getAsLong();
}
There is no point in using allOf and thenApply if you are going to block the current thread anyway with an immediate .get()
return futures.stream()
.map(CompletableFuture::join)
.mapToLong(EmployeeData::salary)
.max()
.getAsLong(); // or throw if futures is empty
allOf approach would be useful if you wanted to return CompletableFuture<Long> and let the clients of your method decide when and in what thread to await completion.
Try this out,
return allDoneFuture.thenApply(v -> futures.stream().map(f -> f.join())).get()
.mapToLong(empData -> empData.salary()).max().getAsLong();
How do i unit test RetryWhen,
public Mono<List<Transaction>> get(String id) {
return class
.get(id).log()
.retryWhen(throwableFlux -> throwableFlux)
.zipWith(Flux.range(min, max + 1), (error, retry) -> new RetryException(error, retry))
.flatMap(retryException -> {
if(retryException.getRetries() == max + 1) {
throw Exceptions.propagate(retryException.getThrowable());
} else if (isClientException(retryException.getThrowable())){
return Flux.empty();
}
return Mono.delay(Duration.ofMinutes( new Double(multiplier * retryException.getRetries()).longValue()));
}));
}
How do i use StepVerifier to test this method?
Another way to implement retry logic,
throwableFlux.takeWhile(throwable -> !isClientException(throwable))
.flatMap(e -> {
if(count.get() >= max + 1) {
throw Exceptions.propagate(e);
}
LOG.info("Retrying in..");
return Mono.delay(Duration.ofMinutes(new Double(multiplier * count.getAndAdd(1)).longValue()));
});
Do you mean testing the RetryHelper applied through retryWhen?
You can certainly use StepVerifier to test such a retryWhen containing sequence, yes. You can also check the number of (re)subscriptions by using an AtomicLong coupled to a doOnSubscribe just before the retryWhen (it will help assert the number of subscriptions made to the source being retried).
Note that we just added such a builder utility for retryWhenand repeatWhen, but in the reactor-extra project (currently in 3.1.0.BUILD-SNAPSHOT)
This is how i was able to test this code.
FirstStep.expectSubscription().expectNoEvent(java.time.Duration.ofMinutes(1)).expectNoEvent(Duration.ofMinutes(3)).verifyError()
We could have used thenAwait(Duration.ofDays(1)) above, but
expectNoEvent has the benefit of guaranteeing that nothing happened
earlier that it should have.
http://projectreactor.io/docs/core/snapshot/reference/docs/index.html#error.handling