How do i unit test RetryWhen,
public Mono<List<Transaction>> get(String id) {
return class
.get(id).log()
.retryWhen(throwableFlux -> throwableFlux)
.zipWith(Flux.range(min, max + 1), (error, retry) -> new RetryException(error, retry))
.flatMap(retryException -> {
if(retryException.getRetries() == max + 1) {
throw Exceptions.propagate(retryException.getThrowable());
} else if (isClientException(retryException.getThrowable())){
return Flux.empty();
}
return Mono.delay(Duration.ofMinutes( new Double(multiplier * retryException.getRetries()).longValue()));
}));
}
How do i use StepVerifier to test this method?
Another way to implement retry logic,
throwableFlux.takeWhile(throwable -> !isClientException(throwable))
.flatMap(e -> {
if(count.get() >= max + 1) {
throw Exceptions.propagate(e);
}
LOG.info("Retrying in..");
return Mono.delay(Duration.ofMinutes(new Double(multiplier * count.getAndAdd(1)).longValue()));
});
Do you mean testing the RetryHelper applied through retryWhen?
You can certainly use StepVerifier to test such a retryWhen containing sequence, yes. You can also check the number of (re)subscriptions by using an AtomicLong coupled to a doOnSubscribe just before the retryWhen (it will help assert the number of subscriptions made to the source being retried).
Note that we just added such a builder utility for retryWhenand repeatWhen, but in the reactor-extra project (currently in 3.1.0.BUILD-SNAPSHOT)
This is how i was able to test this code.
FirstStep.expectSubscription().expectNoEvent(java.time.Duration.ofMinutes(1)).expectNoEvent(Duration.ofMinutes(3)).verifyError()
We could have used thenAwait(Duration.ofDays(1)) above, but
expectNoEvent has the benefit of guaranteeing that nothing happened
earlier that it should have.
http://projectreactor.io/docs/core/snapshot/reference/docs/index.html#error.handling
Related
I'm completely new to reactor programming and I'm really struggling with migrating old integration tests since upgrading to the latest Spring Boot / State Machine.
Most Integration tests have the same basic steps :
Call a method that returns a Mono and starts a state Machine and returns an object containing a generated unique id as well as some other infos related to the initial request.
With the returned object call a method that verifies if a value has been updated in the database (using the information of the object retried in step 1)
Poll at a fixed interval the method that checks in the database if value has changed until either the value has changed or a predefined timeout occurs.
Check another table in the database if another object has been updated
Below an example:
#Test
void testEndToEnd() {
var instance = ServiceInstance.buildDefault();
var updateRequest = UpdateRequest.build(instance);
// retrieve an update Response related to the request
// since a unique id is generated when triggering the update request
// before starting a stateMachine that goes through different steps
var updateResponse = service.updateInstance(updateRequest).block();
await().alias("Check if operation was successful")
.atMost(Duration.ofSeconds(120))
.pollInterval(Duration.ofSeconds(2))
.until(() -> expectOperationState(updateResponse, OperationState.SUCCESS))
// check if values are updated in secondary table
assertValuesInTransaction(updateResponse);
}
This was working fine before but ever since the latest update where it fails with the exception :
java.lang.IllegalStateException: block()/blockFirst()/blockLast() are blocking, which is not supported in thread parallel-6
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:83)
at reactor.core.publisher.Mono.block(Mono.java:1710)
I saw that a good practice to test reactor methods using StepVerifier but I do not see how I can reproduce the part done with Awaitability to poll to see if the value has changed in the DB since the method that checks in the DB returns a Mono and not a flux that keeps sending values.
Any idea on how to accomplish this or to make the spring stack accept blocking operations?
Thanks
My current stack :
Spring Boot 3.0.1
Spring State Machine 3.0.1
Spring 6
Junit 5.9.2
So as discussed in comments here is an example with comments. I used flatMap to subscribe to what expectOperationState returns. Also there is Mono.fromCallable used which check the value from some method and if it fails to emit anything in 3 seconds - the timeout exception is thrown. Also we could try to get rid of this boolean value from expectOperationState and refactor the code to just return Mono<Void> with completed signal but this basically shows how you can achieve what you want.
class TestStateMachine {
#Test
void testUntilSomeOperationCompletes() {
final Service service = new Service();
final UpdateRequest updateRequest = new UpdateRequest();
StepVerifier.create(service.updateInstance(updateRequest)
.flatMap(updateResponse -> expectOperationState(updateResponse, OperationState.SUCCESS))
)
.consumeNextWith(Assertions::assertTrue)
.verifyComplete();
}
private Mono<Boolean> expectOperationState(final UpdateResponse updateResponse, final OperationState success) {
return Mono.fromCallable(() -> {
while (true) {
boolean isInDb = checkValueFromDb(updateResponse);
if (isInDb) {
return true;
}
}
})
.publishOn(Schedulers.single())
//timeout if we not receive any value from callable within 3 seconds so that we do not check forever
.timeout(Duration.ofSeconds(3));
}
private boolean checkValueFromDb(final UpdateResponse updateResponse) {
return true;
}
}
class Service {
Mono<UpdateResponse> updateInstance(final UpdateRequest updateRequest) {
return Mono.just(new UpdateResponse());
}
}
Here is an example without using Mono<Boolean> :
class TestStateMachine {
#Test
void test() {
final Service service = new Service();
final UpdateRequest updateRequest = new UpdateRequest();
StepVerifier.create(service.updateInstance(updateRequest)
.flatMap(updateResponse -> expectOperationState(updateResponse, OperationState.SUCCESS).timeout(Duration.ofSeconds(3)))
)
.verifyComplete();
}
private Mono<Void> expectOperationState(final UpdateResponse updateResponse, final OperationState success) {
return Mono.fromCallable(() -> {
while (true) {
boolean isInDb = checkValueFromDb(updateResponse);
if (isInDb) {
//return completed Mono
return Mono.<Void>empty();
}
}
})
.publishOn(Schedulers.single())
//timeout if we not receive any value from callable within 3 seconds so that we do not check forever
.timeout(Duration.ofSeconds(3))
.flatMap(objectMono -> objectMono);
}
private boolean checkValueFromDb(final UpdateResponse updateResponse) {
return true;
}
}
i currently have this Reactor code where im not sure im doing this the idiomatic way.
My requirements are that for a list of accountIds, I make 2 requests which are done one after the other. One to delete the account data, the other is to trigger an event afterwards. The second request is only made if the first one succeeds.
At the end, i would like to know if all of the sets of requests were successful. I have achieved this with the code below.
Flux.fromIterable(List.of("accountId", "someOtherAccountId"))
.flatMap(accountId -> someWebclient.deleteAccountData(accountId)
.doOnSuccess(response -> log.info("Delete account data success"))
.onErrorResume(e -> {
log.info("Delete account data failure");
return Mono.empty();
})
.flatMap(deleteAccountDataResponse -> {
return eventServiceClient.triggerEvent("deleteAccountEvent")
.doOnSuccess(response -> log.info("Delete account event success"))
.onErrorResume(e -> {
log.info("Delete account event failure");
return Mono.empty();
});
}))
.count()
.subscribe(items -> {
if (items.intValue() == accountIdsToForget.size()) {
log.info("All accountIds deleted and events triggered successfully");
} else {
log.info("Not all accoundIds deleted and events triggered successfully");
}
});
Is there a better way to achieve this?
As the webclients can return errors for 4xx and 5xx, i am having to swallow that up with onErrorResume in order to prevent the error from bubbling up. Similarly, the only way i have been able to capture if all of the accountIds have been processed is by checking the size of the Flux against the size of the List which it was started with
Disclaimer: it is a little subjective how to provide a better solution. In this answer, I will provide my personal choice of error handling, that, in my opinion, provides best extensibility and readability.
I would model a result/report object (kind like Either in functional paradigm), so that each success or error is sent as a "next signal" downstream.
It requires a little more code/boilerplate, but the benefit is that we end up with a flow of successes and failures produced on the fly. It allows to detect errors early, and ease both error recovery and pipeline extensibility (for example, it is then very easy to switch between fail-fast and error silencing strategies, or to build complex reports from upstream results, etc.).
Let's try to apply this to your example. For simplicity, I will mock deletion and notification service with two methods that return an empty result on success:
static Mono<Void> delete(String account) {
if (account.isBlank()) return Mono.error(new IllegalArgumentException("EMPTY ACCOUNT !"));
else return Mono.empty();
}
static Mono<Void> notify(String event) {
if (event.isBlank()) return Mono.error(new IllegalArgumentException("UNKNOWN EVENT !"));
return Mono.empty();
}
I would make this steps:
Create result model:
sealed interface Result { String accountId(); }
sealed interface Error extends Result { Throwable cause(); }
record DeletionError(String accountId, Throwable cause) implements Error {}
record NotifyError(String accountId, Throwable cause) implements Error {}
record Success(String accountId) implements Result {}
Then, we can prepare our pipeline that will wrap our delete and notify operations to make them produce result objects:
static Flux<Result> deleteAndNotify(Flux<String> accounts) {
Function<String, Mono<Result>> safeDelete = account
-> delete(account)
.<Result>thenReturn(new Success(account))
.onErrorResume(err -> Mono.just(new DeletionError(account, err)));
Function<Result, Mono<Result>> safeNotify = deletionResult -> deletionResult instanceof Success
? notify("deleteAccountEvent")
.thenReturn(deletionResult)
.onErrorResume(err -> Mono.just(new NotifyError(deletionResult.accountId(), err)))
: Mono.just(deletionResult);
return accounts.flatMap(safeDelete)
.flatMap(safeNotify);
}
With the code above, you can already receive errors as they arrive. A simple program:
var results = deleteAndNotify(Flux.just("a1", "a2", " ", "a3"));
results.subscribe(System.out::println);
prints:
Success[accountId=a1]
Success[accountId=a2]
DeletionError[accountId= , cause=java.lang.IllegalArgumentException: EMPTY ACCOUNT !]
Success[accountId=a3]
Now, it becomes very simple to adapt your flow of control:
if we want to keep track of errors only, we just have to chain a simple filter: results.filter(it -> it instanceof Error)
To fail-fast, just map error result to a real error: results.flatMap(result -> result instanceof Error err ? Mono.error(err.cause()) : Mono.just(result))
You want to get an idea of the flow throughput ? Just time it: results.timed()
etc.
And if you want to count, you can now directly count errors and successes on the fly. It provides a few advantages:
You are not forced to know the number of accounts to delete in advance to verify if any error happened
You can have a live monitoring of the failed/succeeded operations
We can program counting like that:
record Count(long success, long deleteFailed, long notifyFailed) {
Count() { this(0, 0, 0); }
Count newSuccess() { return new Count(success + 1, deleteFailed, notifyFailed); }
Count newDeletionFailure() { return new Count(success, deleteFailed + 1, notifyFailed); }
Count newNotifyFailure() { return new Count(success, deleteFailed, notifyFailed + 1); }
}
var counting = results.scanWith(Count::new, (count, result) -> switch (result) {
case Success s -> count.newSuccess();
case DeletionError de -> count.newDeletionFailure();
case NotifyError ne -> count.newNotifyFailure();
});
Subscribing to this counting flow using the same input accounts as above would produce that kind of input:
Count[success=0, deleteFailed=0, notifyFailed=0]
Count[success=1, deleteFailed=0, notifyFailed=0]
Count[success=2, deleteFailed=0, notifyFailed=0]
Count[success=2, deleteFailed=1, notifyFailed=0]
Count[success=3, deleteFailed=1, notifyFailed=0]
If you want only a total count, then either use counting.last() or replace scanWith by reduceWith operator.
I hope this answer is of any help to you to better model pipelines/DAG/flows of operations.
You might wonder why don't I use #Transaction which spring already provided. I wanted too, but I'm doing event-sourcing anything that happened can't be changed or rollback. So I need to manually publish another compensation event which solve the previous one that already happened. More over I'm using project reactor so I'm kinda stuck on how should I design to make it works.
Let's say in case of hotel reservation I would have these process. So when the makePayment is failed I should rollback transaction by publish compensation events like this.
// pb stands for publish
Mono.just("start_transaction")
.flatMap { reserveHotelRoom(roomId) } pb RoomReservedEvent v ^ pb RoomUnreservedEvent
.flatMap { applyDiscount(coupon) } pb CouponRedeemedEvent v ^ pb CouponUnredeemedEvent
.flatMap { makePayment(bankAccount) } pb paymentHasbeenMadeEvent v -> oops error happens xD ^
This is what I have tried so far, but it doesn't work. once error has been thrown it still execute the next process so I have to continue working on it, but hope you get an idea of what I'm trying to do so.
class ReactiveTransaction() {
private var aggregators = Mono.just("_start_transaction_")
private var compensationAction = Mono.just("_start_rollback_")
fun addExecution(execution: Mono<String>, onError: Mono<String>): ReactiveTransaction {
// i don't know how to clone mono object so this is the current solution
val temp = compensationAction.flatMap { onError }
compensationAction = temp
aggregators = aggregators.flatMap { execution }
.onErrorResume { temp }
.checkpoint()
return this;
}
fun execute(): Mono<String> {
return aggregators
}
}
fun main() {
ReactiveTransaction().addExecution(reserveHotel, unreserveHotel)
.addExecution(applyDiscount, unapplyDiscount)
.addExecution(makePayment, cancelPayment)
.execute()
}
Is this design valid what should I concern ? or any library that can handle this problem recommend?
update: I use mongodb as my event store with reactive repository
I am working on spring reactive and need to call multiple calls sequentially to other REST API using webclient. The issue is I am able to call multiple calls to other Rest API but response am not able to read without subscribe or block. I can't use subscribe or block due to non reactive programming. Is there any way, i can merge while reading the response and send it as flux.
Below is the piece of code where I am stuck.
private Flux<SeasonsDto> getSeasonsInfo(List<HuntsSeasonsMapping> l2, String seasonsUrl) {
for (HuntsSeasonsMapping s : l2)
{
List<SeasonsJsonDto> list = huntsSeasonsProcessor.appendSeaosonToJson(s.getSeasonsRef());
for (SeasonsJsonDto sjdto:list)
{
Mono<SeasonsDto> mono =new SeasonsAdapter("http://localhost:8087/").callToSeasonsAPI(sjdto.getSeasonsRef());
//Not able to read stream without subscribe an return as Flux object
}
public Mono<SeasonsDto> callToSeasonsAPI(Long long1) {
LOGGER.debug("Seasons API call");
return this.webClient.get().uri("hunts/seasonsInfo/"
+long1).header("X-GoHunt-LoggedIn-User",
"a4d4b427-c716-458b-9bb5-9917b6aa30ff").retrieve().bodyToMono(SeasonsDto.class);
}
Please help to resolve this.
You need to combine the reactive streams using operators such as map, flatMap and concatMap.
private Flux<SeasonsDto> getSeasonsInfo(List<HuntsSeasonsMapping> l2, String seasonsUrl) {
List<Mono<SeasonsDto>> monos = new ArrayList<>();
for (HuntsSeasonsMapping s : l2) {
List<SeasonsJsonDto> list = huntsSeasonsProcessor.appendSeaosonToJson(s.getSeasonsRef());
for (SeasonsJsonDto sjdto:list) {
Mono<SeasonsDto> mono =new SeasonsAdapter("http://localhost:8087/").callToSeasonsAPI(sjdto.getSeasonsRef());
//Not able to read stream without subscribe an return as Flux object
monos.add(mono);
}
}
return Flux.fromIterable(monos).concatMap(mono -> mono);
}
This can further be improved using the steam API, which I suggest you look into, but I didn't want to change too much of your existing code.
I have figured how to do this. I have completely rewrite the code and change in reactive. It means all the for loop has been removed. Below is the code for the same and may be help for others.
public Flux<SeasonsDto> getAllSeasonDetails(String uuid) {
return hunterRepository.findByUuidAndIsPrimaryAndDeleted(uuid, true, false).next().flatMapMany(h1 -> {
return huntsMappingRepository.findByHunterIdAndDeleted(h1.getId(), false).flatMap(k -> {
return huntsMappingRepository.findByHuntReferrenceIdAndDeleted(k.getHuntReferrenceId(), false)
.flatMap(l2 -> {
return huntsSeasonsProcessor.appendSeaosonToJsonFlux(l2.getSeasonsDtl()).flatMap(fs -> {
return seasonsAdapter.callSeasonsAPI(fs.getSeasonsRef(), h1.getId(), uuid).map(k->{
return k;
});
});
});
});
});
}
I'm trying to implement the Reactor Subscriber Context (http://projectreactor.io/docs/core/release/reference/#context) so I can pass values from my SLF4J MDC into a Flux where I then can use the values for logging.
I use the subscriberContext() method to set the value like:
someFlux().subscriberContext(Context.of(MDC_ATTRIBUTE, MDC.get(MDC_ATTRIBUTE)));
I also can access the context in the chain. For example with a flatMap:
.flatMap(r -> Mono.subscriberContext().map(ctx -> {
String name = ctx.getOrDefault(MDC_ATTRIBUTE_NAME, "NO CTX");
return r;
}))
Also doOnEach() works:
.doOnEach(signal -> {
Context ctx = signal.getContext();
if (signal.isOnNext()) {
try (MDC.MDCCloseable closeable = MDC.putCloseable(MDC_ATTRIBUTE_NAME, ctx.getOrDefault(MDC_ATTRIBUTE_NAME, "MAAAAN"))) {
log.debug("FINISHED: {}", requestName);
}
}
})
There is just one problem with it. I want to log something in doOnSubscribe, doOnError and in doOnSuccess. While I could use doOnEach to check for signal.isOnNext() or signal.isOnComplete(), I found out that signal.isOnSubscribe() is never called.
So the question is: How can I get the context in doOnSubscribe() or is this simply not possible?
It is possible, not in 100% use cases and with a bit of a trick:
Flux.just("foo")
.doOnSubscribe(sub -> {
Scannable actual = Scannable.from(sub).scan(Scannable.Attr.ACTUAL);
if (actual instanceof CoreSubscriber) {
Context context = ((CoreSubscriber) actual).currentContext();
System.out.println(context);
}
})
.map(v -> "value: " + v) //below or above doOnSubscribe is fine
.subscriberContext(Context.of("foo", "bar")) //MUST be below doOnSubscribe
.blockLast();