I'm trying to implement the Reactor Subscriber Context (http://projectreactor.io/docs/core/release/reference/#context) so I can pass values from my SLF4J MDC into a Flux where I then can use the values for logging.
I use the subscriberContext() method to set the value like:
someFlux().subscriberContext(Context.of(MDC_ATTRIBUTE, MDC.get(MDC_ATTRIBUTE)));
I also can access the context in the chain. For example with a flatMap:
.flatMap(r -> Mono.subscriberContext().map(ctx -> {
String name = ctx.getOrDefault(MDC_ATTRIBUTE_NAME, "NO CTX");
return r;
}))
Also doOnEach() works:
.doOnEach(signal -> {
Context ctx = signal.getContext();
if (signal.isOnNext()) {
try (MDC.MDCCloseable closeable = MDC.putCloseable(MDC_ATTRIBUTE_NAME, ctx.getOrDefault(MDC_ATTRIBUTE_NAME, "MAAAAN"))) {
log.debug("FINISHED: {}", requestName);
}
}
})
There is just one problem with it. I want to log something in doOnSubscribe, doOnError and in doOnSuccess. While I could use doOnEach to check for signal.isOnNext() or signal.isOnComplete(), I found out that signal.isOnSubscribe() is never called.
So the question is: How can I get the context in doOnSubscribe() or is this simply not possible?
It is possible, not in 100% use cases and with a bit of a trick:
Flux.just("foo")
.doOnSubscribe(sub -> {
Scannable actual = Scannable.from(sub).scan(Scannable.Attr.ACTUAL);
if (actual instanceof CoreSubscriber) {
Context context = ((CoreSubscriber) actual).currentContext();
System.out.println(context);
}
})
.map(v -> "value: " + v) //below or above doOnSubscribe is fine
.subscriberContext(Context.of("foo", "bar")) //MUST be below doOnSubscribe
.blockLast();
Related
i currently have this Reactor code where im not sure im doing this the idiomatic way.
My requirements are that for a list of accountIds, I make 2 requests which are done one after the other. One to delete the account data, the other is to trigger an event afterwards. The second request is only made if the first one succeeds.
At the end, i would like to know if all of the sets of requests were successful. I have achieved this with the code below.
Flux.fromIterable(List.of("accountId", "someOtherAccountId"))
.flatMap(accountId -> someWebclient.deleteAccountData(accountId)
.doOnSuccess(response -> log.info("Delete account data success"))
.onErrorResume(e -> {
log.info("Delete account data failure");
return Mono.empty();
})
.flatMap(deleteAccountDataResponse -> {
return eventServiceClient.triggerEvent("deleteAccountEvent")
.doOnSuccess(response -> log.info("Delete account event success"))
.onErrorResume(e -> {
log.info("Delete account event failure");
return Mono.empty();
});
}))
.count()
.subscribe(items -> {
if (items.intValue() == accountIdsToForget.size()) {
log.info("All accountIds deleted and events triggered successfully");
} else {
log.info("Not all accoundIds deleted and events triggered successfully");
}
});
Is there a better way to achieve this?
As the webclients can return errors for 4xx and 5xx, i am having to swallow that up with onErrorResume in order to prevent the error from bubbling up. Similarly, the only way i have been able to capture if all of the accountIds have been processed is by checking the size of the Flux against the size of the List which it was started with
Disclaimer: it is a little subjective how to provide a better solution. In this answer, I will provide my personal choice of error handling, that, in my opinion, provides best extensibility and readability.
I would model a result/report object (kind like Either in functional paradigm), so that each success or error is sent as a "next signal" downstream.
It requires a little more code/boilerplate, but the benefit is that we end up with a flow of successes and failures produced on the fly. It allows to detect errors early, and ease both error recovery and pipeline extensibility (for example, it is then very easy to switch between fail-fast and error silencing strategies, or to build complex reports from upstream results, etc.).
Let's try to apply this to your example. For simplicity, I will mock deletion and notification service with two methods that return an empty result on success:
static Mono<Void> delete(String account) {
if (account.isBlank()) return Mono.error(new IllegalArgumentException("EMPTY ACCOUNT !"));
else return Mono.empty();
}
static Mono<Void> notify(String event) {
if (event.isBlank()) return Mono.error(new IllegalArgumentException("UNKNOWN EVENT !"));
return Mono.empty();
}
I would make this steps:
Create result model:
sealed interface Result { String accountId(); }
sealed interface Error extends Result { Throwable cause(); }
record DeletionError(String accountId, Throwable cause) implements Error {}
record NotifyError(String accountId, Throwable cause) implements Error {}
record Success(String accountId) implements Result {}
Then, we can prepare our pipeline that will wrap our delete and notify operations to make them produce result objects:
static Flux<Result> deleteAndNotify(Flux<String> accounts) {
Function<String, Mono<Result>> safeDelete = account
-> delete(account)
.<Result>thenReturn(new Success(account))
.onErrorResume(err -> Mono.just(new DeletionError(account, err)));
Function<Result, Mono<Result>> safeNotify = deletionResult -> deletionResult instanceof Success
? notify("deleteAccountEvent")
.thenReturn(deletionResult)
.onErrorResume(err -> Mono.just(new NotifyError(deletionResult.accountId(), err)))
: Mono.just(deletionResult);
return accounts.flatMap(safeDelete)
.flatMap(safeNotify);
}
With the code above, you can already receive errors as they arrive. A simple program:
var results = deleteAndNotify(Flux.just("a1", "a2", " ", "a3"));
results.subscribe(System.out::println);
prints:
Success[accountId=a1]
Success[accountId=a2]
DeletionError[accountId= , cause=java.lang.IllegalArgumentException: EMPTY ACCOUNT !]
Success[accountId=a3]
Now, it becomes very simple to adapt your flow of control:
if we want to keep track of errors only, we just have to chain a simple filter: results.filter(it -> it instanceof Error)
To fail-fast, just map error result to a real error: results.flatMap(result -> result instanceof Error err ? Mono.error(err.cause()) : Mono.just(result))
You want to get an idea of the flow throughput ? Just time it: results.timed()
etc.
And if you want to count, you can now directly count errors and successes on the fly. It provides a few advantages:
You are not forced to know the number of accounts to delete in advance to verify if any error happened
You can have a live monitoring of the failed/succeeded operations
We can program counting like that:
record Count(long success, long deleteFailed, long notifyFailed) {
Count() { this(0, 0, 0); }
Count newSuccess() { return new Count(success + 1, deleteFailed, notifyFailed); }
Count newDeletionFailure() { return new Count(success, deleteFailed + 1, notifyFailed); }
Count newNotifyFailure() { return new Count(success, deleteFailed, notifyFailed + 1); }
}
var counting = results.scanWith(Count::new, (count, result) -> switch (result) {
case Success s -> count.newSuccess();
case DeletionError de -> count.newDeletionFailure();
case NotifyError ne -> count.newNotifyFailure();
});
Subscribing to this counting flow using the same input accounts as above would produce that kind of input:
Count[success=0, deleteFailed=0, notifyFailed=0]
Count[success=1, deleteFailed=0, notifyFailed=0]
Count[success=2, deleteFailed=0, notifyFailed=0]
Count[success=2, deleteFailed=1, notifyFailed=0]
Count[success=3, deleteFailed=1, notifyFailed=0]
If you want only a total count, then either use counting.last() or replace scanWith by reduceWith operator.
I hope this answer is of any help to you to better model pipelines/DAG/flows of operations.
You might wonder why don't I use #Transaction which spring already provided. I wanted too, but I'm doing event-sourcing anything that happened can't be changed or rollback. So I need to manually publish another compensation event which solve the previous one that already happened. More over I'm using project reactor so I'm kinda stuck on how should I design to make it works.
Let's say in case of hotel reservation I would have these process. So when the makePayment is failed I should rollback transaction by publish compensation events like this.
// pb stands for publish
Mono.just("start_transaction")
.flatMap { reserveHotelRoom(roomId) } pb RoomReservedEvent v ^ pb RoomUnreservedEvent
.flatMap { applyDiscount(coupon) } pb CouponRedeemedEvent v ^ pb CouponUnredeemedEvent
.flatMap { makePayment(bankAccount) } pb paymentHasbeenMadeEvent v -> oops error happens xD ^
This is what I have tried so far, but it doesn't work. once error has been thrown it still execute the next process so I have to continue working on it, but hope you get an idea of what I'm trying to do so.
class ReactiveTransaction() {
private var aggregators = Mono.just("_start_transaction_")
private var compensationAction = Mono.just("_start_rollback_")
fun addExecution(execution: Mono<String>, onError: Mono<String>): ReactiveTransaction {
// i don't know how to clone mono object so this is the current solution
val temp = compensationAction.flatMap { onError }
compensationAction = temp
aggregators = aggregators.flatMap { execution }
.onErrorResume { temp }
.checkpoint()
return this;
}
fun execute(): Mono<String> {
return aggregators
}
}
fun main() {
ReactiveTransaction().addExecution(reserveHotel, unreserveHotel)
.addExecution(applyDiscount, unapplyDiscount)
.addExecution(makePayment, cancelPayment)
.execute()
}
Is this design valid what should I concern ? or any library that can handle this problem recommend?
update: I use mongodb as my event store with reactive repository
I'm presently experiencing a really strange and frustrating issue at the moment.
I have some code that is being tested that runs through a reactive call chain containing a series of filtering operations.
As the test runs through the code and a 'false' value is returned, the code still passes through to the next call in the chain instead of just returning.
Since I'm still a 'reactive newbie' I'm figuring I'm probably not doing something incorrectly here in the reactive code chain.
Here is the code:
private Mono<GetCardNumberServiceResponseData> updateCardNumberIfLastFourValidAndShaIsNull(Card card, GetCardNumberServiceResponseData responseData) {
return Mono.just(responseData)
.filter(response -> isValidLastFour(card, response))
.defaultIfEmpty(responseData)
.filter(response -> shaIsNull(card))
.defaultIfEmpty(responseData)
.flatMap(response -> updateCardNumber(card, response));
}
This is the portion that's not evaluating correctly:
.filter(response -> isValidLastFour(card, response))
This is what 'isValidLastFour' currently looks like:
private boolean isValidLastFour(Card card, GetCardNumberServiceResponseData responseData) {
// String cardNumberFromResponse = responseData.getCardNumber();
// String lastFourFromResponse =
// cardNumberFromResponse.substring(cardNumberFromResponse.length() - 4);
// return card.getLastFour().equals(lastFourFromResponse);
return false;
}
So presently I just have it hard-coded to return 'false', but as I step through the test with the debugger, the execution just passes right through as if 'true' is being returned, so I'm really just at a loss at what might be causing this behavior.
As always, any and all help is always greatly appreciated!
If you want responseData to be the default value, in case there is an empty Mono, you have to put defaultIfEmpty at the end of the chain:
return Mono.just(responseData)
.filter(response -> isValidLastFour(card, response))
.filter(response -> shaIsNull(card))
.flatMap(response -> updateCardNumber(card, response))
.defaultIfEmpty(responseData);
Even better, you can merge those filters:
return Mono.just(responseData)
.filter(response -> isValidLastFour(card, response) && shaIsNull(card))
.flatMap(response -> updateCardNumber(card, response))
.defaultIfEmpty(responseData);
I have below code retuning Mono<Foo>:
try {
return userRepository.findById(id) // step 1
.flatMap(user -> barRepository.findByUserId( user.getId()) // step 2
.map(bar-> Foo.builder().msg("Already exists").build()) // step 3
.switchIfEmpty(barRepository.save(Bar.builder().userId(user.getId()).build()) // step 4
.map(bar-> Foo.builder().msg("Created").build()) // step 5
))
.doOnError(throwable -> Mono.just(handleError(throwable)));
} catch(Exception e) {
log.error("from catch block");
return Mono.just(handleError(e));
}
If error occurs in step 1 (e.g. user does not exist by the specified id), will it be caught by doOnError or by try catch block or none of these two?
Same question if error happens in step 2, step3, step 4.
What is the correct code so that error is always caught by doOnError and eliminate try catch?
I am using
public interface UserRepository extends ReactiveMongoRepository<User, String> same for barRepository.
handleError(throwable) simply does log.error(e.getMessage() and retuns Foo.
I think the first error is in the title: "Mono or Flux" is not related with the error handling.
Mono can only emit one item at the most (streams one element)
Flux can emit more complex stuff (i.e. List)
To handle errors you can follow this example:
return webClient.get()
.uri(url)
.retrieve()
.bodyToMono(ModelYouAreRetrieving.class)
.doOnError(throwable -> logger.error("Failed for some reason", throwable))
.onErrorReturn(new ModelYouAreRetrieving(...))
.block();
DoOnError will only perform side effects and assuming the findById are will return a Mono.Error() if it fails something like this should work.
return userRepository.findById(id)
.flatMap ( user ->
barRepository.findByUserId(user.getId())
.map((user,bar)-> Foo.builder().msg("Already exists").build())
.switchIfEmpty(barRepository.save(Bar.builder().userId(user.getId()).build())
.map(bar-> Foo.builder().msg("Created").build())
))
.onErrorReturn(throwable -> Mono.just(handleError(throwable)));
The try catch will only work if you either call a blocking operation of the chain, or a runtime error occurs before you enter the reactive chain. the doOn operations do not modify the chain, they are used for side effects only. Since flatMap expects a producer, you will need to return a Mono from the call, and in this case if an error occurs, then it will just propagate the error. In all reactive chains the error will propagate unless otherwise handled.
Use Exceptions.propagate(e) which wraps a checked exception into a special runtime exception that can be handled by onError
Below Code tries to covers User attributes in upper case. Now, when it encounters kyle the checked exception is throws and MIKE is returned from onErrorReturn
#Test
void Test19() {
Flux.fromIterable(Arrays.asList(new User("jhon", "10000"),
new User("kyle", "bot")))
.map(x -> {
try {
return toUpper(x);
} catch (TestException e) {
throw Exceptions.propagate(e);
}
})
.onErrorReturn(new User("MIKE", "BOT")).subscribe(x -> System.out.println(x));
}
protected final class TestException extends Exception {
private static final long serialVersionUID = -831485594512095557L;
}
private User toUpper(User user) throws TestException{
if (user.getName().equals("kyle")) {
throw new TestException();
}
return new User(user.getName().toUpperCase(), user.getProfession().toUpperCase());
}
Output
User [name=JHON, profession=10000]
User [name=MIKE, profession=BOT]
#Gianluca Pinto's last line of code is also incorrect. The code won't be compiled. onErrorReturn is not suitable for complicated error handling. What you should use is onErrorResume.
see: https://grokonez.com/reactive-programming/reactor/reactor-handle-error#21_By_falling_back_to_another_Flux
onErrorResume will fall back to another Flux and let you catch and manage the exception thrown by previous Flux. if look into the implementation of onErrorReturn, you will find onErrorReturn is actually using onErrorResume.
So here the code should be:
.onErrorResume(throwable -> Mono.just(handleError(throwable)));
The last line of the code of #James Ralston is wrong. The correct code should be:
return userRepository.findById(id)
.flatMap ( user ->
barRepository.findByUserId(user.getId())
.map((user,bar)-> Foo.builder().msg("Already exists").build())
.switchIfEmpty(barRepository.save(Bar.builder().userId(user.getId()).build())
.map(bar-> Foo.builder().msg("Created").build())
))
.onErrorReturn(Mono.just(handleError(throwable)));
While creating the reactive flow, we need to use onError* as it provides a fallback Mono/Flux while doOn* are side-effect operators.
NOTE: The examples are in Kotlin
Below is an example:
fun saveItems(item: Item) = testRepository.save(item)
.onErrorResume {
Mono.error(
onErrorResumeHandler(
it,
"APP-1002",
"Error occurred while saving the something :P, contact admin"
)
)
}
fun onErrorResumeHandler(exception: Throwable, errorCode: String, errorMessage: String) =
if (exception is TestRepositoryException) exception else
TestServiceException(errorCode, errorMessage)
There should be a central exception handler, we can create by extending AbstractErrorWebExceptionHandler. The order is -2 to supersede the default.
Below is an example:
#Component
#Order(-2)
class BaseControllerAdvice(
errorAttributes: ErrorAttributes,
resources: WebProperties.Resources,
applicationContext: ApplicationContext,
serverCodecConfigurer: ServerCodecConfigurer
) : AbstractErrorWebExceptionHandler(errorAttributes, resources, applicationContext) {
val log = logger()
init {
setMessageWriters(serverCodecConfigurer.writers)
}
override fun getRoutingFunction(errorAttributes: ErrorAttributes?) =
router {
RequestPredicates.all().invoke(this#BaseControllerAdvice::renderErrorResponse)
}
//RouterFunctions.route(RequestPredicates.all(),this::renderErrorResponse)
fun renderErrorResponse(
request: ServerRequest
): Mono<ServerResponse> {
val errorPropertiesMap = getErrorAttributes(
request,
ErrorAttributeOptions.defaults()
)
val ex: ApplicationException = getError(request) as ApplicationException
log.info("Error attributes:{}", request)
return ServerResponse.status(HttpStatus.BAD_REQUEST)
.contentType(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromValue(ErrorResponseVO(ex.errorCode, ex.errorMessage)))
}
data class ErrorResponseVO(val errorMessage: String, val errorCode: String)
}
How do i unit test RetryWhen,
public Mono<List<Transaction>> get(String id) {
return class
.get(id).log()
.retryWhen(throwableFlux -> throwableFlux)
.zipWith(Flux.range(min, max + 1), (error, retry) -> new RetryException(error, retry))
.flatMap(retryException -> {
if(retryException.getRetries() == max + 1) {
throw Exceptions.propagate(retryException.getThrowable());
} else if (isClientException(retryException.getThrowable())){
return Flux.empty();
}
return Mono.delay(Duration.ofMinutes( new Double(multiplier * retryException.getRetries()).longValue()));
}));
}
How do i use StepVerifier to test this method?
Another way to implement retry logic,
throwableFlux.takeWhile(throwable -> !isClientException(throwable))
.flatMap(e -> {
if(count.get() >= max + 1) {
throw Exceptions.propagate(e);
}
LOG.info("Retrying in..");
return Mono.delay(Duration.ofMinutes(new Double(multiplier * count.getAndAdd(1)).longValue()));
});
Do you mean testing the RetryHelper applied through retryWhen?
You can certainly use StepVerifier to test such a retryWhen containing sequence, yes. You can also check the number of (re)subscriptions by using an AtomicLong coupled to a doOnSubscribe just before the retryWhen (it will help assert the number of subscriptions made to the source being retried).
Note that we just added such a builder utility for retryWhenand repeatWhen, but in the reactor-extra project (currently in 3.1.0.BUILD-SNAPSHOT)
This is how i was able to test this code.
FirstStep.expectSubscription().expectNoEvent(java.time.Duration.ofMinutes(1)).expectNoEvent(Duration.ofMinutes(3)).verifyError()
We could have used thenAwait(Duration.ofDays(1)) above, but
expectNoEvent has the benefit of guaranteeing that nothing happened
earlier that it should have.
http://projectreactor.io/docs/core/snapshot/reference/docs/index.html#error.handling