Getting reference to context in Spring Reactor - spring

I am using Spring projectreactor reactor-core 3.1.8.RELEASE. I am implementing a logging framework for my microservice to have JSON Audit logs, so used context to store certain fields such as userID, collaboration ID, component Name and few other fields that are common across request life-cycle. Since Threadlocal cannot be used in reactive services to stores these elements, I have to use the context. But getting a reference to the context is apparently very difficult. I can get a reference to the context from the Signal through the doOnEach function call and that's it. If I use doOnEach, it gets called for all signals types and I am unable to isolate on Error, success and so on. Moreover, if an error occurs in between, then all the subsequent doOnEach gets called anyway, so the logs get repeated with several onError log types.
There is very limited documentation regarding how to get a reference to the context object in Spring reactor. Any help regarding a better way to generate audit logs that has collaboration IDs and other request specific IDs stored and propagated across function calls and external invocations is appreciated.
Code Snippets -
In the WebFilter, I am setting few key-value pairs as follows -
override fun filter(exchange: ServerWebExchange, filterChain: WebFilterChain): Mono<Void> {
// add the context variables at the end of the chain as the context moves from
// downstream to upstream.
return filterChain.filter(exchange)
.subscriberContext { context ->
var ctx = context.put(RestRequestInfo::class.java, restRequestInfo(exchange))
ctx = ctx.put(COLLABORATION_ID, UUID.randomUUID().toString())
ctx=ctx.put(COMPONENT_NAME, "sample-component-name")
ctx=ctx.put(USER_NAME, "POSTMAN")
ctx
}
}
Then I want to use the key-value pairs added above in all the subsequent logs so that log aggregators like Splunk can get all the JSON logs associated with this particular request, based on collaboration ID. Right now, the only way to get values out of context is through doOnEach function call, where we get a handle to the SIgnal through which we get handle to context. But all doOnEach gets called during each and every events, irrespective of whether each function call was success or failure
return Mono.just(request)
.doOnEach(**Code to log with context data**)
.map(RequestValidations::validateRequest)
.doOnEach(**Code to log with context data**)
.map(RequestValidations::buildRequest)
.map(RequestValidations::validateQueryParameters)
.doOnEach(**Code to log with context data**)
.flatMap(coverageSummariesGateway::getCoverageSummaries)
.doOnEach(**Code to log with context data**)
.map({ coverageSummaries ->
getCoverageSummariesResponse(coverageSummaries, serviceReferenceId) })
.doOnEach(**Code to log with context data**)
.flatMap(this::renderSuccess)
.doOnEach(**Code to log with context data**)
.doOnError { logger.info("ERROR OCCURRED") }
Thank you!

You could do something like following:
return Mono.just(request)
.doOnEach(**Code to log with context data**)
.flatMap( r -> withMDC(r, RequestValidations::validateRequest))
following method will populate mapping diagnostic context (MDC) so you have it automatically in your logs (depends on you logging pattern). E.g. logback has %X{traceId} where traceId is a key in the tracingContext map.
public static <T, R> Mono<R> withMDC(T value, Function<T, Mono<R>> f) {
return Mono.subscriberContext()
.flatMap( ctx -> {
Optional<Map> tracingContext = ctx.getOrEmpty("tracing-context-key");
if (tracingContext.isPresent()) {
try {
MDC.setContextMap(tracingContext.get());
return f.apply(value);
} finally {
MDC.clear();
}
} else{
return f.apply(value);
}
});
}
It is not quite nice, hope it will be eventually improved by Logging frameworks and context will be injected automatically.

Related

Immediately return first emitted value from two Monos while continuing to process the other asynchronously

I have two data sources, each returning a Mono:
class CacheCustomerClient {
Mono<Entity> createCustomer(Customer customer)
}
class MasterCustomerClient {
Mono<Entity> createCustomer(Customer customer)
}
Callers to my application are hitting a Spring WebFlux controller:
#PostMapping
#ResponseStatus(HttpStatus.CREATED)
public Flux<Entity> createCustomer(#RequestBody Customer customer) {
return customerService.createNewCustomer(entity);
}
As long as either data source successfully completes its create operation, I want to immediately return a success response to the caller, however, I still want my service to continue processing the result of the other Mono stream, in the event that an error was encountered, so it can be logged.
The problem seems to be that as soon as a value is returned to the controller, a cancel signal is propagated back through the stream by Spring WebFlux and, thus, no information is logged about a failure.
Here's one attempt:
public Flux<Entity> createCustomer(final Customer customer) {
var cacheCreate = cacheClient
.createCustomer(customer)
.doOnError(WebClientResponseException.class,
err -> log.error("Customer creation failed in cache"));
var masterCreate = masterClient
.createCustomer(customer)
.doOnError(WebClientResponseException.class,
err -> log.error("Customer creation failed in master"));
return Flux.firstWithValue(cacheCreate, masterCreate)
.onErrorMap((err) -> new Exception("Customer creation failed in cache and master"));
}
Flux.firstWithValue() is great for emitting the first non-error value, but then whichever source is lagging behind is cancelled, meaning that any error is never logged out. I've also tried scheduling these two sources on their own Schedulers and that didn't seem to help either.
How can I perform these two calls asynchronously, and emit the first value to the caller, while continuing to listen for emissions on the slower source?
You can achieve that by transforming you operators to "hot" publishers using share() operator:
First subscriber launch the upstream operator, and additional subscribers get back result cached from the first subscriber:
Further Subscriber will share [...] the same result.
Once a second subscription has been done, the publisher is not cancellable:
It's worth noting this is an un-cancellable Subscription.
So, to achieve your requirement:
Apply share() on each of your operators
Launch a subscription on shared publishers to trigger processing
Use shared operators in your pipeline (here firstWithValue).
Sample example:
import java.time.Duration;
import reactor.core.publisher.Mono;
public class TestUncancellableMono {
// Mock a mono successing quickly
static Mono<String> quickSuccess() {
return Mono.delay(Duration.ofMillis(200)).thenReturn("SUCCESS !");
}
// Mock a mono taking more time and ending in error.
static Mono<String> longError() {
return Mono.delay(Duration.ofSeconds(1))
.<String>then(Mono.error(new Exception("ERROR !")))
.doOnCancel(() -> System.out.println("CANCELLED"))
.doOnError(err -> System.out.println(err.getMessage()));
}
public static void main(String[] args) throws Exception {
// Transform to hot publisher
var sharedQuick = quickSuccess().share();
var sharedLong = longError().share();
// Trigger launch
sharedQuick.subscribe();
sharedLong.subscribe();
// Subscribe back to get the cached result
Mono
.firstWithValue(sharedQuick, sharedLong)
.subscribe(System.out::println, err -> System.out.println(err.getMessage()));
// Wait for subscription to end.
Thread.sleep(2000);
}
}
The output of the sample is:
SUCCESS !
ERROR !
We can see that error message has been propagated properly, and that upstream publisher has not been cancelled.

Using ReactiveSecurityContextHolder inside a Kotlin Flow

I'm working on a Spring Boot (2.2) project using Kotlin, with CouchDB as (reactive) database, and in consequence, async DAO (either suspend functions, or functions returning a Flow). I'm trying to setup WebFlux in order to have async controllers too (again, I want to return Flows, not Flux). But I'm having troubles retrieving my security context from ReactiveSecurityContextHolder.
From what I've read, unlike SecurityContextHolder which is using ThreadLocal to store it, ReactiveSecurityContextHolder relies on the fact that Spring, while making a subscription to my reactive chain, also stored that context inside this chain, thus allowing me to call ReactiveSecurityContextHolder.getContext() from within the chain.
The problem is that I have to transform my Mono<SecurityContext> into a Flow at some point, which makes me loose my SecurityContext. So my question is: is there a way to have a Spring Boot controller returning a Flow while retrieving the security context from ReactiveSecurityContextHolder inside my logic? Basically, after simplification, it should look like this:
#GetMapping
fun getArticles(): Flow<String> {
return ReactiveSecurityContextHolder.getContext().flux().asFlow() // returns nothing
}
Note that if I return the Flux directly (skipping the .asFlow()), or add a .single() or .toList() in the end (hence using a suspend fun), then it works fine and my security context is returned, but again that's not what I want. I guess the solution would be to transfer the context from the Flux (initial reactive chain from ReactiveSecurityContextHolder) to the Flow, but it doesn't seem to be done by default.
Edit: here is a sample project showcasing the problem: https://github.com/Simon3/webflux-kotlin-sample
What you really try to achieve is accessing your ReactorContext from inside a Flow.
One way to do this is to relax the need for returning a Flow and return a Flux instead. This allows you to recover the ReactorContext and pass it to the Flow you are going to use to generate your data.
#ExperimentalCoroutinesApi
#GetMapping("/flow")
fun flow(): Flux<Map<String, String>> = Mono.subscriberContext().flatMapMany { reactorCtx ->
flow {
val ctx = coroutineContext[ReactorContext.Key]?.context?.get<Mono<SecurityContext>>(SecurityContext::class.java)?.asFlow()?.single()
emit(mapOf("user" to ((ctx?.authentication?.principal as? User)?.username ?: "<NONE>")))
}.flowOn(reactorCtx.asCoroutineContext()).asFlux()
}
In the case when you need to access the ReactorContext from a suspend method, you can simply get it back from the coroutineContext with no further artifice:
#ExperimentalCoroutinesApi
#GetMapping("/suspend")
suspend fun suspend(): Map<String,String> {
val ctx = coroutineContext[ReactorContext.Key]?.context?.get<Mono<SecurityContext>>(SecurityContext::class.java)?.asFlow()?.single()
return mapOf("user" to ((ctx?.authentication?.principal as? User)?.username ?: "<NONE>"))
}

spring rector throw Exception if NOT empty

Similar to Spring Reactor: How to throw an exception when publisher emit a value?
I have a finder method in my DAO java findSomePojo which returns result SomePojo . The finder calls amazon db apis and the javasoftware.amazon.awssdk.services.dynamodb.model.GetItemResponse has output of call.
So I am trying this hasElement() check in my service layer createSomePojo method. (Not sure if I am using it correctly- Iwas trying and debugging)
Basically :
I want to check if there is already element, it is illegal to save and I would not call DAOs save. So I need to throw exception.
Assuming that there is already a record of SomePojo in DB, I try to invoke create_SomePjo of service .But I see in logs that filter is not working and is get NPE when reactor invokes createModel_SomePojo making me believe that somehow even after check filter it throws NPE
///service SomePjoService it has create_SomePojo, find_SomePojo etc
Mono<Void> create_SomePojo(reqPojo){
// Before calling DAO 's save I call serivice find (which basically calls DAOs find (Shown befow after this methid)
Mono<Boolean> monoPresent = find_SomePojo(accountId, contentIdExtn)
.filter(i -> i.getId() != null)
.hasElement();
System.out.println("monoPresent="+monoPresent.toString());
if(monoPresent.toString().equals("MonoHasElement")){
//*************it comes here i see that***********//
System.out.println("hrereee monoPresent="+monoPresent);
// Mono<Error> monoCheck=
return monoPresent.handle((next, sink) -> sink.error(new SomeException(ITEM_ALREADY_EXISTS))).then();
} else {
return SomePojoRepo.save(reqPojo).then();
}
}
Mono<SomePojo> find_SomePojo(id){
return SomePojoRepo.find(id);
}
==============================================================
///DAO : SomePojoRepo.java : it has save,find,delete
Mono<SomePojo> find( String id) {
Mono<SomePojo> fallback = Mono.empty();
Mono<GetItemResponse> monoFilteredResponse = monoFuture
.filter(getItemResponse -> getItemResponse.item().size() > 0&& getItemResponse!=null);
Mono<SomePojo> result = monoFilteredResponse
.map(getItemResponse -> createModel_SomePojo(getItemResponse.item()));
Mono<SomePojo> deferedResult = Mono.defer(() -> result.switchIfEmpty(fallback));
return deferedResult;
}
I see there is hasElement() method on Mono . Not sure how to correctly use it.
I can achieve exception if I call DAO save in my service create_SomePojo(reqPojo) directly without doing all this findner check because primary key constraint will take care and throw excpetion and I cna rethrow and then catch in service but what If I want to check in service and throw exception with error codes . The idea is not to pass response error object to dao layer .
Try to use Hooks.onOperatorDebug() hook to get better debugging experience.
Correct way to use hasElement (assuming that find_SomePojo never returns null)
Mono<Boolean> monoPresent = find_SomePojo(accountId, contentIdExtn)
.filter(i -> i.getId() != null)
.hasElement();
return monoPresent.flatMap(isPresent -> {
if(isPresent){
Mono.error(new SomeException(ITEM_ALREADY_EXISTS)));
}else{
SomePojoRepo.save(reqPojo);
}
}).then();
Sidenote
There is a common misconception about what Mono actually is. It does not hold any data - it's just a fragment of pipeline, which transmits signals and data flowing through it. Therefore, line System.out.println("monoPresent="+monoPresent.toString()); makes no sense, because it just prints the hasElements() decorator around the existsing pipeline. Internal name of this decorator is MonoHasElement, no matter what is contained in it(true /false), MonoHasElement would be printed anyway.
Correct ways to print signal (and data transmitted along with them) are:
Mono.log(), Mono.doOnEach/next(System.out::println) or System.out.println("monoPresent="+monoPresent.block());. Beware of third one: it will block whole thread until data is emitted, so use it only if you know what you are doing.
Example with Monos printing to play with:
Mono<String> abc = Mono.just("abc").delayElement(Duration.ofSeconds(99999999));
System.out.println(abc); //this will print MonoDelayElement instantly
System.out.println(abc.block()); //this will print 'abc', if you are patient enough ;^)
abc.subscribe(System.out::println); //this will also print 'abc' after 99999999 seconds, but without blocking current thread

Idiomatic way of verifying a reactive request before actually persisting to the database

I have an endpoint that accepts as well as returns a reactive type. What I'm trying to achieve is to somehow verify that the complete reactive request (that is actually an array of resources) is valid before persisting the changes to the database (read Full-Update of a ressource). The question is not so much concerned with how to actually verify the request but more with how to chain the steps together using which of springs reactive handler methods (map, flatMap and the likes) in the desired order which is basically:
verify correctness of request (the Ressource is properly annotated with JSR-303 annotations)
clear the current resource in case of valid request
persist new resources in the database after clearing the database
Let's assume the following scenario:
val service : ResourceService
#PostMapping("/resource/")
fun replaceResources(#Valid #RequestBody resources:
Flux<RessourceDto>): Flux<RessourceDto> {
var deleteWrapper = Mono.fromCallable {
service.deleteAllRessources()
}
deleteWrapper = deleteWrapper.subscribeOn(Schedulers.elastic())
return deleteWrapper.thenMany<RessourceDto> {
resources
.map(mapper::map) // map to model object
.flatMap(service::createResource)
.map(mapper::map) // map to dto object
.subscribeOn(Schedulers.parallel())
}
}
//alternative try
#PostMapping("/resourceAlternative/")
override fun replaceResourcesAlternative2(#RequestBody resources:
Flux<ResourceDto>): Flux<ResourceDto> {
return service.deleteAllResources()
.thenMany<ResourceDto> {
resources
.map(mapper::map)
.flatMap(service::createResource)
.map(mapper::map)
}
}
Whats the idiomatic way of doing this in a reactive fashion?

How to handle empty event in Spring reactor

Well, this sounds counter-intuitive to what reactive programming is, but I am unable to comprehend a way to handle nulls/exceptions.
private static class Data {
public Mono<String> first() {
return Mono.just("first");
}
public Mono<String> second() {
return Mono.just("second");
}
public Mono<String> empty() {
return Mono.empty();
}
}
I understand that fundamentally unless a publisher publishes an event, a subscriber will not act. So a code like this would work.
Data data = new Data();
data.first()
.subscribe(string -> Assertions.assertThat(string).isEqualTo("first"));
And if the first call returns empty, I can do this.
Data data = new Data();
data.empty()
.switchIfEmpty(data.second())
.subscribe(string -> Assertions.assertThat(string).isEqualTo("second"));
But how do I handle a case when both the calls return empty (typically this is an exception scenario that would need to be propagated to the user).
Data data = new Data();
data.empty()
.switchIfEmpty(data.empty())
.handle((string, sink) -> Objects.requireNonNull(string))
.block();
The handle is not called in the above example since no event was published.
as JB Nizet pointed out, you can chain in a second switchIfEmpty with a Mono.error.
Or, if you're fine with a NoSuchElementException, you could chain in single(). It enforces a strong contract of exactly one element, otherwise propagating that standard exception.

Resources