While implementing an authentication solution based on Spring Security Reactive, I faced an issue where the operations in the chain get duplicated at some point. From that, everything was called twice.
The culprit was the operator .transform at one point of the chain. After editing the called method and replacing the operator by .flatMap, the issue was resolved and everything was only called once.
The question
According to the operator's documentation, the
function is applied to an original operator chain at assembly time to augment it with the encapsulated operators
and
is basically equivalent to chaining the operators directly.
Why did the operator .transform trigger a second subscription to the chain, then ?
The context
This authentication flow takes a trusted username and retrieves its details from a webservice.
The authentication method to implement the ReactiveAuthenticationManager :
#Override
public Mono<Authentication> authenticate(Authentication providedAuthentication) {
String username = (String) providedAuthentication.getPrincipal();
String token = (String) providedAuthentication.getCredentials();
return Mono.just(providedAuthentication)
.doOnNext(x -> LOGGER.debug("Starting authentication of user {}", x))
.doOnNext(AuthenticationValidator.validateProvided)
.then(ReactiveSecurityContextHolder.getContext())
.map(SecurityContext::getAuthentication)
.flatMap(auth -> AuthenticationValidator.validateCoherence(auth, providedAuthentication))
.switchIfEmpty(Mono.defer(() -> {
LOGGER.trace("Switch if empty before retrieving user");
return retrieveUser(username, token);
}))
.doOnNext(logAccess);
}
The duplication of the calls started from the supplier of .switchIfEmpty until the end of the chain.
The method creating the Mono used by .switchIfEmpty :
private Mono<PreAuthenticatedAuthenticationToken> retrieveUser(String username, String token) {
return Mono.just(username)
.doOnNext(x -> LOGGER.trace("Before find by username"))
.then(habileUserDetails.findByUsername(username, token))
.cast(XXXUserDetails.class)
.transform(rolesProvider::provideFor)
.map(user -> new PreAuthenticatedAuthenticationToken(user, GlobalConfiguration.NO_CREDENTIAL, user.getAuthorities()))
.doOnNext(s -> LOGGER.debug("User data retrieved from XXX"));
}
The operator .transform on line 4 has been replaced by .flatMap to resolve the issue.
The original method called by the .transform operator :
public Mono<CompleteXXXUserDetails> provideFor(Mono<XXXUserDetails> user) {
return user
.map(XXXUserDetails::getAuthorities)
.map(l -> StreamHelper.transform(l, GrantedAuthority::getAuthority))
.map(matcher::match)
.map(enricher::enrich)
.map(l -> StreamHelper.transform(l, SimpleGrantedAuthority::new))
.zipWith(user, (authorities, userDetails)
-> CompleteXXXUserDetails.from(userDetails).withAllAuthorities(authorities));
}
Here is a trace of the execution :
DEBUG 20732 --- [ctor-http-nio-3] c.a.s.s.h.a.XXXAuthenticationManager : Starting authentication of user [REDACTED]
TRACE 20732 --- [ctor-http-nio-3] c.a.s.s.h.a.XXXAuthenticationManager : Switch if empty before retrieving user
TRACE 20732 --- [ctor-http-nio-3] c.a.s.s.h.a.XXXAuthenticationManager : Before find by username
TRACE 20732 --- [ctor-http-nio-3] c.a.s.s.xxx.user.UserRetriever : Between request and call
TRACE 20732 --- [ctor-http-nio-3] c.a.s.s.h.u.retriever.UserRetrieverV01: Calling webservice v01
TRACE 20732 --- [ctor-http-nio-3] c.a.s.s.h.a.XXXAuthenticationManager : Before find by username
TRACE 20732 --- [ctor-http-nio-3] c.a.s.s.xxx.user.UserRetriever : Between request and call
TRACE 20732 --- [ctor-http-nio-3] c.a.s.s.h.u.retriever.UserRetrieverV01: Calling webservice v01
For information, I'm using Spring Boot 2.1.2.RELEASE.
This answer doesn't address the root cause but rather explains how a transform could be applied several times when subscribed to several times, which is not the case in OP's issue. Edited the original text into a quote.
That statement is only valid when the transform is applied as a top-level operator in the chain you subscribe to. Here you are applying it within retrieveUser, which is invoked inside a Mono.defer (which goal is to execute that code for each different subscription).
(edit:) so if that defer is subscribed to x times, the transform Function will be applied x times as well.
compose is basically transform-inside-a-defer by the way.
The issue is in the fact that you do a user.whatever(...).zipWith(user, ...).
With transform, this translates to:
Mono<XXXUserDetails> user = Mono.just(username)
.doOnNext(x -> LOGGER.trace("Before find by username"))
.then(habileUserDetails.findByUsername(username, token))
.cast(XXXUserDetails.class);
return user.wathewer(...)
.zipWith(user, ...);
Whereas with flatMap I assume you did something to the effect of flatMap(u -> provideFor(Mono.just(u))? If so, that would translate to:
Mono<XXXUserDetails> user = Mono.just(username)
.doOnNext(x -> LOGGER.trace("Before find by username"))
.then(habileUserDetails.findByUsername(username, token))
.cast(XXXUserDetails.class);
return user.flatMap(u -> {
Mono<XXXUserDetails> capture = Mono.just(u);
return capture.whatever(...)
.zipWith(capture, ...);
}
You can see how both subscribe twice to a Mono<XXXUserDetails due to zipWith.
The reason is seems to subscribe once with flatMap is because it captures the output of the upstream pipeline and applies the provideFor function on that capture. The capture (Mono.just(u)) is subscribed twice but acts as a cache and doesn't bear any logic / logs / etc...
With transform, there is no capture. The provideFor function is applied directly to the upstream pipeline, which makes the fact that it subscribes twice quite visible.
Related
I have my Spring app configured to use a GET parameter for content negotiation. Code is Kotlin but would work the same in Java.
Config:
override fun configureContentNegotiation(configurer: ContentNegotiationConfigurer) {
configurer.favorParameter(true)
.parameterName("format")
.ignoreAcceptHeader(false)
.defaultContentType(MediaType.APPLICATION_JSON)
.mediaType("text/plain", MediaType.TEXT_PLAIN)
.mediaType("application/json", MediaType.APPLICATION_JSON)
.mediaType("application/rdf+xml", MediaType("application", "rdf+xml"))
}
And the following controller methods:
#GetMapping("/test", produces=["text/plain"])
fun testText() : String {
return "Hello"
}
#GetMapping("/test", produces=["application/json"])
fun testJson() : Map<String, String> {
return mapOf("hello" to "world")
}
#GetMapping("/test", produces=["application/rdf+xml"])
fun testRdf(response: HttpServletResponse) {
// dummy response, to demonstrate using output stream.
response.setContentType("application/rdf+xml")
response.outputStream.write("dummy data".toByteArray())
response.outputStream.close()
}
testRdf returns void and uses an output stream to send body data back.
The following works just fine:
http://localhost:8080/test?format=text/plain gives me the plain text
http://localhost:8080/test?format=application/json gives me the JSON
But http://localhost:8080/test?format=application/rdf+xml gives me an HTTP 406 and the logs say
org.apache.tomcat.util.http.Parameters : Start processing with input [format=application/rdf+xml]
o.s.web.servlet.DispatcherServlet : GET "/test?format=application/rdf+xml", parameters={masked}
.w.s.m.s.DefaultHandlerExceptionResolver : Resolved [org.springframework.web.HttpMediaTypeNotAcceptableException: Could not find acceptable representation]
o.s.web.servlet.DispatcherServlet : Completed 406 NOT_ACCEPTABLE
A debugger shows that it doesn't even call my function.
(To prove that the testRdf handler does what's expected, I made the path unique and removed the produces annotation - it works fine outside content negotiation and returns the body as expected.)
As far as I can tell I have indicated that my method is the right handler for that content type, and I have registered the content type correctly.
Why does Spring not consider that my handler meets the content negotiation request?
I found the answer. The parameter characters that needed to be URL encoded, so this works fine:
http://localhost:8080/test?format=application%2Frdf%2Bxml
I am in the process of learning about Spring Reactive and have the following basic reactive demo code.
import org.springframework.web.reactive.function.client.WebClient;
// other imports etc
#Slf4j
class WebClientTests {
private static String baseUrl = "http://localhost:8080";
private static WebClient client = WebClient.create(baseUrl);
#Test
void testWebClient() {
Instant start = Instant.now();
Flux.just(1,2,3)
.map(i -> client.get().uri("/person/{id}", i).retrieve().bodyToFlux(Person.class))
.subscribe(s -> {
log.info("subscribed: {}", s);
});
log.info("Elapsed time: " + Duration.between(start, Instant.now()).toMillis() + "ms");
}
}
It outputs the following.
20:32:55.251 [main] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.targetRecords: 4
20:32:55.652 [main] INFO com.example.reactive.reactivedemo.WebClientTests - subscribed: MonoFlatMap
20:32:55.652 [main] INFO com.example.reactive.reactivedemo.WebClientTests - subscribed: MonoFlatMap
20:32:55.652 [main] INFO com.example.reactive.reactivedemo.WebClientTests - subscribed: MonoFlatMap
20:32:55.668 [main] INFO com.example.reactive.reactivedemo.WebClientTests - Elapsed time: 84ms
However I am unsure why its not outputting the value of the get request? Its not actually triggering the endpoint.
You almost certainly want to use flatMap(), not map() on your .map(i -> client.get().uri... line.
map() is used for synchronous transformations, where you're returning the actual value you want to map to. You're not returning an actual value - you're returning a publisher from your map method, so that publisher is just returned as is - it's never subscribed to, and since nothing happens until you subscribe, your web request never executes.
flatMap() is used for non-blocking transformations where you return a publisher that emits the value, or values, you want to map to. That publisher is subscribed to as part of your reactive chain, and the value emitted by that publisher passed down the chain to the next operator.
Similar to Spring Reactor: How to throw an exception when publisher emit a value?
I have a finder method in my DAO java findSomePojo which returns result SomePojo . The finder calls amazon db apis and the javasoftware.amazon.awssdk.services.dynamodb.model.GetItemResponse has output of call.
So I am trying this hasElement() check in my service layer createSomePojo method. (Not sure if I am using it correctly- Iwas trying and debugging)
Basically :
I want to check if there is already element, it is illegal to save and I would not call DAOs save. So I need to throw exception.
Assuming that there is already a record of SomePojo in DB, I try to invoke create_SomePjo of service .But I see in logs that filter is not working and is get NPE when reactor invokes createModel_SomePojo making me believe that somehow even after check filter it throws NPE
///service SomePjoService it has create_SomePojo, find_SomePojo etc
Mono<Void> create_SomePojo(reqPojo){
// Before calling DAO 's save I call serivice find (which basically calls DAOs find (Shown befow after this methid)
Mono<Boolean> monoPresent = find_SomePojo(accountId, contentIdExtn)
.filter(i -> i.getId() != null)
.hasElement();
System.out.println("monoPresent="+monoPresent.toString());
if(monoPresent.toString().equals("MonoHasElement")){
//*************it comes here i see that***********//
System.out.println("hrereee monoPresent="+monoPresent);
// Mono<Error> monoCheck=
return monoPresent.handle((next, sink) -> sink.error(new SomeException(ITEM_ALREADY_EXISTS))).then();
} else {
return SomePojoRepo.save(reqPojo).then();
}
}
Mono<SomePojo> find_SomePojo(id){
return SomePojoRepo.find(id);
}
==============================================================
///DAO : SomePojoRepo.java : it has save,find,delete
Mono<SomePojo> find( String id) {
Mono<SomePojo> fallback = Mono.empty();
Mono<GetItemResponse> monoFilteredResponse = monoFuture
.filter(getItemResponse -> getItemResponse.item().size() > 0&& getItemResponse!=null);
Mono<SomePojo> result = monoFilteredResponse
.map(getItemResponse -> createModel_SomePojo(getItemResponse.item()));
Mono<SomePojo> deferedResult = Mono.defer(() -> result.switchIfEmpty(fallback));
return deferedResult;
}
I see there is hasElement() method on Mono . Not sure how to correctly use it.
I can achieve exception if I call DAO save in my service create_SomePojo(reqPojo) directly without doing all this findner check because primary key constraint will take care and throw excpetion and I cna rethrow and then catch in service but what If I want to check in service and throw exception with error codes . The idea is not to pass response error object to dao layer .
Try to use Hooks.onOperatorDebug() hook to get better debugging experience.
Correct way to use hasElement (assuming that find_SomePojo never returns null)
Mono<Boolean> monoPresent = find_SomePojo(accountId, contentIdExtn)
.filter(i -> i.getId() != null)
.hasElement();
return monoPresent.flatMap(isPresent -> {
if(isPresent){
Mono.error(new SomeException(ITEM_ALREADY_EXISTS)));
}else{
SomePojoRepo.save(reqPojo);
}
}).then();
Sidenote
There is a common misconception about what Mono actually is. It does not hold any data - it's just a fragment of pipeline, which transmits signals and data flowing through it. Therefore, line System.out.println("monoPresent="+monoPresent.toString()); makes no sense, because it just prints the hasElements() decorator around the existsing pipeline. Internal name of this decorator is MonoHasElement, no matter what is contained in it(true /false), MonoHasElement would be printed anyway.
Correct ways to print signal (and data transmitted along with them) are:
Mono.log(), Mono.doOnEach/next(System.out::println) or System.out.println("monoPresent="+monoPresent.block());. Beware of third one: it will block whole thread until data is emitted, so use it only if you know what you are doing.
Example with Monos printing to play with:
Mono<String> abc = Mono.just("abc").delayElement(Duration.ofSeconds(99999999));
System.out.println(abc); //this will print MonoDelayElement instantly
System.out.println(abc.block()); //this will print 'abc', if you are patient enough ;^)
abc.subscribe(System.out::println); //this will also print 'abc' after 99999999 seconds, but without blocking current thread
I am using Spring projectreactor reactor-core 3.1.8.RELEASE. I am implementing a logging framework for my microservice to have JSON Audit logs, so used context to store certain fields such as userID, collaboration ID, component Name and few other fields that are common across request life-cycle. Since Threadlocal cannot be used in reactive services to stores these elements, I have to use the context. But getting a reference to the context is apparently very difficult. I can get a reference to the context from the Signal through the doOnEach function call and that's it. If I use doOnEach, it gets called for all signals types and I am unable to isolate on Error, success and so on. Moreover, if an error occurs in between, then all the subsequent doOnEach gets called anyway, so the logs get repeated with several onError log types.
There is very limited documentation regarding how to get a reference to the context object in Spring reactor. Any help regarding a better way to generate audit logs that has collaboration IDs and other request specific IDs stored and propagated across function calls and external invocations is appreciated.
Code Snippets -
In the WebFilter, I am setting few key-value pairs as follows -
override fun filter(exchange: ServerWebExchange, filterChain: WebFilterChain): Mono<Void> {
// add the context variables at the end of the chain as the context moves from
// downstream to upstream.
return filterChain.filter(exchange)
.subscriberContext { context ->
var ctx = context.put(RestRequestInfo::class.java, restRequestInfo(exchange))
ctx = ctx.put(COLLABORATION_ID, UUID.randomUUID().toString())
ctx=ctx.put(COMPONENT_NAME, "sample-component-name")
ctx=ctx.put(USER_NAME, "POSTMAN")
ctx
}
}
Then I want to use the key-value pairs added above in all the subsequent logs so that log aggregators like Splunk can get all the JSON logs associated with this particular request, based on collaboration ID. Right now, the only way to get values out of context is through doOnEach function call, where we get a handle to the SIgnal through which we get handle to context. But all doOnEach gets called during each and every events, irrespective of whether each function call was success or failure
return Mono.just(request)
.doOnEach(**Code to log with context data**)
.map(RequestValidations::validateRequest)
.doOnEach(**Code to log with context data**)
.map(RequestValidations::buildRequest)
.map(RequestValidations::validateQueryParameters)
.doOnEach(**Code to log with context data**)
.flatMap(coverageSummariesGateway::getCoverageSummaries)
.doOnEach(**Code to log with context data**)
.map({ coverageSummaries ->
getCoverageSummariesResponse(coverageSummaries, serviceReferenceId) })
.doOnEach(**Code to log with context data**)
.flatMap(this::renderSuccess)
.doOnEach(**Code to log with context data**)
.doOnError { logger.info("ERROR OCCURRED") }
Thank you!
You could do something like following:
return Mono.just(request)
.doOnEach(**Code to log with context data**)
.flatMap( r -> withMDC(r, RequestValidations::validateRequest))
following method will populate mapping diagnostic context (MDC) so you have it automatically in your logs (depends on you logging pattern). E.g. logback has %X{traceId} where traceId is a key in the tracingContext map.
public static <T, R> Mono<R> withMDC(T value, Function<T, Mono<R>> f) {
return Mono.subscriberContext()
.flatMap( ctx -> {
Optional<Map> tracingContext = ctx.getOrEmpty("tracing-context-key");
if (tracingContext.isPresent()) {
try {
MDC.setContextMap(tracingContext.get());
return f.apply(value);
} finally {
MDC.clear();
}
} else{
return f.apply(value);
}
});
}
It is not quite nice, hope it will be eventually improved by Logging frameworks and context will be injected automatically.
I am using the functional endpoints of WebFlux. I translate exceptions sent by the service layer to an HTTP error code using onErrorResume:
public Mono<String> serviceReturningMonoError() {
return Mono.error(new RuntimeException("error"));
}
public Mono<ServerResponse> handler(ServerRequest request) {
return serviceReturningMonoError().flatMap(e -> ok().syncBody(e))
.onErrorResume( e -> badRequest(e.getMessage()));
}
It works well as soon as the service returns a Mono. In case of a service returning a Flux, what should I do?
public Flux<String> serviceReturningFluxError() {
return Flux.error(new RuntimeException("error"));
}
public Mono<ServerResponse> handler(ServerRequest request) {
???
}
Edit
I tried the approach below, but unfortunately it doesn't work. The Flux.error is not handled by the onErrorResume and propagated to the framework. When the exception is unboxed during the serialization of the http response, Spring Boot Exception management catch it and convert it into a 500.
public Mono<ServerResponse> myHandler(ServerRequest request) {
return ok().contentType(APPLICATION_JSON).body( serviceReturningFluxError(), String.class)
.onErrorResume( exception -> badRequest().build());
}
I am actually surprised of the behaviour, is that a bug?
I found another way to solve this problem catching the exception within the body method and mapping it to ResponseStatusException
public Mono<ServerResponse> myHandler(ServerRequest request) {
return ok().contentType(MediaType.APPLICATION_JSON)
.body( serviceReturningFluxError()
.onErrorMap(RuntimeException.class, e -> new ResponseStatusException( BAD_REQUEST, e.getMessage())), String.class);
}
With this approach Spring properly handles the response and returns the expected HTTP error code.
Your first sample is using Mono (i.e. at most one value), so it plays well with Mono<ServerResponse> - the value will be asynchronously resolved in memory and depending on the result we will return a different response or handle business exceptions manually.
In case of a Flux (i.e. 0..N values), an error can happen at any given time.
In this case you could use the collectList operator to turn your Flux<String> into a Mono<List<String>>, with a big warning: all elements will be buffered in memory. If the stream of data is important of if your controller/client relies on streaming data, this is not the best choice here.
I'm afraid I don't have a better solution for this issue and here's why: since an error can happen at any time during the Flux, there's no guarantee we can change the HTTP status and response: things might have been flushed already on the network. This is already the case when using Spring MVC and returning an InputStream or a Resource.
The Spring Boot error handling feature tries to write an error page and change the HTTP status (see ErrorWebExceptionHandler and implementing classes), but if the response is already committed, it will log error information and let you know that the HTTP status was probably wrong.
Though this is an old question, I'd like to answer it for anyone who may stumble upon this Stack Overflow post.
There is another way to address this particular issue (discussed below), without the need to cache / buffer all the elements in memory as detailed in one of the other answers. However, the approach shown below does have a limitation. First, I'll discuss the approach, then the limitation.
The approach
You need to first convert your cold flux into a hot flux. Then on the hot flux call .next(), to return a Mono<Your Object> On this mono, call .flatMap().switchIfEmpty().onErrorResume(). In the flatMap() concatenate the returned Your Object with the hot flux stream.
Here's the original code snippet posted in the question, modified to achieve what is needed:
public Flux<String> serviceReturningFluxError()
{
return Flux.error(new RuntimeException("error"));
}
public Mono<ServerResponse> handler(ServerRequest request)
{
Flux<String> coldStrFlux = serviceReturningFluxError();
// The following step is a very important step. It converts the cold flux
// into a hot flux.
Flux<String> hotStrFlux = coldStrFlux.publish().refCount(1, Duration.ofSeconds(2));
return hotStrFlux.next()
.flatMap( firstStr ->
{
Flux<String> reCombinedFlux = Mono.just(firstStr)
.concatWith(hotStrFlux);
return ServerResponse.ok()
.contentType(MediaType.APPLICATION_JSON)
.body(reCombinedFlux, String.class);
}
)
.switchIfEmpty(
ServerResponse.notFound().build()
)
.onErrorResume( throwable -> ServerResponse.badRequest().build() );
}
The reason for converting from cold to hot Flux is that by doing so, a second redundant HTTP request is not made.
For a more detailed answer please refer to the following Stack Over post, where I've commented upon this in greater detail:
Return relevant ServerResponse in case of Flux.error
Limitation
While the above approach will work for exceptions / Flux.error() streams returned from the service, it will not work for any exceptions that may arise while emitting the individual elements from the flux after the first element is successfully emitted.
The assumption in the above code is simple. If the service throws an exception, then the very first element returned from the service will be a Flux.error() element. This approach does not account for the fact that exceptions may be thrown in the returned Flux stream after the first element, say possibly due to some network connection issue that occurs after the first few elements are already emitted by the Flux stream.