Can I access the request/response body on an ExchangeFilterFunction? - spring

Given an exchange using WebClient, filtered by a custom ExchangeFilterFunction:
#Override
public Mono<ClientResponse> filter(ClientRequest request, ExchangeFunction next) {
return next.exchange(request)
.doOnSuccess(response -> {
// ...
});
}
Trying to access the response body more than once using response.bodyToMono() will cause the underlying HTTP client connector to complain that only one receiver is allowed. AFAIK, there's no way to access the body's Publisher in order to cache() its signals (and I'm not sure it'd be a good idea, resource-wise), as well as no way to mutate or decorate the response object in a manner that allows access to its body (like it's possible with ServerWebExchange on the server side).
That makes sense, but I am wondering if there are any ways I could subscribe to the response body's publisher from a form of filter such as this one. My goal is to log the request/response being sent/received by a given WebClient instance.
I am new to reactive programming, so if there are any obvious no-nos here, please do explain :)

Only for logging you could add a wiretap to the HttpClient as desribed in this answer.
However, your question is also interesting in a more general sense outside of logging.
One possible way is to create a duplicate of the ClientResponse instance with a copy of the previous request body. This might go against reactive principles, but it got the job done for me and I don't see big downsides given the small size of the response bodies in my client.
In my case, I needed to do so because the server sending the request (outside of my control) uses the HTTP status 200 Ok even if requests fail. Therefore, I need to peek into the response body in order to find out if anything went wrong and what the cause was. In my case I evict a session cookie in the request headers from the cache if the error message indicates that the session expired.
These are the steps:
Get the response body as a Mono of a String (cf (1)).
Return a Mono.Error in case an error is detected (cf (2)).
Use the String of the response body to build a copy of the original response (cf (3)).
You could also use a dependency on the ObjectMapper to parse the String into an object for analysis.
Note that I wrote this in Kotlin but it should be easy enough to adapt to Java.
#Component
class PeekIntoResponseBodyExchangeFilterFunction : ExchangeFilterFunction {
override fun filter(request: ClientRequest, next: ExchangeFunction): Mono<ClientResponse> {
return next.exchange(request)
.flatMap { response ->
// (1)
response.bodyToMono<String>()
.flatMap { responseBody ->
if (responseBody.contains("Error message")) {
// (2)
Mono.error(RuntimeException("Response contains an error"))
} else {
// (3)
val clonedResponse = response.mutate().body(responseBody).build()
Mono.just(clonedResponse)
}
}
}
}
}

Related

Access Spring WebClient response body **BEFORE** being parsed

I've got a problem with an URL call response encoding.
So, just before Spring's WebClient converts the body response into an String object, as desired, I need to access the raw body response to parse it with the proper encoding. So, just before:
<execution>.flatMap(servletResponse -> {
Mono<String> mono = servletResponse.bodyToMono(String.class);
}
I need to access the raw URL call response; I think before "flatMap".
So... I've been looking at "codecs" within Spring documentation, but... So; even for testing, I have:
myWriter is an instance of: EncoderHttpMessageWriter.
myReader is an instance of: DecoderHttpMessageReader.
myWriter handles myDecoder, an instance of Decoder.
myReader handles myEncoder, an instance of Encoder.
as per Spring Documentation about Codecs; and testing with both options for the WebClient Builder:
myWebClientBuilder = WebClient.Builder; // provided by Spring Context,
myWebClientBuilder = WebClient.builder(); // "by hand"
So, the relevant part of code looks like this (tried even with register and registerWithDefaultConfig):
WebClient.builder().codecs(consumer -> {
consumer.customCodecs().register(myWriter.getEncoder());
consumer.customCodecs().register(myWriter);
consumer.customCodecs().register(myReader.getDecoder());
consumer.customCodecs().register(myReader);
})
.build();
shows that the codecs load, and internal basic methods are called:
canDecode
canEncode
canRead
canWrite
getDecodableMimeTypes
getDecoder
getEncodableMimeTypes
getEncoder
but... No one of the read/write... Mono<T> / Flux<T> methods are used. Is there anything left for configuring a codec to properly parse the incoming response with the proper encoding?
The response is a String; a old-fashioned String, with all data-fields in a single row, that wll be parsed later, according to rules about positions and lengths of fields; nothing related with JSON or Jackson.
Is there another better way to perform this pre-action?
Many thanks in advance.

Mirror #RequestPart behavior in WebFlux functional router definitions with different content types

Problem
We're developing a Spring Boot service to upload data to different back end databases. The idea is that, in one multipart/form-data request a user will send a "model" (basically a file) and "modelMetadata" (which is JSON that defines an object of the same name in our code).
We got the below to work in the WebFlux annotated controller syntax, when the user sends the "modelMetadata" in the multipart form with the content-type of "application/json":
#PostMapping(consumes = [MediaType.MULTIPART_FORM_DATA_VALUE], produces = [MediaType.APPLICATION_JSON_VALUE])
fun saveModel(#RequestPart("modelMetadata") monoModelMetadata: Mono<ModelMetadata>,
#RequestPart("model") monoModel: Mono<FilePart>,
#RequestHeader headers: HttpHeaders) : Mono<ResponseEntity<ModelMetadata>> {
return modelService.saveModel(monoModelMetadata, monoModel, headers)
}
But we can't seem to figure out how to do the same thing in Webflux's functional router definition. Below are the relevant code snippets we have:
#Bean
fun modelRouter() = router {
accept(MediaType.MULTIPART_FORM_DATA).nest {
POST(ROOT, handler::saveModel)
}
}
fun saveModel(r: ServerRequest): Mono<ServerResponse> {
val headers = r.headers().asHttpHeaders()
val monoModelPart = r.multipartData().map { multiValueMap ->
it["model"] // What do we do with this List<Part!> to get a Mono<FilePart>
it["modelMetadata"] // What do we do with this List<Part!> to get a Mono<ModelMetadata>
}
From everything we've read, we should be able to replicate the same functionality found in the annotation controller syntax with the router functional syntax, but this particular aspect doesn't seem to be well documented. Our goal was to move over to use the new functional router syntax since this is a new application we're developing and there are some nice forward thinking features/benefits as described here.
What we've tried
Googling to the ends of the Earth for a relevant example
this is a similar question, but hasn't gained any traction and doesn't relate to our need to create an object from one piece of the multipart request data
this may be close to what we need for uploading the file component of our multipart request data, but doesn't handle the object creation from JSON
Tried looking at the #RequestPart annotation code to see how things are done on that side, there's a nice comment that seems to hint at how they are converting the parts to objects, but we weren't able to figure out where that code lives or any relevant example of how to use an HttpMessageConverter on the ``
the content of the part is passed through an {#link HttpMessageConverter} taking into consideration the 'Content-Type' header of the request part.
Any and all help would be appreciated! Even just some links for us to better understand Part/FilePart types and there role in multipart requests would be helpful!
I was able to come up with a solution to this issue using an autowired ObjectMapper. From the below solution I could turn the modelMetadata and modelPart into Monos to mirror the #RequestPart return types, but that seems ridiculous.
I was also able to solve this by creating a MappingJackson2HttpMessageConverter and turning the metadataDataBuffer into a MappingJacksonInputMessage, but this solution seemed better for our needs.
fun saveModel(r: ServerRequest): Mono<ServerResponse> {
val headers = r.headers().asHttpHeaders()
return r.multipartData().flatMap {
// We're only expecting one Part of each to come through...assuming we understand what these Parts are
if (it.getOrDefault("modelMetadata", listOf()).size == 1 && it.getOrDefault("model", listOf()).size == 1) {
val modelMetadataPart = it["modelMetadata"]!![0]
val modelPart = it["model"]!![0] as FilePart
modelMetadataPart
.content()
.map { metadataDataBuffer ->
// TODO: Only do this if the content is JSON?
objectMapper.readValue(metadataDataBuffer.asInputStream(), ModelMetadata::class.java)
}
.next() // We're only expecting one object to be serialized from the buffer
.flatMap { modelMetadata ->
// Function was updated to work without needing the Mono's of each type
// since we're mapping here
modelService.saveModel(modelMetadata, modelPart, headers)
}
}
else {
// Send bad request response message
}
}
Although this solution works, I feel like it's not as elegant as the one alluded to in the #RequestPart annotation comments. Thus I will accept this as the solution for now, but if someone has a better solution please let us know and I will accept it!

Idiomatic way of verifying a reactive request before actually persisting to the database

I have an endpoint that accepts as well as returns a reactive type. What I'm trying to achieve is to somehow verify that the complete reactive request (that is actually an array of resources) is valid before persisting the changes to the database (read Full-Update of a ressource). The question is not so much concerned with how to actually verify the request but more with how to chain the steps together using which of springs reactive handler methods (map, flatMap and the likes) in the desired order which is basically:
verify correctness of request (the Ressource is properly annotated with JSR-303 annotations)
clear the current resource in case of valid request
persist new resources in the database after clearing the database
Let's assume the following scenario:
val service : ResourceService
#PostMapping("/resource/")
fun replaceResources(#Valid #RequestBody resources:
Flux<RessourceDto>): Flux<RessourceDto> {
var deleteWrapper = Mono.fromCallable {
service.deleteAllRessources()
}
deleteWrapper = deleteWrapper.subscribeOn(Schedulers.elastic())
return deleteWrapper.thenMany<RessourceDto> {
resources
.map(mapper::map) // map to model object
.flatMap(service::createResource)
.map(mapper::map) // map to dto object
.subscribeOn(Schedulers.parallel())
}
}
//alternative try
#PostMapping("/resourceAlternative/")
override fun replaceResourcesAlternative2(#RequestBody resources:
Flux<ResourceDto>): Flux<ResourceDto> {
return service.deleteAllResources()
.thenMany<ResourceDto> {
resources
.map(mapper::map)
.flatMap(service::createResource)
.map(mapper::map)
}
}
Whats the idiomatic way of doing this in a reactive fashion?

Handling exceptions and returning proper HTTP code with webflux

I am using the functional endpoints of WebFlux. I translate exceptions sent by the service layer to an HTTP error code using onErrorResume:
public Mono<String> serviceReturningMonoError() {
return Mono.error(new RuntimeException("error"));
}
public Mono<ServerResponse> handler(ServerRequest request) {
return serviceReturningMonoError().flatMap(e -> ok().syncBody(e))
.onErrorResume( e -> badRequest(e.getMessage()));
}
It works well as soon as the service returns a Mono. In case of a service returning a Flux, what should I do?
public Flux<String> serviceReturningFluxError() {
return Flux.error(new RuntimeException("error"));
}
public Mono<ServerResponse> handler(ServerRequest request) {
???
}
Edit
I tried the approach below, but unfortunately it doesn't work. The Flux.error is not handled by the onErrorResume and propagated to the framework. When the exception is unboxed during the serialization of the http response, Spring Boot Exception management catch it and convert it into a 500.
public Mono<ServerResponse> myHandler(ServerRequest request) {
return ok().contentType(APPLICATION_JSON).body( serviceReturningFluxError(), String.class)
.onErrorResume( exception -> badRequest().build());
}
I am actually surprised of the behaviour, is that a bug?
I found another way to solve this problem catching the exception within the body method and mapping it to ResponseStatusException
public Mono<ServerResponse> myHandler(ServerRequest request) {
return ok().contentType(MediaType.APPLICATION_JSON)
.body( serviceReturningFluxError()
.onErrorMap(RuntimeException.class, e -> new ResponseStatusException( BAD_REQUEST, e.getMessage())), String.class);
}
With this approach Spring properly handles the response and returns the expected HTTP error code.
Your first sample is using Mono (i.e. at most one value), so it plays well with Mono<ServerResponse> - the value will be asynchronously resolved in memory and depending on the result we will return a different response or handle business exceptions manually.
In case of a Flux (i.e. 0..N values), an error can happen at any given time.
In this case you could use the collectList operator to turn your Flux<String> into a Mono<List<String>>, with a big warning: all elements will be buffered in memory. If the stream of data is important of if your controller/client relies on streaming data, this is not the best choice here.
I'm afraid I don't have a better solution for this issue and here's why: since an error can happen at any time during the Flux, there's no guarantee we can change the HTTP status and response: things might have been flushed already on the network. This is already the case when using Spring MVC and returning an InputStream or a Resource.
The Spring Boot error handling feature tries to write an error page and change the HTTP status (see ErrorWebExceptionHandler and implementing classes), but if the response is already committed, it will log error information and let you know that the HTTP status was probably wrong.
Though this is an old question, I'd like to answer it for anyone who may stumble upon this Stack Overflow post.
There is another way to address this particular issue (discussed below), without the need to cache / buffer all the elements in memory as detailed in one of the other answers. However, the approach shown below does have a limitation. First, I'll discuss the approach, then the limitation.
The approach
You need to first convert your cold flux into a hot flux. Then on the hot flux call .next(), to return a Mono<Your Object> On this mono, call .flatMap().switchIfEmpty().onErrorResume(). In the flatMap() concatenate the returned Your Object with the hot flux stream.
Here's the original code snippet posted in the question, modified to achieve what is needed:
public Flux<String> serviceReturningFluxError()
{
return Flux.error(new RuntimeException("error"));
}
public Mono<ServerResponse> handler(ServerRequest request)
{
Flux<String> coldStrFlux = serviceReturningFluxError();
// The following step is a very important step. It converts the cold flux
// into a hot flux.
Flux<String> hotStrFlux = coldStrFlux.publish().refCount(1, Duration.ofSeconds(2));
return hotStrFlux.next()
.flatMap( firstStr ->
{
Flux<String> reCombinedFlux = Mono.just(firstStr)
.concatWith(hotStrFlux);
return ServerResponse.ok()
.contentType(MediaType.APPLICATION_JSON)
.body(reCombinedFlux, String.class);
}
)
.switchIfEmpty(
ServerResponse.notFound().build()
)
.onErrorResume( throwable -> ServerResponse.badRequest().build() );
}
The reason for converting from cold to hot Flux is that by doing so, a second redundant HTTP request is not made.
For a more detailed answer please refer to the following Stack Over post, where I've commented upon this in greater detail:
Return relevant ServerResponse in case of Flux.error
Limitation
While the above approach will work for exceptions / Flux.error() streams returned from the service, it will not work for any exceptions that may arise while emitting the individual elements from the flux after the first element is successfully emitted.
The assumption in the above code is simple. If the service throws an exception, then the very first element returned from the service will be a Flux.error() element. This approach does not account for the fact that exceptions may be thrown in the returned Flux stream after the first element, say possibly due to some network connection issue that occurs after the first few elements are already emitted by the Flux stream.

Can an Owin Middleware return a response earlier than the Invoke method returns?

I have the following middleware code:
public class UoWMiddleware : OwinMiddleware
{
readonly IUoW uow;
public UoWMiddleware(OwinMiddleware next, IUoW uow) : base(next)
{
this.uow = uow;
}
public override async Task Invoke(IOwinContext context)
{
try
{
await Next.Invoke(context);
}
catch
{
uow.RollBack();
throw;
}
finally
{
if (uow.Status == Base.SharedDomain.UoWStatus.Running)
{
var response = context.Response;
if (response.StatusCode < 400)
{
Thread.Sleep(1000);
uow.Commit();
}
else
uow.RollBack();
}
}
}
}
Occasionally we observe that the response returns to client before calling uow.Commit() via fiddler. For example we put a break point to uow.Commit and we see the response is returned to client despite that we are on the breakpoint waiting. This is somewhat unexpected. I would think the response will strictly return after the Invoke method ends. Am I missing something?
In Owin/Katana the response body (and, of course, the headers) are sent to the client at the precise moment when a middleware calls Write on the Response object of the IOwinContext.
This means that if your next middleware is writing the response body your client will receive it before your server-side code returns from the call to await Next.Invoke().
That's how Owin is designed, and depends on the fact that the Response stream may be written just once in a single Request/Response life-cycle.
Looking at your code, I can't see any major problem in such behavior, because you are simply reading the response headers after the response is written to the stream, and thus not altering it.
If, instead, you require to alter the response written by your next middleware, or you strictly need to write the response after you execute further logic server-side, then your only option is to buffer the response body into a memory stream, and than copy it into the real response stream (as per this answer) when you are ready.
I have successfully tested this approach in a different use case (but sharing the same concept) that you may find looking at this answer: https://stackoverflow.com/a/36755639/3670737
Reference:
Changing the response object from OWIN Middleware

Resources