Webclient raises exceptions on responses with non-200 status codes. How can I return the response anyway? - spring

When using Spring Webclient, any non-200 status code immediately throws an exception. I do not want this behavior.
I have the following API Service:
fun createSomething(body: String): SomeCustomResponse {
// Get signedRequest from another service (see below for that code)
return signedRequest
.retrieve()
.bodyToMono(SomeCustomResponse::class.java)
.block()
}
My response class intentionally contains an error field:
data class SomeCustomResponse(
val message: String? = null,
val error: String? = null
)
I want to pass the errors returned from this request to the front end, so the user can see the error messages. As long as the status code is 200, that works exactly as it should, but any non-200 status code throws immediately.
I've tried using .onStatus, but you can only return Throwable Monos using that, which I do not want to do. That attempted solution looks like this:
return signedRequest
.retrieve()
.onStatus(HttpStatus::is4xxClientError) {
it.bodyToMono(SomeCustomResponse::class.java)
}
.bodyToMono(SomeCustomResponse::class.java)
.block()
But that doesn't compile, complaining about:
Type mismatch: inferred type is Mono<SomeCustomResponse!>! but Mono<out Throwable!>! was expected
Type mismatch: inferred type is SomeCustomResponse! but Throwable! was expected
The signedRequest value is of type RequestHeadersSpec<*>, which is an AWS v4 signed API request. I cannot put custom logic into that service, as it is used for any number of other APIs.
For reference though, that code looks like this:
// Generate Auth headers and then:
return WebClient.create(baseUrl)
.method(method)
.uri { builder: UriBuilder -> builder.path(resourcePath).build() }
.header("X-Amz-Date", headers["X-Amz-Date"])
.header("Authorization", headers["Authorization"])
.body(Mono.just(body), String::class.java)
How can I bypass the default behavior of throwing exceptions here? I just want to return the response exactly as it currently does, regardless of status code.

According to the ResponseSpec.onStatus docs:
To suppress the treatment of a status code as an error and process it as a normal response, return Mono.empty() from the function. The response will then propagate downstream to be processed.
return signedRequest
.retrieve()
.onStatus(HttpStatus::is4xxClientError) {
Mono.empty()
}
.bodyToMono(SomeCustomResponse::class.java)
.block()
Bear in mind though, that if the error is a "real one" (e.g. the URL indeed does not exist) and the response body is not parseable, the method will throw a pretty misleading deserialisation exception.

Related

How to rewrite this code to use retrieve() method?

How to rewrite this code to use retrieve() method?
suspend fun getEntity(): ResponseEntity<EntityDto> {
return restClient
.get()
.uri { ... }
.accept(MediaType.APPLICATION_JSON)
.awaitExchange()
.awaitEntity()
}
When I use retrieve() I receive the following error:
org.springframework.web.reactive.function.UnsupportedMediaTypeException:
Content type 'application/json;charset=UTF-8' not supported for
bodyType=org.springframework.http.ResponseEntity
Your specified content type does not match with the content type in the response. Check the content type from the response and use appropriate one for .accept(MediaType.APPLICATION_JSON). Also check if error has occurred during that call, so that whole expected response do not match because of the error.

How to mock webclient in Kotlin and spring boot for unit tests with mockk framework?

I have the following piece of code in Kotlin (using WebFlux), which I wanna test:
fun checkUser(user: People.User?): Mono<Unit> =
if (user==null) {
Mono.empty()
} else {
webClient.get().uri {
uriBuilder -> uriBuilder
//... building a URI
}.retrieve().bodyToMono(UserValidationResponse::class.java)
.doOnError {
//log something
}.map {
if (!item.isUserValid()) {
throw InvalidUserException()
}
}
}
My unit test so far looks like this:
#Test
fun `Returns error when user is invalid`() {
val user = People.User("name", "lastname", "street", "zip code")
//when
StepVerifier.create(checkUser(user))
//then
.expectError(InvalidUserException::class.java)
.verify()
}
However when I run it, it throw the following error:
io.mockk.MockKException: no answer found for: WebClient(#1).get()
at io.mockk.impl.stub.MockKStub.defaultAnswer(MockKStub.kt:90)
at io.mockk.impl.stub.MockKStub.answer(MockKStub.kt:42)
at io.mockk.impl.recording.states.AnsweringState.call(AnsweringState.kt:16)
at io.mockk.impl.recording.CommonCallRecorder.call(CommonCallRecorder.kt:53)
at io.mockk.impl.stub.MockKStub.handleInvocation(MockKStub.kt:263)
at io.mockk.impl.instantiation.JvmMockFactoryHelper$mockHandler$1.invocation(JvmMockFactoryHelper.kt:25)
at io.mockk.proxy.jvm.advice.Interceptor.call(Interceptor.kt:20)
I guess the error occurs because I havent mocked WebClient(#1).get() but I am not sure how to mock it. So far I have tried:
every { webClient.get() } returns WebClient.RequestHeadersUriSpec
but it doesnt compile. The error says:
Classifier 'RequestHeadersUriSpec' does not have a companion object, and thus must be initialized here
Someone knows how I can mock WebClient(#1).get()? Thanks in advance
Basically you need something like this:
mock ResponseSpec - mock the body or error in whichever way you need for the respective test case
mock RequestHeadersUriSpec - let the retrieve() method return the ResponseSpec mock
mock WebClient - let the get() method return the RequestHeadersUriSpec mock
Here is a full example:
val response = mockk<WebClient.ResponseSpec>()
val spec = mockk<WebClient.RequestHeadersUriSpec<*>>()
val client = mockk<WebClient>()
every { response.bodyToMono(String::class.java) } returns Mono.just("Hello StackOverflow")
every { spec.retrieve() } returns response
every { client.get() } returns spec
println(client.get().retrieve().bodyToMono(String::class.java).block())
This will correctly print the Hello StackOverflow string.
Though it may be a "historical" question, I actually also had this problem recently.
Just as what Krause mentioned, the full call path of WebClient should be mocked. This means the method stream in every{} block should as the same as WebClient call. In your case, it may be something like
every{webClient.get().uri {???}.retrieve().bodyToMono(???)} returns Mono.just(...)
The next question is something about the error message io.mockk.MockKException: no answer found for: RequestBodyUriSpec(#3).uri(......). The key to the question is methods with parameters and without parameters are totally different things.
Thus, for target method, a uri(Function<UriBuilder, URI> uriFunction) is called(a lambda expression is used here to instead of Function interface). However, for mock method, a uri() method without any parameter is called. This is why the error message said , "no answer found for ...". Therefore, in order to match the mocked method, the code should be:
every{webClient.get().uri(any<java.util.function.Function<UriBuilder, URI>>()).retrieve().bodyToMono(???)} returns Mono.just(...)
Or, the any() method can be changed to the real URI which should be as the same as the target method.
Similarly, bodyToMono() should also be mocked with the correct parameter, which may be bodyToMono(any<ParameterizedTypeReference<*>>()).
Finally, the mock code may look like:
every{client.get()
.uri(any<java.util.function.Function<UriBuilder, URI>>())
.retrieve().bodyToMono(any<ParameterizedTypeReference<*>>())}
return Mono.just(...)

Handling exceptions and returning proper HTTP code with webflux

I am using the functional endpoints of WebFlux. I translate exceptions sent by the service layer to an HTTP error code using onErrorResume:
public Mono<String> serviceReturningMonoError() {
return Mono.error(new RuntimeException("error"));
}
public Mono<ServerResponse> handler(ServerRequest request) {
return serviceReturningMonoError().flatMap(e -> ok().syncBody(e))
.onErrorResume( e -> badRequest(e.getMessage()));
}
It works well as soon as the service returns a Mono. In case of a service returning a Flux, what should I do?
public Flux<String> serviceReturningFluxError() {
return Flux.error(new RuntimeException("error"));
}
public Mono<ServerResponse> handler(ServerRequest request) {
???
}
Edit
I tried the approach below, but unfortunately it doesn't work. The Flux.error is not handled by the onErrorResume and propagated to the framework. When the exception is unboxed during the serialization of the http response, Spring Boot Exception management catch it and convert it into a 500.
public Mono<ServerResponse> myHandler(ServerRequest request) {
return ok().contentType(APPLICATION_JSON).body( serviceReturningFluxError(), String.class)
.onErrorResume( exception -> badRequest().build());
}
I am actually surprised of the behaviour, is that a bug?
I found another way to solve this problem catching the exception within the body method and mapping it to ResponseStatusException
public Mono<ServerResponse> myHandler(ServerRequest request) {
return ok().contentType(MediaType.APPLICATION_JSON)
.body( serviceReturningFluxError()
.onErrorMap(RuntimeException.class, e -> new ResponseStatusException( BAD_REQUEST, e.getMessage())), String.class);
}
With this approach Spring properly handles the response and returns the expected HTTP error code.
Your first sample is using Mono (i.e. at most one value), so it plays well with Mono<ServerResponse> - the value will be asynchronously resolved in memory and depending on the result we will return a different response or handle business exceptions manually.
In case of a Flux (i.e. 0..N values), an error can happen at any given time.
In this case you could use the collectList operator to turn your Flux<String> into a Mono<List<String>>, with a big warning: all elements will be buffered in memory. If the stream of data is important of if your controller/client relies on streaming data, this is not the best choice here.
I'm afraid I don't have a better solution for this issue and here's why: since an error can happen at any time during the Flux, there's no guarantee we can change the HTTP status and response: things might have been flushed already on the network. This is already the case when using Spring MVC and returning an InputStream or a Resource.
The Spring Boot error handling feature tries to write an error page and change the HTTP status (see ErrorWebExceptionHandler and implementing classes), but if the response is already committed, it will log error information and let you know that the HTTP status was probably wrong.
Though this is an old question, I'd like to answer it for anyone who may stumble upon this Stack Overflow post.
There is another way to address this particular issue (discussed below), without the need to cache / buffer all the elements in memory as detailed in one of the other answers. However, the approach shown below does have a limitation. First, I'll discuss the approach, then the limitation.
The approach
You need to first convert your cold flux into a hot flux. Then on the hot flux call .next(), to return a Mono<Your Object> On this mono, call .flatMap().switchIfEmpty().onErrorResume(). In the flatMap() concatenate the returned Your Object with the hot flux stream.
Here's the original code snippet posted in the question, modified to achieve what is needed:
public Flux<String> serviceReturningFluxError()
{
return Flux.error(new RuntimeException("error"));
}
public Mono<ServerResponse> handler(ServerRequest request)
{
Flux<String> coldStrFlux = serviceReturningFluxError();
// The following step is a very important step. It converts the cold flux
// into a hot flux.
Flux<String> hotStrFlux = coldStrFlux.publish().refCount(1, Duration.ofSeconds(2));
return hotStrFlux.next()
.flatMap( firstStr ->
{
Flux<String> reCombinedFlux = Mono.just(firstStr)
.concatWith(hotStrFlux);
return ServerResponse.ok()
.contentType(MediaType.APPLICATION_JSON)
.body(reCombinedFlux, String.class);
}
)
.switchIfEmpty(
ServerResponse.notFound().build()
)
.onErrorResume( throwable -> ServerResponse.badRequest().build() );
}
The reason for converting from cold to hot Flux is that by doing so, a second redundant HTTP request is not made.
For a more detailed answer please refer to the following Stack Over post, where I've commented upon this in greater detail:
Return relevant ServerResponse in case of Flux.error
Limitation
While the above approach will work for exceptions / Flux.error() streams returned from the service, it will not work for any exceptions that may arise while emitting the individual elements from the flux after the first element is successfully emitted.
The assumption in the above code is simple. If the service throws an exception, then the very first element returned from the service will be a Flux.error() element. This approach does not account for the fact that exceptions may be thrown in the returned Flux stream after the first element, say possibly due to some network connection issue that occurs after the first few elements are already emitted by the Flux stream.

Can I access the request/response body on an ExchangeFilterFunction?

Given an exchange using WebClient, filtered by a custom ExchangeFilterFunction:
#Override
public Mono<ClientResponse> filter(ClientRequest request, ExchangeFunction next) {
return next.exchange(request)
.doOnSuccess(response -> {
// ...
});
}
Trying to access the response body more than once using response.bodyToMono() will cause the underlying HTTP client connector to complain that only one receiver is allowed. AFAIK, there's no way to access the body's Publisher in order to cache() its signals (and I'm not sure it'd be a good idea, resource-wise), as well as no way to mutate or decorate the response object in a manner that allows access to its body (like it's possible with ServerWebExchange on the server side).
That makes sense, but I am wondering if there are any ways I could subscribe to the response body's publisher from a form of filter such as this one. My goal is to log the request/response being sent/received by a given WebClient instance.
I am new to reactive programming, so if there are any obvious no-nos here, please do explain :)
Only for logging you could add a wiretap to the HttpClient as desribed in this answer.
However, your question is also interesting in a more general sense outside of logging.
One possible way is to create a duplicate of the ClientResponse instance with a copy of the previous request body. This might go against reactive principles, but it got the job done for me and I don't see big downsides given the small size of the response bodies in my client.
In my case, I needed to do so because the server sending the request (outside of my control) uses the HTTP status 200 Ok even if requests fail. Therefore, I need to peek into the response body in order to find out if anything went wrong and what the cause was. In my case I evict a session cookie in the request headers from the cache if the error message indicates that the session expired.
These are the steps:
Get the response body as a Mono of a String (cf (1)).
Return a Mono.Error in case an error is detected (cf (2)).
Use the String of the response body to build a copy of the original response (cf (3)).
You could also use a dependency on the ObjectMapper to parse the String into an object for analysis.
Note that I wrote this in Kotlin but it should be easy enough to adapt to Java.
#Component
class PeekIntoResponseBodyExchangeFilterFunction : ExchangeFilterFunction {
override fun filter(request: ClientRequest, next: ExchangeFunction): Mono<ClientResponse> {
return next.exchange(request)
.flatMap { response ->
// (1)
response.bodyToMono<String>()
.flatMap { responseBody ->
if (responseBody.contains("Error message")) {
// (2)
Mono.error(RuntimeException("Response contains an error"))
} else {
// (3)
val clonedResponse = response.mutate().body(responseBody).build()
Mono.just(clonedResponse)
}
}
}
}
}

Can I define a grpc call with a null request or response?

Does the rpc syntax in proto3 allow null requests or responses?
e.g. I want the equivalent of the following:
rpc Logout;
rpc Status returns (Status);
rpc Log (LogData);
Or should I just create a null type?
message Null {};
rpc Logout (Null) returns (Null);
rpc Status (Null) returns (Status);
rpc Log (LogData) returns (Null);
Kenton's comment below is sound advice:
... we as developers are really bad at guessing what we might want in the future. So I recommend being safe by always defining custom params and results types for every method, even if they are empty.
Answering my own question:
Looking through the default proto files, I came across Empty that is exactly like the Null type I suggested above :)
excerpt from that file:
// A generic empty message that you can re-use to avoid defining duplicated
// empty messages in your APIs. A typical example is to use it as the request
// or the response type of an API method. For instance:
//
// service Foo {
// rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty);
// }
//
message Empty {
}
You also can use predefined:
import "google/protobuf/empty.proto";
package MyPackage;
service MyService {
rpc Check(google.protobuf.Empty) returns (google.protobuf.Empty) {}
}
you can also use another bool property inside the Reply structure. like this
message Reply {
string result = 1;
bool found = 2;
}
so if you dont find the result or some error happened you can return from the service class this
return new Reply()
{
Found = false
};

Resources