How to log spring-webflux WebClient request + response details (bodies, headers, elasped_time)? - spring-boot

Basically, I want to log a request/response informations in one log containing bodies/headers with a Spring WebClient.
With Spring RestTemplate we can do it with a ClientHttpRequestInterceptor. I find about ExchangeFilterFunction for Spring WebClient but haven't managed to do something similar in a clean way. We can use this filter and log the request AND THEN the response but I need both on the same log trace.
Moreover, I haven't managed to get the response body with ExchangeFilterFunction.ofResponseProcessor method.
I expect a log like this (current implementation working with a ClientHttpRequestInterceptor) with all the informations I need:
{
"#timestamp": "2019-05-14T07:11:29.089+00:00",
"#version": "1",
"message": "GET https://awebservice.com/api",
"logger_name": "com.sample.config.resttemplate.LoggingRequestInterceptor",
"thread_name": "http-nio-8080-exec-5",
"level": "TRACE",
"level_value": 5000,
"traceId": "e65634ee6a7c92a7",
"spanId": "7a4d2282dbaf7cd5",
"spanExportable": "false",
"X-Span-Export": "false",
"X-B3-SpanId": "7a4d2282dbaf7cd5",
"X-B3-ParentSpanId": "e65634ee6a7c92a7",
"X-B3-TraceId": "e65634ee6a7c92a7",
"parentId": "e65634ee6a7c92a7",
"method": "GET",
"uri": "https://awebservice.com/api",
"body": "[Empty]",
"elapsed_time": 959,
"status_code": 200,
"status_text": "OK",
"content_type": "text/html",
"response_body": "{"message": "Hello World!"}"
}
Does anyone manage to do something like this with Spring WebClient ? Or how would one proceed to track request/reponses issue with a Spring WebClient ?

You can use filter(), something like this:
this.webClient = WebClient.builder().baseUrl("your_url")
.filter(logRequest())
.filter(logResponse())
.build();
private ExchangeFilterFunction logRequest() {
return (clientRequest, next) -> {
log.info("Request: {} {}", clientRequest.method(), clientRequest.url());
clientRequest.headers()
.forEach((name, values) -> values.forEach(value -> log.info("{}={}", name, value)));
return next.exchange(clientRequest);
};
}
private ExchangeFilterFunction logResponse() {
return ExchangeFilterFunction.ofResponseProcessor(clientResponse -> {
log.info("Response: {}", clientResponse.headers().asHttpHeaders().get("property-header"));
return Mono.just(clientResponse);
});
}

I don't think you can call .bodyToMono twice (once in the filter and then again where you use the client), so you might not be able to log that in the filter. As for the other details...
The WebClient config:
#Configuration
class MyClientConfig {
#Bean
fun myWebClient(): WebClient {
return WebClient
.builder()
.baseUrl(myUrl)
.filter(MyFilter())
.build()
}
}
The filter:
class MyFilter : ExchangeFilterFunction {
override fun filter(request: ClientRequest, next: ExchangeFunction): Mono<ClientResponse> {
return next.exchange(request).flatMap { response ->
// log whenever you want here...
println("request: ${request.url()}, response: ${response.statusCode()}")
Mono.just(response)
}
}
}

Related

Webclient onStatus does not work in case of 406 returned from downstream API

I'm doing a onStatus implementation in my API when I use a webclient (Webflux) to call external API:
//Webclient Call
Flux<Movie> movies = webclient.get().uri(uriBuilder -> uriBuilder.path(api_url)
.build(author))
.retrieve()
.onStatus(HttpStatus::is4xxClientError,
response -> Mono.error(new AcceptHeaderNotsupportedException(response.statusCode().getReasonPhrase())))
.bodyToFlux(Movie.class)
//Global Handler Exception Class
public class GlobalExceptionHandler {
#ExceptionHandler(AcceptHeaderNotsupportedException.class)
public ResponseEntity<?> AcceptHeaderHandling(AcceptHeaderNotsupportedException exception){
ApiException apiException = new ApiException(HttpStatus.NOT_FOUND.value(), exception.getMessage());
return new ResponseEntity<>(ApiException, HttpStatus.NOT_FOUND);
}
}
//AcceptHeaderNotsupportedException Class
public class AcceptHeaderNotsupportedException extends RuntimeException{
public AcceptHeaderNotsupportedException(String message){
super(message);
}
}
//Api custom Exception
public class ApiCustomException{
private int code;
private String message;
}
I am testing a scenario webclient call that return a 406 error from downstream api. So i want to map the response to my object representation and give to my client (postman in this case).
{
code: 406,
"message": error from downstream api
}
but i am getting to client
{
"timestamp": "2021-08-29T14:31:00.944+00:00",
"path": "path",
"status": 406,
"error": "Not Acceptable",
"message": "Could not find acceptable representation",
"requestId": "ba66698f-1",
"trace": "org.springframework.web.server.NotAcceptableStatusException: 406 NOT_ACCEPTABLE \"Could not find acceptable representation\"\n\tat ....}
In case of a 404 error from downstream API the mapping response works fine.
{
code: 404,
"message": not found
}
My question is if i am doing .onStatus(HttpStatus::is4xxClientError should not work for both (404, 406 or other responde code with 4xx ?

Webclient ExchangeFilter not return a defined custom exception class

I have a problem when i do a webclient request (to a external api) and the response is 4xx ou 5xx code. The propose is handling that response and retrieve a response with a custom class
The webclient configuration is
return WebClient.builder()
.baseUrl(baseUrl)
.defaultHeaders(httpHeaders -> {
httpHeaders.setBearerAuth("token");
httpHeaders.add(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
})
.filter(handlingFilter())
.build();
}
handlingFilter.class
private static ExchangeFilterFunction handlingFilter() {
return ExchangeFilterFunction.ofResponseProcessor(clientResponse -> {
if(clientResponse.statusCode()!=null && (clientResponse.statusCode().is5xxServerError() || clientResponse.statusCode().is4xxClientError()) ) {
return Mono.error(new MyException(clientResponse.statusCode().value(), clientResponse.statusCode().getReasonPhrase()));
}else {
return Mono.just(clientResponse);
}
});
}
MyExpcetion.class
public class MyException extends Exception{
private int code;
private String message;
public MyException(String message) {
super(message);
}
}
But my client responses always give me a default format
{
"timestamp": "x",
"path": "x",
"status": "x",
"error": "x",
"message": "x",
"requestId": "x",
}
instead of
{
"code": "x",
"message": "x"
}
what's wrong ?
thanks
To change your client's response (the response of your endpoint), you have to handle the exception properly. Take a look on Spring Documentation about Managing Exceptions
Resuming: if you are using annotation endpoints, you have to create a #ExceptionHandler(MyException.class) on your Controller class or in a #RestControllerAdvice.
If you are using Functional Endpoints, then configure WebExceptionHandler

Handle Spring WebFlux WebClient timeout in Kotlin

I am doing a get http call with Spring WebFlux WebClient (Boot 2.4.3) in Kotlin (1.4.30). When request times out it fails with exception but instead I'd like to return a default value. I see references to onError, onStatus etc. used after retrieve() but they don't seem to be available in my case (only body, toEntity, awaitExchange)
The call:
suspend fun conversation(id: String): Conversation =
client.get().uri("/conversation/{id}", id).retrieve().awaitBody()
WebClient configuration with connect and read timeouts:
fun webClient(url: String, connectTimeout: Int, readTimeout: Long, objectMapper: ObjectMapper): WebClient =
WebClient.builder()
.baseUrl(url)
.exchangeStrategies(
ExchangeStrategies.builder()
.codecs { configurer -> configurer.defaultCodecs().jackson2JsonDecoder(Jackson2JsonDecoder(objectMapper)) }
.build())
.clientConnector(
ReactorClientHttpConnector(
HttpClient.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, connectTimeout)
.doOnConnected { connection ->
connection.addHandlerLast(ReadTimeoutHandler(readTimeout, TimeUnit.MILLISECONDS))
}))
.build()
Response model:
#JsonIgnoreProperties(ignoreUnknown = true)
data class Conversation(
val replyTimestamp: Map<String, String>,
)
How can I return default response (conversation with empty map) in case of timeout instead of failing with an exception?
Update:
I tried suggestion of JArgente below: updated the call with awaitExchange and set valid WireMock response with delay (1010 ms) that is longer that timeout (1000 ms).
Result is still ReadTimeoutException so looking at http status code does not help in this case.
private val defaultConversation = Conversation(emptyMap())
suspend fun conversation(id: String): Conversation =
client.get()
.uri("/conversation/{id}", id)
.awaitExchange {
response -> if (response.statusCode() == HttpStatus.OK) response.awaitBody() else defaultConversation
}
Response:
{
"replyTimestamp": {
"1": "2021-02-23T15:30:28.753Z",
"2": "2021-02-23T16:30:28.753Z"
}
}
Mock config for it:
{
"mappings":
[
{
"priority": 1,
"request": {
"method": "GET",
"urlPathPattern": "/conversation/1"
},
"response": {
"status": 200,
"fixedDelayMilliseconds": 1010,
"headers": {
"content-type": "application/json;charset=utf-8"
},
"bodyFileName": "conversation1.json"
}
}
]
}
You are getting an exception because in your method, you are expecting to get a response of type Conversation, but because you are receiving an error, the body is different.
The way you should handle the response, in this case, should be first, to look at the HTTP status code and then convert the body accordingly.
Here is an example from spring.io
https://docs.spring.io/spring-framework/docs/current/reference/html/web-reactive.html
In your case, when you receive the status code of the error you should create a new empty Conversation and return it
val entity = client.get()
.uri("/conversation/{id}", id)
.accept(MediaType.APPLICATION_JSON)
.awaitExchange {
if (response.statusCode() == HttpStatus.OK) {
return response.awaitBody<Conversation>()
}
else if (response.statusCode().is4xxClientError) {
return response.awaitBody<ErrorContainer>()
}
else {
throw response.createExceptionAndAwait()
}
}
As per Martin's suggestion ended up just wrapping call in a try/catch:
suspend inline fun <reified T : Any> WebClient.ResponseSpec.tryAwaitBodyOrElseLogged(default: T, log: Logger) : T =
try {
awaitBody()
} catch (e: Exception) {
log.warn("Remote request failed, returning default value ($default)", e)
default
}
private val log = LoggerFactory.getLogger(this::class.java)
private val default = Conversation(emptyMap())
suspend fun conversation(id: String): Conversation =
client.get()
.uri("/conversation/{id}", id)
.retrieve()
.tryAwaitBodyOrElseLogged(default, log)
I thought there is some idiomatic way but this works fine.

Spring Boot catch multiple exceptions and send as error response

I am validating an incoming POST request which will create a database entity after validating the request data. I am trying to gather multiple errors in a single request and respond as error response following JSON API spec:
https://jsonapi.org/examples/#error-objects-multiple-errors
HTTP/1.1 400 Bad Request
Content-Type: application/vnd.api+json
{
"errors": [
{
"status": "403",
"source": { "pointer": "/data/attributes/secretPowers" },
"detail": "Editing secret powers is not authorized on Sundays."
},
{
"status": "422",
"source": { "pointer": "/data/attributes/volume" },
"detail": "Volume does not, in fact, go to 11."
},
{
"status": "500",
"source": { "pointer": "/data/attributes/reputation" },
"title": "The backend responded with an error",
"detail": "Reputation service not responding after three requests."
}
]
}
Is it possible to do this by #ControllerAdvice. When Global exception handling is enabled by #ControllerAdvice and throws an exception, the next exception won't be caught.
Not directly, no. Not sure what is your business case/logic, therefore I don't know how you handling these exceptions in service layer, but in general, if you want to pass multiple errors in your #ExceptionHanlder - you could create a custom POJO:
public class MyError {
private String status;
private String source;
private String title;
private String detail;
getters/setters...
}
and then create a custom RuntimeException which would accept list of these POJOs:
public class MyRuntimeException extends RuntimeException {
private final List<MyError> errors;
public MyRuntimeException(List<MyError> errors) {
super();
this.errors = errors;
}
public List<MyError> getErrors() {
return errors;
}
}
And in your service layer you could create list of these POJOs, wrap then in your exception and throw it. Then in #ControllerAdvice you simply catch your exception and call accessor method to iterate against your list of POJOs to construct a payload you want.
Something like:
#ExceptionHandler (MyRuntimeException.class)
#ResponseStatus (BAD_REQUEST)
#ResponseBody
public Map<String, Object> handleMyRuntimeException(MyRuntimeException e) {
return singletonMap("errors", e.getErrors());
}

Get API response error message using Web Client Mono in Spring Boot

I am using webflux Mono (in Spring boot 5) to consume an external API. I am able to get data well when the API response status code is 200, but when the API returns an error I am not able to retrieve the error message from the API. Spring webclient error handler always display the message as
ClientResponse has erroneous status code: 500 Internal Server Error, but when I use PostMan the API returns this JSON response with status code 500.
{
"error": {
"statusCode": 500,
"name": "Error",
"message":"Failed to add object with ID:900 as the object exists",
"stack":"some long message"
}
}
My request using WebClient is as follows
webClient.getWebClient()
.post()
.uri("/api/Card")
.body(BodyInserters.fromObject(cardObject))
.retrieve()
.bodyToMono(String.class)
.doOnSuccess( args -> {
System.out.println(args.toString());
})
.doOnError( e ->{
e.printStackTrace();
System.out.println("Some Error Happend :"+e);
});
My question is, how can I get access to the JSON response when the API returns an Error with status code of 500?
If you want to retrieve the error details:
WebClient webClient = WebClient.builder()
.filter(ExchangeFilterFunction.ofResponseProcessor(clientResponse -> {
if (clientResponse.statusCode().isError()) {
return clientResponse.bodyToMono(ErrorDetails.class)
.flatMap(errorDetails -> Mono.error(new CustomClientException(clientResponse.statusCode(), errorDetails)));
}
return Mono.just(clientResponse);
}))
.build();
with
class CustomClientException extends WebClientException {
private final HttpStatus status;
private final ErrorDetails details;
CustomClientException(HttpStatus status, ErrorDetails details) {
super(status.getReasonPhrase());
this.status = status;
this.details = details;
}
public HttpStatus getStatus() {
return status;
}
public ErrorDetails getDetails() {
return details;
}
}
and with the ErrorDetails class mapping the error body
Per-request variant:
webClient.get()
.exchange()
.map(clientResponse -> {
if (clientResponse.statusCode().isError()) {
return clientResponse.bodyToMono(ErrorDetails.class)
.flatMap(errorDetails -> Mono.error(new CustomClientException(clientResponse.statusCode(), errorDetails)));
}
return clientResponse;
})
Just as #Frischling suggested, I changed my request to look as follows
return webClient.getWebClient()
.post()
.uri("/api/Card")
.body(BodyInserters.fromObject(cardObject))
.exchange()
.flatMap(clientResponse -> {
if (clientResponse.statusCode().is5xxServerError()) {
clientResponse.body((clientHttpResponse, context) -> {
return clientHttpResponse.getBody();
});
return clientResponse.bodyToMono(String.class);
}
else
return clientResponse.bodyToMono(String.class);
});
I also noted that there's a couple of status codes from 1xx to 5xx, which is going to make my error handling easier for different cases
Look at .onErrorMap(), that gives you the exception to look at. Since you might also need the body() of the exchange() to look at, don't use retrieve, but
.exchange().flatMap((ClientResponse) response -> ....);

Resources