On every WebClient request, I have to send the following data: requestMethod, requestHeaders, requestBody, responseHeaders, responseBody to Azure Application Insights to log it. I tried to do it over a WebClient filter, but I can either access ClientRequest or ClientResponse, not both at the same time. Is there a way to access the required data at one place?
Related
I am trying to retrieve the data from strapi latest versio 4 using webclient in spring boot.
The data is one level deeper, as per document following URL will give data.
/api/customer?filters[state][$eq]=karnataka&populate=%2A
/api/customer?filters[state][$eq]=karnataka&populate=* -> This will work for webclient
I tried to use webclient but its not working, if http client then it works.
Below pasted both the approaches, http one works but webclient doesnt give next level data at all
Can u please help me with this?
HttpRequest request =
HttpRequest.newBuilder().GET().uri(URI.create(urlToUse)).build();
java.net.http.HttpClient client =
java.net.http.HttpClient.newBuilder().build(); HttpResponse<String> response
= client.send(request, HttpResponse.BodyHandlers.ofString());
output= new ObjectMapper()
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.readValue(response.body(), Customer.class);
Mono<String> mon = webClient.get()
.uri(urlToUse)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(String.class);
String resp = mon.block(Duration.ofMillis(cmsConfig.getTimeOut()));
output= new ObjectMapper()
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.readValue(resp, Customer.class);
Update on observation:
I did capture packets using wireshark:
With webclient packet i can see request as below:
?filters%5Bstate%5D%5B$eq%5D=karnataka&populate=%252A
With httpclient packet:
?filters[state][$eq]=Karnataka&populate=%2A
%25 is appended to 2A, as % ASCII is %25, how to avoid it? Is this causing an issue?
I want to use webflux to return a single result async. The response doesn't have an id of the object. So when I get the response async back from the remote reply, then I don't have a way to fetch that object from the database to get further information. So is there a way to pass my object id to the async response handler? I couldn't find any way. Here is my sample code
var monoReply = webClient.post().uri(url)
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.body(Mono.just(myRequestObject), MyRequest.class)
.retrieve()
.bodyToMono(MyResponse.class);
monoReply.subscribe(BanlawApiServiceImpl::handleLoginResponse);
private static String handleLoginResponse(MyResponse myResponse) {
String token = myResponse.getToken();
//now I want to know the id of the database object I am dealing with. Response doesn't
have that id
}
You need to continue async flow using flatMap and fetch object from the database. As result handleLoginResponse should return Mono<T>
webClient.post().uri(url)
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.body(Mono.just(myRequestObject), MyRequest.class)
.retrieve()
.bodyToMono(MyResponse.class)
.flatMap(response -> handleLoginResponse(response))
private static Mono<String> handleLoginResponse(MyResponse myResponse) {
...
}
Not sure why you are subscribing to the flow explicitly that usually is anti-pattern and should be avoided. In WebFlux subscription happens behind the scene.
I've been trying to follow the simplest tutorials out there for how to use WebClient, which I understand to be the next greatest thing compared to RestTemplate.
For example, https://www.baeldung.com/spring-5-webclient#4-getting-a-response
So when I try to do the same thing with https://petstore.swagger.io/v2/pet/findByStatus?status=available which is supposed to return some json,
WebClient webClient = WebClient.create();
webClient.get().uri("https://petstore.swagger.io/v2/pet/findByStatus?status=available").exchange().block();
I have absolutely no idea how to proceed from the resultant DefaultClientResponse object. It shouldn't be this convoluted to arrive at the physical response body, but I digress.
How do I get the response body with the code I provided?
In the form you currently have it, and explaining the behaviour..
WebClient webClient = WebClient.create();
webClient.get()
.uri("https://petstore.swagger.io/v2/pet/findByStatus?status=available")
.exchange()
.block();
the block() starts the request by internally synchronously subscribing to the Mono, and returns the resulting ClientResponse. You could also handle this asynchronously by calling subscribe() on the Mono returned by the exchange() method, instead of block().
In this current form, after the block() you now have all the metadata (ie. from the response header) about the response in a ClientResponse object, including the success status. This does not mean that the response body has finished coming through. If you don't care about the response payload, you could confirm the success and leave it at that.
If you further want to look at the response body, you need to convert the response body stream into some class. A this point you can decide whether you want to read everything into a single Mono with bodyToMono or into a stream of objects (Flux) with bodyToFlux, such as in the case where the response is a JSON array that can be parsed into individual separate Java objects.
However, in your case, you just want to see the JSON as-is. So converting to a String is sufficient. You would just use bodyToMono which would return a Mono object.
WebClient webClient = WebClient.create();
String responseJson = webClient.get()
.uri("https://petstore.swagger.io/v2/pet/findByStatus?status=available")
.exchange()
.block()
.bodyToMono(String.class)
.block();
Here you use block() to wait for the response payload to arrive and be parsed into a String, but you could also subscribe to the Mono to receive it reactively when it is complete.
One thing to note is that retrieve() can be used instead of exchange() to shortcut the ClientResponse. In this case you let default behavior handle error responses. Using exchange() puts all the responsibility on the application for responding to error responses on the ClientResponse. Read more in the Javadoc. The retrieve() version would look as follows. No need to block() as you only care about the response data.
WebClient webClient = WebClient.create();
String responseJson = webClient.get()
.uri("https://petstore.swagger.io/v2/pet/findByStatus?status=available")
.retrieve()
.bodyToMono(String.class)
.block();
Here is how you make a request with RestTemplate
String json = new RestTemplate()
.getForEntity("https://petstore.swagger.io/v2/pet/findByStatus?status=available")
.getBody();
Here is how you make a request with requests
import requests
json = requests.get("https://petstore.swagger.io/v2/pet/findByStatus?status=available")
.content
Here is how you make a request with WebClient
String json = WebClient.create()
.get()
.uri("https://petstore.swagger.io/v2/pet/findByStatus?status=available")
.exchange()
.block()
.bodyToMono(String.class)
.block();
I'm currently using ExchangeFilterFunction to log all Headers that come inside the ClientRequest instance and I'm accessing to them doing request.headers().
After my filter is executed, the HttpClient underneath is adding certain headers such as the Accept-Encoding one, thus not getting logged, as they never get added to the ClientRequest instance.
My filter looks like this:
public class WebClientLoggingFilter implements ExchangeFilterFunction {
#Override
public Mono<ClientResponse> filter(final ClientRequest clientRequest, final ExchangeFunction next) {
return Mono.just(clientRequest)
.doOnNext(request -> log(request.headers().toString()))
.flatMap(next::exchange)
.doOnNext(clientResponse -> logData(clientRequestData, message, clientResponse));
}
}
This filter logs everything inside ClientRequest headers, but then HttpClient does its magic, which never arrives at the ClientRequest, even after the response is back. Example of code from Netty.
Is there any other way I can do the logging so I can get access to what it's truly being sent through the network?
Instead of using a filter, I'd recommend utilising the standard loggers by adding these lines to your resources/application.properties:
spring.http.log-request-details=true
logging.level.org.springframework.web.reactive.function.client.ExchangeFunctions=TRACE
However, by default, this will still show headers as {headers masked} (as they may contain sensitive data). To enable header logging for a client, you must explicitly enable it on each WebClient as follows:
return WebClient
.builder()
.exchangeStrategies(ExchangeStrategies.builder().codecs(c ->
c.defaultCodecs().enableLoggingRequestDetails(true)).build()
)
.build()
//carry on using the webclient as normal
You'll then get output similar to the following:
HTTP POST https://a.com/ea, headers=[Content-Type:"application/json", Accept:"application/json", Authorization:"Bearer token blah"]
The configuration property to use has changed to:
spring.codec.log-request-details=true
I've managed to build an SSL connection to the sandbox server and to send the object as a serialised XML object by applying the content type MediaType.APPLICATION_XML. However, this is not enough as the target service only supports SOAP and expects the message properly wrapped in an envelope.
final var webClient = WebClient.builder()
.baseUrl(fmdConfiguration.getSinglePackUrl())
.clientConnector(connector)
.exchangeStrategies(exchangeStrategies)
.filter(logResponseStatus())
.filter(logRequest())
.build();
return webClient
.method(GET)
.contentType(MediaType.APPLICATION_XML)
.body(BodyInserters.fromObject(request))
.retrieve()
.bodyToMono(SinglePackPingResponse.class);
This is the response from the service:
Unable to create envelope from given source because the root element is not named "Envelope"
Unfortunately the the WebClient doesn't support the media type application/soap+xml. When I try to use it, then the WebClient throws the following error:
org.springframework.web.reactive.function.UnsupportedMediaTypeException: Content type 'application/soap+xml;charset=UTF-8' not supported for bodyType=eu.nmvs.SinglePackPingRequest
at org.springframework.web.reactive.function.BodyInserters.unsupportedError(BodyInserters.java:300)
I use:
private void acceptedCodecs(ClientCodecConfigurer clientCodecConfigurer) {
clientCodecConfigurer.customCodecs().encoder(new Jackson2JsonEncoder(new ObjectMapper(), TEXT_XML));
clientCodecConfigurer.customCodecs().decoder(new Jackson2JsonDecoder(new ObjectMapper(), TEXT_XML));
}
and:
webClient = webClientBuilder
.baseUrl(baseUrL)
.filter(logRequest())
.exchangeStrategies(ExchangeStrategies.builder().codecs(this::acceptedCodecs).build()).build();