Pass-through API / Preserve backend headers in Spring Webflux - spring-boot

I am building an application to call a back-end which responds with a mime-type response.
#Override
public Mono<String> getDocument() {
return webClient.get()
.uri(path)
.retrieve()
.bodyToMono(String.class);
}
From this request, I need to preserve the response headers and pass it through as the response. This is mostly because the response headers contain the dynamic content type of the file. I need to forward these headers (all as received) to the API response. For example :
Content-Type : application/pdf
Content-Disposition: attachment; filename="test.pdf"
Following is my handler.
public Mono<ServerResponse> getDocument(ServerRequest request) {
return ServerResponse
.ok()
.contentType(MediaType.APPLICATION_PDF)
.header("Content-Disposition", "attachment; filename=\"test.pdf\"")
.body(BodyInserters.fromPublisher(documentService.getDocument(), String.class));
}
The file is coming through from the API as an attachment as expected, but I do not want to hard code the content-type header. How can I achieve this?
Update with the handler code :
public Mono<ServerResponse> getDocument(ServerRequest request) {
return ServerResponse
.ok()
.contentType(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromPublisher(documentService.getDocument(), String.class));
}

I was able to resolve the problem by returning a ResponseEntity from the service instead of the body and using that to construct the ServerResponse in the handler.
Service :
public Mono<ResponseEntity<String>> getDocument() {
return webClient.get()
.uri(path)
.retrieve()
.toEntity(String.class);
}
Handler :
public Mono<ServerResponse> getDocument(ServerRequest request) {
return documentService
.getDocument()
.flatMap(r -> ServerResponse
.ok()
.headers(httpHeaders -> httpHeaders.addAll(r.getHeaders()))
.body(BodyInserters.fromValue(r.getBody()))
);
}

Related

Spring WebClient - Stop retrying if an exception is thrown in the doOnError

I have the following code to make a request that is going to be retried a max number of times. This request needs an authorization header and I'm caching this information to prevent this method to call the method to retrieve this information every time.
What I'm trying to do is:
When calling myMethod I first retrieve the login information for the service I'm calling, in most cases that will come from the cache when calling the getAuthorizationHeaderValue method.
In the web client, if the response to send this request returns a 4xx response I need to login again to the service I'm calling, before retrying the request. For that, I'm calling the tryToLoginAgain method to set the value for the header again.
After doing that the retry of the request should work now that the header has been set.
If by any chance the call to login again fails I need to stop retrying as there no use on retrying the request.
public <T> T myMethod(...) {
...
try {
AtomicReference<String> headerValue = new AtomicReference<>(loginService.getAuthorizationHeaderValue());
Mono<T> monoResult = webclient.get()
.uri(uri)
.accept(MediaType.APPLICATION_JSON)
.header(HttpHeaders.AUTHORIZATION, headerValue.get())
.retrieve()
.onStatus(HttpStatus::is4xxClientError, response -> throwHttpClientLoginException())
.bodyToMono(type)
.doOnError(HttpClientLoginException.class, e -> tryToLoginAgain(headerValue))
.retryWhen(Retry.backoff(MAX_NUMBER_RETRIES, Duration.ofSeconds(5)));
result = monoResult.block();
} catch(Exception e) {
throw new HttpClientException("There was an error while sending the request");
}
return result;
}
...
private Mono<Throwable> throwHttpClientLoginException() {
return Mono.error(new HttpClientLoginException("Existing Authorization failed"));
}
private void tryToLoginAgain(AtomicReference<String> headerValue) {
loginService.removeAccessTokenFromCache();
headerValue.set(loginService.getAuthorizationHeaderValue());
}
I have some unit tests and the happy path works fine (unauthorized the first time, try to login again and send the request again) but the scenario where the login doesn't work at all is not working.
I thought that if the tryToLoginAgain method throws an Exception that would be caught by the catch I have in myMethod but it never reaches there, it just retries the request again. Is there any way to do what I want?
So at the end I found a way of doing what I wanted and now the code looks like this:
public <T> T myMethod() {
try {
HttpHeaders headers = new HttpHeaders();
headers.setBearerAuth(getAuthorizationHeaderValue());
final RetryBackoffSpec retrySpec = Retry.backoff(MAX_NUMBER_RETRIES, Duration.ofSeconds(5))
.doBeforeRetry(retrySignal -> {
//When retrying, if this was a login error, try to login again
if (retrySignal.failure() instanceof HttpClientLoginException) {
tryToLoginAgain(headers);
}
});
Mono<T> monoResult = Mono.defer(() ->
getRequestFromMethod(httpMethod, uri, body, headers)
.retrieve()
.onStatus(HttpStatus::is4xxClientError, response -> throwHttpClientLoginException())
.bodyToMono(type)
)
.retryWhen(retrySpec);
result = monoResult.block();
} catch (Exception e) {
String requestUri = uri != null ?
uri.toString() :
endpoint;
log.error("There was an error while sending the request [{}] [{}]", httpMethod.name(), requestUri);
throw new HttpClientException("There was an error while sending the request [" + httpMethod.name() +
"] [" + requestUri + "]");
}
return result;
}
private void tryToLoginAgain(HttpHeaders httpHeaders) {
//If there was an 4xx error, let's evict the cache to remove the existing access_token (if it exists)
loginService.removeAccessTokenFromCache();
//And let's try to login again
httpHeaders.setBearerAuth(getAuthorizationHeaderValue());
}
private Mono<Throwable> throwHttpClientLoginException() {
return Mono.error(new HttpClientLoginException("Existing Authorization failed"));
}
private WebClient.RequestHeadersSpec getRequestFromMethod(HttpMethod httpMethod, URI uri, Object body, HttpHeaders headers) {
switch (httpMethod) {
case GET:
return webClient.get()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON);
case POST:
return body == null ?
webClient.post()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON) :
webClient.post()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON)
.contentType(MediaType.APPLICATION_JSON)
.bodyValue(body);
case PUT:
return body == null ?
webClient.put()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON) :
webClient.put()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON)
.contentType(MediaType.APPLICATION_JSON)
.bodyValue(body);
case DELETE:
return webClient.delete()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON);
default:
log.error("Method [{}] is not supported", httpMethod.name());
throw new HttpClientException("Method [" + httpMethod.name() + "] is not supported");
}
}
private String getAuthorizationHeaderValue() {
return loginService.retrieveAccessToken();
}
By using Mono.defer() I can retry on that Mono and make sure I change the headers I'll use with the WebClient. The retry spec will check if the exception was of the HttpClientLoginException type, thrown when the request gets a 4xx status code and in that case it will try to login again and set the header for the next retry. If the status code was different it will retry again using the same authorization.
Also, if there's an error when we try to login again, that will be caught by the catch and it won't retry anymore.

Spring boot - WebFlux - WebTestClient - convert response to responseEntity

I have a Reactive controller which returns:
#ResponseBody
#GetMapping("/data")
public Mono<ResponseEntity<Data>> getData() {
//service call which returns Mono<Data> dataMono
Mono<ResponseEntity<Data>> responseEntityMono = dataMono
.map(data -> ResponseEntity.ok(data))
.doFinally(signalType -> {});
return responseEntityMono;
}
I am using WebTestClient to test this endpoint, but I want to extract the response entity for cucumber to validate further.
I tried this:
#Autowired private WebTestClient webTestClient;
public ResponseEntity get() {
EntityExchangeResult < ResponseEntity > response = webTestClient.get()
.uri(uriBuilder ->
uriBuilder
.path(VISUALIZATION_URL)
.build())
.header("Accepts", "application/json")
.exchange()
.expectStatus().isOk()
.expectHeader().contentType(MediaType.APPLICATION_JSON_VALUE)
.expectBody(ResponseEntity.class)
.returnResult();
return response.getResponseBody();
}
but I am getting an error. I can get the JSON by doing:
public String get() {
BodyContentSpec bodyContentSpec = webTestClient.get()
.uri(uriBuilder ->
uriBuilder
.path(URL)
.build())
.header("Accepts", "application/json")
.exchange()
.expectStatus().isOk()
.expectHeader().contentType(MediaType.APPLICATION_JSON_VALUE)
.expectBody();
return new String(bodyContentSpec.returnResult().getResponseBody());
}
But I am trying to see if I can get the whole ResponseEntity so that I can validate headers, caching headers, and body.
You will never be able to get a ResponseEntity in your test. WebTestClient.ResponseSpec returned from exchange() is the only way to check the answer from your Controller. ResponseEntity is just the object you return in your method but once Jackson serializes it, it is converted to a regular HTTP response (in your case with JSON in his body and regular HTTP headers).

How to convert webclient response to ResponseEntity?

I am able to convert WebClient response to Response Entity with exchange() method which is deprecated now.
Please suggest other way of achieving the same result. Below is my code.
public ResponseEntity<TestClass> getTestDetails() {
ClientResponse clientResponse = webClientBuilder.build()
.get()
.uri("http://localhost:9090/test")
.headers(httpHeaders -> {
httpHeaders.add(Constants.ACCEPT, Constants.APPLICATION_JSON);
})
.exchange()
.block();
return clientResponse.toEntity(TestClass.class).block();
}
I did it in the following way:
public ResponseEntity<TestClass> getTestDetails() {
return webClientBuilder.build()
.get()
.uri("http://localhost:9090/test")
.headers(httpHeaders -> {
httpHeaders.add(Constants.ACCEPT, Constants.APPLICATION_JSON);
})
.retrieve()
.toEntity(TestClass.class)
.block();
}

handling error messages with spring-boot webflux

i'm writing a java adapter for posting something to a webService.
In the http service i use an ExceptionHandler:
#org.springframework.web.bind.annotation.ExceptionHandler
public ResponseEntity<String> handle(BadCommandException ex) {
return ResponseEntity.badRequest().body(ex.getMessage());
}
That works well, if i call it with curl/postman.
But how can i get the error message that is in the body with a reactive WebClient ?
Part of my client-lib:
private Mono<Optional<String>> post(String bearerToken, Model model, IRI outbox) {
return WebClient.builder()
.build()
.post()
.uri(outbox.stringValue())
.contentType(new MediaType("text","turtle"))
.body(BodyInserters.fromValue(modelToTurtleString(model)))
.header("Authorization", "Bearer " + bearerToken)
.header("profile", "https://www.w3.org/ns/activitystreams")
.accept(new MediaType("text", "turtle"))
.retrieve()
.toEntity(String.class)
.map(res->res.getHeaders().get("Location").stream().findFirst());
}
In success case, i want to return the string from the location header. But how can i for example throw an Exception if the http-status is 400 and add the error message from the body to the exception?
Thanks
the spring documentation on webclient retreive() has examples of how to handle errors. I suggest you start there.
Mono<Person> result = client.get()
.uri("/persons/{id}", id).accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(HttpStatus::is4xxClientError, response -> ...)
.onStatus(HttpStatus::is5xxServerError, response -> ...)
.bodyToMono(Person.class);
.onStatus(HttpStatus::is4xxClientError,
clientResponse -> { return clientResponse.createException();
Javadoc:
/**
* Create a {#link WebClientResponseException} that contains the response
* status, headers, body, and the originating request.
* #return a {#code Mono} with the created exception
* #since 5.2
*/
Mono<WebClientResponseException> createException();
My problem was my testcase 'test bad request'. Because of a refactoring it was sending an empty request body instead of a bad one and the annotation #RequestBody has an attribute 'required' which is default true. So the response body was always empty and did not contain my error message as expected ;-)
public Mono<ResponseEntity<Void>> doIt(Principal principal, #PathVariable String actorId, #RequestBody String model) {
...
that cost me hours ;-)

Webflux Spring 2.1.2 customize Content-Type

I am trying to post via WebClient to get microsoft token:
public WebClient getWebclient() {
TcpClient client = TcpClient.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 10000)
.doOnConnected(connection -> connection.addHandlerLast(new ReadTimeoutHandler(15)).addHandlerLast(new WriteTimeoutHandler(15)));
ExchangeStrategies strategies = ExchangeStrategies.builder()
.codecs(configurer -> {
configurer.registerDefaults(true);
FormHttpMessageReader formHttpMessageReader = new FormHttpMessageReader();
formHttpMessageReader.setEnableLoggingRequestDetails(true);
configurer.customCodecs().reader(formHttpMessageReader);
})
.build();
return WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(HttpClient.from(client).followRedirect(true)))
.exchangeStrategies(strategies)
.filter(logRequest())
.filter(logResponse())
.build();
}
MultiValueMap<String, String> credentials = new LinkedMultiValueMap<>();
credentials.add("grant_type", "password");
credentials.add("client_id", oauthClientId);
credentials.add("resource", oauthResource);
credentials.add("scope", oauthScope);
credentials.add("username", oauthUsername);
credentials.add("password", oauthPassword);
Mono<MicrosoftToken> response = webClientService.getWebclient().post()
.uri(oauthUrl)
.contentType(MediaType.APPLICATION_FORM_URLENCODED)
.body(BodyInserters.fromFormData(credentials))
.retrieve()
.onStatus(HttpStatus::is4xxClientError, clientResponse ->
Mono.error(new WebClientException(clientResponse.bodyToMono(String.class), clientResponse.statusCode())))
.bodyToMono(MicrosoftToken.class);
this.cachedToken = response.block();
The problem ist, that microsoft cannot handle a Content-type: application/x-www-form-urlencoded;charset=UTF-8.
Spring is automatically adding the charset=UTF-8 to the request. I need to get rid of this additional charset. I need a Content-Type: application/x-www-form-urlencoded. Is this possible? Otherwise i need to downgrade my spring version to 2.0.0 where the charset is not automatically be added.
My Debug Logs print:
2019-03-14 10:08:42 DEBUG [reactor.netty.channel.ChannelOperationsHandler]:
[id: 0x5d6effce, L:/192.168.148.14:52285 -
R:login.microsoftonline.de/51.4.136.42:443] Writing object
DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
POST /common/oauth2/token HTTP/1.1
user-agent: ReactorNetty/0.8.4.RELEASE
host: login.microsoftonline.de
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 205
2019-03-14 10:08:42 DEBUG [reactor.netty.channel.ChannelOperationsHandler]:
[id: 0x5d6effce, L:/192.168.148.14:52285 -
R:login.microsoftonline.de/51.4.136.42:443] Writing object
I tested this with spring version 2.0.0 and there the charset is not added as in the new version:
POST /common/oauth2/token HTTP/1.1
user-agent: ReactorNetty/0.7.5.RELEASE
host: login.microsoftonline.de
accept-encoding: gzip
Content-Type: application/x-www-form-urlencoded
Content-Length: 205
This took me the best part of a morning to find out, but I finally managed. The problem is that Webflux BodyInserters.fromFormData always sets the content type to application/x-www-form-urlencoded;charset=... regardless of what you set in the headers.
To solve this, first define this method:
/**
* This method is unfortunately necessary because of Spring Webflux's propensity to add {#code ";charset=..."}
* to the {#code Content-Type} header, which the Generic Chinese Device doesn't handle properly.
*
* #return a {#link FormInserter} that doesn't add the character set to the content type header
*/
private FormInserter<String> formInserter() {
return new FormInserter<String>() {
private final MultiValueMap<String, String> data = new LinkedMultiValueMap<>();
#Override public FormInserter<String> with(final String key, final String value) {
data.add(key, value);
return this;
}
#Override public FormInserter<String> with(final MultiValueMap<String, String> values) {
data.addAll(values);
return this;
}
#Override public Mono<Void> insert(final ClientHttpRequest outputMessage, final Context context) {
final ResolvableType formDataType =
ResolvableType.forClassWithGenerics(MultiValueMap.class, String.class, String.class);
return new FormHttpMessageWriter() {
#Override protected MediaType getMediaType(final MediaType mediaType) {
if (MediaType.APPLICATION_FORM_URLENCODED.equals(mediaType)) {
return mediaType;
} else {
return super.getMediaType(mediaType);
}
}
}.write(Mono.just(this.data), formDataType,
MediaType.APPLICATION_FORM_URLENCODED,
outputMessage,
context.hints());
}
};
}
Then, to call your web service, do the following:
final SomeResponseObject response = WebClient
.builder()
.build()
.post()
.uri(someOrOtherUri)
.body(formInserter().with("param1", "value1")
.with("param2", "value2")
)
.retrieve()
.bodyToFlux(SomeReponseObject.class)
.blockLast();
Please note that the block above is mainly for demonstration purposes. You may or may not want to block and wait for the response.
Here's two ways to do it:
webClient
.mutate()
.defaultHeaders(headers -> {
headers.add("Content-Type", ContentType.APPLICATION_FORM_URLENCODED.getMimeType()
}).build()
. uri(uri)
...
OR
webClient
.post()
.uri(uri)
.body(body)
.headers(headers -> getHttpHeaders())
...
private HttpHeaders getHttpHeaders(){
HttpHeaders headers = new HttpHeaders();
headers.add("Content-Type", "application/x-www-form-urlencoded")
return headers;
}
Just a few ways you could utilize the headers consumer in .headers or .defaultHeaders..
But I don't think the charset is the issue to be honest. If you are getting application/json in your response it is probably because Microsoft is forwarding the request with that header through the redirect url you specified in your app registration.
The good news is this is probably desirable, since Microsoft returns the token fields as json, which allows you to call .bodyToMono(MicrosoftToken). I recall having issues with BodyInserters.fromFormData as it did not actually encode the values in the MultiValueMap.
This is what I'm using instead:
private BodyInserter<String, ReactiveHttpOutputMessage> getBodyInserter(Map<String,String> parameters) {
credentials.add("grant_type", encode("password"));
credentials.add("client_id", encode(oauthClientId));
credentials.add("resource", encode(oauthResource));
// and so on..
// note that parameters is a regular Map - not a MultiValueMap
BodyInserter<String, ReactiveHttpOutputMessage> bodyInserter = BodyInserters.fromObject(
parameters.entrySet().stream()
.map(entry -> entry.getKey().concat("=").concat(entry.getValue()))
.collect(Collectors.joining("&", "", "")));
return bodyInserter;
}
private String encode(String str) {
try {
return URLEncoder.encode(str, StandardCharsets.UTF_8.name());
} catch (UnsupportedEncodingException e) {
log.error("Error encoding req body", e);
}
}

Resources