I've an API that is up and running in AWS CloudFront. When I test the API from Postman, it seems to be working fine.
Here's the request from Postman console.
But when invoking the same API from spring's WebClient, it is returning 400 bad request with response:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD><BODY>
<H1>400 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
Bad request.
We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
<BR clear="all">
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
<BR clear="all">
<HR noshade size="1px">
<PRE>
Generated by cloudfront (CloudFront)
Request ID: pGx0NHPSaU3H2EHJwtQrYEdLDjL_UPxO90esPoH3d9efZX_bvzjFQw==
</PRE>
<ADDRESS>
</ADDRESS>
</BODY></HTML>
Here's my code:
#Component
public class SyncTranscriberClient {
private final WebClient webClient;
#Value("${transcriber.sync.username}")
private String username;
#Value("${transcriber.sync.password}")
private String password;
public SyncTranscriberClient(#Value("${transcriber.sync.base-url}") String baseUrl) throws SSLException {
var sslContext = SslContextBuilder
.forClient()
.trustManager(InsecureTrustManagerFactory.INSTANCE)
.build();
var client = HttpClient.create().secure(t -> t.sslContext(sslContext));
webClient = WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(client))
.baseUrl(baseUrl)
.filter(logRequest())
.build();
}
private static ExchangeFilterFunction logRequest() {
return ExchangeFilterFunction.ofRequestProcessor(clientRequest -> {
log.info("Request: {} {}", clientRequest.method(), clientRequest.url());
clientRequest.headers().forEach((name, values) -> values.forEach(value -> log.info("{}={}", name, value)));
return Mono.just(clientRequest);
});
}
public Map<String, String> getCredentials() {
return webClient.post()
.uri("my-url-hidden-for-security-reasons")
.headers(headers -> headers.setBasicAuth(username, password))
.headers(headers -> headers.set("Accept", "*/*"))
.headers(headers -> headers.set("Host", "hidden-for-security-reasons"))
.retrieve()
.onStatus(HttpStatus::isError, response -> response.bodyToMono(String.class) // error body as String or other class
.flatMap(error -> Mono.error(new RuntimeException(error)))
)
.toBodilessEntity()
.map(HttpEntity::getHeaders)
.map(headers -> Map.of(
"Authorization", Objects.requireNonNull(headers.getFirst("Authorization")),
"JSESSIONID", Objects.requireNonNull(headers.getFirst("JSESSIONID"))
))
.block();
}
}
What am I doing wrong here?
I had to send Content-Type: text/plain and an empty body.
webClient.post()
.uri("my-url-hidden-for-security-reasons")
.headers(headers -> {
headers.setBasicAuth(username, password);
headers.set("Content-Type", "text/plain");
})
.body(BodyInserters.fromValue(""))
.retrieve()
.onStatus(
HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(error -> Mono.error(new RuntimeException(error)))
)
.toBodilessEntity()
.map(HttpEntity::getHeaders)
.map(headers -> Map.of(
"Authorization", Objects.requireNonNull(headers.getFirst("Authorization")),
"JSESSIONID", Objects.requireNonNull(headers.getFirst("JSESSIONID"))
))
.block();
Related
I would like to programmatically login to a web application using reactive web client . I could see I am authenticated correctly but could not able to reach redirected url.
I access my application http://localhost:8080/myapplogin
Here my web client response will have a Location header http://localhost:8082/portal/auth/home , and I would like the browser to redirect to this url
I could see the application in 8082 authenticating and redirecting(/auth/home) but do not know how to make my web client return the response and do the redirection through browser. Can someone help
#GetMapping("/myapplogin")
Publisher<ClientResponse> mylogin() {
MultiValueMap<String, String> formData = new LinkedMultiValueMap<>();
formData.add("username", "testuser");
formData.add("password", "test password");
return WebClient.create()
.post()
.uri("http://localhost:8082/portal/auth/login")
.contentType(MediaType.APPLICATION_FORM_URLENCODED)
.body(BodyInserters.fromFormData(formData))
.exchange();
}
You need to return status codes that start with 3, and a Location header holding the URL to redirect to.
#GetMapping("/myapplogin")
public Mono<ResponseEntity> mylogin() {
return login()
.map(res -> ResponseEntity
.status(HttpStatus.PERMANENT_REDIRECT)
.location(URI.create("/auth/home"))
.build()
);
}
where login is
Mono<ClientResponse> login() {
MultiValueMap<String, String> formData = new LinkedMultiValueMap<>();
formData.add("username", "testuser");
formData.add("password", "test password");
return WebClient.create()
.post()
.uri("http://localhost:8082/portal/auth/login")
.contentType(MediaType.APPLICATION_FORM_URLENCODED)
.body(BodyInserters.fromFormData(formData))
.exchange();
}
I am getting intermittent ReadTimeOut from netty with the below error:
The connection observed an error","logger_name":"reactor.netty.http.client.HttpClientConnect","thread_name":"reactor-http-epoll-3","level":"WARN","level_value":30000,"stack_trace":"io.netty.handler.timeout.ReadTimeoutException: null
One observation we made is this particular endpoint for which we are getting this issue is a POST with no request body. I am now sending a dummy json in body now which the downstream system ignores and now I don't see this error anymore at all.
Below is my code:
protected <T, S Mono<S sendMonoRequest (HttpMethod method,
HttpHeaders headers,
T requestBody,
URI uri, Class < S responseClass)
throws ApiException, IOException {
log.info("Calling {} {} {} {}", method.toString(), uri.toString(), headers.toString(),
mapper.writeValueAsString(requestBody));
WebClient.RequestBodySpec requestBodySpec = getWebClient().method(method).uri(uri);
headers.keySet().stream().forEach(headerKey -> headers.get(headerKey).stream().
forEach(headerValue -> requestBodySpec.header(headerKey, headerValue)));
return requestBodySpec
.body(BodyInserters.fromObject(requestBody != null ? requestBody : ""))
.retrieve()
.onStatus(HttpStatus::is4xxClientError, this::doOn4xxError)
.onStatus(HttpStatus::is5xxServerError, this::doOn5xxError)
.onStatus(HttpStatus::isError, this::doOnError)
.bodyToMono(responseClass);
}
protected WebClient getWebClient () {
HttpClient httpClient = HttpClient.create().tcpConfiguration(
client -> client.option(ChannelOption.CONNECT_TIMEOUT_MILLIS,
20000).doOnConnected(conn - conn
.addHandlerLast(new ReadTimeoutHandler(20)).addHandlerLast(new WriteTimeoutHandler(20))));
ClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
return WebClient.builder().clientConnector(connector)
.filter(logResponse())
.build();
}
To resolve the intemrittent timeouts, I have to send a dummy pojo to sendMonoRequest() for request body. Any ideas ?
I have Api calls which uses OAUTH token this auth tokens are specific to different user and have a expiry period of 24 hrs. But sometimes even if the expire time is not up when the api call is made it returns invalid token. Is their a way in which I can retry the api call one more time in the above scenerio by getting the new user access token. The user access token is also got by making an api call. I am using Java reactive webclient with spring boot.
public Mono<Abc> create(Long customerId, Abc abc) {
return profileRepo.findByCustomerId(customerId)
.map(profile -> refreshTokens(customerId)
.flatMap(tokens ->
client.create(token,getProfile(customerId))))
.orElseThrow(ResourceNotFoundException::new);
}
public Mono<Token> refreshTokens(final Long customerId) {
Token token = service.findByCustomerId(customerId);
if (LocalDateTime.now().isAfter(token.getExpiryTime())) {
newToken = client.refresh(token);
}
return newToken;
}
Api call for token refresh and create
public Mono<Token> refresh(final Token token) {
return client.post()
.uri(OAUTH_TOKEN_PATH)
.header(AUTHORIZATION, basicAuth()) // The master token of the service provider
.body(forRefreshToken(new RefreshToken(token.getRefreshToken())))
.retrieve()
.onStatus(HttpStatus::is4xxClientError, response -> response.bodyToMono(String.class)
.flatMap(error -> Mono.error(new ClientResponseException(response.statusCode().value(),response.statusCode(),error))))
.onStatus(HttpStatus::is5xxServerError, response -> response.bodyToMono(String.class)
.flatMap(error -> Mono.error(new ClientResponseException(response.statusCode().value(),response.statusCode(),error))))
.bodyToMono(Token.class);
}
public Mono<Abc> create(final Token token, Profile pro) {
return client.post()
.uri(PATH_V2)
.header(AUTHORIZATION, token.bearer())
.contentType(APPLICATION_JSON)
.body(fromObject(pro))
.retrieve()
.onStatus(HttpStatus::is4xxClientError, response -> response.bodyToMono(String.class)
.flatMap(error -> Mono.error(new ClientResponseException(response.statusCode().value(),response.statusCode(),error))))
.onStatus(HttpStatus::is5xxServerError, response -> response.bodyToMono(String.class)
.flatMap(error -> Mono.error(new ClientResponseException(response.statusCode().value(),response.statusCode(),error))))
.bodyToMono(Abc.class);
}
Thanks in advance,
Sameekshya
I am building an application to call a back-end which responds with a mime-type response.
#Override
public Mono<String> getDocument() {
return webClient.get()
.uri(path)
.retrieve()
.bodyToMono(String.class);
}
From this request, I need to preserve the response headers and pass it through as the response. This is mostly because the response headers contain the dynamic content type of the file. I need to forward these headers (all as received) to the API response. For example :
Content-Type : application/pdf
Content-Disposition: attachment; filename="test.pdf"
Following is my handler.
public Mono<ServerResponse> getDocument(ServerRequest request) {
return ServerResponse
.ok()
.contentType(MediaType.APPLICATION_PDF)
.header("Content-Disposition", "attachment; filename=\"test.pdf\"")
.body(BodyInserters.fromPublisher(documentService.getDocument(), String.class));
}
The file is coming through from the API as an attachment as expected, but I do not want to hard code the content-type header. How can I achieve this?
Update with the handler code :
public Mono<ServerResponse> getDocument(ServerRequest request) {
return ServerResponse
.ok()
.contentType(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromPublisher(documentService.getDocument(), String.class));
}
I was able to resolve the problem by returning a ResponseEntity from the service instead of the body and using that to construct the ServerResponse in the handler.
Service :
public Mono<ResponseEntity<String>> getDocument() {
return webClient.get()
.uri(path)
.retrieve()
.toEntity(String.class);
}
Handler :
public Mono<ServerResponse> getDocument(ServerRequest request) {
return documentService
.getDocument()
.flatMap(r -> ServerResponse
.ok()
.headers(httpHeaders -> httpHeaders.addAll(r.getHeaders()))
.body(BodyInserters.fromValue(r.getBody()))
);
}
I am trying to post via WebClient to get microsoft token:
public WebClient getWebclient() {
TcpClient client = TcpClient.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 10000)
.doOnConnected(connection -> connection.addHandlerLast(new ReadTimeoutHandler(15)).addHandlerLast(new WriteTimeoutHandler(15)));
ExchangeStrategies strategies = ExchangeStrategies.builder()
.codecs(configurer -> {
configurer.registerDefaults(true);
FormHttpMessageReader formHttpMessageReader = new FormHttpMessageReader();
formHttpMessageReader.setEnableLoggingRequestDetails(true);
configurer.customCodecs().reader(formHttpMessageReader);
})
.build();
return WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(HttpClient.from(client).followRedirect(true)))
.exchangeStrategies(strategies)
.filter(logRequest())
.filter(logResponse())
.build();
}
MultiValueMap<String, String> credentials = new LinkedMultiValueMap<>();
credentials.add("grant_type", "password");
credentials.add("client_id", oauthClientId);
credentials.add("resource", oauthResource);
credentials.add("scope", oauthScope);
credentials.add("username", oauthUsername);
credentials.add("password", oauthPassword);
Mono<MicrosoftToken> response = webClientService.getWebclient().post()
.uri(oauthUrl)
.contentType(MediaType.APPLICATION_FORM_URLENCODED)
.body(BodyInserters.fromFormData(credentials))
.retrieve()
.onStatus(HttpStatus::is4xxClientError, clientResponse ->
Mono.error(new WebClientException(clientResponse.bodyToMono(String.class), clientResponse.statusCode())))
.bodyToMono(MicrosoftToken.class);
this.cachedToken = response.block();
The problem ist, that microsoft cannot handle a Content-type: application/x-www-form-urlencoded;charset=UTF-8.
Spring is automatically adding the charset=UTF-8 to the request. I need to get rid of this additional charset. I need a Content-Type: application/x-www-form-urlencoded. Is this possible? Otherwise i need to downgrade my spring version to 2.0.0 where the charset is not automatically be added.
My Debug Logs print:
2019-03-14 10:08:42 DEBUG [reactor.netty.channel.ChannelOperationsHandler]:
[id: 0x5d6effce, L:/192.168.148.14:52285 -
R:login.microsoftonline.de/51.4.136.42:443] Writing object
DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
POST /common/oauth2/token HTTP/1.1
user-agent: ReactorNetty/0.8.4.RELEASE
host: login.microsoftonline.de
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 205
2019-03-14 10:08:42 DEBUG [reactor.netty.channel.ChannelOperationsHandler]:
[id: 0x5d6effce, L:/192.168.148.14:52285 -
R:login.microsoftonline.de/51.4.136.42:443] Writing object
I tested this with spring version 2.0.0 and there the charset is not added as in the new version:
POST /common/oauth2/token HTTP/1.1
user-agent: ReactorNetty/0.7.5.RELEASE
host: login.microsoftonline.de
accept-encoding: gzip
Content-Type: application/x-www-form-urlencoded
Content-Length: 205
This took me the best part of a morning to find out, but I finally managed. The problem is that Webflux BodyInserters.fromFormData always sets the content type to application/x-www-form-urlencoded;charset=... regardless of what you set in the headers.
To solve this, first define this method:
/**
* This method is unfortunately necessary because of Spring Webflux's propensity to add {#code ";charset=..."}
* to the {#code Content-Type} header, which the Generic Chinese Device doesn't handle properly.
*
* #return a {#link FormInserter} that doesn't add the character set to the content type header
*/
private FormInserter<String> formInserter() {
return new FormInserter<String>() {
private final MultiValueMap<String, String> data = new LinkedMultiValueMap<>();
#Override public FormInserter<String> with(final String key, final String value) {
data.add(key, value);
return this;
}
#Override public FormInserter<String> with(final MultiValueMap<String, String> values) {
data.addAll(values);
return this;
}
#Override public Mono<Void> insert(final ClientHttpRequest outputMessage, final Context context) {
final ResolvableType formDataType =
ResolvableType.forClassWithGenerics(MultiValueMap.class, String.class, String.class);
return new FormHttpMessageWriter() {
#Override protected MediaType getMediaType(final MediaType mediaType) {
if (MediaType.APPLICATION_FORM_URLENCODED.equals(mediaType)) {
return mediaType;
} else {
return super.getMediaType(mediaType);
}
}
}.write(Mono.just(this.data), formDataType,
MediaType.APPLICATION_FORM_URLENCODED,
outputMessage,
context.hints());
}
};
}
Then, to call your web service, do the following:
final SomeResponseObject response = WebClient
.builder()
.build()
.post()
.uri(someOrOtherUri)
.body(formInserter().with("param1", "value1")
.with("param2", "value2")
)
.retrieve()
.bodyToFlux(SomeReponseObject.class)
.blockLast();
Please note that the block above is mainly for demonstration purposes. You may or may not want to block and wait for the response.
Here's two ways to do it:
webClient
.mutate()
.defaultHeaders(headers -> {
headers.add("Content-Type", ContentType.APPLICATION_FORM_URLENCODED.getMimeType()
}).build()
. uri(uri)
...
OR
webClient
.post()
.uri(uri)
.body(body)
.headers(headers -> getHttpHeaders())
...
private HttpHeaders getHttpHeaders(){
HttpHeaders headers = new HttpHeaders();
headers.add("Content-Type", "application/x-www-form-urlencoded")
return headers;
}
Just a few ways you could utilize the headers consumer in .headers or .defaultHeaders..
But I don't think the charset is the issue to be honest. If you are getting application/json in your response it is probably because Microsoft is forwarding the request with that header through the redirect url you specified in your app registration.
The good news is this is probably desirable, since Microsoft returns the token fields as json, which allows you to call .bodyToMono(MicrosoftToken). I recall having issues with BodyInserters.fromFormData as it did not actually encode the values in the MultiValueMap.
This is what I'm using instead:
private BodyInserter<String, ReactiveHttpOutputMessage> getBodyInserter(Map<String,String> parameters) {
credentials.add("grant_type", encode("password"));
credentials.add("client_id", encode(oauthClientId));
credentials.add("resource", encode(oauthResource));
// and so on..
// note that parameters is a regular Map - not a MultiValueMap
BodyInserter<String, ReactiveHttpOutputMessage> bodyInserter = BodyInserters.fromObject(
parameters.entrySet().stream()
.map(entry -> entry.getKey().concat("=").concat(entry.getValue()))
.collect(Collectors.joining("&", "", "")));
return bodyInserter;
}
private String encode(String str) {
try {
return URLEncoder.encode(str, StandardCharsets.UTF_8.name());
} catch (UnsupportedEncodingException e) {
log.error("Error encoding req body", e);
}
}