Below is my code:
public static void main(String[] args) {
TcpClient tcpClient = TcpClient.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 100_000)
.proxy(p ->
p.type(ProxyProvider.Proxy.HTTP)
.host("myServerHost.com")
.port(8080))
.doOnConnected ( connection ->
connection.addHandlerLast(new ReadTimeoutHandler(60000, TimeUnit.MILLISECONDS))
.addHandlerLast(new ReadTimeoutHandler(60000, TimeUnit.MILLISECONDS))
);
WebClient myWebClient = WebClient
.builder()
.clientConnector(new ReactorClientHttpConnector(HttpClient.from(tcpClient)))
.baseUrl("https://baseurl.com")
.build();
Mono<String> rs = myWebClient.post()
.uri("uriForRestEndPoints")
.header(HttpHeaders.CONTENT_TYPE.toString(), MediaType.valueOf("text/plain;charset=UTF-8").toString())
.retrieve()
.bodyToMono(String.class)
.doOnEach(t -> {System.out.println("doOnEach: " + t.toString()); })
.doOnSuccess(x -> { System.out.println("Success: ") ;})
.doOnError(x -> { System.out.println("Error: ") ; });
System.out.println("got response "+ rs.block() + "... " );
}
In this sample POC for REST API project - and I am getting 'IllegalStateException' hence I have tried everything from internet. - Like having timeout, proxy, content headers etc.
But still getting same issue "java.lang.IllegalStateException: The underlying HTTP client completed without emitting a response."
Also doOnSuccess, doOnError, doOnEach - is not printing anything.
Can anyone please help me on this.
Related
I am new to Pact Contract testing and I am trying to create a Pact consumer test to validate a method that calls an api with get request. The api request is made using Spring Webclient.
I am not able to create the webclient object by just providing the Pact mockserver eg.
WebClient webClient = WebClient.builder().baseUrl(mockServer.getUrl()).build();
I am getting the exception java.lang.IllegalStateException: No suitable default ClientHttpConnector found. The explanation I get on the internet for that , is to include reactor-netty-http and I was able to get past this issue when i included that in the POM. But I don't think that is the right solution here because I need the mockserver to respond to the webclient request and it is not. Has anyone dealt with this issue before or am I doing this wrong?
Here is the code snippet:
public RequestResponsePact pactMethod(PactDslWithProvider builder) {
Map<String, String> headers = new HashMap<>();
headers.put(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
return builder.given("Consumer request")
.uponReceiving(" getResource call")
.path("/path")
.method("GET")
.willRespondWith()
.status(200)
.headers(headers)
.body(RESPONSE_JSON).toPact();
}
#Test
#PactTestFor(pactMethod = "pactMethod", port = "9999")
public void consumerTest(MockServer mockServer) {
WebClient webClient = WebClient.builder().baseUrl(mockServer.getUrl()).build();
ConsumerServiceClient consumerServiceClient = new ConsumerServiceClient(webClient);
Mono<Data> data = consumerServiceClient.getData();
StepVerifier.create(data)
.assertNext(resp -> {
try {
Value value = resp.getValue();
Assertions.assertFalse( value.isEmpty());
} catch (Exception e) {
log.error("Unable to convert response to Value", e);
Assertions.fail();
}
}).expectComplete()
.verify();
}
The webclient call:
webClient.get()
.uri("/path")
.retrieve()
.onStatus(status -> status == HttpStatus.NOT_FOUND,
res -> Mono.error(new RunTimeException()))
.bodyToMono(clazz);
I have the following code to make a request that is going to be retried a max number of times. This request needs an authorization header and I'm caching this information to prevent this method to call the method to retrieve this information every time.
What I'm trying to do is:
When calling myMethod I first retrieve the login information for the service I'm calling, in most cases that will come from the cache when calling the getAuthorizationHeaderValue method.
In the web client, if the response to send this request returns a 4xx response I need to login again to the service I'm calling, before retrying the request. For that, I'm calling the tryToLoginAgain method to set the value for the header again.
After doing that the retry of the request should work now that the header has been set.
If by any chance the call to login again fails I need to stop retrying as there no use on retrying the request.
public <T> T myMethod(...) {
...
try {
AtomicReference<String> headerValue = new AtomicReference<>(loginService.getAuthorizationHeaderValue());
Mono<T> monoResult = webclient.get()
.uri(uri)
.accept(MediaType.APPLICATION_JSON)
.header(HttpHeaders.AUTHORIZATION, headerValue.get())
.retrieve()
.onStatus(HttpStatus::is4xxClientError, response -> throwHttpClientLoginException())
.bodyToMono(type)
.doOnError(HttpClientLoginException.class, e -> tryToLoginAgain(headerValue))
.retryWhen(Retry.backoff(MAX_NUMBER_RETRIES, Duration.ofSeconds(5)));
result = monoResult.block();
} catch(Exception e) {
throw new HttpClientException("There was an error while sending the request");
}
return result;
}
...
private Mono<Throwable> throwHttpClientLoginException() {
return Mono.error(new HttpClientLoginException("Existing Authorization failed"));
}
private void tryToLoginAgain(AtomicReference<String> headerValue) {
loginService.removeAccessTokenFromCache();
headerValue.set(loginService.getAuthorizationHeaderValue());
}
I have some unit tests and the happy path works fine (unauthorized the first time, try to login again and send the request again) but the scenario where the login doesn't work at all is not working.
I thought that if the tryToLoginAgain method throws an Exception that would be caught by the catch I have in myMethod but it never reaches there, it just retries the request again. Is there any way to do what I want?
So at the end I found a way of doing what I wanted and now the code looks like this:
public <T> T myMethod() {
try {
HttpHeaders headers = new HttpHeaders();
headers.setBearerAuth(getAuthorizationHeaderValue());
final RetryBackoffSpec retrySpec = Retry.backoff(MAX_NUMBER_RETRIES, Duration.ofSeconds(5))
.doBeforeRetry(retrySignal -> {
//When retrying, if this was a login error, try to login again
if (retrySignal.failure() instanceof HttpClientLoginException) {
tryToLoginAgain(headers);
}
});
Mono<T> monoResult = Mono.defer(() ->
getRequestFromMethod(httpMethod, uri, body, headers)
.retrieve()
.onStatus(HttpStatus::is4xxClientError, response -> throwHttpClientLoginException())
.bodyToMono(type)
)
.retryWhen(retrySpec);
result = monoResult.block();
} catch (Exception e) {
String requestUri = uri != null ?
uri.toString() :
endpoint;
log.error("There was an error while sending the request [{}] [{}]", httpMethod.name(), requestUri);
throw new HttpClientException("There was an error while sending the request [" + httpMethod.name() +
"] [" + requestUri + "]");
}
return result;
}
private void tryToLoginAgain(HttpHeaders httpHeaders) {
//If there was an 4xx error, let's evict the cache to remove the existing access_token (if it exists)
loginService.removeAccessTokenFromCache();
//And let's try to login again
httpHeaders.setBearerAuth(getAuthorizationHeaderValue());
}
private Mono<Throwable> throwHttpClientLoginException() {
return Mono.error(new HttpClientLoginException("Existing Authorization failed"));
}
private WebClient.RequestHeadersSpec getRequestFromMethod(HttpMethod httpMethod, URI uri, Object body, HttpHeaders headers) {
switch (httpMethod) {
case GET:
return webClient.get()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON);
case POST:
return body == null ?
webClient.post()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON) :
webClient.post()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON)
.contentType(MediaType.APPLICATION_JSON)
.bodyValue(body);
case PUT:
return body == null ?
webClient.put()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON) :
webClient.put()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON)
.contentType(MediaType.APPLICATION_JSON)
.bodyValue(body);
case DELETE:
return webClient.delete()
.uri(uri)
.headers(httpHeaders -> httpHeaders.addAll(headers))
.accept(MediaType.APPLICATION_JSON);
default:
log.error("Method [{}] is not supported", httpMethod.name());
throw new HttpClientException("Method [" + httpMethod.name() + "] is not supported");
}
}
private String getAuthorizationHeaderValue() {
return loginService.retrieveAccessToken();
}
By using Mono.defer() I can retry on that Mono and make sure I change the headers I'll use with the WebClient. The retry spec will check if the exception was of the HttpClientLoginException type, thrown when the request gets a 4xx status code and in that case it will try to login again and set the header for the next retry. If the status code was different it will retry again using the same authorization.
Also, if there's an error when we try to login again, that will be caught by the catch and it won't retry anymore.
I'm using Spring WebFlux 5.3.6's WebClient to stream a response from a REST endpoint that generates text/csv content.
I can use retrieve() and responseSpec.bodyToFlux to stream the body only like this:
WebClient.ResponseSpec responseSpec = headersSpec.retrieve();
Flux<DataBuffer> dataBufferFlux = responseSpec.bodyToFlux(DataBuffer.class);
DataBufferUtils
.write(dataBufferFlux, outputStream)
.blockLast(Duration.of(20, ChronoUnit.SECONDS));
but I want to get hold of the content-type header and validate it as part of the test. The above code provides access to the response body only, and not the headers.
I've tried to instead use exchangeToFlux() to get more control, and access to the response headers, but what I'm seeing is that the HTTP request is never made. If I add a breakpoint to myResponse.setStatus(clientResponse.rawStatusCode()); it is never hit.
A fuller code sample is below. I've struggled to find any examples of exchangeToFlux that use DataBuffer to stream the result back.
HttpClient httpClient = HttpClient.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 5000)
.responseTimeout(Duration.ofMillis(5000))
.doOnConnected(conn ->
conn.addHandlerLast(new ReadTimeoutHandler(5000, TimeUnit.MILLISECONDS))
.addHandlerLast(new WriteTimeoutHandler(5000, TimeUnit.MILLISECONDS)));
WebClient webClient = WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(httpClient))
.build();
WebClient.RequestHeadersSpec<?> headersSpec = webClient
.get()
.uri("http://localhost:8080/v1/users")
.header(CONTENT_TYPE, "text/csv");
MyResponse<T> myResponse = new MyResponse<>();
CountDownLatch latch = new CountDownLatch(1);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
headersSpec.exchangeToFlux(clientResponse -> {
// Never enters here!
myResponse.setStatus(clientResponse.rawStatusCode());
myResponse.setContentType(clientResponse.headers().contentType());
latch.countDown();
if (clientResponse.statusCode() == HttpStatus.OK) {
Flux<DataBuffer> dataBufferFlux = clientResponse.bodyToFlux(DataBuffer.class);
DataBufferUtils
.write(dataBufferFlux, outputStream)
.blockLast(Duration.of(20, ChronoUnit.SECONDS));
return dataBufferFlux;
}
return Flux.empty();
});
latch.await();
return myResponse;
Seem's like you are not subscribing to the Flux returned from headersSpec.exchangeToFlux.
headersSpec.exchangeToFlux(clientResponse -> {
// Never enters here!
myResponse.setStatus(clientResponse.rawStatusCode());
myResponse.setContentType(clientResponse.headers().contentType());
latch.countDown();
if (clientResponse.statusCode() == HttpStatus.OK) {
Flux<DataBuffer> dataBufferFlux = clientResponse.bodyToFlux(DataBuffer.class);
DataBufferUtils
.write(dataBufferFlux, outputStream)
.blockLast(Duration.of(20, ChronoUnit.SECONDS));
return dataBufferFlux;
}
return Flux.empty();
})
.subscribe(); // <- Subscribe is missing.
I am getting intermittent ReadTimeOut from netty with the below error:
The connection observed an error","logger_name":"reactor.netty.http.client.HttpClientConnect","thread_name":"reactor-http-epoll-3","level":"WARN","level_value":30000,"stack_trace":"io.netty.handler.timeout.ReadTimeoutException: null
One observation we made is this particular endpoint for which we are getting this issue is a POST with no request body. I am now sending a dummy json in body now which the downstream system ignores and now I don't see this error anymore at all.
Below is my code:
protected <T, S Mono<S sendMonoRequest (HttpMethod method,
HttpHeaders headers,
T requestBody,
URI uri, Class < S responseClass)
throws ApiException, IOException {
log.info("Calling {} {} {} {}", method.toString(), uri.toString(), headers.toString(),
mapper.writeValueAsString(requestBody));
WebClient.RequestBodySpec requestBodySpec = getWebClient().method(method).uri(uri);
headers.keySet().stream().forEach(headerKey -> headers.get(headerKey).stream().
forEach(headerValue -> requestBodySpec.header(headerKey, headerValue)));
return requestBodySpec
.body(BodyInserters.fromObject(requestBody != null ? requestBody : ""))
.retrieve()
.onStatus(HttpStatus::is4xxClientError, this::doOn4xxError)
.onStatus(HttpStatus::is5xxServerError, this::doOn5xxError)
.onStatus(HttpStatus::isError, this::doOnError)
.bodyToMono(responseClass);
}
protected WebClient getWebClient () {
HttpClient httpClient = HttpClient.create().tcpConfiguration(
client -> client.option(ChannelOption.CONNECT_TIMEOUT_MILLIS,
20000).doOnConnected(conn - conn
.addHandlerLast(new ReadTimeoutHandler(20)).addHandlerLast(new WriteTimeoutHandler(20))));
ClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
return WebClient.builder().clientConnector(connector)
.filter(logResponse())
.build();
}
To resolve the intemrittent timeouts, I have to send a dummy pojo to sendMonoRequest() for request body. Any ideas ?
I'm trying to figure out how to log exceptions from the webclient, whatever the error status code that is returned from the api that gets called.
I've seen the following implementation:
.onStatus(status -> status.value() != HttpStatus.OK.value(),
rs -> rs.bodyToMono(String.class).map(body -> new IOException(String.format(
"Response HTTP code is different from 200: %s, body: '%s'", rs.statusCode(), body))))
Another example I've seen uses a filter. I guess this filter could be used to log errors as well, aside from requests like in this example:
public MyClient(WebClient.Builder webClientBuilder) {
webClient = webClientBuilder // you can also just use WebClient.builder()
.baseUrl("https://httpbin.org")
.filter(logRequest()) // here is the magic
.build();
}
But are we serious that there is no dedicated exception handler to this thing?
Found it.
bodyToMono throws a WebClientException if the status code is 4xx (client error) or 5xx (Server error).
Full implementation of the service:
#Service
public class FacebookService {
private static final Logger LOG = LoggerFactory.getLogger(FacebookService.class);
private static final String URL_DEBUG = "https://graph.facebook.com/debug_token";
private WebClient webClient;
public FacebookService() {
webClient = WebClient.builder()
.filter(logRequest())
.build();
}
public Mono<DebugTokenResponse> verifyFbAccessToken(String fbAccessToken, String fbAppToken) {
LOG.info("verifyFacebookToken for " + String.format("fbAccessToken: %s and fbAppToken: %s", fbAccessToken, fbAppToken));
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(URL_DEBUG)
.queryParam("input_token", fbAccessToken)
.queryParam("access_token", fbAppToken);
return this.webClient.get()
.uri(builder.toUriString())
.retrieve()
.bodyToMono(DebugTokenResponse.class);
}
private static ExchangeFilterFunction logRequest() {
return ExchangeFilterFunction.ofRequestProcessor(clientRequest -> {
LOG.info("Request: {} {}", clientRequest.method(), clientRequest.url());
clientRequest.headers().forEach((name, values) -> values.forEach(value -> LOG.info("{}={}", name, value)));
return Mono.just(clientRequest);
});
}
#ExceptionHandler(WebClientResponseException.class)
public ResponseEntity<String> handleWebClientResponseException(WebClientResponseException ex) {
LOG.error("Error from WebClient - Status {}, Body {}", ex.getRawStatusCode(), ex.getResponseBodyAsString(), ex);
return ResponseEntity.status(ex.getRawStatusCode()).body(ex.getResponseBodyAsString());
}
}