I have spring reactive a micro-service which call another micro-service to get list of products, but it keep failing with the error
reactor.netty.http.client.PrematureCloseException: Connection prematurely closed DURING response
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Assembly trace from producer [reactor.core.publisher.FluxLift] :
This is how i get WebClient object
public WebClient createWebClient(HttpRequestConfig config, String baseUri) {
Builder clientBuilder = webClientBuilder(config)
.clientConnector(new ReactorClientHttpConnector(HttpClient.newConnection().compress(true)))
.baseUrl(baseUri);
return clientBuilder.build();
}
Any idea about the issue?
Related
Spring security documentation https://docs.spring.io/spring-security/reference/servlet/oauth2/resource-server/jwt.html#oauth2resourceserver-jwt-timeouts
states that:
By default, Resource Server uses connection and socket timeouts of 30 seconds each for coordinating with the authorization server.
I created JwtDecoder in the following way:
#Bean
JwtDecoder jwtDecoder() {
NimbusJwtDecoder jwtDecoder = NimbusJwtDecoder.withJwkSetUri(jwkSetUri).build();
return jwtDecoder;
}
jwkSetUri is set to some non-existent ip.
Now, making request to my resource server gets a timeout (as expected)
and the following exception is thrown:
An error occurred while attempting to decode the Jwt: Couldn't retrieve remote JWK set: org.springframework.web.client.ResourceAccessException: I/O error on GET request: Connect to jwkSetUri [jwkSetUri ] failed: connect timed out; nested exception is org.apache.http.conn.ConnectTimeoutException: Connect to jwkSetUri failed: connect timed out
However, the time after which the exception is thrown does not match what is described in the documentation.
When I run the application on windows, it takes about 20 seconds. When I run on Linux, it takes about 2-3 minutes.
It looks like it depends on the operating system.
However, when I manually set the timeout as follows:
#Bean
public JwtDecoder jwtDecoder(RestTemplateBuilder builder) {
RestOperations rest = builder
.setConnectTimeout(Duration.ofSeconds(5))
.setReadTimeout(Duration.ofSeconds(5))
.build();
NimbusJwtDecoder jwtDecoder = NimbusJwtDecoder.withJwkSetUri(jwkSetUri).restOperations(rest).build();
return jwtDecoder;
}
Then as expected I get a timeout after 5 seconds.
Am I missing something or the default value given in the documentation is incorrect?
I am facing connection error. Log entries are
readAddress(..) failed: Connection reset by peer; nested exception is io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
the connection observed an error
Pending acquire queue has reached its maximum size of 1000; nested exception is reactor.netty.internal.shaded.reactor.pool.PoolAcquirePendingLimitException
Webclient config is:
#Bean
public WebClient webClient(#Autowired ObjectMapperBean objectMapperBean) {
ConnectionProvider provider =
ConnectionProvider
.builder("custom")
.maxConnections(500)
.build();
HttpClient httpClient = HttpClient.create(provider);
ExchangeStrategies exchangeStrategies =
ExchangeStrategies
.builder()
.codecs(codecConfigurer -> codecConfigurer
.defaultCodecs()
.jackson2JsonEncoder(new Jackson2JsonEncoder(objectMapperBean.getObjectMapper(), MediaType.APPLICATION_JSON)))
.build();
return WebClient
.builder()
.clientConnector(new ReactorClientHttpConnector(httpClient))
.exchangeStrategies(exchangeStrategies)
.build();
}
I am not sure where the problem is. Can someone help me on this?
#springboot #webclient
By default the number of pending queue requests allowed for a reactor netty server is
2 * number of connections
if not provided explicitly. You need to take care of these configuration on the basis of your throughput expectations. You can modify this property as follows :
ConnectionProvider.builder(CONNECTION_NAME)
.maxConnections(<MaxConnectionThreads>)
.pendingAcquireTimeout(Duration.ofMillis(<PendingAcquireTimeout>))
.pendingAcquireMaxCount(<maxCount>)
.maxIdleTime(Duration.ofMillis(<MaxIdleTime>))
.build();
I am using webClinet to consume OAuth2 secured services. When services takes time more than default timeout i get below error and request is retried.
io.netty.channel.unix.Errors$NativeEceptionIoException: readAddress failed: Connection reset by peer
The connection observed an error, the request will be retried.
This seem to be IO exception issue with netty.
How do I avoid retry in such scenario?
Here is my webclient configuration-
WebClient webclinet(OAuth2AuthorizedClientManager am){
ExchangeStrategies ex = ExchangeStrategies
.builder()
.codec(c-> c.defaultCodecs().maxInMemorySize(-1)).build();
ServletOAuth2AuthorizedClientExchangeFilterFunction oauth =
new ServletOAuth2AuthorizedClientExchangeFilterFunction(am);
SslContext ssl = new JdkSslContext(SSLContext.getDefault(),true,ClientAuth.REQUIRE);
ReactorClientHttpConnector clientHttpConnector = new ReactorClientHttpConnector(Http.create()
.secure(sslContextSpec-> sslContextSpec.sslContext(ssl)));
return WebClient.builder()
.exchangeStrategies(ex)
.clientConnector(clientHttpConnector)
.filter(oauth)
.build();
There is a property named sslContextSpec.sslContext(ssl).disableRetry(true) which can be used to disable retry functionality of it.
I am using Spring Rest template along with apache's PoolingHttpClientConnectionManager for making API calls. The project in which I am working on requires setting custom timeout for each of the HTTP request I make via rest template. In order to achieve this, I am using CompletableFuture with a separate ExecutorService and calling get(Timeout) method.
try{
CompletableFuture<BidResponse> future = CompletableFuture.supplyAsync(() -> bidderService.getBid(), executorService);
bidResponse = future.get(bidderTimeout, TimeUnit.MILLISECONDS);
} catch (InterruptedException | TimeoutException | ExecutionException e) {
bidResponse = getTimeoutBidResponse();
}
Unfortunately, the problem with this approach is that in cases of timeout, the underlying thread keeps on working until the rest template finishes its call. So I am kind of losing out a thread from the thread pool, as well as a connection from the HTTP connection pool.
Is there a way to close the HTTP connection as soon as we receive a Timeout exception, and return the HTTP connection back to the pool ?
p.s. I also tried using Spring Webclient with Mono.timeout. Turns out it actually closes the HTTP connection immediately, but does not return it back to the HTTP pool.
#Bean
public RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder)
{
return restTemplateBuilder
.setConnectTimeout(...)
.setReadTimeout(...)
.build();
}
I am trying to reverse engineer a working pice of software, to come up with a raw HTTP request, so I can fire it with Postman or other HTTp client.
import org.springframework.security.oauth2.client.OAuth2RestOperations;
private OAuth2RestOperations restTemplate;
#Autowired
public void setRestTemplate(OAuth2RestOperations restTemplate) {
this.restTemplate = restTemplate;
}
private QuoteWorkflowResponse saveQuote(IQuoteRequest request) {
RequestEntity<IQuoteRequest> requestEntity = RequestEntity.post(getPathToPurchaseUrl(request))
.body(request);
ResponseEntity<QuoteWorkflowResponse> responseEntity = restTemplate.exchange(requestEntity, QuoteWorkflowResponse.class);
if (responseEntity.getStatusCode() != HttpStatus.OK) {
handleError(responseEntity);
}
return responseEntity.getBody();
}
This is somehow using Barrer Token, I fetched the token using the debugger and tested it in postman and it works.
My question here is, why restTemplate.exchange is throwing a ResourceAccessException ? what does that means ? and how can I fix it ?
Exception this is throwing:
org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://somehost/aabb/sales/quote": stream is closed; nested exception is java.io.IOException: stream is closed
Thanks to #MrFoll, I have added this to my applicaion.yml
logging:
level:
org:
springframework:
web:
client:
RestTemplate: "DEBUG"
and fixed the issue.