I am using Spring Rest template along with apache's PoolingHttpClientConnectionManager for making API calls. The project in which I am working on requires setting custom timeout for each of the HTTP request I make via rest template. In order to achieve this, I am using CompletableFuture with a separate ExecutorService and calling get(Timeout) method.
try{
CompletableFuture<BidResponse> future = CompletableFuture.supplyAsync(() -> bidderService.getBid(), executorService);
bidResponse = future.get(bidderTimeout, TimeUnit.MILLISECONDS);
} catch (InterruptedException | TimeoutException | ExecutionException e) {
bidResponse = getTimeoutBidResponse();
}
Unfortunately, the problem with this approach is that in cases of timeout, the underlying thread keeps on working until the rest template finishes its call. So I am kind of losing out a thread from the thread pool, as well as a connection from the HTTP connection pool.
Is there a way to close the HTTP connection as soon as we receive a Timeout exception, and return the HTTP connection back to the pool ?
p.s. I also tried using Spring Webclient with Mono.timeout. Turns out it actually closes the HTTP connection immediately, but does not return it back to the HTTP pool.
#Bean
public RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder)
{
return restTemplateBuilder
.setConnectTimeout(...)
.setReadTimeout(...)
.build();
}
Related
I am working on a Spring application that serves up REST endpoints. One of the endpoints essentially acts as a proxy between the HTML client and a third party cloud storage provider. This endpoint retrieves files from the storage provider and proxies them back to the client. Something like the following (note there is a synchronous and asynchronous version of the same endpoint):
#Controller
public class CloudStorageController {
...
#RequestMapping(value = "/fetch-image/{id}", method = RequestMethod.GET, produces = MediaType.IMAGE_JPEG_VALUE)
public ResponseEntity<byte[]> fetchImageSynchronous(#PathVariable final Long id) {
final byte[] imageFileContents = this.fetchImage(id);
return ResponseEntity.ok().body(imageFileContents);
}
#RequestMapping(value = "/fetch-image-async/{id}", method = RequestMethod.GET, produces = MediaType.IMAGE_JPEG_VALUE)
public Callable<ResponseEntity<byte[]>> fetchImageAsynchronous(#PathVariable final Long id) {
return () -> {
final byte[] imageFileContents = this.fetchImage(id);
return ResponseEntity.ok().body(imageFileContents);
};
}
private byte[] fetchImage(final long id) {
// fetch the file from cloud storage and return as byte array
...
}
...
}
Due to the nature of the client app (HTML5 + ajax) and how this endpoint is used, user authentication is supplied to this endpoint differently that the other endpoints. To handle this, a HandlerInterceptor was developed to deal with authentication for this endpoint:
#Component("cloudStorageAuthenticationInterceptor")
public class CloudStorageAuthenticationInterceptor extends HandlerInterceptorAdapter {
#Override
public boolean preHandle(final HttpServletRequest request, final HttpServletResponse response, final Object handler) {
// examine the request for the authentication information and verify it
final Authentication authenticated = ...
if (authenticated == null) {
try {
pResponse.sendError(HttpServletResponse.SC_UNAUTHORIZED);
} catch (IOException e) {
throw new RuntimeException(e);
}
return false;
}
else {
try {
request.login(authenticated.getName(), (String) authenticated.getCredentials());
} catch (final ServletException e) {
throw new BadCredentialsException("Bad credentials");
}
}
return true;
}
}
The interceptor is registered like this:
#Configuration
#EnableWebMvc
public class ApiConfig extends WebMvcConfigurerAdapter {
#Autowired
#Qualifier("cloudStorageAuthenticationInterceptor")
private HandlerInterceptor cloudStorageAuthenticationInterceptor;
#Override
public void addInterceptors(final InterceptorRegistry registry) {
registry.addInterceptor(this.cloudStorageAuthenticationInterceptor)
.addPathPatterns(
"/fetch-image/**",
"/fetch-image-async/**"
);
}
#Override
public void configureAsyncSupport(final AsyncSupportConfigurer configurer) {
final ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(this.asyncThreadPoolCoreSize);
executor.setMaxPoolSize(this.asyncThreadPoolMaxSize);
executor.setQueueCapacity(this.asyncThreadPoolQueueCapacity);
executor.setThreadNamePrefix(this.asyncThreadPoolPrefix);
executor.initialize();
configurer.setTaskExecutor(executor);
super.configureAsyncSupport(configurer);
}
}
Ideally, the image fetching would be done asynchronously (using the /fetch-image-asyc/{id} endpoint) because it has to call a third party web service which could have some latency.
The synchronous endpoint (/fetch-image/{id}) works correctly for all browsers. However, if using the asynchronous endpoint (/fetch-image-async/{id}), Chrome and Firefox work as expect.
However, if the client is Microsoft IE or Microsoft Edge, we seem some strange behavior. The endpoint is called correctly and the response sent successfully (at least from the server's viewpoint). However, it seems that the browser is waiting for something additional. In the IE/Edge DevTools window, the network request for the image shows as pending for 30 seconds, then seems to timeout, updates to successful and the image is successfully display. It also seems the connection to the server is still open, as the server side resources like database connections are not released. In the other browsers, the async response is received and processed in a second or less.
If I remove the HandlerInterceptor and just hard-wire some credentials for debugging, the behavior goes away. So this seems to have something to with the interaction between the HandlerInterceptor and the asynchronous controller method, and is only exhibited for some clients.
Anyone have a suggestion on why the semantics of IE/Edge are causing this behavior?
Based on your description, there are some different behaviors when using IE or Edge
it seems that the browser is waiting for something additional
the connection seems still open
it works fine if remove HandlerInterceptor and use hard code in auth logic
For the first behavior, I would suggest you use fiddler to trace all http requests. It is better if you could compare two different actions via fiddler (1) run on chrome, 2) run on edge ). Check all http headers in requests and responses carefully to see whether there is some different part. For the other behaviors, I would suggest you write logs to find which part spend the most time. It will provide you useful information to troubleshot.
After much tracing on the server and reading through the JavaDocs comments for AsyncHandlerInterceptor, I was able to resolve the issue. For requests to asynchronous controller methods, the preHandle method of any interceptor is called twice. It is called before the request is handed off to the servlet handling the request and again after the servlet has handled the request. In my case, the interceptor was attempting to authenticate the request for both scenarios (pre and post request handling). The application's authentication provider checks credentials in a database. For some reason if the client is IE or Edge, the authentication provider was unable to get a database connection when called from preHandle in the interceptor after the servlet handled the request. The following exception would be thrown:
ERROR o.a.c.c.C.[.[.[.[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.dao.DataAccessResourceFailureException: Could not open connection; nested exception is org.hibernate.exception.JDBCConnectionException: Could not open connection] with root cause
java.sql.SQLTransientConnectionException: HikariPool-0 - Connection is not available, request timed out after 30001ms.
So the servlet would successfully handle the request and send a response, but the filter would get hung up for 30 seconds waiting for the database connection to timeout on the post processing called to preHandle.
So for me, the simple solution was to add a check in preHandle if it is being called after the servlet has already handled the request. I updated the preHandle method as follows:
#Override
public boolean preHandle(final HttpServletRequest pRequest, final HttpServletResponse pResponse, final Object pHandler) {
if (pRequest.getDispatcherType().equals(DispatcherType.REQUEST)) {
... perform authentication ...
}
return true;
}
That solved the issue for me. It doesn't explain everything (i.e., why only IE/Edge would cause the issue), but it seems that preHandle should only do work before the servlet handles the request anyways.
I have the event stream code block below:
#RequestMapping(value = "/stream/{columnId}/data", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
#ResponseBody
public Flux<Activity> streamingData(#PathVariable String columnId, HttpSession httpSession) {
try {
ColumnObject columnObject = streamHelper.findColumnObjectInListById(columnId);
return streamHelper.getStreamData(httpSession.getId(), columnObject);
} catch (Exception e) {
...
}
}
After creating 6 columns through the endpoint, spring server will put all subsequent requests in the pending state.
( get, post methods for example)
#RequestMapping(value = "/session/metrics", method = RequestMethod.GET)
public ResponseEntity<?> keepSessionAliveMetrics(HttpSession httpSession) {
return new ResponseEntity<Void>(HttpStatus.OK); // STATE ONLY PENDING
}
As pointed out by Marten Deinum, this is a typical hard limit: browsers tend to limit the number of concurrent connections to a given domain.
If your application requires a lot of multiplexing, maybe using WebSockets is a better option as many message channels are established on a single TCP connection.
To bypass this limitation of the browsers, you can use Server-Sent Events over HTTP/2: HTTP/2 provides a multiplexing feature that enables to circumvent this HTTP/1 issue.
There is an excellent article about that: https://www.smashingmagazine.com/2018/02/sse-websockets-data-flow-http2/
SpringBoot has an HTTP/2 support but may be not Spring WebFlux (with Netty, at least, at the time of writing): https://docs.spring.io/spring/docs/current/spring-framework-reference/web-reactive.html#webflux-http2
Using SSE over HTTP/2 is certainly the best way to overcome the limitations your are experiencing. You can find more detailed information here: https://www.infoq.com/articles/websocket-and-http2-coexist
I have a spring integration flow defined in the flow DSL syntax. One of my handlers is a Webflux.outboundGateway. When the remote URI is not accessible, an exception is thrown and sent to the "errorChannel". I'm trying to have the flow to retry, but so far with no success (the call is never retried). Here is what my configuration looks like:
#Bean
public IntegrationFlow retriableFlow() {
return IntegrationFlows
.from(...)
.handle(
WebFlux.outboundGateway(m ->
UriComponentsBuilder.fromUriString(remoteGateway + "/foo/bar")
.build()
.toUri(), webClient)
.httpMethod(HttpMethod.POST)
.expectedResponseType(String.class)
.replyPayloadToFlux(true), e -> e.advice(retryAdvice())
)
// [ ... ]
.get();
}
#Bean
public Advice retryAdvice() {
RequestHandlerRetryAdvice advice = new RequestHandlerRetryAdvice();
RetryTemplate retryTemplate = new RetryTemplate();
ExponentialBackOffPolicy retryPolicy = new ExponentialBackOffPolicy();
retryPolicy.setInitialInterval(1000);
retryPolicy.setMaxInterval(20000);
retryTemplate.setBackOffPolicy(retryPolicy);
advice.setRetryTemplate(retryTemplate);
return advice;
}
Should I be using something different than the RequestHandlerRetryAdvice? If so, what should it be?
Webflux is, by definition, async, which means the Mono (reply) is satisfied asynchronously when the request completes/fails, not on the calling thread. Hence the advice won't help because the "send" part of the request is always successful.
You would have to perform retries via a flow on the error channel (assigned somewhere near the start of the flow). With, perhaps, some header indicating how many times you have retried.
The ErrorMessage has properties failedMessage and cause; you can resend the failedMessage.
You could turn off async so the calling thread blocks, but that really defeats the whole purpose of using WebFlux.
We are using Spring-boot with Spring-cloud and Spring-cloud-netflix with Spring-cloud-feign.
We are creating our Gateway application that with the help of Feign will try to communicate with our authentication microservice in order to validate their credentials. Here you can see an example of our Feign authentication client:
#FeignClient(value="auth", configuration = AuthClientConfiguration.class)
public interface AuthClient {
#RequestMapping(method = RequestMethod.GET, value = "/tokens", consumes = MediaType.APPLICATION_JSON_VALUE)
Single<Session> getSession(#RequestHeader("Authorization") String token);
}
The question is, how we can deal with all the exceptions that could be raised by the client? I mean, how we can for example catch that a NetworkException or a TimeoutException has been thrown? We've defined our own ErrorDecoder but it appears that this "kind of listener" only works when the request has arrived and the response returned (in our case from authentication client). So, how we can manage this other exceptions?
Best,
Error decoders are decoding HTTP error responses (500, 404, 401, etc...). Exceptions will bubble up in client calls, so using try/catch should work.
try {
return client.home();
} catch (RuntimeException e) {
e.printStackTrace();
throw e;
}
I've read the Jersey documentation, and it says Jersey automatically closes a connection after an entity is read (e.g. response.readEntity(SomeObject.class))
But when an exception is thrown, either a bad request or a socket timeout, does Jersey automatically close the connection, or should I have a finally clause that calls client.close()?
No. Neither does Jersey call client.close() in case of an exception nor does the JerseyClient implement AutoCloseable.
You can easily test this. A client throws a IllegalStateException if you invoke a method after closing:
Client client = ClientBuilder.newClient();
client.close();
client.target("http://stackoverflow.com").request().get(); // IllegalStateException
But you can invoke a method after catching an exception:
Client client = ClientBuilder.newClient();
try {
client.target("http://foo.bar").request().get(); // java.net.ConnectException: Operation timed out
} catch (Exception ex) {
client.target("http://stackoverflow.com").request().get(); // works
}
So closing is your job.
Update: JAX-RS 2.1 will use AutoClosables.