We have a use case that a single request from end user will trigger 1 or more http requests to other remote systems. All responses from remote systems will be aggregated into one single response to end user. We had this by using commonj workmanager in websphere.
Now, we want to use asyncHttpClient in order to have lesser threads to service multiple concurrent http calls.
So my question is:
How to integrate asyncHttpClient with commonj workmanager? We have to use commonj as thread pool as it's the only way to have managed threads in wehsphere.
Please advise.
All you need to do is create a new ExecutorService that leverages the work manager as shown below. More details...
Lookup the WorkManager and construct the WASThreadFactory
InitialContext ctx = new InitialContext();
WorkManager wm = (WorkManager)ctx.lookup("java:comp/env/wm/default");
ThreadFactory tf = new WASThreadFactory(wm);
Create a ThreadPoolExecutor using a bounded buffer
BlockingQueue q = new ArrayBlockingQueue(50);
ExecutorService myOwnThreadPool= = new ThreadPoolExecutor(
1, 10, 5000, TimeUnit.MILLISECONDS, q, tf);
According to their documentation, you can specify the ExecutorService to be used as below
Builder builder = new AsyncHttpClientConfig.Builder();
builder.setExecutorService(myOwnThreadPool);
AsyncHttpClient client = new AsyncHttpClient(builder.build());
NOTE: Didn't verify for compilation. I've however used this in my earlier project.
Related
I have one issue in PRD. we recently released a springboot application and it has REST API exposed. Mobile/web APP call a legacy spring application which is in spring [not sprintboot] and it is a web applicationwhich then routes and makes a call to the these failing apis in new springboot. We are seeing timeout exception for these apis only .
there are lots of other OUTBOUND api calls made from spring legacy web application to other applications eg : login API [which has apis heavy traffic but these legacy apis work well and call other legacy applications.
There are no exception/error in logs in springboot application which has these REST API exposed. Infact we only see timeout in spring web application -meaning connection is exhausted but that does not explain why other apis OUTBOUND call are not failing which use same wrapper HTTPClient. Those which fail with timeout dont have request logs in springboot [ obviously because they dont leave spring web application tomcat JVM and die there due to timeout ]
So if we say connection pool is exhausted, the other heavey traffic OUTBOUnd calls should also face same issue but we dont see that.
All API call OUTWARD use HTTPCLient [apache.]
Not clear what is causing issue. I also explicitly defined below in new springboot for server side [I just did it to see if that makes difference but in vain]:
server:
tomcat:
connection-timeout: 10s
max-connections: 20000
max-threads: 500
min-spare-threads: 10
tomcat Log at spring web applicaiton [caller]:
org.apache.http.conn.ConnectionPoolTimeoutException
org.apache.http.conn.ConnectionPoolTimeoutException.Timeout waiting for connection from pool
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:313)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:279)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:191)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:221)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:140)
at
Any inputs?
Code snippet of Wrapper HTTPClient :
SSLContext sslContext = SSLContexts.createDefault();
HostnameVerifier hostnameVerifier = new DefaultHostnameVerifier();
SSLConnectionSocketFactory secureSSLConnectionSocketFactory = new SSLConnectionSocketFactory(
sslContext,
sslProtocolsArray,
ciphersArray,
hostnameVerifier);
ConnectionSocketFactory nonSecureConnectionSocketFactory = PlainConnectionSocketFactory.getSocketFactory();
Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder
.<ConnectionSocketFactory>create()
.register("https", secureSSLConnectionSocketFactory)
.register("http", nonSecureConnectionSocketFactory)
.build();
securePoolingConnectionManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);
securePoolingConnectionManager.setMaxTotal(this.connectionMgrMaxTotalSecure);
securePoolingConnectionManager.setDefaultMaxPerRoute(this.connectionMgrMaxPerRouteSecure);
SocketConfig secureSocketConfig = SocketConfig
.custom()
.setSoKeepAlive(true)
.setTcpNoDelay(true)
.build();
secureHttpsClient = HttpClients
.custom()
.setSSLSocketFactory(secureSSLConnectionSocketFactory)
.setConnectionManager(securePoolingConnectionManager)
.setDefaultRequestConfig(secureRequestConfig)
.setDefaultSocketConfig(secureSocketConfig)
.disableAutomaticRetries()
.build();
Stacktrace after above is just failing at wrapper HTTPClient method where call is invoked :
protected String execute(HttpClient httpclient, HttpRequestBase http) throws IOException {
String result;
ResponseHandler<String> responseHandler = new BasicResponseHandler();
result = httpclient.execute(http, responseHandler);
return result;
}
So I have to dig in another wrapper which was also using this HTTP pool and was being used in our legacy which was leaking. Closing this. Fortunately there was pool statistics api exposed so that I can see leased connection count which confirmed leaking. Since this second wrapper was used rerely and we had used in this release this was suspect and removing it solved the issue. It is another matter to dig that wrapper and find out how the pool was handled but the cause was caught.
I'd like to monitor the connection pool of a Apache httpComponents PoolingHttpClientConnectionManager but cannot find a way to access it.
We create our HttpClient using the following builder:
HttpClient httpClient = HttpClientBuilder.create()
.setMaxConnTotal(maxConnections)
.setMaxConnPerRoute(maxConnectionsPerRoute)
.build();
This creates an instance of an InternalHttpClient, which holds an instance of a PoolingHttpClientConnectionManager, which holds an instance of a CPool.
CPool would give us access to T getRoutes() and PoolStats getStats(T) — which looks promising to me. But I can't really find out how to get access to this CPool.
HttpClient.getConnectionManager() is deprecated. InternalHttpClient.getConnectionManager() is not deprecated, but returns a custom connection manager, which only exposes some of the methods of the real connection manager instance behind it.
So, how to get access to these stats? These would be very helpful for us.
Use ConnPoolControl implemented by PoolingHttpClientConnectionManager
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
ConnPoolControl<HttpRoute> connPoolControl = cm;
CloseableHttpClient client = HttpClients.custom()
.setConnectionManager(cm)
.build();
I have one question regarding Spring WebClient
In my application I need to do many similar API calls, sometimes I need change headers in the calls (Authentication token). So the question arises, what would be better of the two options:
To create one WebClient for all incoming requests to MyService.class, by making it private final field, like code below:
private final WebClient webClient = WebClient.builder()
.baseUrl("https://another_host.com/api/get_inf")
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.defaultHeader(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE)
.build();
Here arises another question: is WebClient thread-safe? (because service is used by many threads)
To create new WebClient for each new request incoming to service class.
I want to provide maximum performance, and to use it in right way, but I don't know how WebClient works inside it, and how it expects to be used.
Thank you.
Two key things here about WebClient:
Its HTTP resources (connections, caches, etc) are managed by the underlying library, referenced by the ClientHttpConnector that you can configure on the WebClient
WebClient is immutable
With that in mind, you should try to reuse the same ClientHttpConnector across your application, because this will share the connection pool - this is arguably the most important thing for performance. This means you should try to derive all WebClient instances from the same WebClient.create() call. Spring Boot helps you with that by creating and configuring for you a WebClient.Builder bean that you can inject anywhere in your app.
Because WebClient is immutable it is thread-safe. WebClient is meant to be used in a reactive environment, where nothing is tied to a particular thread (this doesn't mean you cannot use in a traditional Servlet application).
If you'd like to change the way requests are made, there are several ways to achieve that:
configure things in the builder phase
WebClient baseClient = WebClient.create().baseUrl("https://example.org");
configure things on a per-request basis
Mono<ClientResponse> response = baseClient.get().uri("/resource")
.header("token", "secret").exchange();
create a new client instance out of an existing one
// mutate() will *copy* the builder state and create a new one out of it
WebClient authClient = baseClient.mutate()
.defaultHeaders(headers -> {headers.add("token", "secret");})
.build();
From my experience, if you are calling an external API on a server you have no control over, don't use WebClient at all, or use it with the pooling mechanism turned off. Any performance gains from connection pooling are greatly overweighed by the assumptions built into the (default reactor-netty) library that will cause random errors on one API call when another was abruptly terminated by the remote host, etc. In some cases, you don't even know where the error occurred because the calls are all made from a shared worker thread.
I made the mistake of using WebClient because the doc for RestTemplate said it would be deprecated in the future. In hindsight, I would go with regular HttpClient or Apache Commons HttpClient, but if you are like me and already implemented with WebClient, you can turn off the pooling by creating your WebClient as follows:
private WebClient createWebClient(int timeout) {
TcpClient tcpClient = TcpClient.newConnection();
HttpClient httpClient = HttpClient.from(tcpClient)
.tcpConfiguration(client -> client.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, timeout * 1000)
.doOnConnected(conn -> conn.addHandlerLast(new ReadTimeoutHandler(timeout))));
return WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(httpClient))
.build();
}
*** Creating a separate WebClient does not mean that WebClient will have a separate connection pool. Just look at the code for HttpClient.create - it calls HttpResources.get() to get the global resources. You could provide the pool settings manually but considering the errors that occur even with the default setup, I don't consider it worth the risk.
Just wondering if RestTemplate out of the box uses connection pooling or does it simply establish a new connection each time ?
Yes, Spring RestTemplateBuilder uses Apache HttpClient for pooling (usage).
RestTemplateBuilder creates HttpComponentsClientHttpRequestFactory and uses HttpClientBuilder.
HttpClientBuilder, by default, sets pool size per route (host) to 5 and total pool size to 10 (source):
s = System.getProperty("http.maxConnections", "5");
int max = Integer.parseInt(s);
poolingmgr.setDefaultMaxPerRoute(max);
poolingmgr.setMaxTotal(2 * max);
To check connection pool logging set logging level as follows:
org.apache.http.impl.conn.PoolingHttpClientConnectionManager=TRACE
I believe RestTemplate doesn’t use a connection pool to send requests, it uses a SimpleClientHttpRequestFactory that wraps a standard JDK’s HttpURLConnection opening and closing the connection.
Indeed you can configure RestTemplate to use a pooled implementation such as HttpComponentsClientHttpRequestFactory but most-likely you might also need to configure some settings to prevent requests from timing out.
I have blogged about this issue at Troubleshooting Spring's RestTemplate Requests Timeout
By default RestTemplate creates new Httpconnection every time and closes the connection once done.
If you need to have a connection pooling under rest template then you may use different implementation of the ClientHttpRequestFactory that pools the connections.
new RestTemplate(new HttpComponentsClientHttpRequestFactory())
You can create a Bean for RestTemplate and config there :
#Bean
public RestTemplate restTemplate() {
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(100);
connectionManager.setDefaultMaxPerRoute(20);
RequestConfig requestConfig = RequestConfig
.custom()
.setConnectionRequestTimeout(5000) // timeout to get connection from pool
.setSocketTimeout(5000) // standard connection timeout
.setConnectTimeout(5000) // standard connection timeout
.build();
HttpClient httpClient = HttpClientBuilder.create()
.setConnectionManager(connectionManager)
.setDefaultRequestConfig(requestConfig).build();
ClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(httpClient);
return new RestTemplate(requestFactory);
}
And there are a lot config you can do. Refer to https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/HttpClientBuilder.html
EDIT
If you want to use micrometer metrics you should also use a RestTemplateBuilder for constructing the RestTemplate.
In case of using Spring Boot configured with Apache HttpClient (having org.apache.httpcomponents:httpclient library in dependencies)
There is a default connection pool configured by PoolingHttpClientConnectionManager
Default concurrent settings for connections (you can find more about default settings here https://hc.apache.org/httpcomponents-client-4.5.x/current/tutorial/html/connmgmt.html):
PoolingHttpClientConnectionManager maintains a maximum limit of connections on a per route basis and in total. Per default this implementation will create no more than 2 concurrent connections per given route and no more 20 connections in total. For many real-world applications these limits may prove too constraining, especially if they use HTTP as a transport protocol for their services.
We can use okhttpclient underneath spring's rest template to use connection pooling. A detailed blog on this below
https://www.bytesville.com/changing-httpclient-in-spring-resttemplate/
I want to use AsyncRestTemplate for making a REST call in my service. According to Spring documentation, this class has 5 constructors( refer http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/web/client/AsyncRestTemplate.html)
AsyncRestTemplate(AsyncClientHttpRequestFactory asyncRequestFactory)-Using this AsyncClientHttpRequestFactory argument I will be able to configure the connection pool.
AsyncRestTemplate(AsyncListenableTaskExecutor taskExecutor)-Using this I will be able to configure the thread pool for the async operation.
I want to know if there is a way I could configure both connection pool and thread pool in AsyncRestTemplate.
Thank you very much in adance.
You can set task executor in SimpleClientHttpRequestFactory also:
ThreadPoolTaskScheduler taskExecutor = new ThreadPoolTaskScheduler();
taskExecutor.setPoolSize(10);
SimpleClientHttpRequestFactory simpleClientHttpRequestFactory = new SimpleClientHttpRequestFactory();
simpleClientHttpRequestFactory.setTaskExecutor(taskExecutor);
new AsyncRestTemplate(simpleClientHttpRequestFactory);