I am using Okhttp Dispatcher in my application to send out HTTP2 request to a simulator which is an Reactor Netty HTTPSever . Actually we are doing a PT with 10 simulators , out of which 9 simulators have a delay of 50 ms , things work fine in this case , but when i introduce the 10th simulator with 10 seconds delay then immediately all of the requests are failing with executor rejected error response. Below is the dispatcher configuration used
Dispatcher dispatcher = new Dispatcher(new ThreadPoolExecutor(
1,
350,
60L,
TimeUnit.SECONDS,
new SynchronousQueue(),
Util.threadFactory(HTTP_THREAD_NAME_DEFAULT, false)));
dispatcher.setMaxRequestsPerHost(300);
dispatcher.setMaxRequests(500);
I have tried increasing the ThreadPoolExecutor max threads to 500 , MaxRequestsPerHost to 500 and MaxRequests to 800, but things don't work
NOTE: we are testing with 4k tps and its an springboot application not mobile application
OkHttp uses the parameters maxRequests and maxRequestsPerHost to determine concurrency. You should keep the default dispatcher and use these tuning parameters instead.
Related
RabbitMQ consumers are not adding up
I get the exact same issue, I've a RabittMQ listener defined with the following annotation and using Spring AMQP.
#RabbitListener(queues = "#{documentCreationQueue.name}", concurrency = "10-20")
It creates correctly the 10 consumers, but then it never increase to more that 10 even if I sent 20 requests and the 10 first one are in progress.
Did you find a solution to your problem couple of years ago ?
#GaryRussell any suggestions ?
I try to define a SimpleRabbitListenerContainerFactory with the annotation containerFactory and try multiple settings (prefectchCount, consecutiveActiveTrigger, taskExecutor with a thread pool....) without any success.
I have one issue in PRD. we recently released a springboot application and it has REST API exposed. Mobile/web APP call a legacy spring application which is in spring [not sprintboot] and it is a web applicationwhich then routes and makes a call to the these failing apis in new springboot. We are seeing timeout exception for these apis only .
there are lots of other OUTBOUND api calls made from spring legacy web application to other applications eg : login API [which has apis heavy traffic but these legacy apis work well and call other legacy applications.
There are no exception/error in logs in springboot application which has these REST API exposed. Infact we only see timeout in spring web application -meaning connection is exhausted but that does not explain why other apis OUTBOUND call are not failing which use same wrapper HTTPClient. Those which fail with timeout dont have request logs in springboot [ obviously because they dont leave spring web application tomcat JVM and die there due to timeout ]
So if we say connection pool is exhausted, the other heavey traffic OUTBOUnd calls should also face same issue but we dont see that.
All API call OUTWARD use HTTPCLient [apache.]
Not clear what is causing issue. I also explicitly defined below in new springboot for server side [I just did it to see if that makes difference but in vain]:
server:
tomcat:
connection-timeout: 10s
max-connections: 20000
max-threads: 500
min-spare-threads: 10
tomcat Log at spring web applicaiton [caller]:
org.apache.http.conn.ConnectionPoolTimeoutException
org.apache.http.conn.ConnectionPoolTimeoutException.Timeout waiting for connection from pool
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:313)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:279)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:191)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:221)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:140)
at
Any inputs?
Code snippet of Wrapper HTTPClient :
SSLContext sslContext = SSLContexts.createDefault();
HostnameVerifier hostnameVerifier = new DefaultHostnameVerifier();
SSLConnectionSocketFactory secureSSLConnectionSocketFactory = new SSLConnectionSocketFactory(
sslContext,
sslProtocolsArray,
ciphersArray,
hostnameVerifier);
ConnectionSocketFactory nonSecureConnectionSocketFactory = PlainConnectionSocketFactory.getSocketFactory();
Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder
.<ConnectionSocketFactory>create()
.register("https", secureSSLConnectionSocketFactory)
.register("http", nonSecureConnectionSocketFactory)
.build();
securePoolingConnectionManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);
securePoolingConnectionManager.setMaxTotal(this.connectionMgrMaxTotalSecure);
securePoolingConnectionManager.setDefaultMaxPerRoute(this.connectionMgrMaxPerRouteSecure);
SocketConfig secureSocketConfig = SocketConfig
.custom()
.setSoKeepAlive(true)
.setTcpNoDelay(true)
.build();
secureHttpsClient = HttpClients
.custom()
.setSSLSocketFactory(secureSSLConnectionSocketFactory)
.setConnectionManager(securePoolingConnectionManager)
.setDefaultRequestConfig(secureRequestConfig)
.setDefaultSocketConfig(secureSocketConfig)
.disableAutomaticRetries()
.build();
Stacktrace after above is just failing at wrapper HTTPClient method where call is invoked :
protected String execute(HttpClient httpclient, HttpRequestBase http) throws IOException {
String result;
ResponseHandler<String> responseHandler = new BasicResponseHandler();
result = httpclient.execute(http, responseHandler);
return result;
}
So I have to dig in another wrapper which was also using this HTTP pool and was being used in our legacy which was leaking. Closing this. Fortunately there was pool statistics api exposed so that I can see leased connection count which confirmed leaking. Since this second wrapper was used rerely and we had used in this release this was suspect and removing it solved the issue. It is another matter to dig that wrapper and find out how the pool was handled but the cause was caught.
I wanted to learn non blocking REST, but first I wrote blocking controller for comparison. To my surprise Spring doesn't block incoming requests.
Simple blocking service:
#Service
public class BlockingService {
public String blocking() {
try {
Thread.sleep(10000L);
} catch (InterruptedException ign) {}
return "Blocking response";
}
}
Simple REST controller:
#Slf4j
#RestController
public class BlockingRestController {
private final BlockingService blockingService;
#Autowired
public BlockingRestController(BlockingService blockingService) {
this.blockingService = blockingService;
}
#GetMapping("blocking")
public String blocking() {
log.info("Starting blocking request processing...");
return blockingService.blocking();
}
}
And I was thinking that when I send 4 requests using curl from 4 separated terminals I get:
1. Starting blocking request processing... (console where spring is running)
2. (4 terminals waiting)
3. "Blocking response" (in 1st terminal)
4. Starting blocking request processing... (console where spring is running)
5. (3 terminals waiting)
6. "Blocking response" (in 2nd terminal)
And so on...
But to my surprise I got:
1. Starting blocking request processing... (console where spring is running)
2. Starting blocking request processing... (console where spring is running)
3. Starting blocking request processing... (console where spring is running)
4. Starting blocking request processing... (console where spring is running)
5. "Blocking response" (in 1st terminal)
6. "Blocking response" (in 2nd terminal)
7. "Blocking response" (in 3rd terminal)
8. "Blocking response" (in 4th terminal)
Why first request doesn't block processing requests? Yet I don't create new threads and I don't process anything asynchronous?
Why do I ask about it? Because I want to learn how to use DeferredResult, but now I don't see a need.
It's blocking in the sense that it blocks one thread: the thread taken out of the pool of threads by the servlet container (Tomcat, Jetty, etc., not Spring) to handle your request. Fortunately, many threads are used concurrently to handle requests, otherwise the performance of any Java web application would be dramatic.
If you have, let's say, 500 concurrent requests all taking 1 minute to complete, and the pool of threads has 300 threads, then 200 requests will be queued, waiting for one of the threads to become available.
Absolutely NO since NIO!!!
Spring Web runs in a web container Tomcat or Netty, it is tomcat or Netty's work to create thread, not spring mvc or spring webflux.
If you use tomcat in a BIO model, it is definitely new thread per request.
Netty is of course NIO, tomcat supports NIO and APR, both are non-blocking,
Spring boot webmvc tomcat is default NIO, no need to worry about creating to many threads since nio.
Tomcat 8 NIO,how it works?
I don't want to block the message consumer with a receiver timeout, because as I understand it this "ties up" a thread in the queue server pool (and we have multiple queues configured). So how can I delay the message from being sent without a custom thread (the work is being performed by an EJB 2 session bean)? If it helps, JBoss Messaging 1.x is the provider.
Scheduled delivery is a feature of JBoss Messaging...
long now = System.currentTimeMillis();
Message msg =
sess.createMessage();
msg.setLongProperty(JBossMessage.JMS_JBOSS_SCHEDULED_DELIVERY_PROP_NAME, now + 1000 * 60 * 60 * 2);
prod.send(msg);
We have a use case that a single request from end user will trigger 1 or more http requests to other remote systems. All responses from remote systems will be aggregated into one single response to end user. We had this by using commonj workmanager in websphere.
Now, we want to use asyncHttpClient in order to have lesser threads to service multiple concurrent http calls.
So my question is:
How to integrate asyncHttpClient with commonj workmanager? We have to use commonj as thread pool as it's the only way to have managed threads in wehsphere.
Please advise.
All you need to do is create a new ExecutorService that leverages the work manager as shown below. More details...
Lookup the WorkManager and construct the WASThreadFactory
InitialContext ctx = new InitialContext();
WorkManager wm = (WorkManager)ctx.lookup("java:comp/env/wm/default");
ThreadFactory tf = new WASThreadFactory(wm);
Create a ThreadPoolExecutor using a bounded buffer
BlockingQueue q = new ArrayBlockingQueue(50);
ExecutorService myOwnThreadPool= = new ThreadPoolExecutor(
1, 10, 5000, TimeUnit.MILLISECONDS, q, tf);
According to their documentation, you can specify the ExecutorService to be used as below
Builder builder = new AsyncHttpClientConfig.Builder();
builder.setExecutorService(myOwnThreadPool);
AsyncHttpClient client = new AsyncHttpClient(builder.build());
NOTE: Didn't verify for compilation. I've however used this in my earlier project.