I have one simple web service (post) when running the tomcat with jdk8 151, getting the higher throughput while sending the request through JMeter. while with jdk8 161/171 throughput is comparatively very low.
Related
I running a Spring Boot 2.6.x application (bundled with Tomcat 9.56.x) with the following configuration:
server.tomcat.accept-count = 100
server.tomcat.threads.max = 1000
server.tomcat.threads.min-spare = 10
on a machine with 16 CPU cores and 32GB of RAM
I testing a performance load of my server, during which I'm opening multiple (500) connections and each one sends a HTTP request every 1 second.
Expected behavior: tomcat will attempt to use as much threads as possible in order to maximize a throughput.
Actual behavior: tomcat always stick to 10 threads (which are configured by "min-spare") and never adding threads above that configured amount. I know that by observing its JMX endpoint (currentThreadCount is always 10). This is despite that it definitely not able to process all requests in time, since I have growing amount of pending requests in my client.
Does anyone can explain me such behavior? Based on what Tomcat (the NIO thread pool) supposed to decide whether to add threads?
Turns out the issue was in my client.
For issuing requests, I was using RestTemplate which internally was using HttpClient. Well, HttpClient internally managing connections and by default it has ridiculously low limits configured - max 20 concurrent connections...
I solved the issue by configuring PoolingHttpClientConnectionManager (which supposed to deliver better throughput in multi-threaded environment) and increased limits:
HttpClientBuilder clientBuilder = HttpClientBuilder.create();
PoolingHttpClientConnectionManager connManager
= new PoolingHttpClientConnectionManager();
connManager.setMaxTotal(10000);
connManager.setDefaultMaxPerRoute(10000);
clientBuilder.setConnectionManager(connManager);
HttpClient httpClient = clientBuilder.build();
After doing that, I greatly increased issued requests per second which made Tomcat to add new threads - as expected
After upgrading to spring-boot 2.5, CancelledServerWebExchangeException started to appear in prometheus http_server_requests_seconds metrics quite frequently (up to 10% server responses end up with it, according to graphics). It appears in my own API metrics, as well as actuator endpoints metrics (health, info, prometheus).
Example:
http_server_requests_seconds_count{exception="CancelledServerWebExchangeException",method="GET",outcome="UNKNOWN",status="200",uri="/actuator/health"} 137.0
Kind of strange combination of outcome="UNKNOWN" & status="200"
The problem is: all these requests have successful responses.
Questions: what is this exception for and why may it occur so often?
How to reproduce: start application locally and put some load on it (I used 50 threads in jmeter accessing actuator endpoints)
I have developed a microservice using Spring Boot and deployed it as a docker container and when doing performance testing the service I see that the maximum number of threads that created to the service is 20 at any point in time even though the number of calls made is much higher. I have even set the max thread to 4000 and also set Max connection to 10000 along with all db configuration and server has 24 core and 64 gb ram still no improvement Are there any limitations with respect to number of calls that can be made to a microservice developed using Spring Boot or is the issue is with docker container? Or is this normal?
Am using spring boot Netflix zuul router version 2.1.1, we are doing a performance test with two services and the two services have a stubbed response. Our requirement Is to test with a Tps of 15 concurrently both the services. And the Response time of stub is 8 seconds, I checked the elapsed time for both the services and I did find that both services are taking an 8.4 seconds to respond. While we reached the TPS to 10.5 one of the service reached average response time to 11.5 seconds and the second service took like 9.5 seconds. Our Target is 15 TPS and test the service till 40 TPS to check the limit of our services. Am suspecting zuul Netflix router, as I tweaked the threads to 100 for tomcat I saw improvement from 12.5 to 11.5 seconds. Please let me know where am missing. What should I do to improve performance. Am not using eureka and I use routes to connect. Currently the limit is 200 with a refresh limit of 1 second.
I did simple performance comparison of SpringBoot versus Jakarta MicroProfile servers. Projects is available at:
https://github.com/HotswapProjects/pingperf
https://github.com/HotswapProjects/pingperf-spring-boot
The test uses Docker for simple server embedding, and JMeter as a client. The SpringBoot and Microprofiles based on Tomcat9 are using default Tomcat's thread pool settings (maxThreads=200). There is one problem I can't resolve. If the server is under heavy load from JMeter (50 threads, hardware Ryzen 1600, os linux), then CPU utilization is only 60% (checked in htop). I did the tests also out of Docker and there is no improvement, CPU utilization is still 60%. Is it possible to tune Tomcat9 settings to reach 100% CPU utilization in this test or is it platform problem?