What is the max number of concurrent websocket connections a tomcat instance can support? We are looking to serve 20000 connections at any given time. What would be the recommended number of tomcat instances to support the load?
Take a look at this: https://github.com/cloudmark/spring-websockets-performance-testing
They use Tomcat 8 but i think there will not be a fundamental difference in performance.
Related
How to configure spring boot server threads (for both IO and Worker) for undertow embedded server?
Thanks in advance.
As per the documentation:
server.undertow.threads.io //Number of I/O threads to create for the worker. The default is derived from the number of available processors.
server.undertow.threads.worker //Number of worker threads. The default is 8 times the number of I/O threads.
This is probably a rather peculiar question. I am using Spring Boot 2.0.2 with the default Tomcat container. In order to set up a test in our QA environment that simulates many servers, I would like to set up a Spring Boot-based REST service that listens on a very large number of ports simultaneously. I'm able to do this using the technique previously described in in another SO post (Configure Spring Boot with two ports) which basically adds connectors using TomcatServletWebServerFactory.addAdditionalTomcatConnectors().
The difficulty is that a large number of threads seem to be created for each additional port activated; some empirical measurements show the total to be 17 + (15 * number of ports). This means listening on 250 ports result in 3767 threads created and 500 ports result in 7517 threads created and I would like to go somewhat beyond that number. The test program used to take the above measurements is the bare minimum to bring up a Spring service and there is no code that creates threads explicitly so insofar, as I know, all of those threads were created by Spring/Tomcat.
Is there a way to accomplish this using Spring that doesn't use so many threads per active port? Would an alternate container like Jetty be more efficient?
You can configure the embedded tomcat container properties in spring boot configuration file and set the appropriate numbers for these properties to limit the threads created by Tomcat container -
server.tomcat.max-threads=200 # Maximum amount of worker threads.
server.tomcat.min-spare-threads=10 # Minimum amount of worker threads.
I am using spring boot 2.0.4.RELEASE. My doubt is whether my application is running in event loop style or not. I am using tomcat as my server.
I am running some performance tests in my application and after a certain time I see a strange behaviour. After the request reaches 500 req/second , my application is not able to serve more than 500 req/second. Via prometheus I was able to figure out max thread for tomcat were 200 by default. Looks like all the threads were consumed and that's why , it was not able to server more than 500 req/second. Please correct me if am wrong.
Can the tomcat server run in event-loop style ?
How can I change the event-loop size for tomcat server if possible.
Tried changing it to jetty still the same issue. Wondering if my application is running in event loop style.
Hey i think that you are doing something wrong in your project maybe one of your dependency does not support reactive programming. If you want to benefit from async programing(reactive) your code must be 100 reactive even for security you must use reactive spring security.
Normally a reactive spring application will run on netty not in tomcat so check your dependency because tomcat is not reactive
This is more of a analysis. After running some performance test on my local machine , I was able to figure out what was actually happening inside my application.
What I did was, ran performance test on my local machine and analysed the application through JConsole.
As I said I scheduled all my blocking dB calls to schedulers.elastic. What I realised that I it is causing the bottleneck. since my dB connections are limited and I am using hikari for connection pooling so it doesn’t matter the number of threads I create out of elastic pool.
Since reactive programming is more about consuming resource to the fullest with lesser number of threads, since the threads were being created in unbounded way so it was no different from normal application .
So what I did as part of resolution limited the number of threads to 100 that were supposed to be used by for dB calls. And bang number jumped from 500 tps to 2300 tps.
I know this is not the number which one should expect out of reactive application , it has much more capability. Since right now I do not have any choice but to bear with non reactive drivers .Waiting for production grade availability of reactive drivers for mssql server.
I am working on a application (Banking) which has a TPS requirement of 100 and multiple concurrent users.
Will Spring Boot 1.x.x allow me to achieve this?
Note: I would have used Spring Boot 2.x.x which supports Reactive paradigm but there is some legacy code which I have to use and it does not work on 2.x.x.
You can hit these numbers running a Java application on any reasonable hardware. LMAX claims that Disruptor can do over 100k TPS with 1ms latency. Spring Boot, or Java in general, won't be the limiting factor.
What will be the problem are the business requirements. If your application is to produce complex reports from over utilised database that's located in another data centre, well just the packet round-trip from CA to Netherlands is 150ms. If your SQL queries will take 30+ seconds, you are toast.
You can take a look at Tuning Tomcat For A High Throughput, Fail Fast System. It gives a good insight what can be tuned in a standard Tomcat deployment (assuming you will use Tomcat in Spring Boot). However it's unlikely that HTTP connections (assuming you will expose HTTP API) will be the initial bottleneck.
Many applications use connection pools for both HTTP and JDBC calls for resiliency. But using and configuring these 2 types of pools is very different. This duplicates the complexity of implementing resiliency patterns that are common to both - such as timeouts, retries, caching / alerting fallbacks, circuit breaking, and monitoring.
To my mind Hystrix offers common approaches of configuring and implementing these same resiliency patterns for both HTTP and JDBC calls.
My questions are:
Could Hystrix theoretically replace existing HTTP and JDBC
connection pools entirely?
If so, what are the pros and cons of doing so?
Replacing them entirely reduces the world of complexity that surrounds these connection pools - with their attendant timeout and validation query properties etc. However I am hazy about how Hystrix could "keep alive" JDBC / HTTP connections - and therefore avoid expensive connection setup costs - without delegating to existing libraries specialized for these tasks.
For context I have a DropWizard app, which uses Tomcat DBCP for its JDBC connection pool and Apache HttpClient for its HTTP connection pools.
No, Hystrix can not replace your connection pools.
Hystrix' main features are:
Limiting the number of calls to a service by using a limited thread pool or semaphores.
The possibility to time out calls to a service to avoid application threads being locked up waiting for slow/hung services.
Adding bulkheads so that one slow service minimally affects the rest of the application.
Circuit breaking slow/hung services.
The is no support for pooling connections.
I guess you can argue that the first point is somewhat related to a connection pool in that both Hystrix and a connection pool can limit the load against an other system. However, the main reason to have a connection pool is the performance gain of pooling connection. This load-limiting behavior is basically a bonus of connection pooling.
Hystrix could however compliment connection pools by providing the fail-fast timeout behavior and bulkheads if added in front of your connection pools, as you suggest in your question.