I have a Spring boot app that use HikariCP for Postgres connection pooling.
Recently I've set up tracing to collect some data how time is spent when handling a request to a specific endpoint.
My assumptions are that when using HikariCP:
The first connection to the database while handling the request might be a bit slower
Subsequent connections to the database should be fast (< 10 ms)
However, as the trace shows, the first connection is fast (< 10 ms). And while some subsequent connections during the same request handling are also fast (< 10 ms), I frequently see some subsequent connections taking 50-100ms, which seems quite slow to me, although I'm not sure if this is to be expected or not.
Is there anything I can configure to improve this behavior?
Maybe good to know:
The backend in question doesn't really see any other traffic right now, so it's only handling traffic when I manually send requests to it
I've changed maximumPoolSize to 1 to rule out that the issue is that it uses different connections in the context of 1 request and that's what causes the issue. The same behavior is still seen.
I use the default Hikari settings, I don't change them.
I do think something is wrong with your pool configuration or your usage of the pool if it takes roughly 10 ms to get an already initialized connection from your pool. I would expect it to be sub-millisecond... Are you sure you are using the pool correctly?
Make sure you are using as new versions of pool and driver as possible, and make sure that connectionTestQuery is not set, as that would execute a query every time the connection is obtained from the pool. The defaults should be good enough for the rest of the settings.
Debug logs could be one thing help figure out what is happening, metrics on the pool another. Have a look at Spring Boot Actuator, it will help you with that...
To answer your actual question on how you can improve the situation given it actually takes roughly 10 ms to obtain a connection: Do not obtain and return the connection to the pool for every query... If you do not want to pass the connection around in your code, and if it suits your use case, you can make this happen easily by making sure your whole request is wrapped in a transaction. See the Spring guide on managing transactions.
Related
I have a simple rest endpoint that executes Postgres procedure.
This procedure returns the current state of device.
For example:
20 devices.
Client app connect to API and make 20 responses to that endpoint every second.
For x clients there are x*20 requests.
For 2 clients 40 requests.
It causes a big cpu load on Postgres server only if there are many clients and/or many devices.
I didn’t create it but I need to redesign it.
How to limit concurrent queries to db only for it? It would be a hot fix.
My second idea is to create background worker that executes queries only one in the same time. Then the endpoint fetches data from memory.
I would try the simple way first. Try to reduce
the amount of database connections in the pool OR
the amount of working threads in the build-in Tomcat.
More flexible option would be to put the logic behind a thread pool limiting the amount of working threads. It is not trivial, if the Spring context and database is used inside a worker. Take a look on a Spring annotation #Async.
Offtopic: The solution we are discussing here looks like a workaround. The discussed solution alone will most probably increase the throughput only by factor 2 maybe 3. It is not JEE conform and it will be most probably not very stable. It is better to refactor the application avoiding such a problem. Another option would be to buy a new database server.
Update: JEE compliant solution would be to implement some sort of bulkhead pattern. It will limit the amount of concurrent running requests and reject it, if the some critical number is reached. The server application answers with "503 Service Unavailable". The client application catches this status and retries a second later (see "exponential backoff").
Can someone help me understand the Hikari implementation better. We currently have the following settings in use:
spring.datasource.hikari.maxLifetime=600000
spring.datasource.hikari.maximumPoolSize=3
spring.datasource.hikari.minimumIdle=1
What we see, is that after 10 min, the applications connections are cleared, and then, it struggles to make new connections to the DB, resulting in sessions exceeded errors, from the DB, we understand what they are, but it seems that rather than use the existing sessions, Hikari is trying to create new sessions, and the DB is not allowing that as is it supposed to.
Why is Hikari trying to establish new connections and not use connections from the available sessions ont he DB?
How does the maxlifetime work? reading the documents on the Hikari Git hub page, it seems straightforward enough.
We have also set
spring.datasource.hikari.idle-timeout=10000
on the idea that it may not kill those connections quite so quickly, but it still seems to remove them...
Thanks
I have a spring boot webflux application which by default uses netty.
One of the business requirements that we have mandates that requests should time out within 2 seconds.
When very few requests are sent to the app, everything is fine but when the request load is increased (Like over 40 or 50 concurrent per second by Jmeter) sometimes all of them time out due to each taking longer than the 2-second threshold.
I have spent a long time reading things online and looking into what could be causing this issue but with no success. When requests are sent concurrently most end up taking a long time and the problematic part is where an external HTTTP request is made to other microservice. All my tests are local and I have tested the microservices and they seem fast enough to handle a big load so the microservices themselves are not the issue.
I know that netty uses event loop and does not create a thread per request.
I believe there are likely synchronous tasks that are blocking those few netty threads. For this reason I have done massive refactoring and have ".publishOn(Schedulers.boundedElastic())" or ".subscribeOn(Schedulers.boundedElastic())" in the Mono reactive chains. After the refactoring Most of the operations seem to be running on elastic threads and not the "reactor-http-nio-x" (According to the logs) but doing so has not helped the main issue and the problem still exists.
It will be a huge help if someone could direct me to what I should be doing. At this point, I have no more improvements to make, and think I might have been looking at this the wrong way and my approach has not been correct.
I have not attached any code sine the application is big and I do not still know where the actual problem lies.
I've encountered the same problem. I've didn't find the root cause of this, but when I switched from WebClient to RestTemplate with dedicated thread pool per client (external service) then the problem was solved. I've run a blockhound to find if I block somewhere in the stream, but it didn't find anything. I've also tried deploying my application with increased number of NIO worker thread pool (by default it's equal to cores number) and there was some improvement, but after all RestTemplate yielded the best performance. So I'm still on Webflux stack, but I don't use WebClient anymore and the performance on high load is fine.
Everything with my co-located cache works fine as long as there is one request at a time. But when I hit my service with several concurrent requests, my cache doesn't seem to work.
Preliminary analysis led me to this - https://azure.microsoft.com/en-us/documentation/articles/cache-dotnet-how-to-use-service/
Apparently, I would have to use maxConnectionsToServer to allow multiple concurrent connections to cache. But the document also talks about a useLegacyProtocol parameter which has to be set to false to enable connection pooling.
I have the following questions:
My service would be getting a few hundred concurrent requests. Would this be a
good setting for such a scenario:
<dataCacheClient name="default" maxConnectionsToServer="100"
useLegacyProtocol="false">
This is my understanding of the behavior I would get with this configuration - Each time a request comes in, an attempt would be made to retrieve a connection from the pool. If there is no available connection, a new connection would be created if there are less than 100 connections currently, else the request would fail. Please confirm if this is correct.
The above documentation says that one connection would be used per instance of DataCacheFactory. I have a cache manager class which manages all interactions with cache. This is a singleton class. It creates a DataCacheFactory object and uses it to get a handle to the cache during its instantiation. My service would have 2 instances. Looks like I would need only 2 connections to server. Is this correct? Do I even need connection pooling?
What is the maximum value maxConnectionsToServer can accept and what would be an ideal value for the given scenario?
I also see a boolean paramater named "ConnectionPool". This looks complementary to "useLegacyProtocol". Is this not redundant? How is setting useLegacyProtocol="false" different from connectionPool="true"? I am confused as to whether or not and how to use this parameter.
Are maxConnectionsToServer and ConnectionPool parameters related in any way? What does it mean when I have maxConnectionsToServer set to 5 and ConnectionPool=true?
Can somebody explain what ajax-push is? From what I understand it involves leaving HTTP connections open for a long time and reconnecting as needed. It seems to be used in chat systems a lot.
I have also heard when using ajax-push in Java it is important to use something with the NIO-connetors or grizzle serlvet api? Again, I'm just researching what it exactly.
In normal AJAX (call it pull) you ask the server for something and you get it immediately. This is fine when you want to get some data from the server now. But what if something happens on the server and the server wants to push that event to the client(s)?
Technically this is implemented using so called long polling - the browser opens the HTTP connection and waits for the response. As long as there is nothing interesting on the server side, it waits. But when something happens, the server sends the response and the client receives it immediately. This is a huge advantage over normal polling where you ask the server every few seconds - it generates a lot of traffic and still introduces noticeable latency.
The only problem with this approach is the number of pending HTTP connections. Old-school Java servlet containers aren't quite capable of handling such amount of connections due to one-thread-per-connection limitation - they quickly run out of memory. Even though the HTTP threads aren't doing anything (waiting for some other part of the system to wake them up and give them the response), they occupy memory.
However there are plenty of solutions nowadays:
Tomcat NIO connectors
Atmosphere Ajax Push/Comet library
Servlet 3.0 #Async (most portable)
Container-specific features, but Servlet 3.0, if available, should be considered superior.