I'm using springboot+hikari+hibernate to build a rest API.
Some queries are really have and the database is really polluted and worst, there is a clear connection leak.
This is a legacy system which I wasn't part of design nor development but I need to do maintenance and I would like to debug where is the connection leak.
As far as I can see all transactions are annotated #Transaction... so there shouldn't be any connection "forgotten open"...
Is there any way for me to build a configuration class or annotate a method with something in order to be invoked right before any thread request a connection and right after the thread releases it? This way I could try to trace where is the leak source.
I've this settings on
spring.datasource.hikari.connection-timeout=60000
spring.datasource.hikari.maximum-pool-size=30
spring.datasource.hikari.idle-timeout=120000
spring.datasource.hikari.minimum-idle=10
spring.datasource.hikari.maxLifetime=1800000
spring.datasource.hikari.leakDetectionThreshold=60000
logging.level.com.zaxxer.hikari.HikariConfig=DEBUG
logging.level.com.zaxxer.hikari=TRACE
But this generates a very verbose log output and this system is in production so is almost impossible to track the problem.... I would like to create just a method for every connection acquire and release so I could myself write the log and find the source.
Related
I have come across some datasource properties for Spring Boot, that are specified in the application.properties. I am having a hard time understanding the purpose of the datasource properties.
Also, I am not able to find the explanation of these properties here for datasource - https://docs.spring.io/spring-boot/docs/1.1.2.RELEASE/reference/html/common-application-properties.html and I am using Spring Boot version 1.
spring.datasource.test-while-idle=true
spring.datasource.initial-size=10
spring.datasource.min-idle=10
spring.datasource.time-between-eviction-runs-millis=300000
spring.datasource.validation-query=SELECT 1 from DUAL
spring.datasource.test-on-borrow=true
spring.datasource.test-on-connect=true
spring.datasource.validation-interval=300000
The properties are validation-interval and time-between-eviction-runs-millis. What is the difference between them? The latter runs to evict the dead connections. But what about validation-interval?
Difference between test-on-borrow and test-on-connect?
I can't seem to find the right place to see their documentation or their purpose, or I am looking at the wrong place.
Please advise.
Following explanation is based on my experience, for your reference.
spring.datasource.time-between-eviction-runs-millis means the time interval to recycle idle connection. It is usually used with spring.datasource.test-while-idle.
spring.datasource.min-evictable-idle-time-millis means the effective time of idle connection in connection pool.
spring.datasource.test-on-borrow means whether testing the connection while borrowing the connection from connection pool. And it may have some impact on the performance.
spring.datasource.test-on-connect means whether testing the connection while creating the connection.
I'm looking at performance issues with a Grails application, and the suggestion is to remove the transactions from the services.
Is there a way that I can measure the change in the service?
Is there a place that has data on how expensive transactions are? [Time and resource-wise]
If someone told you that removing transactions from your services was a good way to help performance, you should not listen to any future advice from that person. You should look at the time spent in transactions and determine what the real overhead is, and find methods and entire services that are run in transactions but don't need to be and fix those to be nontransactional. But removing all transactions would be irresponsible.
You would be intentionally adding sporadic errors in method return values and making your data inconsistent, and this will get worse when you have a lot of traffic. A somewhat faster but buggy app or web site is not going to be popular, and if this doesn't help performance (or not much) then you still have to do the real work of finding the bottlenecks, missing indexes, and other things that are genuinely causing problems.
I would remove all #Transactional annotations and database writes from all controllers though; not for performance reasons, but to keep the application tiers sensible and not polluted with unrelated code and logic.
If you find one or more service methods that don't require transactions, switch to annotating each transactional method as needed but omit the annotation at class scope so un-annotated methods inherit nothing and aren't transactional. You could also move those methods to non-transactional services.
Note that services are only non-transactional if there are no #Transactional annotations and there is a transactional property disabling the feature:
static transactional = false
If you don't have that property and have no annotations, it will look like it's ok, but transactional defaults to true if not specified.
There's also something else that can help a lot (and already does). The dataSource bean is actually a proxy of a proxy - one proxy returns the connection from the pool that's a being used by an open Hibernate session or transaction so you can see uncommitted data and do your queries and updates in the same connection. The other is more related to your question: org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy which has been in Spring for years but only used in Grails since 2.3. It helps with methods that start or participate in a transaction but do no database work. For the case of a single method call that unnecessarily starts and commits an 'empty' transaction, the overhead involved includes getting the pooled connection, then calling set autocommit false, setting the transaction isolation level, etc. All of these are small costs but they add up. The class works by giving you a proxied connection that caches these method calls, and only gets a real connection and invokes these method on it when a query is actually run. If there are no queries and the only calls are those transaction-related setup methods, there's basically no cost at all. You shouldn't rely on this and should be intentional with the use of #Transactional annotations, but if you miss one this pool proxy will help avoid unnecessary work.
I've been looking for someone else doing this same thing, but haven't seen a scenario that's quite like this so I thought I'd see if anyone here has any good ideas on how to accomplish this.
My group builds and maintains an open-source neuroimaging data archive tool called XNAT. Previous versions of our application have always required users to run a builder application that took in a build.properties file and used that to initialize the database server configuration, among other things. We're really trying to get down to a single installable war file that we can make available on the NeuroDebian repository. In order to do this, we need to be able to start the application WITHOUT any database configuration information, run through a configuration wizard a la Wordpress or Drupal installations that includes the user inputting the database configuration, and finally setting this configuration information SOMEWHERE and re-starting or re-initializing the application context so that it gets its data source started up, Hibernate entity scans run, all auto-wired or injected dependencies that require the data source or Hibernate transaction manager resolved, and services scanned for #Transactional annotations, and so on.
I can easily see how we can use the new Spring Framework WebApplicationInitializer to detect whether the user has already set up the database configuration and initialize the app properly based on that:
If database has not been configured, create an servlet that just supports the UI for the initialization wizard
If database has been configured, create the regular application context
The problem in the first case is what happens once the user has completed the initialization wizard? We can store the database configuration somewhere and now we're ready to go but... how do we get the regular application context working? Can we just take the code that we'd call in the already initialized scenario and call that? Will that initialize the application properly then, with component scans and so on all being handled or...?
The only solution we have currently is to have the user restart the server manually (it's usually Tomcat) or use the server manager application to restart just our application. That's not very aesthetically pleasing though.
My end goal here will be to write a simple test app that takes in the database credentials and then tries to initialize everything else afterwards, but I'm hoping to see if anyone's thought about this particular issue and/or tried it and has any advice on how to handle it. Any help would be greatly appreciated!
I have a very interesting situation. I am slightly new to JBoss and Oracle, having worked mostly with Weblogic on DB2. That said, what I am trying to do is pretty simple.
I have a local-tx-datasource to an Oracle database. From my Java I code, I invoke datasource.getConnection() after retrieving the datasource using the appropriate JNDI name. The local-tx-datasource declaration in my -ds.xml file does not have any explicit reference to autocommit behaviour.
After getting the connection, I execute a create/update query and I get back the correct update count. Subsequently, for a short duration, I am even able to retrieve this record. However, after that the database pretends it never got the record in the first place, and there is nothing at all.
My experience with connections suggests that this happens when the connection does not commit its work, and so only that connection itself will be able to see the data in its transaction. From what I read, JBoss too follows the specification that the Connection returned is an autocommit one. I even verified this from my Java code, and it states the autocommit behaviour is set to true. However, if that was the case, why are my records not getting created / updated?
Following this, I set the Connection's autocommit behaviour to false (again from Java code), and then did the commit explicitly. Since then, there has been no issue.
What could possibly be going wrong? Is my understanding of autocommit here incorrect or does JBoss have some other interpretation of it. Please note, I do not have any transactions at all. These are very simple single record insert queries.
Please note, I do not have any transactions at all.
Wrong assumption. The local-tx-datasource starts a JTA transaction in your behalf. I'm not sure how the autocommit works in this scenario, but I suppose that autocommit applies only when you are using exclusively JDBC transactions, not JTA transactions.
In JTA, if you don't commit a transaction[*], it will be rolled back after the timeout. This explains the scenario that you are experiencing. So, I'd try to either change the local-tx-datasource to no-tx-datasource or to manually commit the transaction.
Note, however, that not managing your transactions is a bad thing. Autocommit should always be avoided. There's no better party to determine when to commit than your application. Leaving this responsibility to the driver/container is, IMO, not very responsible :-)
[*] One exception is for operations inside EJBs, whose business methods are "automatically" wrapped in a JTA transaction. So, you don't need to explicitly commit the transaction.
In WebSphere, if you code opens two concurrent database connections, you get an error of the form:
J2CA0086W: Shareable connection MCWrapper id 556e556e Managed connection WSRdbManagedConnectionImpl#52365236 State:STATE_TRAN_WRAPPER_INUSE
from resource jdbc/abc was used within a local transaction containment boundary.
Our framework allows us to do so (nested transactions which can be on a separate connection or multiple named transactions). I've seen lots of references to turning off some switch in WebSphere to turn on connection sharing but no details on how to set this flag. Can someone point me to the steps to achieve this?
Specifically, if you see this article: http://www-01.ibm.com/support/docview.wss?rs=180&context=SSEQTP&dc=DB520&dc=D600&dc=DB530&dc=D700&dc=DB500&dc=DB540&dc=DB510&dc=DB550&q1=j2ca0086w&uid=swg21121449&loc=en_US&cs=utf-8&lang=en
under "Resolving the problem" I want to know how to set the connection pool to be unshareable (assuming that indeed solves the problem).
What version of IBM WAS are you using? If you have WAS 8, go to
Resources-> JDC-> Datasources-> your datasources -> WebSphere Application Server properties -> Datasources no transactional.
Sorry for my english.
The message happens when dataSource.getConnection() is called twice in
a servlet. The datasource jdbc/oracle is lookup from local reference.
Call it once and reuse the connection or call con.close() before doing the 2nd getConnection()