I have come across some datasource properties for Spring Boot, that are specified in the application.properties. I am having a hard time understanding the purpose of the datasource properties.
Also, I am not able to find the explanation of these properties here for datasource - https://docs.spring.io/spring-boot/docs/1.1.2.RELEASE/reference/html/common-application-properties.html and I am using Spring Boot version 1.
spring.datasource.test-while-idle=true
spring.datasource.initial-size=10
spring.datasource.min-idle=10
spring.datasource.time-between-eviction-runs-millis=300000
spring.datasource.validation-query=SELECT 1 from DUAL
spring.datasource.test-on-borrow=true
spring.datasource.test-on-connect=true
spring.datasource.validation-interval=300000
The properties are validation-interval and time-between-eviction-runs-millis. What is the difference between them? The latter runs to evict the dead connections. But what about validation-interval?
Difference between test-on-borrow and test-on-connect?
I can't seem to find the right place to see their documentation or their purpose, or I am looking at the wrong place.
Please advise.
Following explanation is based on my experience, for your reference.
spring.datasource.time-between-eviction-runs-millis means the time interval to recycle idle connection. It is usually used with spring.datasource.test-while-idle.
spring.datasource.min-evictable-idle-time-millis means the effective time of idle connection in connection pool.
spring.datasource.test-on-borrow means whether testing the connection while borrowing the connection from connection pool. And it may have some impact on the performance.
spring.datasource.test-on-connect means whether testing the connection while creating the connection.
Related
I have a Spring Boot application (version 1.5.1.RELEASE) and I am using spring-boot-starter-data-jpa as a dependency to manage my database. I am using postgres as my database and configured it using the below properties.
spring.datasource.url=${POSTGRES_URL}
spring.datasource.username=${POSTGRES_USER}
Now when I run my tests which are almost 120, I get too many client already open error for abou 10 test cases while starting the test case itself and it fails.(remaining 100 test cases pass with success as they are able to get a connection to database)
First thing I did is increased my default postgres max connections count from 100 to 200 in the postgres server config file and my tests pass successfully after this change.
Now I investigated a bit and tried setting the parameters for connection pooling properties such as :
spring.datasource.tomcat.max-active=200
spring.datasource.tomcat.test-on-borrow=true
spring.datasource.tomcat.max-wait=10000
However these properties do not work and the tests fails again giving the same error as above. I tried reading from multiple different blogs and spring documentation for setting the connection pool properties but did not find what might be going wrong with me.
I also think that if I set the above property spring.datasource.tomcat.max-active to 100 connections it should work with the help of tomcat jdbc pooling as i think in current scenario it is trying to open a new connection to database for each test case and I am in a fear that this same scenario might happen when I deploy this code to production environment and a new connection will be opened to the database for each request.
Does anyone have faced this problem before or is there something wrong I am doing.
Thanks in advance for the help.
Try upgrading Spring boot version, 1.5.10-RELEASE is the current version.
Also, I found the connection pool properties for my application were not being applied when the property prefix tomcat was included. If you are still having issues try removing that.
i.e.
spring.datasource.tomcat.max-active=200
Becomes
spring.datasource.max-active=200
See https://artofcode.wordpress.com/2017/10/19/spring-boot-configuration-for-tomcats-pooling-data-source
I have two Java Web Applications running under Tomcat (6.0) and using the Tomcat & c3p0 connection pool as Tomcat Data-source.If I define two Resources (server.xml) for two different Oracle Connection and use c3p0 for connection pool as below which my we applications use, my questions are:
<Resource
name="jdbc/OracleDB"
auth="Container"
type="com.mchange.v2.c3p0.ComboPooledDataSource"
driverClass="oracle.jdbc.OracleDriver"
factory="xxx"
jdbcUrl="jdbc:oracle:thin:#(DESCRIPTION= (LOAD_BALANCE=on)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=xxx) (PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=xxx)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=xxx)))"
maxPoolSize="10"
minPoolSize="0"
maxIdleTime="60"
maxConnectionAge="600"
acquireIncrement="1"
user="xxx="
password="xxx=" />
<Resource name="jdbc/xxx2DB"
auth="Container"
type="com.mchange.v2.c3p0.ComboPooledDataSource"
driverClass="oracle.jdbc.OracleDriver"
factory="xxx"
jdbcUrl="jdbc:oracle:thin:#xxx:1527:xxx"
maxPoolSize="10"
minPoolSize="0"
acquireIncrement="1"
maxIdleTime="60"
maxConnectionAge="600"
user="xxx"
password="xxx"
/>
Q1. Does the below in server.xml mean that two connection pools are existing in the Tomcat
memory to two different Oracle Instances?
Q2. Do I have to specify any configuration properties (ref:http://www.mchange.com/projects/c3p0/index.html#configuration_properties) , in my case I have a connection to an Oracle RAC instance and another to a single instance of Oracle. Should I take into account
any additional configuration properties in an enterprise environment ?
Q3. Are the below setting valid enough?
Q4. How do I enable c3p0 logging (i have only the jar in Tomcat lib and the above setting for now?
Any advice?
Thanks in advance.
A1: Yes.
A2: Not sure if I got the question right, but you should only specify those connection properties which defaults do not make sense to you. It's probably best to do this in the separate c3p0.properties file and specify which connection which property applies to. Answer to 'additional configuration properties' question requires to know the specifics of your environment. I would recommend looking at idle_test_period setting as these can typically have wrong defaults (make sure this matches relevant setting on the DB end).
A3: Yeah, this looks OK.
A4: You need to specify logging preferences in c3p0.properties or define them in dynamically as System properties. Refer to this chapter of the c3p0 manual.
A1: Yes
A2: Looks right at first glance. You need to be careful with RAC in case you would use distributed transactions - then I believe you would have to specify additional properties.
A3: Looks ok but depending on your task and estimated load I would play with maxIdleTime, acquireIncrement and maxPoolSize. For example with current settings after 60 seconds of inactivity your connection will be closed so next operation requesting connection after 60 seconds would incur penalty in waiting for new connection to open.
A4: http://www.mchange.com/projects/c3p0/index.html#configuring_logging
Advice:
There is a better connection pool from Tomcat 7, which could be used in Tomcat 6 as well. Consider it, especially if you need high performance
Consider connection testing so your application don't get (or get less frequently) exception when db connectivity goes down
It might be preferable to package connection pool and it's configuration within your application vs. in Tomcat. This way application WAR would be self contained so it could be just dropped into wabapps dir of tomcat.
i am trying to use c3p0 in my current Spring based application. I am no database pooling expert and was wondering if there is a more or less a good example of a c3p0 configuration (in applicationContext.xml) that can be used against a MySQL database?
The pool should detect database connection loss and retries.
c3o0 is dead, do not use it. Use jdbc-pool
Here's the docs, (scroll down to How to use) http://people.apache.org/~fhanik/jdbc-pool/jdbc-pool.html
Here's a good article on why is this better http://www.tomcatexpert.com/blog/2010/03/12/explaining-jdbc-pool-high-concurrency-alternative-connection-pooling-module
You mean c3p0 or hibernate-c3p0?
Hibernate has taken development for C3P0 and it is very well alive.
It just published version 4.0.0 CR1, which will be out with Hibernate 4.
I'm reading about c3p0 and dbcp for handling jdbc connections and heard alot of problems and people saying dbcp is dead but c3p0 can't do jdbc4 and so on. But I don't know if these posts are out of date.
Now I hit on BoneCP which explains how broken connections are handled here. Connections are wrapped and BoneCP does some prechecking of exceptions before they get passed to the application. If something is wrong with the connection BoneCP removes it from the pool.
1.) Do all these pools have that kind of connection handling?
2.) This question gets asked over and over again, but I couldn't find any answer from 2011. What one shall I use for a new application, that will be maintained for the next 10 years, if not more.
1.) I don't know about dbcp but with regards to C3P0 this functionality does exist in the C3P0PooledConnection class (look in the invoke method, the exception is caught and handled, if you want more details on the exact handling I can add them). I also needed to know if it contained it to drop testOnCheckin/Checkout and I verified it contains this behaviour.
2.) Really hard to say, since on the one hand C3P0 is widely used in many production sites and the maintainer has resumed active development but on the other hand BoneCP seems to have some very interesting design principles (pool sharding for examples) and some flattering benchmarks. Since you usually have a pretty good indirection from your connection pool library (mostly the dependency is contained to a config file or two) I can suggest to start with one and once you have a production with actual data try to optimize it and compare it with another library (also optimized of course). It very well may be that either library you choose will be sufficient for your needs.
In WebSphere, if you code opens two concurrent database connections, you get an error of the form:
J2CA0086W: Shareable connection MCWrapper id 556e556e Managed connection WSRdbManagedConnectionImpl#52365236 State:STATE_TRAN_WRAPPER_INUSE
from resource jdbc/abc was used within a local transaction containment boundary.
Our framework allows us to do so (nested transactions which can be on a separate connection or multiple named transactions). I've seen lots of references to turning off some switch in WebSphere to turn on connection sharing but no details on how to set this flag. Can someone point me to the steps to achieve this?
Specifically, if you see this article: http://www-01.ibm.com/support/docview.wss?rs=180&context=SSEQTP&dc=DB520&dc=D600&dc=DB530&dc=D700&dc=DB500&dc=DB540&dc=DB510&dc=DB550&q1=j2ca0086w&uid=swg21121449&loc=en_US&cs=utf-8&lang=en
under "Resolving the problem" I want to know how to set the connection pool to be unshareable (assuming that indeed solves the problem).
What version of IBM WAS are you using? If you have WAS 8, go to
Resources-> JDC-> Datasources-> your datasources -> WebSphere Application Server properties -> Datasources no transactional.
Sorry for my english.
The message happens when dataSource.getConnection() is called twice in
a servlet. The datasource jdbc/oracle is lookup from local reference.
Call it once and reuse the connection or call con.close() before doing the 2nd getConnection()