Validating Connection Before Handing over to WebApp in ConnectionPooling - oracle

I have connection pooling implemented in spring using Oracle Data Source. Currently we are facing an issue where connections are becoming invalid after a period of time. (May be Oracle is dropping those idle connections after a while). Here are my questions:
Can Oracle database be configured to drop idle connections automatically after a specific period of time. Since we expect those connections to lie idle for a while; if there is any such configuration; it may be happening.
In our connection pooling properties in spring we didn't have "validateConnection" property. I understand that it validates the connection before handing it over to web application? But does that mean that if a connection passes validateConnection test then it'll always connect to database correctly. I ask this, as I read following problem here:
http://forum.springsource.org/showthread.php?t=69759
If suppose validateConnection doesn't do the whole 9 yards of ensuring that connection is valid, is there any other option like "testBeforBorrow" in DBCP , which runs a test query to ensure that connection is active before handing it over to webapp?
I'll be grateful if you could provide answers to one ore more queries listed above.
Cheers

You don't say what application server you are using, or how you are configuring the datasource, so I can't give you specific advice.
Connection validation often sounds like a good idea, but you have to be careful with it. For example, we once used it in our JBoss app servers to validate connections in the pool before handing them to the application. This Oracle-proprietary mechanism used the ping() method on the Oracle JDBC driver, which checks that the connection is still alive. It worked fine, but it turns out that ping() executes "select 'x' from dual' on the server, which is a surprisingly expensive query when it's run dozens of times per second.
So the moral is, if you have a high-traffic server, be very careful with connection validation, it can actually bring your database server to its knees.
As for DBCP, that has the ability to validate connections as their borrowed from the pool, as well as returned to the pool, and you can tell it what SQL to send to the database to perform this validation. However, if you're not using DBCP for your connection pooling, then that's not much use to you. C3PO does something similar.
If you're using an app server's data source mechanism, then you have to find out if you can configure that to validate connections, and that's specific to your server.
One last thing: Spring isn't actually involved here. Spring just uses the DataSource that you give it, it's up to the DataSource implementation to perform connection validation.

Configuration of data source "was" as follows:
<bean id="datasource2"
class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName">
<value>org.apache.commons.dbcp.BasicDataSource</value>
</property>
<property name="url">
<value>ORACLE URL</value>
</property>
<property name="username">
<value>user id</value>
</property>
<property name="password">
<value>user password</value>
</property>
<property name="initialSize" value="5"/>
<property name="maxActive" value="20"/>
</bean>
have changed it to:
<bean id="connectionPool1" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
<property name="connectionCachingEnabled" value="true" />
<property name="URL">
<value>ORACLE URL</value>
</property>
<property name="user">
<value>user id</value>
</property>
<property name="password">
<value>user password</value>
</property>
<property name="connectionCacheProperties">
<value>
MinLimit:1
MaxLimit:5
InitialLimit:1
ConnectionWaitTimeout:120
InactivityTimeout:180
ValidateConnection:true
</value>
</property>
</bean>

Related

How to check DB connections and make reconnect in case of need?

I have an application that connects to the DB using scan address. There are two identical Oracle instances of DB under this scan address. If one instance is down for some reason or crashes the app should recover automatically and try to connect to the other working instance of the DB.
I use HikariCP, Hibernate, jdbc:thin client and Oracle DB.
Which of these I should force to check if the connection to the DB is alive and if it's not try to establish new one?
The scan address above the two database instances works fine. When my app crashes and I refresh the page it automatically creates a pool of connections on the other working DB, but I need this process to be done automatically.
I use Hikari xml configuration:
<bean id="dataSource" class="com.zaxxer.hikari.HikariDataSource" destroy-method="close">
<property name="driverClassName" value="${jdbc.driverClassName}"/>
<property name="jdbcUrl" value="${jdbc.url}"/>
<property name="username" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
<property name="minimumIdle" value="${jdbc.minPoolSize:0}" />
<property name="maximumPoolSize" value="${jdbc.maxPoolSize:10}" />
<property name="idleTimeout" value="${jdbc.maxIdleTimeSeconds:10000}" />
</bean>
There are various thing to consider:
HikariCP checks every connections before borrowing it to application. Unless there is a delay less that 10 seconds between borrows.
Oracle's JDBC drivers implement functionality: Application Continuity(AC). JDBC drivers remembers inserts/updates sent to a database, when transaction is open. When you loose this connection, JDBC driver transparently re-connects and replays SQLs. This feature has some limitations, but in basic situations data is not lost even in situations when transaction is not COMMITed yet.
Oracle introduced few handy features in JDBC 4.x standards. One of them is Connection.isValid(). On Oracle's case is does the same thing as OCIPing. It exchanges a single packet roundtrip between Application and DB server. Having a particular SQL to be executed on each Connection borrow belongs to past. Just execute your JDBC driver and it will tell you when JDBC version it supports.
$ java -jar ojdbc8.jar
Oracle 12.2.0.1.0 JDBC 4.2 compiled with javac 1.8.0_91 on Tue_Dec_13_06:08:31_PST_2016
#Default Connection Properties Resource
#Mon Jun 10 20:13:17 CEST 2019
Aside from HikariCP you can also use Oracle's UCP. This connection pooling library implementation can receive events from RAC cluster, can load balance connections between nodes. can close connection to a particular db node when requested. You can even have zero-downtime DB server patching, when configured properly.

Oracle connection pooling takes a lot of time for first call

In spring I have a datasource defined in this way:
<bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
<property name="URL" value="${jdbc.url}" />
<property name="user" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
<property name="connectionCachingEnabled" value="true"/>
</bean>
<tx:annotation-driven transaction-manager="transactionManager" />
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
This datasource is used by my REST service and everything works fine...anyway first REST call is very slow (about ~5 secs), after that EVERY other call is fast.
I think this is an initialization related problem, in the sense that initialization is made when first DB call is received.
Is there a way to tell spring to initialize this datasource on server startup?
I think this is an initialization related problem, in the sense that
initialization is made when first DB call is received.
With your current config I think that's what's happening.
Is there a way to tell spring to initialize this datasource on server
startup?
It's the behavior of the connection pool, not Spring. Spring is creating the bean when your app starts (you aren't using lazy-init="true" on the bean). However, the connection pool isn't creating connections to the database when Spring instantiates it. From the Oracle docs:
The initial pool size property specifies the number of available
connections that are created when the connection pool is initially
created or re-initialized. This property is typically used to reduce
the ramp-up time incurred by priming the pool to its optimal size.
A value of 0 indicates that no connections are pre-created. The
default value is 0.
Try setting a non-zero value for initialPoolSize.
Edit: Try setting ConnectionCacheProperties instead:
<property name="connectionCacheProperties">
<props merge="default">
<prop key="InitialLimit">5</prop>
</props>
</property>

Oracle stale connection

Every once in a while in my app I get this error:
ERROR 2015-04-09 08:30:13,724 [http-bio-8080-exec-2] mojo.jdbc.MojoAlertDataAccess: Invalid or Stale Connection found in the Connection Cache
java.sql.SQLException: Invalid or Stale Connection found in the Connection Cache
at oracle.jdbc.pool.OracleImplicitConnectionCache.getConnection(OracleImplicitConnectionCache.java:421)
at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:395)
at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:179)
at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:157)
at mojo.jdbc.MojoAlertDataAccess.getAllAlertTypes(MojoAlertDataAccess.java:807)
So currently I have a datasource in Spring defined as:
<bean id="globalDSRead" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
<property name="connectionCachingEnabled" value="true" />
<property name="URL" value="${mojo.jdbc.read.url}"/>
<property name="user" value="${mojo.jdbc.read.username}"/>
<property name="password" value="${mojo.jdbc.read.password}"/>
<property name="connectionCacheProperties">
<value>
PropertyCheckInterval:10
MinLimit:1
MaxLimit:200
InitialLimit:1
ConnectionWaitTimeout:30
InactivityTimeout:30
ValidateConnection:true
</value>
</property>
</bean>
As far as I can tell this happens when the app has been sitting idle for a while, but it's not all together easy to reproduce.
Also - Another thing that occurs in this function is the retrieval of the connection can take a long time to come back, again this usually happens when the app's been sitting idle for a long time.
Anyone have any ideas whatcould be misconfigured?
Thanks
Yes, I have seen that before. I recommend to use Apache DBCP (https://commons.apache.org/proper/commons-dbcp/) for the connection pool management.

Spring JMS Connections: performance considerations

I have the need to send/receive messages towards/from different topics stored on a single JMS Server.
I would like to use JmsTemplate for sending and MessageListenerContainer for registering asyncronous listeners.
My configuration looks like this:
<bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate">
<property name="environment">
<props>
<prop key="java.naming.factory.initial">xxx</prop>
<prop key="java.naming.provider.url">yyy</prop>
</props>
</property>
</bean>
<bean id="connectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref ="jndiTemplate"/>
<property name="jndiName" value="TopicConnectionFactory"/>
</bean>
<bean id="singleConnectionFactory" class="org.springframework.jms.connection.SingleConnectionFactory">
<constructor-arg ref="connectionFactory"/>
</bean>
<bean id="tosJmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="singleConnectionFactory"/>
<property name="destinationResolver" ref="destinationResolver"/>
<property name="pubSubDomain" value="true"/>
</bean>
As far as I understood, the singleConnectionFactory, returning always the same connection instance, helps reducing the overhead of creating and closing
a connection each time a jmsTemplate needs (for example) to send/receive a message (as it would be when using a normal ConnectionFactory).
My first question is: if I create multiple jmsTemplate(s), can they all share a ref to a singleConnectionFactory? Or do they have to receive a distinct instance each (singleConnectionFactory1, singleConnectionFactory2, etc)?
Reading the API for SingleConnectionFactory, I found this:
Note that Spring's message listener containers support the use of a shared Connection
within each listener container instance. Using SingleConnectionFactory in combination only really makes sense for sharing a single JMS Connection across multiple listener containers.
This sound a bit cryptic to me. As far as I know, it is possible to register only 1 Listener per MessageListenerContainer, so I don't understand to what extent is a connection shared.
Suppose I want to register N Listeners: I will need to repeat N times something like this:
<bean
class="org.springframework.jms.listener.SimpleMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory" />
<property name="destinationName" value="destX" />
<property name="messageListener" ref="listener1outOfN" />
</bean>
How many Connections are created in such case from connectionFactory? One for each ListenerContainer or just a pool of Connections? And what if I provide the SimpleMessageListenerContainer-s with a ref to singleConnectionFactory?
What is the best approach (from the point of view of the performances, of course) in this case?
if I create multiple jmsTemplate(s), can they all share a ref to a singleConnectionFactory?
Yes, this is fine. The javadoc for SingleConnectionFactory says:
According to the JMS Connection model, this is perfectly thread-safe (in contrast to e.g. JDBC).
JMS Connection objects are thread-safe, and can be used by multiple threads concurrenctly. So there's no need to use multiple SingleConnectionFactory beans.
As far as I know, it is possible to register only 1 Listener per MessageListenerContainer, so I don't understand to what extent is a connection shared.
This is true; however, each MessageListenerContainer can have multiple threads processing messages concurrently, all using the same MessageListener object. The MessageListenerContainer will use a single, shared Connection for all of these threads (unless configured to do otherwise).
Note that Spring's message listener containers support the use of a shared Connection within each listener container instance. Using SingleConnectionFactory in combination only really makes sense for sharing a single JMS Connection across multiple listener containers.
In other words, if all you have is a single MessageListenerContainer, then SingleConnectionFactory is unnecessary, since the single connection is managed internally to MessageListenerContainer. if you have multiple listener containers, and want them all to share a connection, then SingleConnectionFactory is required. Also, if you want to share a connection between listening and sending, as you do, then SingleConnectionFactory is also necessary.

How to configure c3p0 in hibernate to auto-refresh stale DB connections

I am using hibernate 3, c3p0 9.1.2, Oracle 11g in my application. If I restart the Oracle then the stale connections are not getting refresh and I am getting exception "java.sql.SQLRecoverableException: Closed Connection". Below is my hibernate.cfg.xml.
I am a beginner in Hibernate API. Can you please suggest how to configure hibernate to automatically refresh the stale connections on a specified time.
Here is my hibernate.cfg.xml
oracle.jdbc.driver.OracleDriver
jdbc:oracle:thin:#localhost:1521:ems
emsman
<property name="hibernate.c3p0.idle_test_period">60</property> <!-- seconds -->
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">20</property>
<property name="hibernate.c3p0.timeout">1800</property>
<property name="hibernate.c3p0.max_statements">50</property>
<property name="show_sql">false</property>
<property name="dialect">org.hibernate.dialect.OracleDialect</property>
<property name="c3p0.validate">true</property>
<mapping resource="<package-name>/GroupOpWorkflow.hbm.xml"/>
<mapping resource="<package-name>/GroupOperation.hbm.xml"/>
<mapping resource="<package-name>/GroupOpNode.hbm.xml"/>
<mapping resource="<package-name>/NodeStatusLog.hbm.xml"/>
</session-factory>
It's c3p0, your database connection pool, that you need to configure - not hibernate. Try setting idleConnectionTestPeriod and an appropriate preferredTestQuery, e.g., select 1 from dual. The validate property has been deprecated and it's recommended that you not use that.
See http://community.jboss.org/wiki/HowToConfigureTheC3P0ConnectionPool for more information. You'll get the most control if you create a c3p0.properties file in WEB-INF/classes but you need to make sure not to override those properties in your hibernate.cfg.xml.
After gone through the document { http://community.jboss.org/wiki/HowToConfigureTheC3P0ConnectionPool } I found C3P0 was not a all used by hibernate.
So wrote a new C3P0 xml file and used the below system properties:
C3P0_SYS_PROPS="-Dcom.mchange.v2.c3p0.cfg.xml=<FILE-PATH>/c3p0-config.xml -Dcom.mchange.v2.log.MLog=com.mchange.v2.log.FallbackMLog -Dcom.mchange.v2.log.FallbackMLog.DE
FAULT_CUTOFF_LEVEL=WARNING"
So here is the final working configuration
hibernate.cfg.xml
<session-factory>
<property name="hibernate.connection.driver_class">oracle.jdbc.driver.OracleDriver</property>
<property name="hibernate.connection.url">jdbc:oracle:thin:#localhost:1521:ems</property>
<property name="hibernate.connection.username">emsman</property>
<property name="hibernate.connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</property>
<property name="hibernate.connection.autoReconnect">true</property>
<property name="show_sql">false</property>
<property name="dialect">org.hibernate.dialect.Oracle10gDialect</property>
<property name="hibernate.c3p0.idle_test_period">300</property> <!-- In seconds -->
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">20</property>
<property name="hibernate.c3p0.timeout">1800</property>
<property name="hibernate.c3p0.max_statements">50</property>
.....
<c3p0-config>
<default-config>
<!-- Configuring Connection Testing -->
<!-- property name="automaticTestTable">TEST_EMS_HIBERNATE_CONN</property -->
<property name="checkoutTimeout">0</property>
<property name="testConnectionOnCheckout">true</property>
<property name="testConnectionOnCheckin">false</property>
<property name="preferredTestQuery">SELECT 1 from dual</property>
<!-- Configuring Recovery From Database Outages -->
<property name="acquireRetryAttempts">0</property>
<property name="acquireRetryDelay">1000</property>
<property name="breakAfterAcquireFailure">false</property>
<!-- Configuring to Debug and Workaround Broken Client Apps -->
<property name="unreturnedConnectionTimeout">1800</property>
<property name="debugUnreturnedConnectionStackTraces">true</property>
</default-config>
c3p0-config.xml
<c3p0-config>
<default-config>
<!-- Configuring Connection Testing -->
<!-- property name="automaticTestTable">TEST_EMS_HIBERNATE_CONN</property -->
<property name="checkoutTimeout">0</property>
<property name="testConnectionOnCheckout">true</property>
<property name="testConnectionOnCheckin">false</property>
<property name="preferredTestQuery">SELECT 1 from dual</property>
<!-- Configuring Recovery From Database Outages -->
<property name="acquireRetryAttempts">0</property>
<property name="acquireRetryDelay">1000</property>
<property name="breakAfterAcquireFailure">false</property>
<!-- Configuring to Debug and Workaround Broken Client Apps -->
<property name="unreturnedConnectionTimeout">1800</property>
<property name="debugUnreturnedConnectionStackTraces">true</property>
</default-config>
You can try the following, it's a simple advice from the author of c3p0 taken here:
The best thing to do is usually to try Step 3, see if it helps
(however you measure performance), see if it hurts (is your
application troubled by broken Connections? does it recover from
database restarts well enough?), and then decide.
Step 3: If you'd like to improve performance by eliminating
Connection testing from clients' code path:
Set testConnectionOnCheckout to false
Set testConnectionOnCheckin to true
Set idleConnectionTestPeriod to 30, fire up you application and
observe. This is a pretty robust setting, all Connections will tested
on check-in and every 30 seconds thereafter while in the pool. Your
application should experience broken or stale Connections only very
rarely, and the pool should recover from a database shutdown and
restart quickly. But there is some overhead associated with all that
Connection testing.
If database restarts will be rare so quick recovery is not an issue,
consider reducing the frequency of tests by idleConnectionTestPeriod
to, say, 300, and see whether clients are troubled by stale or broken
Connections. If not, stick with 300, or try an even bigger number.
Consider setting testConnectionOnCheckin back to false to avoid
unnecessary tests on checkin. Alternatively, if your application does
encounter bad Connections, consider reducing idleConnectionTestPeriod
and set testConnectionOnCheckin back to true. There are no correct or
incorrect values for these parameters: you are trading off overhead
for reliability in deciding how frequently to test. The exact numbers
are not so critical. It's usually easy to find configurations that
perform well. It's rarely worth spending time in pursuit of "optimal"
values here.

Resources