I have a web application which using PostgreSQL as database.
Assume the scenario like i am trying to Save() data using Hibernate. Now just before save the data if database server will down then what happened ?
Hibernate will re-try save again or do i have to write the re-try code.
Any help will be appreciated.
No, Hibernate will not re-try to save your data. The db failure will cause your transaction to fail, and all the change will be rollbacked.
Even worse, if you don't configure correctly your connection pool, you will not be able to re-establish the connection when the db will be running again. Take a look to this question for more info. I copy and paste c3p0 configuration.
<c3p0-config>
<default-config>
<!-- Configuring Connection Testing -->
<!-- property name="automaticTestTable">TEST_EMS_HIBERNATE_CONN</property -->
<property name="checkoutTimeout">0</property>
<property name="testConnectionOnCheckout">true</property>
<property name="testConnectionOnCheckin">false</property>
<property name="preferredTestQuery">SELECT 1 from dual</property>
<!-- Configuring Recovery From Database Outages -->
<property name="acquireRetryAttempts">0</property>
<property name="acquireRetryDelay">1000</property>
<property name="breakAfterAcquireFailure">false</property>
<!-- Configuring to Debug and Workaround Broken Client Apps -->
<property name="unreturnedConnectionTimeout">1800</property>
<property name="debugUnreturnedConnectionStackTraces">true</property>
</default-config>
Related
I have an application that connects to the DB using scan address. There are two identical Oracle instances of DB under this scan address. If one instance is down for some reason or crashes the app should recover automatically and try to connect to the other working instance of the DB.
I use HikariCP, Hibernate, jdbc:thin client and Oracle DB.
Which of these I should force to check if the connection to the DB is alive and if it's not try to establish new one?
The scan address above the two database instances works fine. When my app crashes and I refresh the page it automatically creates a pool of connections on the other working DB, but I need this process to be done automatically.
I use Hikari xml configuration:
<bean id="dataSource" class="com.zaxxer.hikari.HikariDataSource" destroy-method="close">
<property name="driverClassName" value="${jdbc.driverClassName}"/>
<property name="jdbcUrl" value="${jdbc.url}"/>
<property name="username" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
<property name="minimumIdle" value="${jdbc.minPoolSize:0}" />
<property name="maximumPoolSize" value="${jdbc.maxPoolSize:10}" />
<property name="idleTimeout" value="${jdbc.maxIdleTimeSeconds:10000}" />
</bean>
There are various thing to consider:
HikariCP checks every connections before borrowing it to application. Unless there is a delay less that 10 seconds between borrows.
Oracle's JDBC drivers implement functionality: Application Continuity(AC). JDBC drivers remembers inserts/updates sent to a database, when transaction is open. When you loose this connection, JDBC driver transparently re-connects and replays SQLs. This feature has some limitations, but in basic situations data is not lost even in situations when transaction is not COMMITed yet.
Oracle introduced few handy features in JDBC 4.x standards. One of them is Connection.isValid(). On Oracle's case is does the same thing as OCIPing. It exchanges a single packet roundtrip between Application and DB server. Having a particular SQL to be executed on each Connection borrow belongs to past. Just execute your JDBC driver and it will tell you when JDBC version it supports.
$ java -jar ojdbc8.jar
Oracle 12.2.0.1.0 JDBC 4.2 compiled with javac 1.8.0_91 on Tue_Dec_13_06:08:31_PST_2016
#Default Connection Properties Resource
#Mon Jun 10 20:13:17 CEST 2019
Aside from HikariCP you can also use Oracle's UCP. This connection pooling library implementation can receive events from RAC cluster, can load balance connections between nodes. can close connection to a particular db node when requested. You can even have zero-downtime DB server patching, when configured properly.
I have an OSGi bundle which needs to persist data in a database. As described in a previous stackoverflow question I have found that in order for transactions to work as expected I need to use an XADataSource to connect to the database. When I do so however I see that the connections to the database that are opened by my application are never closed, which of course results in the database not being able to accept any more connections after a while.
My setup is the following:
I have a bundle which creates the datasource and which only includes a blueprint.xml file with the following content
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">
<bean id="dataSource" class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource">
<property name="url" value="jdbc:mysql://localhost:3306/myschema"/>
<property name="user" value="user"/>
<property name="password" value="pass"/>
</bean>
<service interface="javax.sql.XADataSource" ref="dataSource">
<service-properties>
<entry key="osgi.jndi.service.name" value="jdbc/mysqlds"/>
</service-properties>
</service>
</blueprint>
Then in the bundle that persists my data I have a persistence.xml
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
<persistence-unit name="mypu" transaction-type="JTA">
<jta-data-source>osgi:service/javax.sql.DataSource/(osgi.jndi.service.name=jdbc/mysqlds)
</jta-data-source>
</persistence-unit>
</persistence>
And I specify that my service methods should run in a transaction in my blueprint.xml
<bean id="MyServiceImpl"
class="com.test.impl.MyServiceImpl">
<jpa:context property="em" unitname="mypu" />
<tx:transaction method="*" value="Required" />
</bean>
<service id="MyService" ref="MyServiceImpl" interface="com.test.api.MyService" />
I deploy by bundles in Karaf, using Aries and OpenJPA for persistence, while I have also deployed the Aries transaction wrapper bundle (org.apache.aries.transaction.wrappers) in order to enlist my XA resources with the transaction manager.
Any ideas what I am missing in my configuration?
Edit:
After some more searching I found this DBCP issue which suggests that the problem I'm having is a bug of DBCP with MySQL. However I'm at a loss on how to replace DBCP with some other Connection Pool implementation OpenJPA can work with. Any suggestions are more than welcome.
I used commons-dbcp to have a connection pool that also enlists XA Connections with the following configuration:
<bean id="myXAEnabledConnectionPoolDataSource" class="org.apache.commons.dbcp.managed.BasicManagedDataSource" destroy-method="close">
<property name="xaDataSourceInstance" ref="mysqlXADataSourceBean" />
<property name="transactionManager" ref="transactionManager" />
</bean>
You can get the transaction manager as a reference based on the interface javax.transaction.TransactionManager.
In this way commons-dbcp will handle the lifecycle of the connections properly. Please note that the destroy method is there so when the blueprint container stops the connection pool will be closed.
Edit:
1-2 years ago I had the same problem but with PostgreSQL. I debugged aries.transaction.wrapper at that time a lot but I cannot remember exactly the cause why I left it out. I think the motivation was behind that commons-dbcp is a solution that worked for me in previous projects while I could not fix aries.transaction.wrapper even after analyzing it's code a lot.
Please note that MysqlDataSource is not a connection pool. It gives back a new connection always when you need one. It is also not XA enabled. MysqlXADatasource is XA enabled so you should probably instantiate an object from that class. However, an XADataSource is responsible only to give back XAConnections for you but not for enlisting them. That is where a ManagedConnectionPool can help. A managed connection pool does the followings:
Wraps all provided Connection objects with a custom managed connection class
In case close is called on a connection, it is not closed if there is an ongoing transaction. It is only closed (added back to the pool) when a transaction commit or rollback is done
In case a connection is queried from the pool and there was a connection also provided in the same transaction, the same transaction will be returned (that was a difficult sentence :))
Sometimes JDBC drivers provide connection pools and even managed connection pools, however, it is better to use the JDBC driver only to get new connections and wrap it with a 3rdparty library that was tested in several projects and works for sure.
Previously, my Java web projects used Eclipse-ordinary structure, and at the start of the container (in case, Tomcat), Hibernate generated the schemes correctly.
Now I'm using Maven infrastructure. I've relocated the needed files and configured all well (I think, because all is working right: Spring is starting, Hibernate is connecting the database - when it was previously created and there's some data to fetch). I've tested all CRUD operations and it's working.
The problem is that Hibernate refuses to generate the schemes (DDL) as it did when over Eclipse-ordinary infrastructure.
Additional information:
My persistence.xml is almost empty (as always) because Spring applicationContext.xml is starting it. I have not changed the file, it continues the same way as before.
<!-- Location: src/main/resources/META-INF/persistence.xml -->
<persistence>
<persistence-unit name="jpa-persistence-unit" transaction-type="RESOURCE_LOCAL"/>
</persistence>
Part of the Spring configuration goes here (applicationContext.xml):
<!-- Location: src/main/webapp/WEB-INF/applicationContext.xml -->
<!-- ... -->
<bean id="jpaVendorAdapter" class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="database" value="[DATABASE-NAME]" />
<property name="showSql" value="true" />
<property name="generateDdl" value="true" /> <!-- THIS CONFIGURATION WORKED PREVIOUSLY, NOW WITH MAVEN, IT'S IGNORED -->
<property name="databasePlatform" value="[DIALECT]" />
</bean>
<!-- ... -->
I'm not using any Maven Hibernate plugin, because I just want the default behavior that occurred earlier.
Did Maven invalidate this "generateDdl" property!? Why!? What should I do!? I can't find any solution.
I found out the solution.
Maven has any fault about that.
Hibernate was not able to create my database because the "DIALECT" was wrong.
I remembered that I changed the dialect from MySQL to MySQL-InnoDB. Hibernate was logging this problem but I couldn't see it because the slf4j-simple dependency was not explicity imported.
Thank you for your time, Shawn.
I am using hibernate 3, c3p0 9.1.2, Oracle 11g in my application. If I restart the Oracle then the stale connections are not getting refresh and I am getting exception "java.sql.SQLRecoverableException: Closed Connection". Below is my hibernate.cfg.xml.
I am a beginner in Hibernate API. Can you please suggest how to configure hibernate to automatically refresh the stale connections on a specified time.
Here is my hibernate.cfg.xml
oracle.jdbc.driver.OracleDriver
jdbc:oracle:thin:#localhost:1521:ems
emsman
<property name="hibernate.c3p0.idle_test_period">60</property> <!-- seconds -->
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">20</property>
<property name="hibernate.c3p0.timeout">1800</property>
<property name="hibernate.c3p0.max_statements">50</property>
<property name="show_sql">false</property>
<property name="dialect">org.hibernate.dialect.OracleDialect</property>
<property name="c3p0.validate">true</property>
<mapping resource="<package-name>/GroupOpWorkflow.hbm.xml"/>
<mapping resource="<package-name>/GroupOperation.hbm.xml"/>
<mapping resource="<package-name>/GroupOpNode.hbm.xml"/>
<mapping resource="<package-name>/NodeStatusLog.hbm.xml"/>
</session-factory>
It's c3p0, your database connection pool, that you need to configure - not hibernate. Try setting idleConnectionTestPeriod and an appropriate preferredTestQuery, e.g., select 1 from dual. The validate property has been deprecated and it's recommended that you not use that.
See http://community.jboss.org/wiki/HowToConfigureTheC3P0ConnectionPool for more information. You'll get the most control if you create a c3p0.properties file in WEB-INF/classes but you need to make sure not to override those properties in your hibernate.cfg.xml.
After gone through the document { http://community.jboss.org/wiki/HowToConfigureTheC3P0ConnectionPool } I found C3P0 was not a all used by hibernate.
So wrote a new C3P0 xml file and used the below system properties:
C3P0_SYS_PROPS="-Dcom.mchange.v2.c3p0.cfg.xml=<FILE-PATH>/c3p0-config.xml -Dcom.mchange.v2.log.MLog=com.mchange.v2.log.FallbackMLog -Dcom.mchange.v2.log.FallbackMLog.DE
FAULT_CUTOFF_LEVEL=WARNING"
So here is the final working configuration
hibernate.cfg.xml
<session-factory>
<property name="hibernate.connection.driver_class">oracle.jdbc.driver.OracleDriver</property>
<property name="hibernate.connection.url">jdbc:oracle:thin:#localhost:1521:ems</property>
<property name="hibernate.connection.username">emsman</property>
<property name="hibernate.connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</property>
<property name="hibernate.connection.autoReconnect">true</property>
<property name="show_sql">false</property>
<property name="dialect">org.hibernate.dialect.Oracle10gDialect</property>
<property name="hibernate.c3p0.idle_test_period">300</property> <!-- In seconds -->
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">20</property>
<property name="hibernate.c3p0.timeout">1800</property>
<property name="hibernate.c3p0.max_statements">50</property>
.....
<c3p0-config>
<default-config>
<!-- Configuring Connection Testing -->
<!-- property name="automaticTestTable">TEST_EMS_HIBERNATE_CONN</property -->
<property name="checkoutTimeout">0</property>
<property name="testConnectionOnCheckout">true</property>
<property name="testConnectionOnCheckin">false</property>
<property name="preferredTestQuery">SELECT 1 from dual</property>
<!-- Configuring Recovery From Database Outages -->
<property name="acquireRetryAttempts">0</property>
<property name="acquireRetryDelay">1000</property>
<property name="breakAfterAcquireFailure">false</property>
<!-- Configuring to Debug and Workaround Broken Client Apps -->
<property name="unreturnedConnectionTimeout">1800</property>
<property name="debugUnreturnedConnectionStackTraces">true</property>
</default-config>
c3p0-config.xml
<c3p0-config>
<default-config>
<!-- Configuring Connection Testing -->
<!-- property name="automaticTestTable">TEST_EMS_HIBERNATE_CONN</property -->
<property name="checkoutTimeout">0</property>
<property name="testConnectionOnCheckout">true</property>
<property name="testConnectionOnCheckin">false</property>
<property name="preferredTestQuery">SELECT 1 from dual</property>
<!-- Configuring Recovery From Database Outages -->
<property name="acquireRetryAttempts">0</property>
<property name="acquireRetryDelay">1000</property>
<property name="breakAfterAcquireFailure">false</property>
<!-- Configuring to Debug and Workaround Broken Client Apps -->
<property name="unreturnedConnectionTimeout">1800</property>
<property name="debugUnreturnedConnectionStackTraces">true</property>
</default-config>
You can try the following, it's a simple advice from the author of c3p0 taken here:
The best thing to do is usually to try Step 3, see if it helps
(however you measure performance), see if it hurts (is your
application troubled by broken Connections? does it recover from
database restarts well enough?), and then decide.
Step 3: If you'd like to improve performance by eliminating
Connection testing from clients' code path:
Set testConnectionOnCheckout to false
Set testConnectionOnCheckin to true
Set idleConnectionTestPeriod to 30, fire up you application and
observe. This is a pretty robust setting, all Connections will tested
on check-in and every 30 seconds thereafter while in the pool. Your
application should experience broken or stale Connections only very
rarely, and the pool should recover from a database shutdown and
restart quickly. But there is some overhead associated with all that
Connection testing.
If database restarts will be rare so quick recovery is not an issue,
consider reducing the frequency of tests by idleConnectionTestPeriod
to, say, 300, and see whether clients are troubled by stale or broken
Connections. If not, stick with 300, or try an even bigger number.
Consider setting testConnectionOnCheckin back to false to avoid
unnecessary tests on checkin. Alternatively, if your application does
encounter bad Connections, consider reducing idleConnectionTestPeriod
and set testConnectionOnCheckin back to true. There are no correct or
incorrect values for these parameters: you are trading off overhead
for reliability in deciding how frequently to test. The exact numbers
are not so critical. It's usually easy to find configurations that
perform well. It's rarely worth spending time in pursuit of "optimal"
values here.
I have connection pooling implemented in spring using Oracle Data Source. Currently we are facing an issue where connections are becoming invalid after a period of time. (May be Oracle is dropping those idle connections after a while). Here are my questions:
Can Oracle database be configured to drop idle connections automatically after a specific period of time. Since we expect those connections to lie idle for a while; if there is any such configuration; it may be happening.
In our connection pooling properties in spring we didn't have "validateConnection" property. I understand that it validates the connection before handing it over to web application? But does that mean that if a connection passes validateConnection test then it'll always connect to database correctly. I ask this, as I read following problem here:
http://forum.springsource.org/showthread.php?t=69759
If suppose validateConnection doesn't do the whole 9 yards of ensuring that connection is valid, is there any other option like "testBeforBorrow" in DBCP , which runs a test query to ensure that connection is active before handing it over to webapp?
I'll be grateful if you could provide answers to one ore more queries listed above.
Cheers
You don't say what application server you are using, or how you are configuring the datasource, so I can't give you specific advice.
Connection validation often sounds like a good idea, but you have to be careful with it. For example, we once used it in our JBoss app servers to validate connections in the pool before handing them to the application. This Oracle-proprietary mechanism used the ping() method on the Oracle JDBC driver, which checks that the connection is still alive. It worked fine, but it turns out that ping() executes "select 'x' from dual' on the server, which is a surprisingly expensive query when it's run dozens of times per second.
So the moral is, if you have a high-traffic server, be very careful with connection validation, it can actually bring your database server to its knees.
As for DBCP, that has the ability to validate connections as their borrowed from the pool, as well as returned to the pool, and you can tell it what SQL to send to the database to perform this validation. However, if you're not using DBCP for your connection pooling, then that's not much use to you. C3PO does something similar.
If you're using an app server's data source mechanism, then you have to find out if you can configure that to validate connections, and that's specific to your server.
One last thing: Spring isn't actually involved here. Spring just uses the DataSource that you give it, it's up to the DataSource implementation to perform connection validation.
Configuration of data source "was" as follows:
<bean id="datasource2"
class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName">
<value>org.apache.commons.dbcp.BasicDataSource</value>
</property>
<property name="url">
<value>ORACLE URL</value>
</property>
<property name="username">
<value>user id</value>
</property>
<property name="password">
<value>user password</value>
</property>
<property name="initialSize" value="5"/>
<property name="maxActive" value="20"/>
</bean>
have changed it to:
<bean id="connectionPool1" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
<property name="connectionCachingEnabled" value="true" />
<property name="URL">
<value>ORACLE URL</value>
</property>
<property name="user">
<value>user id</value>
</property>
<property name="password">
<value>user password</value>
</property>
<property name="connectionCacheProperties">
<value>
MinLimit:1
MaxLimit:5
InitialLimit:1
ConnectionWaitTimeout:120
InactivityTimeout:180
ValidateConnection:true
</value>
</property>
</bean>