how to efficiently Update using entityManager with set timeout - oracle

Question

I am running some scheduled cron jobs in a springboot app to update few tables.
Scheduler Class :
#Component
public class ScheduledTasks {
#Autowired
ScheduledTasksServiceRepository scheduledTasksServiceRepository;
//Cron Job runs in every 3 hours to move recently deactivated records from ACTIVE Partition Category to RECENT
//#Scheduled(cron="0 */3 * * * ?", zone = "GMT")
//Cron Job runs in every 2 hours with initial delay of 1 minute to move recently deactivated records from ACTIVE Partition Category to RECENT
#Scheduled(fixedDelay = 7200000, initialDelay = 60000)
public void scheduledUpdateQueriesJob() {
TableNames tables = new TableNames(); // 5 tables in total
tableNames.getTableNames().stream().forEach(x -> scheduledTasksServiceRepository.runQuery(x));
} }
Repository for running scheduled queries:
#Repository
#Transactional(propagation = Propagation.REQUIRES_NEW, timeout = 60)
#Async("AppTaskExecutor")
public class ScheduledTasksServiceRepository {
#PersistenceContext
private EntityManager entityManager;
String query = "some update query";
public void runQuery(tableName)
{
try{
Integer status = entityManager.createNativeQuery(query).executeUpdate();
}catch (QueryTimeoutException queryTimeoutException){
....
}catch (PersistenceException persistenceException){
...
}catch (Exception exception){
...
}
}
Now this query is taking lots of time in updating and creating a blocking_session in database.
If I keep timeout too long like 5000 seconds, it gives PersistenceException which says connection closed and not able to roll back but transaction is not committed even. This job is running in every 2 hours and creating a new thread which again block a new session object.
persistanceException :
2021-02-15 15:36:37,563 WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper [App-2] SQL Error: 17008, SQLState: 08003
2021-02-15 15:36:37,564 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper [App-2] Closed Connection
2021-02-15 15:36:37,588 ERROR com.csg.cs.crs.dante.ext.service.riskevent.persistence.ScheduledTasksServiceRepository [App-2] persistenceException occurred while running Query for table : TABLE_2
javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: could not execute statement
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:154)
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:181)
at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1621)
at com.csg.cs.crs.dante.ext.service.riskevent.persistence.ScheduledTasksServiceRepository.runQuery(ScheduledTasksServiceRepository.java:58)
at com.csg.cs.crs.dante.ext.service.riskevent.persistence.ScheduledTasksServiceRepository$$FastClassBySpringCGLIB$$11e07c76.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.hibernate.exception.JDBCConnectionException: could not execute statement
at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:112)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:99)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:200)
at org.hibernate.engine.query.spi.NativeSQLQueryPlan.performExecuteUpdate(NativeSQLQueryPlan.java:107)
at org.hibernate.internal.SessionImpl.executeNativeUpdate(SessionImpl.java:1507)
at org.hibernate.query.internal.NativeQueryImpl.doExecuteUpdate(NativeQueryImpl.java:295)
at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1612)
... 18 common frames omitted
Caused by: java.sql.SQLRecoverableException: Closed Connection
at oracle.jdbc.driver.PhysicalConnection.needLine(PhysicalConnection.java:3525)
at oracle.jdbc.driver.OracleStatement.closeOrCache(OracleStatement.java:1478)
at oracle.jdbc.driver.OracleStatement.close(OracleStatement.java:1461)
at oracle.jdbc.driver.OracleStatementWrapper.close(OracleStatementWrapper.java:122)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.close(OraclePreparedStatementWrapper.java:98)
at oracle.ucp.jdbc.proxy.oracle.StatementProxy.close(StatementProxy.java:145)
at oracle.ucp.jdbc.proxy.oracle$1ucp$1jdbc$1proxy$1oracle$1StatementProxy$2oracle$1jdbc$1internal$1OraclePreparedStatement$$$Proxy.close(Unknown Source)
at oracle.ucp.jdbc.proxy.oracle.ConnectionProxyBase.handleSQRecoverableException(ConnectionProxyBase.java:94)
at oracle.ucp.jdbc.proxy.oracle.StatementProxy.onError(StatementProxy.java:255)
at oracle.ucp.jdbc.proxy.oracle$1ucp$1jdbc$1proxy$1oracle$1StatementProxy$2oracle$1jdbc$1internal$1OraclePreparedStatement$$$Proxy.executeUpdate(Unknown Source)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:197)
... 22 common frames omitted
2021-02-15 15:36:37,608 ERROR org.springframework.aop.interceptor.SimpleAsyncUncaughtExceptionHandler [App-2] Unexpected exception occurred invoking async method: public void com.csg.cs.crs.dante.ext.service.riskevent.persistence.ScheduledTasksServiceRepository.runQuery(java.lang.String)
org.springframework.orm.jpa.JpaSystemException: Unable to rollback against JDBC Connection; nested exception is org.hibernate.TransactionException: Unable to rollback against JDBC Connection
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:353)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:255)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:538)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:633)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:386)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.hibernate.TransactionException: Unable to rollback against JDBC Connection
at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.rollback(AbstractLogicalConnectionImplementor.java:127)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.rollback(JdbcResourceLocalTransactionCoordinatorImpl.java:304)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:263)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:534)
... 12 common frames omitted
Caused by: java.sql.SQLRecoverableException: Closed Connection
at oracle.jdbc.driver.PhysicalConnection.getAutoCommit(PhysicalConnection.java:2089)
at oracle.jdbc.driver.PhysicalConnection.rollback(PhysicalConnection.java:2218)
at oracle.ucp.jdbc.proxy.oracle$1ucp$1jdbc$1proxy$1oracle$1ConnectionProxy$2oracle$1jdbc$1internal$1OracleConnection$$$Proxy.rollback(Unknown Source)
at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.rollback(AbstractLogicalConnectionImplementor.java:121)
... 16 common frames omitted
2021-02-15 15:50:33,046 WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper [App-6] SQL Error: 17008, SQLState: 08003
2021-02-15 15:50:33,047 WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper [App-9] SQL Error: 17008, SQLState: 08003
2021-02-15 15:50:33,047 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper [App-6] Closed Connection
2021-02-15 15:50:33,047 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper [App-9] Closed Connection
Timeout Exception :
2021-02-15 19:28:06,929 ERROR com.csg.cs.crs.dante.ext.service.riskevent.persistence.ScheduledTasksServiceRepository [App-2] QueryTimeoutException occurred while running Query for table : TABLE_2
javax.persistence.QueryTimeoutException: could not execute statement
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:113)
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:181)
at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1621)
at com.csg.cs.crs.dante.ext.service.riskevent.persistence.ScheduledTasksServiceRepository.runQuery(ScheduledTasksServiceRepository.java:58)
at com.csg.cs.crs.dante.ext.service.riskevent.persistence.ScheduledTasksServiceRepository$$FastClassBySpringCGLIB$$11e07c76.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.hibernate.QueryTimeoutException: could not execute statement
at org.hibernate.dialect.Oracle8iDialect$3.convert(Oracle8iDialect.java:577)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:99)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:200)
at org.hibernate.engine.query.spi.NativeSQLQueryPlan.performExecuteUpdate(NativeSQLQueryPlan.java:107)
at org.hibernate.internal.SessionImpl.executeNativeUpdate(SessionImpl.java:1507)
at org.hibernate.query.internal.NativeQueryImpl.doExecuteUpdate(NativeQueryImpl.java:295)
at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1612)
... 18 common frames omitted
Caused by: java.sql.SQLTimeoutException: ORA-01013: user requested cancel of current operation
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:509)
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:461)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1104)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:550)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:268)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:655)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:270)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:91)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:970)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1205)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3666)
at oracle.jdbc.driver.T4CPreparedStatement.executeInternal(T4CPreparedStatement.java:1426)
at oracle.jdbc.driver.OraclePreparedStatement.executeLargeUpdate(OraclePreparedStatement.java:3756)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3736)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1063)
at oracle.ucp.jdbc.proxy.oracle$1ucp$1jdbc$1proxy$1oracle$1StatementProxy$2oracle$1jdbc$1internal$1OraclePreparedStatement$$$Proxy.executeUpdate(Unknown Source)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:197)
... 22 common frames omitted
Caused by: oracle.jdbc.OracleDatabaseException: ORA-01013: user requested cancel of current operation
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:513)
... 38 common frames omitted
How can we :
flush/save/commit all the records updated before timeout/connection close. Partial update would also do for me as it will reduce no of records to be updated when same job will run again.
kill the session used to run this query which created a lock in database. Need to kill session before timeout.
Run this scheduled task only in single instance. Multiple instances of this service gets deployed.
Also, How can I get default timeout and connection closure time values for this session/entity manager. As of now, nothing explicitly set in application.yml.