Dealing with "Database is already closed" with H2 embedded database and spring-jdbc - h2

Event though the built-in H2 EmbeddedDatabaseConfigurer provides a JDBC connection with DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false options set, the annoying exception is caused by AbstractEmbeddedDatabaseConfigurer.shutdown().
There are already plenty of discussions around this topic on stackoverflow. I have tried to play a bit with the options and have come to the conclusion that the rather odd settings (DB_CLOSE_DELAY=20;DB_CLOSE_ON_EXIT=false) provided with a custom DataSourceFactory reduce the number of occurrences but do not completely eliminate it.
/**
* This factory tries to solve the problem that Spring tries to close a data source that has already been closed by the H2 engine/H2 driver.
*/
public class H2DataSourceFactory implements DataSourceFactory {
private final SimpleDriverDataSource dataSource = new SimpleDriverDataSource();
#Override
public ConnectionProperties getConnectionProperties() {
return new ConnectionProperties() {
#Override
public void setDriverClass(Class<? extends Driver> driverClass) {
dataSource.setDriverClass(driverClass);
}
#Override
public void setUrl(String url) {
// ignore the URL and use a custom URL
dataSource.setUrl("jdbc:h2:mem:%s;DB_CLOSE_DELAY=20;DB_CLOSE_ON_EXIT=false;MODE=PostgreSQL");
}
#Override
public void setUsername(String username) {
dataSource.setUsername(username);
}
#Override
public void setPassword(String password) {
dataSource.setPassword(password);
}
};
}
#Override
public DataSource getDataSource() {
return this.dataSource;
}
}
Has someone else come to a solution that really gets rid of this effect?
Here is the stacktrace:
14:12:32.691 [localhost-startStop-2] WARN o.s.j.d.e.H2EmbeddedDatabaseConfigurer - Could not shutdown embedded database
org.h2.jdbc.JdbcSQLException: Database is already closed (to disable automatic closing at VM shutdown, add ";DB_CLOSE_ON_EXIT=FALSE" to the db URL) [90121-160]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:329) ~[h2-1.3.160.jar:1.3.160]
at org.h2.message.DbException.get(DbException.java:169) ~[h2-1.3.160.jar:1.3.160]
at org.h2.message.DbException.get(DbException.java:146) ~[h2-1.3.160.jar:1.3.160]
at org.h2.message.DbException.get(DbException.java:135) ~[h2-1.3.160.jar:1.3.160]
at org.h2.jdbc.JdbcConnection.checkClosed(JdbcConnection.java:1394) ~[h2-1.3.160.jar:1.3.160]
at org.h2.jdbc.JdbcConnection.checkClosed(JdbcConnection.java:1369) ~[h2-1.3.160.jar:1.3.160]
at org.h2.jdbc.JdbcConnection.createStatement(JdbcConnection.java:191) ~[h2-1.3.160.jar:1.3.160]
at org.springframework.jdbc.datasource.embedded.AbstractEmbeddedDatabaseConfigurer.shutdown(AbstractEmbeddedDatabaseConfigurer.java:42) ~[spring-jdbc-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory.shutdownDatabase(EmbeddedDatabaseFactory.java:177) [spring-jdbc-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory$EmbeddedDataSourceProxy.shutdown(EmbeddedDatabaseFactory.java:250) [spring-jdbc-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_25]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_25]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_25]
at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_25]
at org.springframework.beans.factory.support.DisposableBeanAdapter.invokeCustomDestroyMethod(DisposableBeanAdapter.java:350) [spring-beans-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:273) [spring-beans-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578) [spring-beans-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554) [spring-beans-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:900) [spring-beans-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523) [spring-beans-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:907) [spring-beans-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:908) [spring-context-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:884) [spring-context-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:836) [spring-context-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:579) [spring-web-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:115) [spring-web-4.1.5.RELEASE.jar:4.1.5.RELEASE]
at org.apache.catalina.core.StandardContext.listenerStop(StandardContext.java:4773) [tomcat-embed-core-8.0.20.jar:8.0.20]
at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5385) [tomcat-embed-core-8.0.20.jar:8.0.20]
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232) [tomcat-embed-core-8.0.20.jar:8.0.20]
at org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1425) [tomcat-embed-core-8.0.20.jar:8.0.20]
at org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1414) [tomcat-embed-core-8.0.20.jar:8.0.20]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_25]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]

Related

Why are my connections not returning to the pool with Hikari?

We have this database layer with JdbcTemplate:
#Override
#Transactional
protected SomeObj insert(SomeObj obj) {
this.getNamedParameterJdbcTemplate().update(INSERT_QUERY, MyJdbcUtils.prepareQueryParams(obj));
synchronized (obj.getSomeList()) { decidable.getReasons().forEach(someItem -> { if (someItem != null) getJdbcTemplate().update(INSERT_ITEM_QUERY, obj.getId(), someItem ); }); }
return obj;
}
#Override
#Transactional
public SomeObj update(SomeObj obj) {
this.getNamedParameterJdbcTemplate().update(UPDATE_QUERY, MyJdbcUtils.prepareQueryParams(obj));
synchronized (obj.getSomeList()) { decidable.getReasons().forEach(someItem -> { if (someItem != null) getJdbcTemplate().update(UPDATE_ITEM_QUERY, obj.getId(), someItem ); }); }
return obj;
}
#Override
public SomeObj upsert(SomeObj obj) {
try {
return this.insert(obj);
} catch (DuplicateKeyException exception) {
final SomeObj existingObj = this.find(obj.getSomeId()).get(0);
// Populate some fields from existing object
}
return this.update(transaction);
}
MyJdbcUtils is as follows:
public static Map<String, ?> prepareQueryParams(SomeObj obj) {
return new HashMap<String, Object>() {{
put("someColumn", obj.getSomeColumn());
...
}};
}
My database related application.properties are as follows:
spring.datasource.jdbc-url=...
spring.datasource.username=...
spring.datasource.password=...
spring.datasource.initialization-mode=never
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.leakDetectionThreshold=5000
spring.datasource.maximumPoolSize=100
spring.datasource.minimumIdle=20
spring.datasource.idleTimeout=30000
After a while, Hikari pool logs looks like this:
HikariPool-1 - Timeout failure stats (total=100, active=100, idle=0, waiting=2)
with a lot of:
Connection leak detection triggered for org.postgresql.jdbc.PgConnection#17b19afa on thread http-nio-8080-exec-100, stack trace follows
java.lang.Exception: Apparent connection leak detected
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128) ~[HikariCP-3.4.1.jar!/:?]
at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122) ~[hibernate-core-5.4.6.Final.jar!/:5.4.6.Final]
at org.hibernate.internal.NonContextualJdbcConnectionAccess.obtainConnection(NonContextualJdbcConnectionAccess.java:38) ~[hibernate-core-5.4.6.Final.jar!/:5.4.6.Final]
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:104) ~[hibernate-core-5.4.6.Final.jar!/:5.4.6.Final]
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:134) ~[hibernate-core-5.4.6.Final.jar!/:5.4.6.Final]
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getConnectionForTransactionManagement(LogicalConnectionManagedImpl.java:250) ~[hibernate-core-5.4.6.Final.jar!/:5.4.6.Final]
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.begin(LogicalConnectionManagedImpl.java:258) ~[hibernate-core-5.4.6.Final.jar!/:5.4.6.Final]
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.begin(JdbcResourceLocalTransactionCoordinatorImpl.java:246) ~[hibernate-core-5.4.6.Final.jar!/:5.4.6.Final]
at org.hibernate.engine.transaction.internal.TransactionImpl.begin(TransactionImpl.java:83) ~[hibernate-core-5.4.6.Final.jar!/:5.4.6.Final]
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.beginTransaction(HibernateJpaDialect.java:184) ~[spring-orm-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:402) ~[spring-orm-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747) ~[spring-aop-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) ~[spring-aop-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at com.company.project.core.domain.transaction.JdbcTransactionDaoImpl$$EnhancerBySpringCGLIB$$87fab16e.update(<generated>) ~[classes!/:?]
at com.company.project.core.integration.spring.TransactionIntegrationConfiguration.asyncPersistenceService(TransactionIntegrationConfiguration.java:208) ~[classes!/:?]
at sun.reflect.GeneratedMethodAccessor326.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171) ~[spring-messaging-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120) ~[spring-messaging-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.integration.gateway.GatewayProxyFactoryBean.invokeGatewayMethod(GatewayProxyFactoryBean.java:508) ~[spring-integration-core-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.integration.gateway.GatewayProxyFactoryBean.doInvoke(GatewayProxyFactoryBean.java:478) ~[spring-integration-core-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.integration.gateway.GatewayProxyFactoryBean$Invoker.get(GatewayProxyFactoryBean.java:1012) ~[spring-integration-core-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_212]
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134) ~[?:1.8.0_212]
at org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor.submit(ThreadPoolTaskExecutor.java:341) ~[spring-context-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.integration.gateway.GatewayProxyFactoryBean.invoke(GatewayProxyFactoryBean.java:451) ~[spring-integration-core-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at com.sun.proxy.$Proxy181.processTransaction(Unknown Source) ~[?:?]
at com.company.project.core.services.transaction.TransactionServiceImpl.async(TransactionServiceImpl.java:101) ~[classes!/:?]
at com.company.project.core.services.transaction.TransactionServiceImpl$$FastClassBySpringCGLIB$$73b9cf27.invoke(<generated>) ~[classes!/:?]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769) ~[spring-aop-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747) ~[spring-aop-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93) ~[spring-aop-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747) ~[spring-aop-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) ~[spring-aop-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at com.company.project.core.services.transaction.TransactionServiceImpl$$EnhancerBySpringCGLIB$$eae305f2.async(<generated>) ~[classes!/:?]
at com.company.project.core.controllers.TransactionController.async(TransactionController.java:52) ~[classes!/:?]
at sun.reflect.GeneratedMethodAccessor323.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) ~[spring-web-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) ~[spring-web-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.0.RELEASE.jar!/:5.2.0.RELEASE]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-embed-core-9.0.27.jar!/:9.0.27]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
until:
HikariPool-1 - Connection is not available, request timed out after 30000ms
DBA confirms that the longest query is ~1second and I'm sure that the queries should run around that long. What am I doing wrong? Why aren't my connections returning to the pool and are still marked as active?
Update:
As per comments, I switched the logic to:
private final TransactionTemplate transactionTemplate;
#Autowired
public MyDao (PlatformTransactionManager transactionManager, DataSource dataSource) {
setDataSource(dataSource); // Class extends NamedParameterJdbcDaoSupport
this.transactionTemplate = new TransactionTemplate(transactionManager);
}
#Override
protected SomeObj insert(SomeObj obj) {
return this.transactionTemplate.execute(status -> {
this.getNamedParameterJdbcTemplate().update(INSERT_QUERY, MyJdbcUtils.prepareQueryParams(obj));
for (String someItem : obj.getSomeList()) if (someItem != null) getJdbcTemplate().update(INSERT_ITEM_QUERY, obj.getId(), someItem );
return obj;
});
}
#Override
public SomeObj update(SomeObj obj) {
return this.transactionTemplate.execute(status -> {
this.getNamedParameterJdbcTemplate().update(UPDATE_QUERY, MyJdbcUtils.prepareQueryParams(obj));
for (String someItem : obj.getSomeList()) if (someItem != null) getJdbcTemplate().update(UPDATE_ITEM_QUERY, obj.getId(), someItem );
return obj;
});
}
However the issue did not resolve.

SCDF - Create Metadata tables in different schema and work the Batch Job with different schema

I'm using Spring Batch which loads data from Oracle and put it into the MongoDB. I'm looking to use the Spring Cloud Data Flow, but SCDF doesn't have support for the MongoDB.
Is there any way if we can maintain the SCDF Metadata into the Postgres (as its has good support) or H2, in this example I used H2, and run the actual batch jobs which load data from Oracle to MongoDB.
When I tried to run this combination, I get the below error, certainly some config changes needs to be tweek. Any quick workaround ?
Error:
org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:152) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader$$FastClassBySpringCGLIB$$ebb633d0.invoke(<generated>) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.batch.item.database.JdbcCursorItemReader$$EnhancerBySpringCGLIB$$3c62d122.open(<generated>) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:103) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:311) ~[spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:399) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:313) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:144) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:137) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at com.sun.proxy.$Proxy66.run(Unknown Source) [na:na]
at com.mastercard.customer.data.management.refdata.ReferenceMongoBatchApplication.run(ReferenceMongoBatchApplication.java:63) [classes!/:1.0]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:813) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:797) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:324) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
at com.mastercard.customer.data.management.refdata.ReferenceMongoBatchApplication.main(ReferenceMongoBatchApplication.java:51) [classes!/:1.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [reference-mongo-batch-1.0.jar:1.0]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [reference-mongo-batch-1.0.jar:1.0]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [reference-mongo-batch-1.0.jar:1.0]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [reference-mongo-batch-1.0.jar:1.0]
application.properties
# ORACLE DATASOURCE
spring.datasource.url=jdbc:oracle:thin:#//XXXXXX:1527/XXXX
spring.datasource.username=user
spring.datasource.password=password
spring.datasource.driver-class-name=oracle.jdbc.OracleDriver
spring.datasource.validation-query= select 1
#spring.datasource.hikari.validationTimeout=30000
spring.data.mongodb.host=localhost
spring.data.mongodb.port=27017
spring.data.mongodb.database=mgdb_drefdata_99
Config
#Configuration
public class EmployeeJob {
#Value( "${spring.chunk.size}")
private String chunkSize;
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private JdbcCursorItemReader<Employee> EmployeeReader;
#Autowired
public AsyncItemProcessor<Employee, Employee> asyncItemProcessor;
#Autowired
public AsyncItemWriter<Employee> asyncItemWriter;
#Bean
public EmployeeStepExecuListner EmployeeStepExecuListner() {
return new EmployeeStepExecuListner();
}
#Bean("readEmployeeJob")
#Primary
public Job readEmployeeJob() {
return jobBuilderFactory.get("readEmployeeJob")
.incrementer(new RunIdIncrementer())
.start(EmployeeStepOne())
.build();
}
#SuppressWarnings({ "unchecked", "rawtypes" })
#Bean
public Step EmployeeStepOne() {
return stepBuilderFactory.get("EmployeeStepOne")
.<Employee, Employee>chunk(Integer.parseInt(chunkSize))
.reader(EmployeeReader)
.processor((ItemProcessor) asyncItemProcessor)
.writer(asyncItemWriter)
.build();
}
}
Config
#Configuration
public class EmployeeBatchConfig {
#Autowired
#Qualifier(value="oracleDS")
private DataSource dataSource;
#Bean(destroyMethod = "")
#StepScope
public JdbcCursorItemReader<Employee> EmployeeReader() throws Exception {
JdbcCursorItemReader<Employee> reader = new JdbcCursorItemReader<>();
reader.setDataSource(this.dataSource);
reader.setSql(SELECT_Employee_SQL);
reader.setRowMapper(new EmployeeRowMapper());
reader.afterPropertiesSet();
return reader;
}
#Bean
public ItemProcessor<Employee, Employee> EmployeeProcessor() {
return new EmployeeProcessor();
}
#Bean
public AsyncItemProcessor<Employee, Employee> asyncItemProcessor() throws Exception{
AsyncItemProcessor<Employee, Employee> asyncItemProcessor = new AsyncItemProcessor<>();
asyncItemProcessor.setDelegate(EmployeeProcessor());
asyncItemProcessor.setTaskExecutor(new SimpleAsyncTaskExecutor());
asyncItemProcessor.afterPropertiesSet();
return asyncItemProcessor;
}
#Bean
public EmployeeWriter EmployeeWriter() {
return new EmployeeWriter();
}
#Bean
public AsyncItemWriter<Employee> asyncItemWriter() throws Exception{
AsyncItemWriter<Employee> asyncItemWriter = new AsyncItemWriter<>();
asyncItemWriter.setDelegate(EmployeeWriter());
asyncItemWriter.afterPropertiesSet();
return asyncItemWriter;
}
}

Invalid property 'id' of bean class FlatFileItemReader, file read and Entity used has id

I'm currently working on a batch program that reads csv files, then inserts the data using hibernate and I believe JPA.
The error that I'm running into is the following:
org.springframework.beans.NotReadablePropertyException: Invalid property 'id' of bean class [org.springframework.batch.item.file.FlatFileItemReader]: Could not find field for property during fallback access!
I've tried the following to resolve the issue:
The error was coming up when reading a file that didn't have a column named id. Tried only files that had an id filed. Same issue remained
Found on here that the issue might be missing an id field (found on an SO thread), so added getter and setters wit the following for id fields in the Entity class.
#Id
#Column(name = "id", nullable = false)
private Long id;
and for files that didn't have an id
#Id
#Column(name = "id")
#GeneratedValue(strategy = GenerationType.SEQUENCE)
private Long id;
but the same issue occurred.
The getters and setters are structured as such.
public Long getId() { return id; }
public void setId(final Long id) { this.id = id; }
Another thread here suggested using Long instead of int (what they were originally) was the cause. Turns out it was.
The code that the issue was occurring at in the debugger referred to this code, namely the save method
public interface LCBOInventoryRepository extends CrudRepository {
#Override
#SuppressWarnings("unchecked")
public LCBOInventory save(LCBOInventory lcboInventoryObject);
but every object that is being passed into that method should have the Id information available as mention earlier.
As requested, here is the code for the reader and writer being used for the process that this issue is occuring on.
Reader:
#Component
public class LCBOInventoryReader extends AbstractLCBOReader implements ItemReader, InterstepDataRetriever {
#Autowired
public LCBOInventoryReader(final String currentCSVFilePathKey ) {
this.currentCSVFilePathKey = currentCSVFilePathKey;
}
private static final Logger log = LoggerFactory.getLogger(LCBOInventoryReader.class);
#Override
public ItemReader<LCBOInventory> read() throws UnexpectedInputException, ParseException, NonTransientResourceException {
Class<LCBOInventory> classType = LCBOInventory.class;
return createCSVReader(classType, currentCSVFilePath, inventoryTrackerProperties.getLCBOFilPropertiess().getInventory());
}
#Override
public void retrieveInterstepDataFromJobContext(final ExecutionContext jobContext) {
this.currentCSVFilePath = (String) jobContext.get(currentCSVFilePathKey);
}
}
And here is the Abstract class that is extended:
public abstract class AbstractLCBOReader {
#Autowired
protected LCBOInventoryTrackerProperties inventoryTrackerProperties;
protected String currentCSVFilePathKey;
protected String currentCSVFilePath;
private static final Logger log = LoggerFactory.getLogger(AbstractLCBOReader.class);
protected <T> ItemReader<T> createCSVReader(final Class<T> classType,
final String currentCSVFilePath,
final LCBOFileDetailsProperties properties) {
FlatFileItemReader<T> reader = new FlatFileItemReader<>();
// Skip a line to ignore the header information in these files
reader.setLinesToSkip(properties.getNumberOfLinesToSkipInFile());
reader.setResource(new FileSystemResource(currentCSVFilePath + File.separator + properties.getFileName()));
reader.setLineMapper(createLineMapper(classType, properties));
reader.setRecordSeparatorPolicy(new DefaultRecordSeparatorPolicy());
reader.setEncoding("utf8");
return reader;
}
private <T> LineMapper<T> createLineMapper(final Class<T> classType, final LCBOFileProperties.LCBOFileDetailsProperties properties) {
DefaultLineMapper<T> lineMapper = new DefaultLineMapper<>();
lineMapper.setLineTokenizer(createLineTokenizer(properties));
lineMapper.setFieldSetMapper(createFieldSetMapper(classType));
return lineMapper;
}
private <T> FieldSetMapper<T> createFieldSetMapper(final Class<T> classType) {
BeanWrapperFieldSetMapper<T> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
fieldSetMapper.setTargetType(classType);
return fieldSetMapper;
}
private LineTokenizer createLineTokenizer(final LCBOFileProperties.LCBOFileDetailsProperties properties) {
LCBOFileProperties.Column[] columns = properties.getColumns();
int[] columnIndexes = new int[columns.length];
String[] columnNames = new String[columns.length];
// populating the columnIndexes
for (int i = 0; i < columns.length; i++) {
columnIndexes[i] = columns[i].getColumnIndex();
columnNames[i] = columns[i].getColumnName();
}
DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer();
lineTokenizer.setIncludedFields(columnIndexes);
lineTokenizer.setNames(columnNames);
lineTokenizer.setDelimiter(",");
lineTokenizer.setQuoteCharacter('"');
return lineTokenizer;
}
}
And the interface InterstepDataRetriever that the reader implements
public interface InterstepDataRetriever {
#BeforeStep
default void retrieveValuesFromStepContext(StepExecution stepExecution) {
JobExecution jobExecution = stepExecution.getJobExecution();
ExecutionContext jobContext = jobExecution.getExecutionContext();
retrieveInterstepDataFromJobContext(jobContext);
}
void retrieveInterstepDataFromJobContext(ExecutionContext jobContext);
}
The latter shouldn't have any effect on this issue, but was included for completeness (besides you never know).
Writer(Old):
#Component
public class LCBOInventoryWriter implements ItemWriter<LCBOInventory> {
#Autowired
private LCBOInventoryRepository inventoryDAO;
#Override
public void write(List<? extends LCBOInventory> lcboInventoryItem) throws Exception {
inventoryDAO.save(lcboInventoryItem);
}
}
Writer(New - this was put in the same file as the Step ):
#Bean
public ItemWriter<LCBOStore> writer() {
HibernateItemWriter writer = new HibernateItemWriter();
writer.setSessionFactory(sessionFactory);
System.out.println("writing stuff");
return writer;
}
Here's the stack trace when executing the code (edit: stacktrace has been updated):
org.hibernate.MappingException: Unknown entity: org.springframework.batch.item.file.FlatFileItemReader
at org.hibernate.internal.SessionFactoryImpl.getEntityPersister(SessionFactoryImpl.java:783) ~[hibernate-core-5.0.12.Final.jar:5.0.12.Final]
at org.hibernate.internal.SessionImpl.getEntityPersister(SessionImpl.java:1520) ~[hibernate-core-5.0.12.Final.jar:5.0.12.Final]
at org.hibernate.engine.internal.ForeignKeys.isTransient(ForeignKeys.java:225) ~[hibernate-core-5.0.12.Final.jar:5.0.12.Final]
at org.hibernate.event.internal.AbstractSaveEventListener.getEntityState(AbstractSaveEventListener.java:510) ~[hibernate-core-5.0.12.Final.jar:5.0.12.Final]
at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.performSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:83) ~[hibernate-core-5.0.12.Final.jar:5.0.12.Final]
at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:73) ~[hibernate-core-5.0.12.Final.jar:5.0.12.Final]
at org.hibernate.internal.SessionImpl.fireSaveOrUpdate(SessionImpl.java:648) ~[hibernate-core-5.0.12.Final.jar:5.0.12.Final]
at org.hibernate.internal.SessionImpl.saveOrUpdate(SessionImpl.java:640) ~[hibernate-core-5.0.12.Final.jar:5.0.12.Final]
at org.hibernate.internal.SessionImpl.saveOrUpdate(SessionImpl.java:635) ~[hibernate-core-5.0.12.Final.jar:5.0.12.Final]
at org.springframework.batch.item.database.HibernateItemWriter.doWrite(HibernateItemWriter.java:140) ~[spring-batch-infrastructure-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.item.database.HibernateItemWriter.write(HibernateItemWriter.java:113) ~[spring-batch-infrastructure-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProcessor.writeItems(SimpleChunkProcessor.java:175) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProcessor.doWrite(SimpleChunkProcessor.java:151) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProcessor.write(SimpleChunkProcessor.java:274) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:199) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:75) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133) ~[spring-tx-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374) ~[spring-batch-infrastructure-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144) ~[spring-batch-infrastructure-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:392) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) [spring-aop-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) [spring-aop-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at com.sun.proxy.$Proxy54.run(Unknown Source) [na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:214) [spring-boot-autoconfigure-1.5.1.RELEASE.jar:1.5.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:231) [spring-boot-autoconfigure-1.5.1.RELEASE.jar:1.5.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:123) [spring-boot-autoconfigure-1.5.1.RELEASE.jar:1.5.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:117) [spring-boot-autoconfigure-1.5.1.RELEASE.jar:1.5.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:776) [spring-boot-1.5.1.RELEASE.jar:1.5.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:760) [spring-boot-1.5.1.RELEASE.jar:1.5.1.RELEASE]
at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:747) [spring-boot-1.5.1.RELEASE.jar:1.5.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-1.5.1.RELEASE.jar:1.5.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1162) [spring-boot-1.5.1.RELEASE.jar:1.5.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1151) [spring-boot-1.5.1.RELEASE.jar:1.5.1.RELEASE]
at com.lcbo.config.LCBOBatchConfig.main(LCBOBatchConfig.java:173) [classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92]
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) [idea_rt.jar:na]
So in a nutshell I'm at a loss as to what id field it needs to work, why it thinks that it's an entity (there's no #Entity annotation), and how i can either satisfy what it needs, or fix it so that it's not required.
Are you sure that you're passing in the correct Class<T> classType to the BeanWrapperFieldSetMapper?
This error makes it appear as if you're passing in the FlatFileItemReader class rather than LCBOInventory.
NotReadablePropertyException: Invalid property 'id' of bean class
[org.springframework.batch.item.file.FlatFileItemReader]:
UPDATE: Please check the write method of your writer. The stack trace shows the error is coming from there, not from your reader. Is it possible you're trying to persist a FlatFileItemReader object?
LCBOInventoryWriter.write(LCBOInventoryWriter.java:19)

Hibernate transaction block issue

Here is code from my relevant DAO class
#Autowired
private SessionFactory sessionFactory;
public void getTabletbyNameAndSave(String []selectedMedicines) {
sessionFactory.getCurrentSession().beginTransaction();
EPrescriber ePrescriber=new EPrescriber();
List<SelectedTablets> selectedTabletsList=new ArrayList<SelectedTablets>();
for (String item : selectedMedicines) {
Tablets tablets=null;
String hql="from Tablets t where t.category='"+item.trim()+"' ";
Query queryList = sessionFactory.getCurrentSession().createQuery(hql);
tablets=(Tablets)queryList.uniqueResult();
SelectedTablets selectedTablets=new SelectedTablets();
selectedTablets.setTablets(tablets);
selectedTablets.setePrescriberid(ePrescriber);
selectedTabletsList.add(selectedTablets);
sessionFactory.getCurrentSession().save(selectedTablets);
}
ePrescriber.setPatientName("Kasun");
ePrescriber.setSelectedTablets(selectedTabletsList);
sessionFactory.getCurrentSession().save(ePrescriber);
sessionFactory.getCurrentSession().getTransaction().commit();
}
Here is my error. It seems that my transaction block is not handled correctly. Please help advise me on this problem. Thanks!
org.springframework.transaction.TransactionSystemException: Could not commit Hibernate transaction; nested exception is org.hibernate.TransactionException: Transaction not successfully started
at org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:660)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:393)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621)
at com.priyan.patients.EPrescriberDAO$$EnhancerByCGLIB$$3ce885f5.getTabletbyNameAndSave(<generated>)
at com.priyan.patients.ContactsControllers.setTabletsNames(ContactsControllers.java:328)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:176)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:426)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:414)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:560)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:879)
at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:617)
at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1760)
at java.lang.Thread.run(Unknown Source)
Caused by: org.hibernate.TransactionException: Transaction not successfully started
at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:131)
at org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:656)
... 33 more
I would try manually assigning the transaction to a local variable to make sure you are running the final commit on the correct transaction. Something like the following:
#Autowired private SessionFactory sessionFactory;
public void getTabletbyNameAndSave(String []selectedMedicines) {
Session currentSession = sessionFactory.getCurrentSession();
Transaction tx = currentSession.beginTransaction();
EPrescriber ePrescriber=new EPrescriber();
List<SelectedTablets> selectedTabletsList=new ArrayList<SelectedTablets>();
for (String item : selectedMedicines) {
Tablets tablets=null;
String hql="from Tablets t where t.category='"+item.trim()+"' ";
Query queryList = currentSession.createQuery(hql);
tablets=(Tablets)queryList.uniqueResult();
SelectedTablets selectedTablets=new SelectedTablets();
selectedTablets.setTablets(tablets);
selectedTablets.setePrescriberid(ePrescriber);
selectedTabletsList.add(selectedTablets);
currentSession.save(selectedTablets);
}
ePrescriber.setPatientName("Kasun");
ePrescriber.setSelectedTablets(selectedTabletsList);
currentSession.save(ePrescriber);
tx.commit();
}

How to mock SessionFactory or Session by PowerMockito in a project using Spring and Hibernate?

I am working on a project that uses Spring and Hibernate. I have an abstract DAO that contains SessionFactory. I need to mock SessionFactory or Session for my unit test. But I have not been successful so far. Below is the code. Anyone has any idea? Thank you.
Here is my AbstractDAO
public abstract class AbstractDAO {
protected JdbcTemplate jdbcTemplate;
protected SessionFactory sessionFactory;
protected NamedParameterJdbcTemplate namedParameterJdbcTemplate;
...
#Autowired
public void setSessionFactory(SessionFactory sessionFactory) {
this.sessionFactory = sessionFactory;
}}
And here is my ConcreteDAO
public class LoginDAO extends AbstractDAO implements InitializingBean {
...
public User getLoggedinUserByUserid(Long userid){
log.info("in getLoggedinUserByUserid");
User result = null;
Session session = sessionFactory.openSession();
try {
session.beginTransaction();
result = (User) session.get(User.class, userid);
session.getTransaction().rollback();
session.close();
} catch (Exception e) {
log.error(e,e);
session.getTransaction().rollback();
session.close();
}
return result;
}
...}
and here is my test class
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = { "LoginDAOTest-context.xml" })
#PrepareForTest({SessionFactory.class, Session.class, Transaction.class})
public class LoginDAOTest2 extends BaseDAO {
private static final Logger log = Logger.getLogger(LoginDAOTest2.class);
#Rule
public PowerMockRule rule = new PowerMockRule();
private LoginDAO loginDAO = new LoginDAO();
private SessionFactory mockedSessionFactory;
private Session mockedSession;
private Transaction mockedTransaction;
#Autowired
public void setSessionFactory(SessionFactory sessionFactoryCore) {
mockedSessionFactory = PowerMockito.mock( sessionFactoryCore.getClass());
mockedSession = PowerMockito.mock(Session.class);
PowerMockito.doReturn(mockedSession).when(mockedSessionFactory).openSession();
PowerMockito.doReturn(mockedTransaction).when(mockedSession).beginTransaction();
loginDAO.setSessionFactory(this.mockedSessionFactory);
}
#Test
public void shouldRollbackInGetLoggedinUserByUseridWhenSessionThrowsException() {
// Given
PowerMockito.doThrow(new Exception()).when(mockedSession).get(User.class, 12L);
// When
loginDAO.getLoggedinUserByUserid(12L);
// Then
verify(mockedTransaction, times(1)).rollback();
}
}
and here is the result of junit.
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.aeon.ps.dao.LoginDAOTest2': Injection of autowired dependencies failed; nested exception is java.lang.NoClassDefFoundError: org/mockito/internal/MockitoInvocationHandler
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:287)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireBeanProperties(AbstractAutowireCapableBeanFactory.java:374)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:110)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:75)
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:321)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:211)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:288)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:290)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.lang.NoClassDefFoundError: org/mockito/internal/MockitoInvocationHandler
at org.powermock.api.mockito.PowerMockito.mock(PowerMockito.java:138)
at com.aeon.ps.dao.LoginDAOTest2.setSessionFactory(LoginDAOTest2.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:586)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:87)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284)
... 26 more
Caused by: java.lang.ClassNotFoundException: org.mockito.internal.MockitoInvocationHandler
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 35 more
Why don't do this :
#Before
public void setUp() {
mockedSessionFactory = Mockito.mock(SessionFactory.class);
mockedSession = Mockito.mock(Session.class);
mockedTransaction = Mockito.mock(Transaction.class);
Mockito.when(mockedSessionFactory.openSession()).thenReturn(mockedSession);
Mockito.when(mockedSession.beginTransaction()).thenReturn(mockedTransaction);
loginDAO.setSessionFactory(this.mockedSessionFactory);
}
instead of :
#Autowired
public void setSessionFactory(SessionFactory sessionFactoryCore) {
mockedSessionFactory = PowerMockito.mock( sessionFactoryCore.getClass());
mockedSession = PowerMockito.mock(Session.class);
PowerMockito.doReturn(mockedSession).when(mockedSessionFactory).openSession();
PowerMockito.doReturn(mockedTransaction).when(mockedSession).beginTransaction();
loginDAO.setSessionFactory(this.mockedSessionFactory);
}
In addition, it allows you to not use powermock.
Interesting part is here :
Caused by: java.lang.NoClassDefFoundError: org/mockito/internal/MockitoInvocationHandler
at org.powermock.api.mockito.PowerMockito.mock(PowerMockito.java:138)
at com.aeon.ps.dao.LoginDAOTest2.setSessionFactory(LoginDAOTest2.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:586)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:87)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284)
... 26 more
It means that powermock is using an internal class that is not available in the mockito distribution your are using.
From that information I gather you are using Mockito 1.9.5-rc1, with some other recent Powermock release. As of now, Johan din't released a newer version of Powermock that take care of the changes that were made in Mockito 1.9.5-rc1.
These refactoring changes in Mockito were made to help the introduction of the MockMaker API, which allows a third party to provide his own mock factory, bytecode engine or else to create mocks.
See the documentation here : http://docs.mockito.googlecode.com/hg/1.9.5-rc1/org/mockito/Mockito.html
So for now, you should use Mockito 1.9.0 if you are using Powermock.

Resources