Spring batch scope issue while using spring boot - spring

Question

I'm having Standalone spring batch job. This works perfectly fine when in JUNIT
#RunWith(SpringJUnit4ClassRunner.class)
//#SpringApplicationConfiguration(classes = KPBootApplication.class)
#ContextConfiguration(locations={"classpath:kp-sb.xml"})
public class KPBootApplicationTests {
private final static Logger LOG=LoggerFactory.getLogger(KPBootApplicationTests.class);
#Autowired
ApplicationContext context;
#Autowired
private JobLauncher jobLauncher;
#Autowired
private Job job;
#Test
public void testJob() {
final JobParameters jobParameters = new JobParametersBuilder()
.toJobParameters();
JobExecution execution;
try {
execution = jobLauncher.run(job, jobParameters);
final ExitStatus status = execution.getExitStatus();
if (ExitStatus.COMPLETED.getExitCode().equals(status.getExitCode())) {
LOG.info("Job completed Yeaaaaaaaa!!!!!");
} else {
final List<Throwable> exceptions = execution
.getAllFailureExceptions();
for (final Throwable throwable : exceptions) {
LOG.error(throwable.getMessage(), throwable);
}
}
} catch (JobExecutionAlreadyRunningException e) {
LOG.error(e.getMessage(), e);
} catch (JobRestartException e) {
LOG.error(e.getMessage(), e);
} catch (JobInstanceAlreadyCompleteException e) {
LOG.error(e.getMessage(), e);
} catch (JobParametersInvalidException e) {
LOG.error(e.getMessage(), e);
}
}
}
And the configuration file
<!-- Below code till Job Repo is commented out during spring-boot -->
<context:property-placeholder
properties-ref="kpProps" />
<util:properties id="kpProps">
<prop key="app.file.path">
D:/temp/kp1/all
</prop>
<prop key="app.max_thread_num">
10
</prop>
</util:properties>
<!--
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<property name="transactionManager" ref="batchTransactionManager" />
</bean>
-->
<bean id="batchTransactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager">
<property name="rollbackOnCommitFailure" value="false" />
</bean>
<bean id="multiResourcePartitionerReq"
class="org.springframework.batch.core.partition.support.MultiResourcePartitioner">
<property name="resources" value="file:${app.file.path}/kp_http_request*" />
</bean>
<bean id="multiResourcePartitionerRes"
class="org.springframework.batch.core.partition.support.MultiResourcePartitioner">
<property name="resources" value="file:${app.file.path}/kp_http_response*" />
</bean>
<bean id="kpPool"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor"
destroy-method="destroy">
<property name="maxPoolSize" value="${app.max_thread_num}" />
</bean>
<bean id="idIncrementer1"
class="org.springframework.batch.core.launch.support.RunIdIncrementer" />
<batch:job id="kpGenJob" incrementer="idIncrementer1"
job-repository="jobRepository">
<batch:split id="splitStep" next="step4">
<batch:flow>
<batch:step id="step2">
<batch:partition partitioner="multiResourcePartitionerReq"
step="step2.slave">
<batch:handler task-executor="kpPool" />
</batch:partition>
</batch:step>
</batch:flow>
<batch:flow>
<batch:step id="step3">
<batch:partition partitioner="multiResourcePartitionerRes"
step="step3.slave">
<batch:handler task-executor="kpPool" />
</batch:partition>
</batch:step>
</batch:flow>
</batch:split>
<batch:step id="step4">
<batch:tasklet transaction-manager="batchTransactionManager">
<ref bean="kptasklet" />
</batch:tasklet>
</batch:step>
</batch:job>
<batch:step id="step2.slave">
<batch:tasklet transaction-manager="batchTransactionManager">
<batch:chunk reader="reqItemReader" writer="cvsFileItemWriter"
commit-interval="10000" />
</batch:tasklet>
</batch:step>
<batch:step id="step3.slave">
<batch:tasklet transaction-manager="batchTransactionManager">
<batch:chunk reader="resItemReader" writer="cvsFileItemWriter"
commit-interval="10000" />
</batch:tasklet>
</batch:step>
<bean id="reqItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"
scope="step">
<property name="resource" value="#{stepExecutionContext['fileName']}" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<!-- split it -->
<property name="lineTokenizer">
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="includedFields" value="5,6,8,10,11"></property>
<property name="names"
value="f1,f2,f3,f4,f5" />
<property name="strict" value="false" />
</bean>
</property>
<property name="fieldSetMapper">
<bean
class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
<property name="prototypeBeanName" value="tblHttpData" />
</bean>
</property>
</bean>
</property>
<property name="linesToSkip" value="1"></property>
</bean>
<bean id="tblHttpData" class="com.kp.batch.batch.job.domain.TblHttpData"
scope="prototype" />
<bean id="resItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"
scope="step">
<property name="resource" value="#{stepExecutionContext['fileName']}" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<!-- split it -->
<property name="lineTokenizer">
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="includedFields" value="3,4"></property>
<property name="names" value="f1,f2" />
<property name="strict" value="false" />
</bean>
</property>
<property name="fieldSetMapper">
<bean
class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
<property name="prototypeBeanName" value="tblHttpData" />
<property name="strict" value="true" />
</bean>
</property>
</bean>
</property>
<property name="linesToSkip" value="1"></property>
</bean>
<bean id="kptasklet" class="com.kp.batch.batch.job.step.KPTasklet" />
<bean id="cvsFileItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"
scope="step">
<!-- write to this csv file -->
<property name="resource" value="#{stepExecutionContext['fileName']}.tmp" />
<property name="shouldDeleteIfExists" value="true" />
<property name="lineAggregator">
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter" value="," />
<property name="fieldExtractor">
<bean
class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<property name="names"
value="server,refferer,application,baseApplication,httpSize" />
</bean>
</property>
</bean>
</property>
</bean>
The Main Class
#SpringBootApplication
#EnableBatchProcessing
#ImportResource(value={"classpath:spring-context.xml"})
public class KPBootApplication {
private final static Logger LOG = LoggerFactory
.getLogger(KPBootApplication.class);
public static void main(String[] args) {
ApplicationContext ctx = SpringApplication.run(
KPBootApplication.class, args);
LOG.info("Application KPBOOT Started");
SayHello hello = (SayHello) ctx.getBean("sayHello");
if (hello != null) {
LOG.debug("hello is not null");
LOG.info("Got message {}", hello.getMessage());
} else {
LOG.debug("hello is null");
}
LOG.info("Done");
}
}
When I comment out contextconfiguration annotation and enable spring-boot by enabling #SpringApplicationConfiguration( note: I've enabled spring batch using annotation #EnableBatchProcessing) and comment out beans which are automcatically created by spring-boot for spring batch such as JobRepo I get following error saying no context holder for step scope
java.lang.IllegalStateException: Failed to load ApplicationContext
at org.springframework.test.context.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:94)
at org.springframework.test.context.DefaultTestContext.getApplicationContext(DefaultTestContext.java:72)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:117)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:83)
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:212)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:200)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:252)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:254)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:217)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:83)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:68)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:163)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'step2': Cannot resolve reference to bean 'step2.slave' while setting bean property 'step'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'step2.slave': Cannot resolve reference to bean 'reqItemReader' while setting bean property 'itemReader'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'reqItemReader': Scope 'step' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for step scope
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1469)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1214)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:743)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:691)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:321)
at org.springframework.boot.test.SpringApplicationContextLoader.loadContext(SpringApplicationContextLoader.java:98)
at org.springframework.test.context.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:68)
at org.springframework.test.context.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:86)
... 25 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'step2.slave': Cannot resolve reference to bean 'reqItemReader' while setting bean property 'itemReader'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'reqItemReader': Scope 'step' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for step scope
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1469)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1214)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
... 42 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'reqItemReader': Scope 'step' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for step scope
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:352)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
... 52 common frames omitted
Caused by: java.lang.IllegalStateException: No context holder available for step scope
at org.springframework.batch.core.scope.StepScope.getContext(StepScope.java:160)
at org.springframework.batch.core.scope.StepScope.get(StepScope.java:99)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:337)
... 54 common frames omitted
Not sure why spring boot throws error
NOTE:spring-boot version 1.2.1

Answer 1

This may be a bug (we're still investigating), however we do have a work around. The cause of this is that when using #EnableBatchProcessing the StepScope that is automatically configured assumes java config and therefore does not proxy the step scoped beans, causing them to be created too soon. The work around is to manually configure a StepScope in your XML configuration with the following configuration:
<bean id="stepScope" class="org.springframework.batch.core.scope.StepScope">
<property name="autoProxy" value="true"/>
</bean>

Answer 2

Michael's comment is working for me, I am also providing JavaConfig copy-paste alternative for lazy people like me :)
#Bean
public StepScope stepScope() {
final StepScope stepScope = new StepScope();
stepScope.setAutoProxy(true);
return stepScope;
}

Answer 3

Seeing as you are using #RunWith(SpringRunner.class), declaring #TestExecutionListeners({..., StepScopeTestExecutionListener.class}) above your class will setup the scopes for you.
Same with #TestExecutionListeners({..., JobScopeTestExecutionListener.class}) for jobScope.

Answer 4

You can solve by 2 Ways:
A.
Use spring.main.allow-bean-definition-overriding=true in application.properties
B.
To solve spring boot batch scope issue and avoid the use of spring.main.allow-bean-definition-overriding=true:
Solutions:
Disable autoProxy of StepScope. To perform that use following code snippet as a reference:
<bean id="stepScope" class="org.springframework.batch.core.scope.StepScope">
<property name="autoProxy" value="false"/>
</bean>
Now perform following Spring bean configuration:
A.)
To configure step scope via XML-Based configuration:
<bean id="userPreferences" class="com.foo.UserPreferences" scope="step">
<aop:scoped-proxy/>
</bean>
B.)
To configure step scope via Java-Based configuration:
#StepScope
#Component(value = "userPreferences")
public class UserPreference {}

Answer 5

Don't forget to configure
spring.main.allow-bean-definition-overriding=true

Answer 6

Simply adding #SpringBatchTest on your test class should works.