Passing arguments from BatchJob to Tasklet in Spring Batch - spring

To all Spring enthusiasts, here's a good challenge. Hope someone can crack it!!!
I use Spring to batch the extraction process. I have two classes 'ExtractBatchJob' and 'TaskletImpl'
public class ExtractBatchJob {
/** Logger for current class */
private static Log log = LogFactory.getLog(Extractor.class);
public static void main(String[] args)
throws IOrganizationServiceRetrieveMultipleOrganizationServiceFaultFaultFaultMessage,
IOException {
ApplicationContext context = new ClassPathXmlApplicationContext(
"/META-INF/cxf/batch-context.xml");
SpringBusFactory factory = new SpringBusFactory(context);
Bus bus = factory.createBus();
SpringBusFactory.setDefaultBus(bus);
IOrganizationService service = (IOrganizationService) factory
.getApplicationContext().getBean("service");
JobLauncher jobLauncher = (JobLauncher)context.getBean("jobLauncher");
Job job = (Job) context.getBean("firstBatchJob");
try {
JobExecution execution = jobLauncher.run(job, new JobParameters());
}catch (Exception e){
e.printStackTrace();
}
}
The second class TaskletImpl implements the Spring Tasklet interface.
public class TaskletImpl implements Tasklet {
/** Logger for current class */
private static Log log = LogFactory.getLog(CRMExtractor.class);
/* (non-Javadoc)
* #see org.springframework.batch.core.step.tasklet.Tasklet#execute(org.springframework.batch.core.StepContribution, org.springframework.batch.core.scope.context.ChunkContext)
*/
#Overridepublic RepeatStatus execute(StepContribution arg0, ChunkContext arg1)
throws Exception {
// TODO Auto-generated method stub
log.info("************ CRM Extraction Batch Job is executing!!! *******");
//QUESTION: To Extract Entity from third party
// web service need object reference for
//factory and service from ExtractBatchjob class
List<Entity> orderEntities = getEntities("orderQueryImpl", factory, service);
OrderDao orderDao = (OrderDao) factory.getApplicationContext()
.getBean("orderDao");
orderDao.batchInsert(orderEntities);*/
return RepeatStatus.FINISHED;
}
public static List<Entity> getEntities(String queryImpl, SpringBusFactory factory,
IOrganizationService service)
throws IOrganizationServiceRetrieveMultipleOrganizationServiceFaultFaultFaultMessage,
IOException {
QueryBuilder queryBuilder = (QueryBuilderTemplate) factory
.getApplicationContext().getBean(queryImpl);
QueryExpression queryExpr = queryBuilder.getQuery();
EntityCollection result = service
.retrieveMultiple(queryExpr);
return result.getEntities().getEntities();
}
}
Below is the snippet of the context file
`<import resource="cxf.xml" />
<bean id="firstBatch" class="com.abc.model.TaskletImpl" />
<batch:step id="firstBatchStepOne">
<batch:tasklet ref="firstBatch" />
</batch:step>
<batch:job id="firstBatchJob">
<batch:step id="stepOne" parent="firstBatchStepOne" />
</batch:job>`
My question is quite straightforward, how do I pass the two variables/objects 'service' and 'factory' to the TaskletImpl class from the ExtractBatchJob class.

The cleanest solution is to wire service and factory using Spring injection mechanism.
You have two solution:
Create SpringBusFactory as Spring bean and wire it into tasklet
Define a ContextBean (as singleton) for you job, create SpringBusFactory and set it as property of ContextBean; wire this bean to your tasklet
If you want to use object created outside Spring context (with new I meant) must be injected into Spring context.

Related

Spring AOP with prototype beans

I am using Spring AOP to fire metrics in our application. I have created an annotation #CaptureMetrics which has an #around advice associated with it. The advice is invoked fine from all the methods tagged with #CaptureMetrics except for a case when a method is invoked on a prototype bean.
The annotation has #Target({ElementType.TYPE, ElementType.METHOD})
PointCut expression:
#Around(value = "execution(* *.*(..)) && #annotation(captureMetrics)",
argNames = "joinPoint,captureMetrics")
Prototype bean creation
#Bean
#Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
public DummyService getDummyServicePrototypeBean(int a, String b) {
return new DummyService(a, b);
}
DummyService has a method called dummyMethod(String dummyString)
#CaptureMetrics(type = MetricType.SOME_TYPE, name = "XYZ")
public Response dummyMethod(id) throws Exception {
// Do some work here
}
When dummyService.dummyMethod("123") is invoked from some other service, the #Around advice is not called.
Config class
#Configuration
public class DummyServiceConfig {
#Bean
public DummyServiceRegistry dummyServiceRegistry(
#Value("${timeout}") Integer timeout,
#Value("${dummy.secrets.path}") Resource dummySecretsPath) throws IOException {
ObjectMapper mapper = new ObjectMapper();
Map<String, String> transactionSourceToTokens = mapper.readValue(
dummySecretsPath.getFile(), new TypeReference<Map<String, String>>() {
});
DummyServiceRegistry registry = new DummyServiceRegistry();
transactionSourceToTokens.forEach((transactionSource, token) ->
registry.register(transactionSource,
getDummyServicePrototypeBean(timeout, token)));
return registry;
}
#Bean
#Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
public DummyService getDummyServicePrototypeBean(int a, String b) {
return new DummyService(a, b);
}
}
Singleton Registry class
public class DummyServiceRegistry {
private final Map<String, DummyService> transactionSourceToService = new HashMap<>();
public void register(String transactionSource, DummyService dummyService) {
this.transactionSourceToService.put(transactionSource, dummyService);
}
public Optional<DummyService> lookup(String transactionSource) {
return Optional.ofNullable(transactionSourceToService.get(transactionSource));
}
}
Any advice on this please?
Note:
The prototype Dummy service is used to call a third party client. It is a prototype bean as it has a state that varies based on whose behalf it is going to call the third party.
A singleton registry bean during initialization builds a map of {source_of_request, dummyService_prototype}. To get the dummyService prototype it calls getDummyServicePrototypeBean()
The configuration, registry and prototype dummy bean were correct.
I was testing the flow using an existing integration test and there instead of supplying a prototype Bean, new objects of DummyService were instantiated using the new keyword. It wasn't a spring managed bean.
Spring AOP works only with Spring managed beans.

Spring transactioninterceptor is invoked when no class or method annotated with transactional

I am working on a gigaspace xap application which uses spring under the hood. The jini transaction manager provided by gigaspaces does not support serializable.
I have a class which uses spring-batch to process a file. Below is how it is invoking the job
public class FileProcessor implements BasicFileProcessor {
#Value("${feeddownload.basedir}")
private String baseDir;
#Autowired
private JobLauncher jobLauncher;
#Autowired
private Job cmJob;
#Autowired
private MapJobRepositoryFactoryBean repositoryFactoryBean;
#Override
public void process(RiskRunCompletion riskRunCompletion, VersionedSliceName versionedSliceName, GigaSpace gigaSpace) {
Transaction tx = gigaSpace.getCurrentTransaction();
try {
//Adding current time to the parameter, to enable multiple times calling job with same parameters
long currentTimeInMillis = System.currentTimeMillis();
JobParameter currentTimeInMillinsParam = new JobParameter(currentTimeInMillis);
Map parameterMap = new LinkedHashMap();
addDirectoryParams(valuationSliceRun, parameterMap);
parameterMap.put(CURRENT_TIME, currentTimeInMillinsParam);
JobParameters paramMap = new JobParameters(parameterMap);
JobExecution cmExecution = launchJobWithParameters(paramMap);
for (Throwable t : cmExecution.getAllFailureExceptions()) {
throw new RuntimeException(t);
}
} catch (Exception e) {
throw new RuntimeException("Exception during batch job", e);
} finally {
repositoryFactoryBean.clear();
}
}
private JobExecution launchJobWithParameters(JobParameters paramMap) throws Exception {
return jobLauncher.run(cmJob, paramMap);
}
}
The process method is invoked from a different class as below
public class FileBasedProcessingEventListener implements ApplicationContextAware {
#Value("${feeddownload.basedir}")
private String baseDir;
#Autowired
private BasicFileProcessor cmProcessor;
#Autowired
private FileBasedProcessingExceptionHandler fileBasedProcessingExceptionHandler;
public void handle(FileBasedProcessingEvent fileBasedProcessingEvent, GigaSpace gigaSpace) throws IOException {
LOGGER.info("Processing file based processing event : " + fileBasedProcessingEvent);
createLockFiles(fileBasedProcessingEvent);
handleEvent(fileBasedProcessingEvent, gigaSpace);
}
private void handleEvent(FileBasedProcessingEvent fileBasedProcessingEvent, GigaSpace gigaSpace) {
Transaction tx = gigaSpace.getCurrentTransaction();
cmProcessor.process(fileBasedProcessingEvent.getRiskRunCompletion(), versionedSliceName, gigaSpace);
}
}
Handle method is called from the framework. Now i am not sure why i am getting the exception as below
Caused by: org.springframework.transaction.InvalidIsolationLevelException: Jini Transaction Manager does not support serializable isolation level
at org.openspaces.core.transaction.manager.AbstractJiniTransactionManager.applyIsolationLevel(AbstractJiniTransactionManager.java:271)
at org.openspaces.core.transaction.manager.AbstractJiniTransactionManager.doJiniBegin(AbstractJiniTransactionManager.java:251)
at org.openspaces.core.transaction.manager.AbstractJiniTransactionManager.doBegin(AbstractJiniTransactionManager.java:207)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:372)
at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:417)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:255)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
None of the classes are marked as transactional, i am not sure why TransactionInterceptor is being invoked when i have not marked any class or any method as transaction it should not be of any concern. I also used Transaction tx = gigaSpace.getCurrentTransaction(); to check that transaction is not active it comes as null only
i am confused when none of the classes are marked as transactional why is spring trying to invoke this method under transaction
Looks like Gigaspaces transaction manager is based upon Spring transaction management infrastructure as can be inferred from the documentation here - so if you are using Gigaspaces transaction API then you are using Spring transaction management. Also worth looking is any transaction manager configuration inside xml configuration files which would point the exact transaction manager class.
Its indeed seems that transactions are in use, check if you define a transaction manger in your pu.xml, in particular check jobLauncher initialization.

Accessing Beans outside of the Step Scope in Spring Batch

Is it possible to access beans defined outside of the step scope? For example, if I define a strategy "strategyA" and pass it in the job parameters I would like the #Value to resolve to the strategyA bean. Is this possible? I am currently working round the problem by getting the bean manually from the applicationContext.
#Bean
#StepScope
public Tasklet myTasklet(
#Value("#{jobParameters['strategy']}") MyCustomClass myCustomStrategy)
MyTasklet myTasklet= new yTasklet();
myTasklet.setStrategy(myCustomStrategy);
return myTasklet;
}
I would like to have the ability to add more strategies without having to modify the code.
The sort answer is yes. This is more general spring/design pattern issue rater then Spring Batch.
The Spring Batch tricky parts are the configuration and understanding scope of bean creation.
Let’s assume all your Strategies implement Strategy interface that looks like:
interface Strategy {
int execute(int a, int b);
};
Every strategy should implements Strategy and use #Component annotation to allow automatic discovery of new Strategy. Make sure all new strategy will placed under the correct package so component scan will find them.
For example:
#Component
public class StrategyA implements Strategy {
#Override
public int execute(int a, int b) {
return a+b;
}
}
The above are singletons and will be created on the application context initialization.
This stage is too early to use #Value("#{jobParameters['strategy']}") as JobParameter wasn't created yet.
So I suggest a locator bean that will be used later when myTasklet is created (Step Scope).
StrategyLocator class:
public class StrategyLocator {
private Map<String, ? extends Strategy> strategyMap;
public Strategy lookup(String strategy) {
return strategyMap.get(strategy);
}
public void setStrategyMap(Map<String, ? extends Strategy> strategyMap) {
this.strategyMap = strategyMap;
}
}
Configuration will look like:
#Bean
#StepScope
public MyTaskelt myTasklet () {
MyTaskelt myTasklet = new MyTaskelt();
//set the strategyLocator
myTasklet.setStrategyLocator(strategyLocator());
return myTasklet;
}
#Bean
protected StrategyLocator strategyLocator(){
return = new StrategyLocator();
}
To initialize StrategyLocator we need to make sure all strategy were already created. So the best approach would be to use ApplicationListener on ContextRefreshedEvent event (warning in this example strategy names start with lower case letter, changing this is easy...).
#Component
public class PlugableStrategyMapper implements ApplicationListener<ContextRefreshedEvent> {
#Autowired
private StrategyLocator strategyLocator;
#Override
public void onApplicationEvent(ContextRefreshedEvent contextRefreshedEvent) {
ApplicationContext applicationContext = contextRefreshedEvent.getApplicationContext();
Map<String, Strategy> beansOfTypeStrategy = applicationContext.getBeansOfType(Strategy.class);
strategyLocator.setStrategyMap(beansOfTypeStrategy);
}
}
The tasklet will hold a field of type String that will be injected with Strategy enum String using #Value and will be resolved using the locator using a "before step" Listener.
public class MyTaskelt implements Tasklet,StepExecutionListener {
#Value("#{jobParameters['strategy']}")
private String strategyName;
private Strategy strategy;
private StrategyLocator strategyLocator;
#BeforeStep
public void beforeStep(StepExecution stepExecution) {
strategy = strategyLocator.lookup(strategyName);
}
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
int executeStrategyResult = strategy.execute(1, 2);
}
public void setStrategyLocator(StrategyLocator strategyLocator) {
this.strategyLocator = strategyLocator;
}
}
To attach the listener to the taskelt you need to set it in your step configuration:
#Bean
protected Step myTaskletstep() throws MalformedURLException {
return steps.get("myTaskletstep")
.transactionManager(transactionManager())
.tasklet(deleteFileTaskelt())
.listener(deleteFileTaskelt())
.build();
}
jobParameters is holding just a String object and not the real object (and I think is not a good pratice store a bean definition into parameters).
I'll move in this way:
#Bean
#StepScope
class MyStategyHolder {
private MyCustomClass myStrategy;
// Add get/set
#BeforeJob
void beforeJob(JobExecution jobExecution) {
myStrategy = (Bind the right strategy using job parameter value);
}
}
and register MyStategyHolder as listener.
In your tasklet use #Value("#{MyStategyHolder.myStrategy}") or access MyStategyHolder instance and perform a getMyStrategy().

Spring Junit test issue

I am creating Junit test cases in Spring.
private static BeanFactory servicefactory = null;
private static BookingProcessService bookingProcessService;
#BeforeClass
public static void setUpBeforeClass() throws Exception{
try{
String[] configFiles = {"applicationcontext-Service.xml","applicationcontext-Hibernate.xml","applicationContext-dao.xml"};
ClassPathXmlApplicationContext appContext = new ClassPathXmlApplicationContext(configFiles);
servicefactory = (BeanFactory) appContext;
bookingProcessService = (BookingProcessService) servicefactory.getBean("bookingProcessService");
}catch (Exception e){
e.printStackTrace();
}
}
Here BookingProcessService is an interface and its implementation class is BookingProcessServiceImpl.java.
In spring configuration files, there is no bean id defined for that.
Is there any way I can use the 'bookingProcessService' for invoking the actual method definition written in BookingProcessImp.java in my test methods?
If u are configuring this test I recommend to use Annotations, will be more clear.
Usually in your "applicationcontext-Service.xml" must have the bean defined with the Implementation, and is the classes where you would use the Interface of this class.
With that you would not coupling code with implementation.
In this example, simple must do the next:
#Autowired
private BookingProcessService bookingProcessService;
The annotation #Autowired from Spring will do the IOC from your xml configuration "applicationcontext-Service.xml".

Storing in JobExecutionContext from tasklet and accessing in another tasklet

I have a requirement in which a tasklet, stores all the files in the directories in an arraylist. The size of the list is stored in the job execution context. Later this count is accessed from another tasklet in another step. How do it do this. I tried to store in jobexecution context, at runtime throws unmodifiable collection exception,
public RepeatStatus execute(StepContribution arg0, ChunkContext arg1)
throws Exception {
StepContext stepContext = arg1.getStepContext();
StepExecution stepExecution = stepContext.getStepExecution();
JobExecution jobExecution = stepExecution.getJobExecution();
ExecutionContext jobContext = jobExecution.getExecutionContext();
jobContext.put("FILE_COUNT",150000);
also stored the stepexection reference in beforestep annotation .still not possioble.kindly let me know ,how to share data between two tasklets.
you have at least 4 possibilities:
use the ExecutionPromotionListener to pass data to future steps
use a (spring) bean to hold inter-step data, e.g. a ConcurrentHashMap
without further action this data won't be accessible for a re-start
access the JobExecutionContext in your tasklet, should be used with caution, will cause thread problems for parallel steps
use the new jobscope (introduced with spring batch 3)
Code Example for accessing JobExecution from Tasklet:
setting a value
public class ChangingJobExecutionContextTasklet implements Tasklet {
/** {#inheritDoc} */
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
// set variable in JobExecutionContext
chunkContext
.getStepContext()
.getStepExecution()
.getJobExecution()
.getExecutionContext()
.put("value", "foo");
// exit the step
return RepeatStatus.FINISHED;
}
}
extracting a value
public class ReadingJobExecutionContextTasklet implements Tasklet {
private static final Logger LOG = LoggerFactory.getLogger(ChangingJobExecutionContextTasklet.class);
/** {#inheritDoc} */
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
// pull variable from JobExecutionContext
String value = (String) chunkContext
.getStepContext()
.getStepExecution()
.getJobExecution()
.getExecutionContext()
.get("value");
LOG.debug("Found value in JobExecutionContext:" + value);
// exit the step
return RepeatStatus.FINISHED;
}
}
i created code examples for the first 3 solutions in my spring-batch-examples github repository, see module complex and package interstepcommunication
Another way is to use StepExecutionListener which is called after step execution.
Your tasklet can implements it and share local attribute.
public class ReadingJobExecutionContextTasklet implements Tasklet, StepExecutionListener {
private String value;
#Override
public ExitStatus afterStep(StepExecution stepExecution) {
ExecutionContext jobExecutionContext = stepExecution.getJobExecution().getExecutionContext();
jobExecutionContext.put("key", value);
//Return null to leave the old value unchanged.
return null;
}
}
So, in the step, your bean is a tasklet and a listener like bellow.
You should also configure the scope of you step to "step" :
<batch:step id="myStep" next="importFileStep">
<batch:tasklet>
<ref bean="myTasklet"/>
<batch:listeners>
<batch:listener ref="myTasklet"/>
</batch:listeners>
</batch:tasklet>
</batch:step>
<bean id="myTasklet" class="ReadingJobExecutionContextTasklet" scope="step">

Resources