Spring Batch MultiReader - spring

I am trying to for multi-read using spring batch approach in which i have to get result from multiple queries and need to do some processing withe result and write them into file. I tried for a sample code but its giving me a error. I believe i have almost there but something i am missing due to my limited knowledge in spring batch
Below is my Code
MultipleReader-batch-context.xml
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:batch="http://www.springframework.org/schema/batch" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-3.0.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd">
<import resource="classpath:context-datasource.xml" />
<!-- JobRepository and JobLauncher are configuration/setup classes -->
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" />
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<bean id= "result1" class="org.springframework.batch.item.database.JdbcCursorItemReader" >
<property name="dataSource" ref="dataSource" />
<property name="sql" value="SELECT * from SNG_CORE.LOOK_UP" />
<property name="rowMapper"> <bean class="com.websystique.springbatch.ExamResultRowMapper" />
</property>
</bean>
<bean id ="result2" class="org.springframework.batch.item.database.JdbcCursorItemReader" >
<property name="dataSource" ref="dataSource" />
<property name="sql" value= "SELECT * from SNG_CORE.LOOK_UP_2"/>
<property name="rowMapper"> <bean class="com.websystique.springbatch.ExamResultRowMapper" />
</property>
</bean>
<bean id="customItemReader" class="com.websystique.springbatch.MyCustomReader" scope="step">
<property name="result1" ref="result1" />
<property name="result2" ref="result2" />
</bean>
<job id="MultipleReaderJob" xmlns="http://www.springframework.org/schema/batch">
<step id="step1">
<tasklet>
<chunk reader="customItemReader" writer="flatFileItemWriter"
commit-interval="1" />
</tasklet>
</step>
<property name="resource" value="file:csv/examResult.txt" />
<property name="lineAggregator">
<!-- An Aggregator which converts an object into delimited list of strings -->
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter" value="|" />
<property name="fieldExtractor">
<!-- Extractor which returns the value of beans property through reflection -->
<bean
class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<property name="names" value="studentName, studentName1, studentName2" />
</bean>
</property>
</bean>
</property>
</bean>
====
ExamResultRowMapper
public class ExamResultRowMapper implements RowMapper<ExamResult>{
#Override
public ExamResult mapRow(ResultSet rs, int rowNum) throws SQLException {
ExamResult result = new ExamResult();
result.setStudentName(rs.getString("I_LKUP"));
result.setStudentName1(rs.getString("C_LKUP_TYP"));
result.setStudentName2(rs.getString("C_LKUP_KEY"));
return result;
}
==================
Custom Reader
public class MyCustomReader implements ItemReader<ExamResult> {
ItemReader<ExamResult> result1;
ItemReader<ExamResult> result2;
#Override
public ExamResult read()
throws Exception, UnexpectedJobExecutionException, ParseException, NonTransientResourceException {
ExamResult application1 = result1.read();
ExamResult application2 = result2.read();
/***
* I want to perform some operation with two result set
* and passs the final data to writter
* /
ExamResult summary = new ExamResult();
return summary;
}
}
===Exam result===
public class ExamResult {
private String studentName;
private String studentName1;
private String studentName2;
public String getStudentName() {
return studentName;
}
public void setStudentName(String studentName) {
this.studentName = studentName;
}
public String getStudentName1() {
return studentName1;
}
public void setStudentName1(String studentName1) {
this.studentName1 = studentName1;
}
public String getStudentName2() {
return studentName2;
}
public void setStudentName2(String studentName2) {
this.studentName2 = studentName2;
}
#Override
public String toString() {
return " [studentName=" + studentName + ", dob=" + studentName1 + ", Std=" + studentName2 + "]";
}
}
====================================================================
When ever i execute this i am getting
Encountered an error executing step step1 in job MultipleReaderJob
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.customItemReader' defined in class path resource [MultipleReader-batch-context.xml]: Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'result1' of bean class [com.websystique.springbatch.MyCustomReader]: Bean property 'result1' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
Note : All works good if i run my job to read with 1 result set .
<job id="MultipleReaderJob" xmlns="http://www.springframework.org/schema/batch">
<step id="step1">
<tasklet>
<chunk reader="result1" writer="flatFileItemWriter"
commit-interval="1" />
</tasklet>
</step>

Related

Spring Batch partition doesnt work composite itemprocessor

I have a Spring Batch partition job. I’m using CompositeProcessor, read data from DB and save these items into an CopyOnWriteArrayList. Because the environment is concurrent but my CopyOnWriteArrayList is being utilized for other threads and mix information, I don’t know why and what I am doing bad, and the output writing them into files for each thread.
public class CustomerItemProcessor implements ItemProcessor<beangenerico,CopyOnWriteArrayList<beanCustomer>> {
private CustomerDAO customerDAO;
private CopyOnWriteArrayList<beanCustomer> listbean;
public CopyOnWriteArrayList<beanCustomer> process(beangenerico rangos) throws Exception {
listbean = customerDAO.getAccAgentes(rangos);
if(listbean != null) {
//customer.setId(currentCustomer.getId());
return listbean;
}else{
return null;
}
}
The configuration of my batch im XML:
<batch:job id="partitionJob" xmlns="http://www.springframework.org/schema/batch">
<batch:step id="masterStep">
<batch:partition step="slave" partitioner="rangePartitioner">
<batch:handler grid-size="10" task-executor="taskExecutor"/>
</batch:partition>
</batch:step>
</batch:job>
<!-- each thread will run this job, with different stepExecutionContext values. -->
<batch:step id="slave" xmlns="http://www.springframework.org/schema/batch">
<batch:tasklet task-executor="taskExecutor" throttle-limit="1">
<batch:chunk reader="beaniniendreader" writer="tempRecordsWriter" processor="completeItemProcessor" commit-interval="1" />
</batch:tasklet>
</batch:step>
<bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor" />
<bean id="rangePartitioner" class="my.package.springbatch.RangePartitioner" />
<bean id="beaniniendreader" class="my.package.springbatch.FormiikReader" scope="step"></bean>
<bean id="beanprocessor" class="my.package.springbatch.FormiikProcessor" scope="step">
<property name="accountExecutiveDao" ref="accountExecutiveDao"/>
</bean>
<bean id="beanprocessor2" class="my.package.springbatch.CustomerItemProcessor" scope="step">
<property name="customerDAO" ref="customerAccDao"/>
</bean>
<bean id="completeItemProcessor" class="org.springframework.batch.item.support.CompositeItemProcessor">
<property name="delegates">
<list>
<ref bean="beanprocessor2"/>
<ref bean="accItemprocessor"/>
<ref bean="beanaccDataItem"/>
</list>
</property>
</bean>
<bean id="tempRecordsWriter" class="my.package.springbatch.ListDelegateWriter" scope="step">
<property name="delegate" ref="flatFileItemWriterPartition"/>
</bean>
<!-- csv file writer -->
<bean id="flatFileItemWriterPartition" class="org.springframework.batch.item.file.FlatFileItemWriter"
scope="step" >
<property name="resource"
value="file:csv/outputs/users.processed#{stepExecutionContext[fromId]}-#{stepExecutionContext[toId]}.csv" />
<property name="appendAllowed" value="false" />
<property name="lineAggregator">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter" value="," />
<property name="fieldExtractor">
<bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<property name="names" value="cuenta, name, purchasedPackage" />
</bean>
</property>
</bean>
</property>
</bean>
I going back to take the subject of my code, it is advised to me using Threadlocal for storing thread-specific data whereby it´s works . Here I put my code again. Thanks for your replies.
public class CustomerItemProcessor implements ItemProcessor<beangenerico,ThreadLocal<CopyOnWriteArrayList<beanCustomer>>> {
private CustomerDAO customerDAO;
private ThreadLocal<CopyOnWriteArrayList<beanCustomer>> listbean = new ThreadLocal<CopyOnWriteArrayList<beanCustomer>>();
public ThreadLocal<CopyOnWriteArrayList<beanCustomer>> process(beangenerico rangos) throws Exception {
listbean.set(new CopyOnWriteArrayList<beanCustomer>());
listbean = customerDAO.getAccAgentes(rangos);
if(listbean != null) {
return listbean;
} else {
return null;
}
}
public void setCustomerDAO(CustomerDAO customerDAO) {
this.customerDAO = customerDAO;
}
}

Unable to use Repository class of spring data jpa on a class method in Quartz why?

I'm developing multiple-jobs-in-quartz-spring-example which is combination of Spring + Quartz + Spring Data JPA. I'm looking to developed a code which will run in 5 sec, it will hit to DB and fetches a records from DB.
I am close to making it working but I see small problem. In my JobA.class, why don't I get an instance of CustomerRepository.java? It's always null and thats why code unable to hit to DB.
Source code at: http://www.github.com/test512/multiple-jobs-in-quartz-spring-example.gi‌​t.
JobA.java
#Service
public class JobA extends QuartzJobBean {
private CustomerRepository customerRepository = null;
#Autowired
public JobA(CustomerRepository customerRepository) {
this.customerRepository = customerRepository;
}
public JobA() { }
#Override
protected void executeInternal(JobExecutionContext executionContext) throws JobExecutionException {
System.out.println("~~~~~~~ Job A is runing ~~~~~~~~");
Trigger trigger = executionContext.getTrigger();
System.out.println(trigger.getPreviousFireTime());
System.out.println(trigger.getNextFireTime());
getCustomerList();
}
private List<Customer> getCustomerList(){
List<Customer> customers = customerRepository.findAll();
for (Customer customer : customers) {
System.out.println("------------------------------");
System.out.println("ID : "+customer.getId());
System.out.println("NAME : "+customer.getName());
System.out.println("STATUS : "+customer.getStatus());
}
return customers;
}
}
Customer.java
#Entity
#Table(name="customer")
public class Customer {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name="ID")
private Long id;
#Column(name="NAME")
private String name;
#Column(name="STATUS")
private String status;
public long getId() {
return id;
}
public void setId(long id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getStatus() {
return status;
}
public void setStatus(String status) {
this.status = status;
}
}
CustomerRepository.java
public interface CustomerRepository extends JpaRepository<Customer, Long>{
}
dataSourceContext.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jdbc="http://www.springframework.org/schema/jdbc"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd
http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc.xsd">
<context:property-placeholder location="classpath:database.properties"/>
<!-- <jdbc:initialize-database data-source="dataSource" enabled="true">
<jdbc:script location="classpath:db-schema.sql" />
<jdbc:script location="classpath:db-test-data.sql" />
</jdbc:initialize-database> -->
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="${mysql.driver.class.name}" />
<property name="url" value="${mysql.url}" />
<property name="username" value="${mysql.username}" />
<property name="password" value="${mysql.username}" />
</bean>
<bean id="jpaVendorAdapter" class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="showSql" value="true"/>
<property name="generateDdl" value="true"/>
<property name="database" value="MYSQL"/>
</bean>
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource"/>
<property name="jpaVendorAdapter" ref="jpaVendorAdapter"/>
<!-- spring based scanning for entity classes-->
<property name="packagesToScan" value="com.mkyong.*"/>
</bean>
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager"/>
</beans>
Spring-Quartz.xml
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jpa="http://www.springframework.org/schema/data/jpa"
xmlns:repository="http://www.springframework.org/schema/data/repository"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/data/jpa http://www.springframework.org/schema/data/jpa/spring-jpa.xsd
http://www.springframework.org/schema/data/repository http://www.springframework.org/schema/data/repository/spring-repository.xsd">
<import resource="classpath:dataSourceContext.xml"/>
<jpa:repositories base-package="com.mkyong.repository" />
<context:component-scan base-package="com.mkyong.*" />
<context:annotation-config />
<bean id="jobA" class="com.mkyong.job.JobA" />
<bean id="jobB" class="com.mkyong.job.JobB" />
<bean id="jobC" class="com.mkyong.job.JobC" />
<bean id="autowiredA" class="com.mkyong.job.JobASpringBeanJobFactory" />
<!-- ~~~~~~~~~ Quartz Job ~~~~~~~~~~ -->
<bean name="JobA" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass" value="com.mkyong.job.JobA" />
</bean>
<bean name="JobB" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass" value="com.mkyong.job.JobB" />
</bean>
<bean name="JobC" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass" value="com.mkyong.job.JobC" />
</bean>
<!-- ~~~~~~~~~~~ Cron Trigger, run every 5 seconds ~~~~~~~~~~~~~ -->
<bean id="cronTriggerJobA" class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="JobA" />
<property name="cronExpression" value="0/5 * * * * ?" />
</bean>
<bean id="cronTriggerJobB" class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="JobB" />
<property name="cronExpression" value="0/5 * * * * ?" />
</bean>
<bean id="cronTriggerJobC" class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="JobC" />
<property name="cronExpression" value="0/5 * * * * ?" />
</bean>
<!-- ~~~~~~~~~~~~~~~~ Scheduler bean Factory ~~~~~~~~~~~~~~~~ -->
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="jobFactory" ref="autowiredA"/>
<property name="triggers">
<list>
<ref bean="cronTriggerJobA" />
<!-- <ref bean="cronTriggerJobB" />
<ref bean="cronTriggerJobC" /> -->
</list>
</property>
</bean>
</beans>
JobASpringBeanJobFactory.java
public class JobASpringBeanJobFactory extends SpringBeanJobFactory implements ApplicationContextAware {
private transient AutowireCapableBeanFactory beanFactory;
#Override
public void setApplicationContext(final ApplicationContext context) {
beanFactory = context.getAutowireCapableBeanFactory();
}
#Override
protected Object createJobInstance(final TriggerFiredBundle bundle) throws Exception {
final Object job = super.createJobInstance(bundle);
beanFactory.autowireBean(job);
return job;
}
}
App.java
public class App {
public static void main(String[] args) throws Exception {
ApplicationContext context = new ClassPathXmlApplicationContext("Spring-Quartz.xml");
}
}
Going through the following link http://codrspace.com/Khovansa/spring-quartz-with-a-database/ should help.
Quoting from the above link,
Quartz creates a new job instance on each invocation. It means that Quartz jobs are not regular Spring beans and we can't expect the Spring magic to take an effect automatically (and Spring's 'JobDetailFactoryBean' is not smart enough to do it for us). So we'll have to implement our own job factory that would overwrite the default SpringBeanJobFactory.
So you need to have a custom SpringBeanJobFactory by extending SpringBeanJobFactory & implementing ApplicationContextAware and finally invoke beanFactory.autowireBean(job)

Spring #ExceptionHandler not working

I am trying to handle an exception using #ExceptionHandler but it is not working and I don't know why. The thing is right now I am getting this response from my web service: {"message":"The date provided 2013-02-30 is invalid","code":500,"ids":null}. And I want that to be a 400 exception instead of 500.
Here is my code:
#Controller
public class WsController {
#RequestMapping(value={"/getDeletedUsers"}, method=RequestMethod.GET)
public List<Integer> getDeletedUsers(#RequestParam(value = "date", required = true) String dateStr) throws WebServiceException {
if (dateStr == null) {
throw new WebServiceException("The date provided is null");
} else if (StringUtils.isEmpty(dateStr)) {
throw new WebServiceException("The date provided is empty");
} else {
SimpleDateFormat sdf = new SimpleDateFormat();
sdf.applyPattern("yyyy-MM-dd");
sdf.setLenient(false);
try {
sdf.parse(dateStr);
} catch (ParseException e) {
throw new WebServiceException("The date provided " + dateStr + " is invalid");
}
return service.getDeletedUsers(dateStr);
}
}
#ExceptionHandler(WebServiceException.class)
public void handleWebServiceException() {
System.out.println("PLEASE DO SOMETHING!");
}
}
public class WebServiceException extends Exception {
//Constructors and serialVersionUID
}
disparcher-servlet.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd">
<bean class="org.springframework.web.context.support.ServletContextPropertyPlaceholderConfigurer">
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/>
<property name="searchContextAttributes" value="true"/>
<property name="contextOverride" value="true"/>
<property name="ignoreResourceNotFound" value="true"/>
<property name="locations">
<list>
<value>classpath:application.properties</value>
<value>file:c://rt//properties//webservices-application.properties</value>
</list>
</property>
</bean>
<import resource="classpath:keepalive.xml" />
<import resource="classpath:controller.xml" />
<bean id="exceptionResolver" class="com.company.project.webservices.spring.ExceptionResolver"/>
<bean id="error" class="com.company.project.webservices.spring.ErrorView"/>
<bean id="readOnlyModeError" class="com.company.project.webservices.spring.ReadOnlyModeErrorView"/>
<bean id="methodUnavailableError" class="com.company.project.webservices.spring.MethodUnavailableErrorView"/>
<bean id="badRequestError" class="com.company.project.webservices.spring.BadRequestErrorView"/>
<!-- View Resolver -->
<bean id="viewResolver" class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
<property name="order" value="1" />
<property name="mediaTypes">
<map>
<entry key="json" value="application/json"/>
</map>
</property>
<!-- Ticket-3245 Spring 60 -->
<property name="defaultContentType" value="application/json" />
<property name="useNotAcceptableStatusCode" value="true"/>
<property name="viewResolvers">
<list>
<bean class="org.springframework.web.servlet.view.BeanNameViewResolver">
<property name="order" value="2" />
</bean>
</list>
</property>
<property name="defaultViews">
<list>
<bean class="org.springframework.web.servlet.view.json.MappingJacksonJsonView" />
</list>
</property>
</bean>
<bean id="messageSource" class="org.springframework.context.support.ResourceBundleMessageSource">
<property name="basename" value="messages"/>
</bean>
<!-- Dispatches requests mapped to POJO #Controllers implementations -->
<bean class="org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter">
<property name="messageConverters">
<util:list id="beanList">
<ref bean="jsonHttpMessageConverter"/>
</util:list>
</property>
</bean>
<bean id="jsonHttpMessageConverter"
class="org.springframework.http.converter.json.MappingJacksonHttpMessageConverter">
</bean>
<bean class="org.springframework.web.servlet.mvc.annotation.DefaultAnnotationHandlerMapping">
<property name="useDefaultSuffixPattern" value="false"/>
</bean>
<!-- Dispatches requests mapped to non-annotated controllers -->
<bean class="org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter" />
<context:annotation-config/>
<context:component-scan base-package="com.company.project" />
When I try to debug the handleWebServiceException method it doesn't even stop there.
Any tips will be appreciated.
As jny suggested, I took a look at the dispatcher-servlet.xml and realized that there was a exceptionResolver bean that I haven't seen that looks like this:
public class ExceptionResolver extends AbstractHandlerExceptionResolver {
private static Log logger = LogFactory.getLog(ExceptionResolver.class);
#Override
protected ModelAndView doResolveException(HttpServletRequest request,
HttpServletResponse response, Object handler, Exception ex) {
ModelAndView mv = null;
if (ex instanceof ReadOnlyModeException) {
mv = new ModelAndView("readOnlyModeError");
} else if (ex instanceof MethodUnavailableException) {
mv = new ModelAndView("methodUnavailableError");
} else if (ex instanceof BadRequestException) {
mv = new ModelAndView("badRequestError");
} else {
logger.error("caught an exception: ", ex);
mv = new ModelAndView("error");
}
ResultBean resultBean = new ResultBean();
resultBean.setCode(500);
resultBean.setMessage(ex.getMessage());
mv.addObject("result", resultBean);
return mv;
}
}
And there I was able to make the suitable changes.
Thanks again to jny for the hint.

How to create a SocketConfig class in Spring using factory-method?

I am using apache commons httpclient 4.3.x along with spring3. I am trying to wire up a connectionpool with it's associated socketconfig instance.
http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html
http://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/config/SocketConfig.html?is-external=true
My code looks like this:
<bean id="socketConfig" class="org.apache.http.config.SocketConfig" factory-method="custom" init-method="build">
<property name="soTimeout" value="60000"/>
<property name="soLinger" value="5" />
</bean>
<bean name="poolingHttpConnectionManager" class="org.apache.http.impl.conn.PoolingHttpClientConnectionManager" depends-on="socketConfig">
<property name="maxTotal" value="20" />
<property name="defaultMaxPerRoute" value="20" />
<property name="defaultSocketConfig" ref="socketConfig" />
</bean>
However, this is not working. The instance type that is used to setDefaultSocketConfig() on PoolingHttpClientConnectionManager is of type SocketConfig.Builder instead of SocketConfig.
What I want to have happen is as follows:
SocketConfig config = SocketConfig.custom()
.setSoTimeout(60000)
.setSoLinger(5)
.build()
So, I expect that the socketConfig bean type should be a SocketConfig instance, not a SocketConfig.Builder instance.
As per spring docs, I thought this should work.
http://docs.spring.io/spring/docs/3.0.x/spring-framework-reference/htmlsingle/spring-framework-reference.html#beans-factory-class-static-factory-method
is there anything I am doing wrong? Or is this just not supported in spring?
It turns out that the socketconfig builder instance is not designed to work with spring very well.
I had to use a spring beanfactory implementation to create the instance.
The bean class:
import org.apache.http.config.SocketConfig;
import org.springframework.beans.factory.FactoryBean;
public class SocketConfigFactoryBean implements FactoryBean<SocketConfig> {
int soLinger;
int soTimeout;
public SocketConfig getObject() throws Exception {
return SocketConfig.custom()
.setSoLinger(soLinger)
.setSoTimeout(soTimeout)
.build();
}
public Class<?> getObjectType() {
return SocketConfig.class;
}
public boolean isSingleton() {
return true;
}
public int getSoLinger() {
return soLinger;
}
public void setSoLinger(int soLinger) {
this.soLinger = soLinger;
}
public int getSoTimeout() {
return soTimeout;
}
public void setSoTimeout(int soTimeout) {
this.soTimeout = soTimeout;
}
}
The bean definition
<bean name="poolingHttpConnectionManager" class="org.apache.http.impl.conn.PoolingHttpClientConnectionManager">
<property name="maxTotal" value="20" />
<property name="defaultMaxPerRoute" value="20" />
<property name="defaultSocketConfig">
<bean class="org.apache.http.config.SocketConfig" factory-method="custom" init-method="build">
<bean class="com.ex.spring.beans.factory.SocketConfigFactoryBean">
<property name="soTimeout" value="60000"/>
<property name="soLinger" value="5" />
</bean>
</property>
</bean>
I was able to achieve it by doing the next configuration in spring:
<bean id="socketConfig" class="org.apache.http.config.SocketConfig" factory-method="custom">
<property name="soTimeout" value="1000" />
<property name="soLinger" value="5" />
</bean>
<bean name="poolingHttpConnectionManager" class="org.apache.http.impl.conn.PoolingHttpClientConnectionManager">
<property name="maxTotal" value="20" />
<property name="defaultMaxPerRoute" value="20" />
<property name="defaultSocketConfig">
<bean factory-bean="socketConfig" factory-method="build" />
</property>
</bean>

write method in spring batch not called

I have a problem whereby the write method is not called.
Spring Context
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:batch="http://www.springframework.org/schema/batch" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/batch
http://www.springframework.org/schema/batch/spring-batch-2.2.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
">
<!-- spring batch context -->
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<property name="transactionManager" ref="transactionManager" />
</bean>
<bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<!-- Must set this -->
<bean
class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
<property name="jobRegistry" ref="jobRegistry" />
</bean>
<bean id="jobRegistry"
class="org.springframework.batch.core.configuration.support.MapJobRegistry" />
<!-- spring batch context -->
<bean id="report" class="core.Report" scope="prototype" />
<bean id="customWriter" class="spring.jobs.CustomWriter" />
<batch:job id="reportJob">
<batch:step id="step1">
<batch:tasklet>
<batch:chunk reader="cvsFileItemReader" writer="customWriter"
commit-interval="10">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<bean id="cvsFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
<!-- Read a csv file -->
<property name="resource" value="file:input/report.csv" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="names" value="id,impressions" />
</bean>
</property>
<property name="fieldSetMapper">
<bean
class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
<property name="prototypeBeanName" value="report" />
</bean>
</property>
</bean>
</property>
</bean>
<!-- run every 5 seconds -->
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<bean id="cronTrigger"
class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="jobDetail" />
<property name="cronExpression" value="*/5 * * * * ?" />
</bean>
</property>
</bean>
<bean id="jobDetail"
class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass" value="quartz.JobLauncherDetails" />
<property name="group" value="quartz-batch" />
<property name="durability" value="true" />
<property name="jobDataAsMap">
<map>
<entry key="jobName" value="reportJob" />
<entry key="jobLocator" value-ref="jobRegistry" />
<entry key="jobLauncher" value-ref="jobLauncher" />
<entry key="param1" value="j1" />
<entry key="param2" value="j2" />
</map>
</property>
</bean>
</beans>
CustomWriter.java
import java.util.List;
import org.springframework.batch.item.ItemWriter;
import core.Report;
public class CustomWriter implements ItemWriter<Report> {
#Override
public void write(List<? extends Report> items) throws Exception{
System.out.println("writer..." + items.size());
for(Report item : items){
System.out.println(item);
}
}
}
quartz.JobLauncherDetails class
package quartz;
import java.util.Date;
import java.util.Map;
import java.util.Map.Entry;
import org.quartz.JobExecutionContext;
import org.springframework.batch.core.JobExecutionException;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.configuration.JobLocator;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.scheduling.quartz.QuartzJobBean;
public class JobLauncherDetails extends QuartzJobBean {
static final String JOB_NAME = "jobName";
private JobLocator jobLocator;
private JobLauncher jobLauncher;
public JobLauncherDetails() {
System.out.println("JobLauncherDetails initialize...");
}
public void setJobLocator(JobLocator jobLocator) {
this.jobLocator = jobLocator;
}
public void setJobLauncher(JobLauncher jobLauncher) {
this.jobLauncher = jobLauncher;
}
#SuppressWarnings("unchecked")
protected void executeInternal(JobExecutionContext context) {
System.out.println("executeInternal...");
Map<String, Object> jobDataMap = context.getMergedJobDataMap();
String jobName = (String) jobDataMap.get(JOB_NAME);
JobParameters jobParameters = getJobParametersFromJobMap(jobDataMap);
try {
jobLauncher.run(jobLocator.getJob(jobName), jobParameters);
} catch (JobExecutionException e) {
e.printStackTrace();
}
}
// get params from jobDataAsMap property, job-quartz.xml
private JobParameters getJobParametersFromJobMap(
Map<String, Object> jobDataMap) {
JobParametersBuilder builder = new JobParametersBuilder();
for (Entry<String, Object> entry : jobDataMap.entrySet()) {
String key = entry.getKey();
Object value = entry.getValue();
if (value instanceof String && !key.equals(JOB_NAME)) {
builder.addString(key, (String) value);
} else if (value instanceof Float || value instanceof Double) {
builder.addDouble(key, ((Number) value).doubleValue());
} else if (value instanceof Integer || value instanceof Long) {
builder.addLong(key, ((Number) value).longValue());
} else if (value instanceof Date) {
builder.addDate(key, (Date) value);
} else {
// JobDataMap contains values which are not job parameters
// (ignoring)
}
}
// need unique job parameter to rerun the same job
builder.addDate("run date", new Date());
return builder.toJobParameters();
}
}
In this case, the writer.... in write method is not printed.. Couldn't find the problem up till now..
I tried your config with latest version of batch (2.2.1.RELEASE), spring (3.2.3.RELEASE) and quartz (2.2.0) and all works fine substituting file:input/report.csv with file:${user.dir}/src/main/java/input/report.csv.
Are you sure file exists and doesn't throw a
IllegalStateException: Input resource must exist (reader is in
'strict' mode): URL [file:input/report.csv]
You can try use classpath:input/report.csv or build the real path of file (or check if your file is not empty...)

Resources