pass job parameters to custom writer Spring batch - spring

I have a custom writer with a FlatFileItemWriter and i want to pass a job parameter( a output file) defined in the main class
How can i deal with this ?
Thank you very much
CustomWriter
public class PersonItemWriter implements ItemWriter<Person> {
private FlatFileItemWriter<String> flatFileItemWriter = new FlatFileItemWriter<String>();
private Resource resource;
#Override
public void write(List<? extends Person> personList) throws Exception {
flatFileItemWriter.setResource(new FileSystemResource(resource.getFile()));
PassThroughLineAggregator<String> aggregator = new PassThroughLineAggregator<String();
flatFileItemWriter.setLineAggregator(aggregator);
flatFileItemWriter.open(new ExecutionContext());
flatFileItemWriter.write(Arrays.asList(aggregator.aggregate("test")));
flatFileItemWriter.close();
}
public void setResource(Resource resource) {
this.resource = resource;
}
}
Launcher
JobLauncher jobLauncher = (JobLauncher) applicationContext.getBean("jobLauncher");
Job job = (Job) applicationContext.getBean("personJob");
/* Parameters sent to job */
JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();
jobParametersBuilder.addString("outputFileName", "file:" + personFile); // pass this to the itemWriter
configuration job xml
<bean id="personWriter" class="com.dev.writer.PersonItemWriter" scope="step>
<property name="resource" value="#{jobParameters[outputFileName]}" />
</bean>

You have to declare the bean with either step scope or job scope so you can have late binding of a property based on the job parameter:
<bean id="personWriter" class="com.dev.writer.PersonItemWriter" scope="step">
<property name="resource" value="#{jobParameters[outputFileName]}" />
</bean>
These scopes are not available by default, you need to include them either by either using the batch namespace or defining the following bean:
<bean class="org.springframework.batch.core.scope.StepScope" />
Update:
Here's the complete writer:
public class PersonItemWriter implements ItemWriter<Person> {
FlatFileItemWriter<String> flatFileItemWriter = new FlatFileItemWriter<String>();
private Resource resource;
#Override
public void write(List<? extends Person> personList) throws Exception {
flatFileItemWriter.setResource(resource);// how the pass the job parameter file here
PassThroughLineAggregator<String> aggregator = new PassThroughLineAggregator<String();
flatFileItemWriter.setLineAggregator(aggregator);
aggregator.aggregate("test"); // do not save in output file
}
public FlatFileItemWriter<String> getFlatFileItemWriter() {
return flatFileItemWriter;
}
public void setFlatFileItemWriter(FlatFileItemWriter<String> flatFileItemWriter) {
this.flatFileItemWriter = flatFileItemWriter;
}
public void setResource(Resource resource) {
this.resource = resource;
}
}

You can define a HashMap and use this HashMap instead of jobParameter.
<bean id="paramBean" class="java.util.HashMap"/>
<bean id="personWriter" class="com.dev.writer.PersonItemWriter" scope="step">
<property name="resource" value="#{paramBean[outputFileName]}" />
</bean>
Write the setter method in ItemWriter and set the values in the HashMap.
private HashMap paramBean;
public void setParamBean(HashMap paramBean) {
this.paramBean= paramBean;
}
paramBean.set(<key>,<value>);

Related

How to bring Quartz scheduler to standby

I have quartz job implementation using spring. My scheduler works fine and jobs are getting executed perfectly.
My question is how to bring this scheduler to standby? So that no jobs gets triggered once I decide to bring the scheduler to standby mode.
Below is the job class
public class MyJobClass extends QuartzJobBean {
//my job logic
}
Snippet from applicationContext_Scheduler.xml
<bean name="myJobBean"
class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass" value="MyJobClass" />
</bean>
<bean id="rsHourlyJobCronTrigger"
class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="myJobBean" />
<property name="cronExpression" value="00 0/1 * * * ?" />
</bean>
Instead of creating the scheduler bean in xml. I'd make it programatically as follows:
#Configuration
public class QuartzSchedulerConfiguration {
#Autowired
private ApplicationContext applicationContext;
#Bean
public JobFactory jobFactory() {
ApplicationContextHolder jobFactory = new ApplicationContextHolder();
jobFactory.setApplicationContext(applicationContext);
return jobFactory;
}
#Bean
public SchedulerFactoryBean schedulerFactory() {
SchedulerFactoryBean factory = new SchedulerFactoryBean();
factory.setAutoStartup(true);
factory.setSchedulerName("Quartz Scheduler");
factory.setOverwriteExistingJobs(true);
factory.setJobFactory(jobFactory());
return factory;
}
}
#Component
public final class ApplicationContextHolder extends SpringBeanJobFactory implements ApplicationContextAware {
private static ApplicationContext context;
private transient AutowireCapableBeanFactory beanFactory;
#Override
public void setApplicationContext(ApplicationContext ctx) throws BeansException {
beanFactory = ctx.getAutowireCapableBeanFactory();
context = ctx;
}
#Override
protected Object createJobInstance(final TriggerFiredBundle bundle) throws Exception {
final Object job = super.createJobInstance(bundle);
beanFactory.autowireBean(job);
return job;
}
public static ApplicationContext getContext() {
return context;
}
}
This will create your quartz scheduler bean that is application context aware allowing you to autowire spring objects into your quartz jobs.
Then create some Scheduling service like:
#Service
public class SchedulerService {
#Autowired
private SchedulerFactoryBean schedulerFactory;
private Scheduler scheduler;
#PostConstruct
private void init() {
scheduler = schedulerFactory.getScheduler();
}
public void standBy() throws Exception {
if (scheduler != null && !scheduler.isInStandbyMode()) {
scheduler.standby();
}
}
}
Then add other methods that you need for scheduling.

How to stop file transferring in spring-batch

I have created a spring-batch job for reading files from local directory and upload it to remote directory through ftp using Camel-spring-batch. I am doing the same using chunk.
My spring batch job configuration looks like :
<bean id="consumerTemplate" class="org.apache.camel.impl.DefaultConsumerTemplate" init-method="start" destroy-method="stop">
<constructor-arg ref="camelContext"/>
</bean>
<bean id="producerTemplate" class="org.apache.camel.impl.DefaultProducerTemplate" scope="step" init-method="start" destroy-method="stop">
<constructor-arg ref="camelContext"/>
</bean>
<bean id="localFileReader" class="com.camel.springbatch.reader.LocalFileReader" scope="step" destroy-method="stop">
<constructor-arg value="file:#{jobParameters['dirPath']}"/>
<constructor-arg ref="consumerTemplate"/>
</bean>
<bean id="ftpFileWriter" class="com.camel.springbatch.writer.FtpFileWriter" scope="step">
<constructor-arg ref="producerTemplate"/>
<constructor-arg value="ftp://#{jobParameters['host']}?username=#{jobParameters['user']}&password=#{jobParameters['password']}"/>
</bean>
Job configuration :
<batch:job id="ftpReadWrite">
<batch:step id="readFromLocalWriteToFtp" next="readFromFtpWriteToLocal">
<batch:tasklet>
<batch:chunk reader="localFileReader" writer="ftpFileWriter" commit-interval="5" />
</batch:tasklet>
</batch:step>
And my "Localfilereader" and "ftpFileWriter" looks like :
import org.apache.camel.ConsumerTemplate;
import org.apache.camel.component.spring.batch.support.CamelItemReader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class LocalFileReader extends CamelItemReader {
private Logger log= LoggerFactory.getLogger(this.getClass());
ConsumerTemplate consumerTemplate;
String endpointUri;
public LocalFileReader(ConsumerTemplate consumerTemplate, String endpointUri) {
super(consumerTemplate, endpointUri);
this.consumerTemplate=consumerTemplate;
this.endpointUri=endpointUri;
}
#Override
public Object read() throws Exception {
Object item = consumerTemplate.receiveBody(endpointUri);
return item;
}
}
"Ftp File Writer"
import org.apache.camel.ProducerTemplate;
import org.apache.camel.component.spring.batch.support.CamelItemWriter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.List;
public class FtpFileWriter extends CamelItemWriter {
private Logger log= LoggerFactory.getLogger(this.getClass());
ProducerTemplate producerTemplate;
String endpointUri;
public FtpFileWriter(ProducerTemplate producerTemplate, String endpointUri) {
super(producerTemplate, endpointUri);
this.producerTemplate=producerTemplate;
this.endpointUri=endpointUri;
}
#Override
public void write(List items) throws Exception {
System.out.println("************************Writing item to ftp "+items);
for (Object item : items) {
System.out.println("writing item [{}]..."+item);
producerTemplate.sendBody(endpointUri, item);
log.debug("wrote item");
}
}
}
It works fine if I have only 5 file in my local directory. It read the all 5 file from my local directory and it send to the writer and writer send it to the ftp server as my commit-interval=5. If I have 6 file in in local directory then it send first chunk of 5 file to writer and again it start reading the remaining file and this time there is only one file remaining. It read 1 file and start waiting for 4 files and never send to writer. I tried it with commit-interval=1 now it send all 6 files to server and again start waiting for next file. Here I need to stop the process once all file have been processed.
Please help me to resolved this issue...
From ConsumerTemplate's javadoc receiveBody waits until there is a response; you need to work with timeout (check TimeoutPolicy in spring-batch) or a different way to mark reader as 'exhausted' (return null from reader) to stop reader from reading
You could use receiveBodyNoWait instead of receiveBody.
Then you have to check if there are files remaining inside the consumer endpoint. I code this for a tasklet that consumes big-xml file into smaller pieces.
The tasklet :
public class MyCamelTasklet extends ServiceSupport implements Tasklet, InitializingBean{
private static final Logger LOG = LoggerFactory.getLogger(MyCamelTasklet.class);
private final CamelContext camelContext;
private final ConsumerTemplate consumerTemplate;
private final File workingDir;
private final Route xmlSplitRoute;
public MyCamelTasklet(ConsumerTemplate consumerTemplate) {
super();
this.consumerTemplate = consumerTemplate;
this.camelContext = consumerTemplate.getCamelContext();
this.xmlSplitRoute = this.camelContext.getRoutes().get(0);
this.workingDir = new File(xmlSplitRoute.getRouteContext().getFrom().getUri().replace("file:", ""));
}
#Override
public RepeatStatus execute(StepContribution arg0, ChunkContext arg1)
throws Exception {
LOG.debug("reading new item...");
Endpoint endpointXmlSplitRoute = xmlSplitRoute.getEndpoint();
while(getNbFilesToConsume(this.workingDir) > 0) {
consumerTemplate.receiveBodyNoWait(endpointXmlSplitRoute);
}
return RepeatStatus.FINISHED;
}
private int getNbFilesToConsume(File workingDir){
return FileUtils.listFiles(workingDir, new String[]{"xml"}, false).size();
}
#Override
protected void doStart() throws Exception {
ServiceHelper.startService(consumerTemplate);
}
#Override
protected void doStop() throws Exception {
ServiceHelper.stopService(consumerTemplate);
}
#Override
public void afterPropertiesSet() throws Exception {
ObjectHelper.notNull(camelContext, "CamelContext", this);
camelContext.addService(this);
}
}
The unit test for the preceding tasklet:
public class SplitTaskletTest {
#Test public void execute() throws Exception {
CamelContext camelContext = new DefaultCamelContext();
camelContext.addRoutes(new RouteBuilder() {
public void configure() {
Namespaces ns = new Namespaces("nsl", "http://www.toto.fr/orders");
from("file:data/inbox").id("inbox-road").
split().
xtokenize("//nsl:order", 'w', ns, 1000).
streaming().
to("file:data/outbox?fileName=${file:name.noext}-${exchangeId}.${file:ext}");
}
});
camelContext.start();
ConsumerTemplate consumer =new DefaultConsumerTemplate(camelContext);
consumer.start();
MyCamelTasklet tasklet = new MyCamelTasklet(consumer);
long debutTraitement = System.currentTimeMillis();
tasklet.execute(null, null);
long finTraitement = System.currentTimeMillis();
long total = finTraitement-debutTraitement;
File outputDir = new File("data/outbox");
outputDir.mkdir();
int nbGeneratesFiles = FileUtils.listFiles(outputDir, new String[]{"xml"}, false).size();
System.out.println("Traitement total en secondes : "+total/1000);
Assert.assertTrue(nbGeneratesFiles>0);
}
}

spring-integration unit test outbound-channel adapter

i have the following configuration
<int:channel id="notificationChannel" datatype="com.mycompany.integration.NotificationMessage">
<int:queue message-store="jdbc-message-store" capacity="1000" />
</int:channel>
<int:outbound-channel-adapter ref="notificationHandler"
method="handle" channel="notificationChannel" >
<int:poller max-messages-per-poll="100" fixed-delay="60000"
time-unit="MILLISECONDS" >
<int:transactional isolation="DEFAULT" />
</int:poller>
</int:outbound-channel-adapter>
now i want to unit-test this, i need to wait for the message being processed correctly in the test, i tried it with an interceptor but that doesn't work because i could only sync on message delivery but not on successful processing of the message. implement sending a reply when the procesing is done but this would mean that would implement this only to make my unit-test work, in production there wouldn't be a replyChannel set in the message-header. how can i realize syncing on successful processing of the request without implementing it in the messageHandler?
If you are using Spring Integration 2.2.x, you can do this with an advice...
public class CompletionAdvice extends AbstractRequestHandlerAdvice {
private final CountDownLatch latch = new CountDownLatch(1);
#Override
protected Object doInvoke(ExecutionCallback callback, Object target, Message<?> message) throws Exception {
Object result = callback.execute();
latch.countDown();
return result;
}
public CountDownLatch getLatch() {
return latch;
}
}
In your test environment, add the advice to the adapter's handler with a bean factory post processor.
public class AddCompletionAdvice implements BeanFactoryPostProcessor {
private final Collection<String> handlers;
private final Collection<String> replyProducingHandlers;
public AddCompletionAdvice(Collection<String> handlers, Collection<String> replyProducingHandlers) {
this.handlers = handlers;
this.replyProducingHandlers = replyProducingHandlers;
}
#Override
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
for (String beanName : handlers) {
defineAdviceAndInject(beanFactory, beanName, beanName + "CompletionAdvice");
}
for (String beanName : replyProducingHandlers) {
String handlerBeanName = beanFactory.getAliases(beanName + ".handler")[0];
defineAdviceAndInject(beanFactory, handlerBeanName, beanName + "CompletionAdvice");
}
}
private void defineAdviceAndInject(ConfigurableListableBeanFactory beanFactory, String beanName, String adviceBeanName) {
BeanDefinition serviceHandler = beanFactory.getBeanDefinition(beanName);
BeanDefinition advice = new RootBeanDefinition(CompletionAdvice.class);
((BeanDefinitionRegistry) beanFactory).registerBeanDefinition(adviceBeanName, advice);
serviceHandler.getPropertyValues().add("adviceChain", new RuntimeBeanReference(adviceBeanName));
}
}
Add the post processor to the config <bean class="foo.AddCompletionAdvice" />.
Finally, inject the advice(s) into your test case
#ContextConfiguration
#RunWith(SpringJUnit4ClassRunner.class)
public class TestAdvice {
#Autowired
private CompletionAdvice fooCompletionAdvice;
#Autowired
private CompletionAdvice barCompletionAdvice;
#Autowired
private MessageChannel input;
#Test
public void test() throws Exception {
Message<?> message = new GenericMessage<String>("Hello, world!");
input.send(message);
assertTrue(fooCompletionAdvice.getLatch().await(1, TimeUnit.SECONDS));
assertTrue(barCompletionAdvice.getLatch().await(1, TimeUnit.SECONDS));
}
}
and wait for the latch(es).
<int:publish-subscribe-channel id="input"/>
<int:outbound-channel-adapter id="foo" channel="input" ref="x" method="handle"/>
<int:service-activator id="bar" input-channel="input" ref="x"/>
<bean class="foo.AddCompletionAdvice">
<constructor-arg name="handlers">
<list>
<value>foo</value>
</list>
</constructor-arg>
<constructor-arg name="replyProducingHandlers">
<list>
<value>bar</value>
</list>
</constructor-arg>
</bean>
<bean id="x" class="foo.Foo" />
I added these classes to a Gist
EDIT: Updated to provide a general case for ultimate consumers (no reply) and reply producing consumers.

Spring JavaMailSenderImpl throws an TypeMismatchException

I configured JavaMail with Spring framework using JavaMailSenderImpl in my application.Actually I tried to load mailing properties from database and done little bit changes at spring config.xml file.
But i got error
"Initialization of bean failed; nested exception is
org.springframework.beans.TypeMismatchException: Failed to convert
property value of type [com.core.springexamples.UCMSMailImpl] to
required type [org.springframework.mail.MailSender] for property
'mailSender'; nested exception is java.lang.IllegalArgumentException:
Cannot convert value of type [com.core.springexamples.UCMSMailImpl] to
required type [org.springframework.mail.MailSender] for property
'mailSender': no matching editors or conversion strategy found"
whatever changes are implemented in my application,those are mentioned in below.
Step 1:
<bean id="javaMailImpl" class="org.springframework.mail.javamail.JavaMailSenderImpl"></bean>
Step 2:-
<bean id="mailSender" class="com.core.springexamples.UCMSMailImpl" scope="prototype" init-method="configuredProperties">
<property name="javaMailImpl" ref="javaMailImpl"></property>
</bean>
com.core.springexamples.UCMSMailImpl:-
public class UCMSMailImpl {
private JavaMailSenderImpl javaMailImpl;
private ConfigDAO configDAO;
public void configuredProperties(){
System.out.println("UCMSMailImpl::configuredProperties");
Properties props=new Properties();
String[] mildata=configDAO.getMailingPropData();
props.put("mail.smtp.auth", mildata[0]);
props.put("mail.smtp.starttls.enable", mildata[2]);
props.put("mail.smtp.host", mildata[3]);
props.put("mail.smtp.port", mildata[4]);
props.put("mail.smtp.host", mildata[5]);
props.put("username", mildata[6]);
props.put("password",mildata[7]);
getJavaMailImpl().setJavaMailProperties(props);
}
public JavaMailSenderImpl getJavaMailImpl() {
return javaMailImpl;
}
public void setJavaMailImpl(JavaMailSenderImpl javaMailImpl) {
this.javaMailImpl = javaMailImpl;
}
public void setConfigDAO(ConfigDAO configDAO){
this.configDAO=configDAO;
}
public ConfigDAO getConfigDAO(){
return configDAO;
}
Step 3:-I am trying send the mail from MailSender.send using UCMSMailImpl java class.I refered the UCMSMailImpl bean.
<bean id="sendMail" class="com.core.springexamples.JavaMailing">
<property name="mailSender" ref="mailSender"></property>
</bean>
public class JavaMailing {
private MailSender mailSender;
public void sendMail(String from,String to,String text,String subject){
SimpleMailMessage message = new SimpleMailMessage();
message.setFrom(from);
message.setTo(to);
message.setSubject(subject);
message.setText(text);
mailSender.send(message);
}
/**
* #return the mailSender
*/
public MailSender getMailSender() {
return mailSender;
}
/**
* #param mailSender the mailSender to set
*/
public void setMailSender(MailSender mailSender) {
this.mailSender = mailSender;
}
Step 4:- I trying to test the sendMail bean
ApplicationContext context =new ClassPathXmlApplicationContext("applicationContext-mail.xml");
JavaMailing m=(JavaMailing)context.getBean("sendMail");
m.sendMail("john.ch#gmail.com", "john.c#gmail.com", "TEST MAIL", "TEST MAIL");
But i got exception is TypeMismatchException: Failed to convert property value of type [com.core.springexamples.UCMSMailImpl] to required type [org.springframework.mail.MailSender] for property
Please help me.
You cannot assign a class to an interface, if it doesn't implement the interface. UCMSMailImpl does not implement MailSender. Keep the rest as it is and change your UCMSMailImpl like this:
public class UCMSMailImpl implements MailSender {
private JavaMailSenderImpl javaMailImpl;
private ConfigDAO configDAO;
// do your property initialization
// ...
// implement interface methods
void send(SimpleMailMessage simpleMessage) throws MailException {
this.javaMailImpl.send(simpleMessage);
}
void send(SimpleMailMessage[] simpleMessages) throws MailException {
this.javaMailImpl.send(simpleMEssages);
}
}
If you cannot change UCMSMailImpl, extend it:
public class MyUCMSMailImpl extends UCMSMailImpl implements MailSender {
void send(SimpleMailMessage simpleMessage) throws MailException {
this.getgetJavaMailImpl().send(simpleMessage);
}
void send(SimpleMailMessage[] simpleMessages) throws MailException {
this.getgetJavaMailImpl().send(simpleMEssages);
}
}
and change your configuration:
<bean id="mailSender" class="your.package.MyUCMSMailImpl" scope="prototype" init-method="configuredProperties">
<property name="javaMailImpl" ref="javaMailImpl"></property>
</bean>

how to call method using spring?

I want to call a method after creating the bean using spring, for example I create Factory and schema but I want to call the same method of Factory before creating a schema bean
<!-- schemaFactory-->
<bean id="schemaFact" class="javax.xml.validation.SchemaFactory"
factory-method="newInstance">
<constructor-arg value="http://www.w3.org/2001/XMLSchema" />
</bean>
<!-- schema -->
<bean id="schema" class="javax.xml.validation.Schema"
factory-bean="schemaFact" factory-method="newSchema">
<constructor-arg value="3DSecure.xsd" />
</bean>
You could use your own factory bean instead of javax.xml.validation.SchemaFactory, which will delegate to javax.xml.validation.SchemaFactory and the call the method:
public class MySchemaFactory {
public SchemaFactory newInstance() {
SchemaFactory factory = SchemaFactory.newInstance();
factory.callSomeMethod();
return factory;
}
}
<bean id="mySchemaFactory" class="com.foo.bar.MySchemaFactory"/>
<bean id="schemaFact"
factory-bean="mySchemaFactory"
factory-method="newInstance"/>
If you are using Spring 3.0/3.1 you can take advantage of Java configuration:
#Configuration
class Config {
#Bean
public SchemaFactory schemaFact() throws SAXNotSupportedException, SAXNotRecognizedException {
final SchemaFactory schemaFactory = SchemaFactory.newInstance("http://www.w3.org/2001/XMLSchema");
schemaFactory.setFeature("apache.org/xml/features/validation/schema-full-checking", false);
return schemaFactory;
}
#Bean
public Schema schema() throws SAXException {
return schemaFact().newSchema(new File("3DSecure.xsd"));
}
}
But which method on SchemaFactory do you want to call? Seems like all of them are either getters or setters, so you can use normal XML injection... Alternatively create your own FactoryBean:
class SchemaFactoryFactoryBean implements FactoryBean<SchemaFactory> {
#Override
public SchemaFactory getObject() throws Exception
{
final SchemaFactory schemaFactory = SchemaFactory.newInstance("http://www.w3.org/2001/XMLSchema");
schemaFactory.setFeature("apache.org/xml/features/validation/schema-full-checking", false);
return schemaFactory;
}
#Override
public Class<?> getObjectType()
{
return SchemaFactory.class;
}
#Override
public boolean isSingleton()
{
return true;
}
}
And use it like this (no factory-method needed):
<bean id="schemaFact" class="SchemaFactoryFactoryBean"/>

Resources