spring batch - Write object data to multiple files - spring

In spring batch, I have a requirement where i have list of customer objects from database and from this list i need to create multiple text files.
clas Customer{
long customerId;
string name;
Address add;
Phone phn;
}
class Address{
string address;
long pincode;
string street;
}
class Phone{
string phoneNumber;
string desc;
}
I want to write the data of each customer to different text files such as
customer.txt -> customerId, name
address.txt -> address, pincode, street
phone.txt -> phoneNumber, desc
i tried using CompositeItemWriter which can delegate object to different writer but i could not get how to retrive specific object or properties in the writer.
I couldnt get any sample or explaination to implement CompositeWriter. can someone please help me with the writer implementation? Is there any other better way of doing it?
Thanks in advance.

I was able to write data of each customer to different text file using CompositeItemWriter. Below is how i did for customer phone extract. I hope it helps someone.
<bean id="compositeCustomerWriter"
class="org.springframework.batch.item.support.CompositeItemWriter">
<property name="delegates">
<list>
<ref bean="customerWriter" />
<ref bean="customerPhoneWriter" />
<ref bean="customerAddressWriter" />
</list>
</property>
</bean>
<bean id="customerPhoneWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
<property name="resource" value="file:customer_phone_extract.txt" />
<property name="lineAggregator" ref="customerPhonelineAggregator" />
</bean>
<bean id="customerPhonelineAggregator"
class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter" value="|" />
<property name="fieldExtractor">
<bean
class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<property name="names"
value="phn.phoneNumber,phn.desc" />
</bean>
</property>
</bean>

Related

spring new arrayList injection

how can i inject an empty (new) arraylist of "double/string/object" in spring ?
public class Pack{
private List<String> names;
private List<Course> courses;
private List<double> prices;
private int comp;
}
xml file
<bean id="pack" class="com.classes.Pack">
<property name="compt" value="0" />
<property name="courses" ref="courses" />
<property name="prices" ref="prices" />
<property name="names" ref="names" />
</bean>
You can inject an empty list like this, however this is probably unnecessary, unless you're trying to setup an example template spring XML config perhaps.
<property name="courses">
<list></list>
</property>

Spring Batch - Last item from the reader alone is getting updated

I have to read from a file (FlatFile) and update a column if that ID present in the file matches the id in the column.The file is being read properly but only the last id value is getting updated here . Please find the snippet
Job-Config.xml
<bean id="abcitemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
<property name="resource" value="file:datafile/outputs/ibdData.txt" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="names" value="ID,NAM,TYPE" />
<property name="delimiter" value="|"/>
</bean>
</property>
<property name="fieldSetMapper">
<bean class="com.pershing.intraware.springbatch.mapper.abcFieldsetMapper" />
</property>
</bean>
</property>
</bean>
<bean id="abcitemWriter" class="org.springframework.batch.item.database.JdbcBatchItemWriter" scope="step">
<property name="dataSource" ref="dataSource" />
<property name="sql"><value>UPDATE TEST_abc SET BIZ_ARNG_CD = CASE WHEN ID IN (SELECT ID FROM TEST_abc WHERE ID= ? and MONTH=(to_char(sysdate, 'MM')) AND YR =(to_char(sysdate, 'YY'))) THEN 'Y' ELSE 'N' END</value></property>
<!-- It will take care matching between object property and sql name parameter -->
<property name="itemPreparedStatementSetter" ref="testPrepStatementSetter" />
</bean>
</beans>
Setter.java
public class IDItemPreparedStatementSetter implements ItemPreparedStatementSetter<Test> {
#Override
public void setValues(Test item, PreparedStatement ps) throws SQLException {
// TODO Auto-generated method stub
ps.setString(1, item.getID());
}
}
Your query is updating each row of database every time it is fired. You need to restrict that. Currently; it must be setting the BIZ_ARNG_CD to 'Y' for records with ID equal to the ID of the last record passed to the writer.
You can fix this in 2 ways -
Default the database column to 'N' and don't set it to 'N' in the update statement
Add where clause in update script ( BIZ_ARNG_CD != 'Y')

read only selective columns from csv file using spring batch

there is one csv file having 100 columns, but we need only 3-5 columns which needs to be loaded into database.
I dont want to specify all the 100 columns in linetokenizer in job xml.
Please suggest how we can proceed in this case
Try using a custom fieldSetMapper. You can use it similar to a ResultSet with indexes.
You have to list all the column names only if you want automatic mapping.
Specify only the delimiter, in your case ","
<bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
<property name="resource" value="YOURFILE" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="fieldSetMapper">
<bean class="CUSTOMFIELDSETMAPPER" />
</property>
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="delimiter" value="," />
</bean>
</property>
</bean>
</property>
</bean>
The Custom Mapper could be something like this, say if you want to read the 1st column and 25th column:
public class CustomMapper implements FieldSetMapper<CustomPOJO>{
#Override
public CustomPOJO mapFieldSet(FieldSet fieldSet) throws BindException {
CustomPOJO result = new CustomPOJO();
result.setName(fieldSet.readString(0));
result.setAddress(fieldSet.readString(24));
return result;
}
}
For further explanation on how to use the reader, please refer to this tutorial

Reading in UTF-8 encoding in Java results in issues at only some places

I am reading a UTF_8 encoded file byte by byte in Java Spring Batch and saving the content in mongodb. The reading is delegated to FlatFileItemReader and the encoding is injected into the bean as UTF-8.
What I do see is that only at one or two specific places, the umlaut character is getting stored with an encoding problem in mongodb. I verified that the content in the original file is okay. What is interesting is that everywhere else, the umlaut character in the original file is interpreted correctly and stored correctly in mongodb.
Could you help with this?
<bean id="recordReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
<property name="resource" value="file:#{jobExecutionContext['inputfile']}" />
<property name="linesToSkip" value = "1" />
<property name= "encoding" value ="UTF-8"/>
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper" >
<property name = "lineTokenizer">
<bean class = "com.xyz.NoQuoteDelimitedLineTokenizer">
<property name = "strict" value = "false" />
<property name = "names" value = "Col1,Col2,Col3,Col4,Col5,Col6,Col7,Col8" />
<property name="delimiter">
<util:constant static-field="org.springframework.batch.item.file.transform.DelimitedLineTokenizer.DELIMITER_TAB" />
</property>
</bean>
</property>
<property name = "fieldSetMapper">
<bean class = "com.xyz.MyFieldSetMapper" >
<property name = "ctx" value="#{jobExecutionContext['contextBean']}"/>
</bean>
</property>
</bean>
</property>
</bean>
FieldSetMapper
private void validateFields (String id, ConsumerDO ci, FieldSet fieldset_p) throws Exception
{try {
ci.setCaption(fieldset_p.readString("newCaption"));
}
catch (Exception e) {
}
}

Spring 3.2 with MVC, ContentNegotation, REST and PDF Generator

Let's say, I have a REST styled controller mapping
#RequestMapping(value="users", produces = {MediaType.APPLICATION_JSON_VALUE})
public List<User> listUsers(#ReqestParams Integer offset, #ReqestParams Integer limit, #ReqestParams String query) {
return service.loadUsers(query, offset, limit);
}
Serving JSON (or even XML) is not an issue, this is easy using ContentNegotation and MessageConverters
<bean id="contentNegotiationManager" class="org.springframework.web.accept.ContentNegotiationManagerFactoryBean">
<property name="favorPathExtension" value="true" />
<property name="favorParameter" value="false" />
<property name="ignoreAcceptHeader" value="false" />
<property name="mediaTypes" >
<value>
html=text/html
json=application/json
xml=application/xml
</value>
</property>
</bean>
Now, I need to add support for PDF. Naturally, I want to use (Spring) MVC + REST as much as possible. Most examples I have found implement this with an explicit definition not using REST style, e.g.
#RequestMapping(value="users", produces = {"application/pdf"})
public ModelAndView listUsersAsPdf(#ReqestParams Integer offset, #ReqestParams Integer limit, #ReqestParams String query) {
List<User> users = listUsers(offset, limit, query); // delegated
return new ModelAndView("pdfView", users);
}
That works, but is not very comfortable because for every alternate output (PDF, Excel, ...) I would add a request mapping.
I have already added application/pdf to the content negotation resolver; unfortunately any request with a suffix .pdf or the Accept-Header application/pdf were be responded with 406.
What is the ideal setup for a REST/MVC style pattern to integrate alternate output like PDF?
You can create a WEB-INF/spring/pdf-beans.xml like below.
<bean id="listofusers" class="YourPDFBasedView"/>
And your controller method will return view name as listofusers.
#RequestMapping(value="users")
public ModelAndView listUsersAsPdf(#ReqestParams Integer offset, #ReqestParams Integer limit, #ReqestParams String query) {
List<User> users = listUsers(offset, limit, query); // delegated
return new ModelAndView("listofusers", users);
}
And you can use contentNegotiationViewResolver in this way:
<bean class="org.springframework.web.servlet.view.XmlViewResolver">
<property name="order" value="1"/>
<property name="location" value="WEB-INF/spring/pdf-views.xml"/>
</bean>
<!--
View resolver that delegates to other view resolvers based on the content type
-->
<bean class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
<!-- All configuration is now done by the manager - since Spring V3.2 -->
<property name="contentNegotiationManager" ref="cnManager"/>
</bean>
<!--
Setup a simple strategy:
1. Only path extension is taken into account, Accept headers are ignored.
2. Return HTML by default when not sure.
-->
<bean id="cnManager" class="org.springframework.web.accept.ContentNegotiationManagerFactoryBean">
<property name="ignoreAcceptHeader" value="true"/>
<property name="defaultContentType" value="text/html" />
</bean>
For JSON: Create a generic JSON view resolver like below and register it as bean in context file.
public class JsonViewResolver implements ViewResolver {
/**
* Get the view to use.
*
* #return Always returns an instance of {#link MappingJacksonJsonView}.
*/
#Override
public View resolveViewName(String viewName, Locale locale) throws Exception {
MappingJacksonJsonView view = new MappingJacksonJsonView();
view.setPrettyPrint(true); // Lay the JSON out to be nicely readable
return view;
}
}
Same for XML:
public class MarshallingXmlViewResolver implements ViewResolver {
private Marshaller marshaller;
#Autowired
public MarshallingXmlViewResolver(Marshaller marshaller) {
this.marshaller = marshaller;
}
/**
* Get the view to use.
*
* #return Always returns an instance of {#link MappingJacksonJsonView}.
*/
#Override
public View resolveViewName(String viewName, Locale locale)
throws Exception {
MarshallingView view = new MarshallingView();
view.setMarshaller(marshaller);
return view;
}
}
and register above xml view resolver in context file like this:
<oxm:jaxb2-marshaller id="marshaller" >
<oxm:class-to-be-bound name="some.package.Account"/>
<oxm:class-to-be-bound name="some.package.Customer"/>
<oxm:class-to-be-bound name="some.package.Transaction"/>
</oxm:jaxb2-marshaller>
<!-- View resolver that returns an XML Marshalling view. -->
<bean class="some.package.MarshallingXmlViewResolver" >
<constructor-arg ref="marshaller"/>
</bean>
You can find more information at this link:
http://spring.io/blog/2013/06/03/content-negotiation-using-views/
Using all view resolver techniques, you can avoid writing duplicate methods in controller, such as one for xml/json, other for excel, other for pdf, another for doc, rss and all.
Knalli, if you replace #ResponseBody with ModelAndView(), you can achieve both the features.
Is there any reason you want to keep #ResponseBody ? I just want to know if I am missing anything, just want to learn.
Other option is to write HttpMessageConverters then:
Some samples are here.
Custom HttpMessageConverter with #ResponseBody to do Json things
http://www.javacodegeeks.com/2013/07/spring-mvc-requestbody-and-responsebody-demystified.html
This is working sample. I have configured contentnegotiationviewresolver for this, and give highest order. After that I have ResourceBundleViewResolver for JSTL and Tiles View, then XmlViewResolver for excelResolver, pdfResolver, rtfResolver. excelResolver, pdfResolver, rtfResolver. XmlViewResolver and ResourceBundleViewResolver works only with MAV only, but MappingJacksonJsonView and MarshallingView takes care for both MAV and #ResponseBody return value.
<bean id="contentNegotiatingResolver" class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
<property name="order"
value="#{T(org.springframework.core.Ordered).HIGHEST_PRECEDENCE}" />
<property name="mediaTypes">
<map>
<entry key="json" value="application/json" />
<entry key="xml" value="application/xml" />
<entry key="pdf" value="application/pdf" />
<entry key="xlsx" value="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" />
<entry key="doc" value="application/msword" />
</map>
</property>
<property name="defaultViews">
<list>
<!-- JSON View -->
<bean class="org.springframework.web.servlet.view.json.MappingJacksonJsonView" />
<!-- XML View -->
<bean class="org.springframework.web.servlet.view.xml.MarshallingView">
<constructor-arg>
<bean id="jaxbMarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
<property name="classesToBeBound">
<list>
<value>Employee</value>
<value>EmployeeList</value>
</list>
</property>
</bean>
</constructor-arg>
</bean>
</list>
</property>
<property name="ignoreAcceptHeader" value="true" />
</bean>
<bean class="org.springframework.web.servlet.view.ResourceBundleViewResolver"
id="resourceBundleResolver">
<property name="order" value="#{contentNegotiatingResolver.order+1}" />
</bean>
<bean id="excelResolver" class="org.springframework.web.servlet.view.XmlViewResolver">
<property name="location">
<value>/WEB-INF/tiles/spring-excel-views.xml</value>
</property>
<property name="order" value="#{resourceBundleResolver.order+1}" />
</bean>
<bean id="pdfResolver" class="org.springframework.web.servlet.view.XmlViewResolver">
<property name="location">
<value>/WEB-INF/tiles/spring-pdf-views.xml</value>
</property>
<property name="order" value="#{excelResolver.order+1}" />
</bean>
<bean id="rtfResolver" class="org.springframework.web.servlet.view.XmlViewResolver">
<property name="location">
<value>/WEB-INF/tiles/spring-rtf-views.xml</value>
</property>
<property name="order" value="#{excelResolver.order+1}" />
</bean>
And our XMLViewResolver spring-pdf-views.xml looks like this.
<bean id="employees"
class="EmployeePDFView"/>
And EmployeePDFView will have code for generating pdf and writing pdf byte stream on Response object. This will resolve to rest url that will end with .pdf extension, and when you return MAV with "employees" id.

Resources