org.springframework.batch.item.file.MultiResourceItemWriter saves currentResourceItemCount to executionContext, but inside the update function, which is always called before write function and as result when I am getting always currentResourceItemCount equal to 0.
What I want to achieve is to get the value of how many items have bean written to the file and put it to the footer.
public class FooterCallback extends StepExecutionListenerSupport implements FlatFileFooterCallback {
private StepExecution stepExecution;
#Override
public void writeFooter(Writer writer) throws IOException {
Integer itemCount = stepExecution.getExecutionContext().getInt("MultiResourceItemWriter.resource.item.count");
writer.write("F;" + itemCount );
}
}
Config is:
<bean id="csvGenerateWriter" class="org.springframework.batch.item.file.MultiResourceItemWriter" scope="step">
<property name="resource" value="file:#{jobExecutionContext[outputPath]}#{jobExecutionContext[outputFile]}"/>
<property name="itemCountLimitPerResource" value="3" />
<property name="delegate" ref="delegateWriter" />
</bean>
<bean id="delegateWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
<property name="footerCallback" ref="footerCallback" />
</bean>
but with the above code, I am always getting 0 in itemCount. Is is a bug or a feature? Why update furntion in MultiResourceItemWriter is not called after the write?
Create a custom writer with delegation; also this writer will be responsible for footer and should take care of written items count.
public class FooterWriter implements ResourceAwareItemWriterItemStream<Item>,FlatFileFooterCallback {
ResourceAwareItemWriterItemStream<Item> delegate;
int count;
public void write(List<? extends Item> items) throws Exception {
count += items.size();
delegate.write(items);
}
public void setDelegate(ResourceAwareItemWriterItemStream<Item> delegate) {
this.delegate = delegate;
}
public void writeFooter(Writer writer) throws IOException {
writer.write("F;" + count);
}
public void open(ExecutionContext executionContext) throws ItemStreamException {
count = executionContext.getInt("fwc", 0);
delegate.open(executionContext);
}
public void update(ExecutionContext executionContext) throws ItemStreamException {
executionContext.putInt("fwc", count);
delegate.update(executionContext);
}
public void close() throws ItemStreamException {
// call close() before reset count because close() calls writeFooter()!
delegate.close();
this.count = 0;
}
public void setResource(Resource resource) {
delegate.setResource(resource);
}
}
<bean id="csvGenerateWriter" class="org.springframework.batch.item.file.MultiResourceItemWriter" scope="step">
<property name="resource" value="file:output.txt"/>
<property name="itemCountLimitPerResource" value="3" />
<property name="delegate" ref="delegateWriter" />
</bean>
<bean id="flatFileItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
<property name="footerCallback" ref="delegateWriter" />
<property name="lineAggregator">
<bean class="org.springframework.batch.item.file.transform.PassThroughLineAggregator" />
</property>
</bean>
<bean id="delegateWriter" class="footercallback.FooterWriter">
<property name="delegate" ref="flatFileItemWriter" />
</bean>
Related
I am novice in spring Transaction Management, I am using it in batch insert but the batch processing is not working, its committing line by line when autoCommit is true, and if autoCommit is false then the data is not inserting in D.B.
below is the insert function.
#Override
#Transactional(propagation= Propagation.REQUIRED, rollbackFor=Exception.class)
public void insertInviteContactMsisdn(final long ettId, final List<String> msisdnList, final List<String> allreadyMsisdnList) {
try {
String qr = "insert into InviteContactMsisdn (id,ettIdAparty,msisdn,status) values(0,?,?,?)";
getJdbcTemplate().batchUpdate(qr, new BatchPreparedStatementSetter() {
#Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
ps.setLong(1, ettId);
ps.setString(2, msisdnList.get(i));
ps.setBoolean(3, allreadyMsisdnList.contains(msisdnList.get(i)));
}
#Override
public int getBatchSize() {
return msisdnList.size();
}
});
}catch(DataAccessException dataAccessException) {
log.error("error in processing insertInviteContactMsisdn ettId:"+ettId);
throw dataAccessException;
}
}
also the dataSource configuration:-
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="${db.url}" />
<property name="username" value="${db.user}" />
<property name="password" value="${db.pass}" />
<property name="removeAbandoned" value="true"/>
<property name="initialSize" value="1" />
<property name="maxActive" value="1" />
<property name="defaultAutoCommit" value="false" />
Thanks.
I know I am doing something seriously wrong.
when I using programmatic Transaction Management the same problem occurring.
#Override
public void insertInviteContactMsisdn(final long ettId, final List<String> msisdnList, final List<String> allreadyMsisdnList) {
// TODO Auto-generated method stub
TransactionDefinition transactionDefinition = new DefaultTransactionDefinition();
TransactionStatus transactionStatus = transactionManager.getTransaction(transactionDefinition);
try {
String qr = "insert into InviteContactMsisdn (id,ettIdAparty,msisdn,status) values(0,?,?,?)";
jdbcTemplateObject.batchUpdate(qr, new BatchPreparedStatementSetter() {
#Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
ps.setLong(1, ettId);
ps.setString(2, msisdnList.get(i));
ps.setBoolean(3, allreadyMsisdnList.contains(msisdnList.get(i)));
}
#Override
public int getBatchSize() {
return msisdnList.size();
}
});
transactionManager.commit(transactionStatus);
}catch(DataAccessException dataAccessException) {
log.error("error in processing insertInviteContactMsisdn ettId:"+ettId);
transactionManager.rollback(transactionStatus);
throw dataAccessException;
}
}
I use JpaRepository to save data, but the hibernate.show_sql shows "select" and won't save data. Following is my service:
#Autowired
private UserRepository userRepository;
#PostConstruct
public void init() {
User admin = new User();
admin.setDisplayName("admin");
admin.setEmailAddress("admin#admin");
admin.setPassword("admin___");
admin.setRegisteredAt(new Date());
admin.setLastAccessAt(new Date());
admin.setUuid(UUID.randomUUID().toString());
try {
System.out.println("before save");
userRepository.save(admin);
System.out.println("after save");
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
The output looks like this:
========before save======
Hibernate: select user0_.uuid as uuid1_0_0_, user0_.display_name as display_2_0_0_, user0_.email_address as email_ad3_0_0_, user0_.last_access_at as last_acc4_0_0_, user0_.password as password5_0_0_, user0_.registered_at as register6_0_0_ from User user0_ where user0_.uuid=?
========after save=======
Following is my applicationContext.xml:
<context:component-scan base-package="test">
<context:exclude-filter type="annotation"
expression="org.springframework.stereotype.Controller" />
</context:component-scan>
<bean id="myDataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/helloworld" />
<property name="username" value="root" />
<property name="password" value="password" />
</bean>
<bean id="myEmf"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="packagesToScan" value="test.entity"></property>
<property name="dataSource" ref="myDataSource" />
<property name="jpaProperties">
<props>
<prop key="hibernate.show_sql">true</prop>
<prop key="hibernate.hbm2ddl.auto">update</prop>
</props>
</property>
<property name="persistenceProvider">
<bean class="org.hibernate.jpa.HibernatePersistenceProvider"></bean>
</property>
</bean>
<tx:annotation-driven transaction-manager="transactionManager" />
<bean id="transactionManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="myDataSource" />
</bean>
<jpa:repositories base-package="test.repository"
entity-manager-factory-ref="myEmf" transaction-manager-ref="transactionManager"></jpa:repositories>
Attached is my class generated by JPA Tools:
#Entity
#NamedQuery(name="User.findAll", query="SELECT u FROM User u")
public class User implements Serializable {
private static final long serialVersionUID = 1L;
#Id
private String uuid;
#Column(name="display_name")
private String displayName;
#Column(name="email_address")
private String emailAddress;
#Temporal(TemporalType.TIMESTAMP)
#Column(name="last_access_at")
private Date lastAccessAt;
private String password;
#Temporal(TemporalType.TIMESTAMP)
#Column(name="registered_at")
private Date registeredAt;
public User() {
}
public String getUuid() {
return this.uuid;
}
public void setUuid(String uuid) {
this.uuid = uuid;
}
public String getDisplayName() {
return this.displayName;
}
public void setDisplayName(String displayName) {
this.displayName = displayName;
}
public String getEmailAddress() {
return this.emailAddress;
}
public void setEmailAddress(String emailAddress) {
this.emailAddress = emailAddress;
}
public Date getLastAccessAt() {
return this.lastAccessAt;
}
public void setLastAccessAt(Date lastAccessAt) {
this.lastAccessAt = lastAccessAt;
}
public String getPassword() {
return this.password;
}
public void setPassword(String password) {
this.password = password;
}
public Date getRegisteredAt() {
return this.registeredAt;
}
public void setRegisteredAt(Date registeredAt) {
this.registeredAt = registeredAt;
}
}
Since you're using JPA, the transaction manager should be a JpaTransactionManager, not a DataSourceTransactionManager.
This is not a duplicate of this question. So please don't close it for "is duplicate of" reasons..
I am trying to autowire a private field in my service class using this tutorial. My problem is that restaurantOwnerRepository remains null and does not get initialized.
servlet-context.xml
<context:component-scan base-package="com.mahlzeit.web.server" />
<bean id="sessionFactory"
class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="configLocation" value="classpath:hibernate.cfg.xml" />
</bean>
<tx:annotation-driven />
<bean id="transactionManager"
class="org.springframework.orm.hibernate4.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
<bean id="restaurantOwnerRepository" class="com.mahlzeit.web.server.dao.RestaurantOwnerRepository">
<constructor-arg>
<ref bean="sessionFactory" />
</constructor-arg>
</bean>
Service code:
#Component
public class RestaurantInformationServiceImpl extends XsrfProtectedServiceServlet implements RestaurantInformationService {
private static final long serialVersionUID = -4088840947018614411L;
#Autowired
private RestaurantOwnerRepository restaurantOwnerRepository;
private final static Logger logger = Logger.getLogger(RestaurantInformationServiceImpl.class);
#Override
public List<RestaurantDTO> getAvailableRestaurants() {
// restaurantOwnerRepository is 'null'
List<Restaurant> availableRestaurants = restaurantOwnerRepository.getAvailableRestaurants(getSessionId());
return null;
}
private String getSessionId() {
HttpServletRequest httpRequest = getThreadLocalRequest();
return httpRequest.getSession().getId();
}
}
RestaurantOwnerRepository.java
public class RestaurantOwnerRepository implements RestauranOwnerDAO {
private SessionFactory sessionFactory;
public RestaurantOwnerRepository(SessionFactory sessionFactory) {
this.sessionFactory = sessionFactory;
}
// ..
}
What could be the reason for this?
here is a sample controller for your example , you should define the bean in your context.xml or if you place it in this package : com.mahlzeit.web.server it will be managed by spring automatically , cause as i see you have placed the context:component-scan
#Controller
public class RestaurantInformationServiceImpl {
#Autowired
private RestaurantOwnerRepository restaurantOwnerRepository;
#RequestMapping(value="/")
public #ResponseBody ModelAndView getRestaurants(
HttpServletRequest request,
HttpServletResponse response) {
ModelAndView model = new ModelAndView("yourPage");
List<?> rests = restaurantOwnerRepository.getAvailableRestaurants(httpRequest.getSession().getId());
model.addObject("restList", rests );
return model;
}
}
When I am prining data in console with , it is working.
But when I am trying to write data in output text file, it is not working but there is no error.
Can anyone help me that what is issue here??
Bellow is my coding:-
Java Class:
It should be used during writting in file.
import org.springframework.batch.item.file.transform.FieldExtractor;
import com.poc.customer.vo.Person;
public class PersonFieldSetExtractor implements FieldExtractor<Person> {
#Override
public Object[] extract(Person person) {
return new Object[] { person.getSrcId(), person.getSrcCode(),
person.getFirstName(), person.getLastName(),
person.getPincode(), person.getStreet()};
}
}
Java Class:
This will be used after reading from file and during bean population.
import org.springframework.batch.item.file.mapping.FieldSetMapper;
import org.springframework.batch.item.file.transform.FieldSet;
import org.springframework.validation.BindException;
import com.poc.customer.vo.Person;
public class PersonFieldSetMapper implements FieldSetMapper<Person> {
#Override
public Person mapFieldSet(FieldSet fieldSet) throws BindException {
Person person = new Person();
person.setSrcId(fieldSet.readString(0));
person.setSrcCode(fieldSet.readString(1));
person.setFirstName(fieldSet.readString(2));
person.setLastName(fieldSet.readString(3));
person.setPincode(fieldSet.readString(4));
//person.setStreet(fieldSet.readString(5));
return person;
}
}
public class Person implements Serializable, Comparable<Person> {
#Override
public int compareTo(Person arg0) {
return this.firstName.compareTo(arg0.getFirstName());
}
private String srcId;
private String srcCode;
private String firstName;
private String lastName;
private String pincode;
private String street;
public String getSrcId() {
return srcId;
}
public void setSrcId(String srcId) {
this.srcId = srcId;
}
public String getSrcCode() {
return srcCode;
}
public void setSrcCode(String srcCode) {
this.srcCode = srcCode;
}
public String getFirstName() {
return firstName;
}
public void setFirstName(String firstName) {
this.firstName = firstName;
}
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
public String getPincode() {
return pincode;
}
public void setPincode(String pincode) {
this.pincode = pincode;
}
public String getStreet() {
return street;
}
public void setStreet(String street) {
this.street = street;
}
}
Spring Configuration:-
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:batch="http://www.springframework.org/schema/batch"
xmlns:task="http://www.springframework.org/schema/task"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/batch
http://www.springframework.org/schema/batch/spring-batch-2.2.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.2.xsd">
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<property name="transactionManager" ref="transactionManager" />
</bean>
<bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<batch:job id="personJob">
<batch:step id="step1">
<batch:tasklet>
<batch:chunk reader="txtFileItemReader" writer="txtFileItemWriter" commit-interval="1">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<bean id="txtFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
<bean id="txtFileItemReader" class="com.poc.batch.util.FlatFileReader">
<!-- Read a csv file -->
<property name="resource" value="classpath:com/cts/poc/batch/resource/Test_Data.txt" />
<property name="lineMapper" ref="lineMapper" />
</bean>
<bean id="lineMapper" class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="fieldSetMapper" ref="fieldSetMapper" />
<property name="lineTokenizer" ref="lineTokenizer" />
</bean>
<bean id="lineTokenizer" class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="delimiter" value="|"/>
</bean>
<bean id="fieldSetMapper" class="com.poc.batch.util.PersonFieldSetMapper"/>
<!-- Writer to write output txt file -->
<bean id="txtFileItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
<property name="resource" value="file:com/cts/poc/batch/resource/Test_Out_Data.txt" />
<property name="appendAllowed" value="true" />
<property name="lineAggregator" ref="lineAggregator"/>
</bean>
<bean id="lineAggregator" class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter" value="|" />
<property name="fieldExtractor" ref="personFieldSetExtractor" />
</bean>
<bean id="personFieldSetExtractor" class="com.poc.batch.util.PersonFieldSetExtractor"/>
</beans>
Why use AbstractRoutingDataSource can not dynamic switch DataSource
This is the configuration information
public class DynamicSwitch {
public static final ThreadLocal<String> local=new ThreadLocal<String>();
public static void setDB(String id){
local.set(id);
}
public static String getDB(){
return local.get();
}
public static void removeDB(){
local.remove();
}
}
public class DynamicSource extends AbstractRoutingDataSource implements InitializingBean{
#Override
protected Object determineCurrentLookupKey() {
// TODO Auto-generated method stub
return DynamicSwitch.getDB();
}
}
<bean id="dynamic" class="com.aware.DynamicSource">
<property name="targetDataSources">
<map key-type="java.lang.String">
<entry key="1" value-ref="dataSource"></entry>
<entry key="2" value-ref="localdataSource"></entry>
</map>
</property>
<property name="defaultTargetDataSource" ref="dataSource"></property>
</bean>
<bean id="methodService" class="com.test.service.MethodServiceImpl">
<property name="sqlMapClient" ref="sqlMapClient"></property>
</bean>
<bean id="test" class="com.test.Test" scope="prototype"></bean>
public class Test2 extends ActionSupport{
public String execute() throws Exception {
// TODO Auto-generated method stub
DynamicSwitch.setDB("2");
MethodService methodService=(MethodService)ApplicationAware.getBean("methodService");
Map<String, String> map=new HashMap<String, String>();
List list=methodService.testList("Service_ks_missionSpace.getService_ks_missionList", map);
System.out.println(list.size());
return SUCCESS;
}
Invoke DynamicSwitch.setDB("2") find can not Switch DataSource.
DataSource or to default dataSource
Why