I am very new to Spring Batch. I just tried to run a simple spring batch example which i got from net in vain.The example just tries to read from a flat file and write the contents into a separate flat file.
Below is my xml which is loaded into context (through a ContextLoaderListener).
test-batch.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://172.18.7.57/spring-beans.dtd">
<beans>
<bean id="simpleJob"
class="org.springframework.batch.core.job.SimpleJo b">
<property name="name" value="simpleJob" />
<property name="steps">
<list>
<ref local="step" />
</list>
</property>
<property name="jobRepository" ref="jobRepository" />
</bean>
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.suppo rt.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<bean id="jobRepository"
class="org.springframework.batch.core.repository.s upport.SimpleJobRepository">
<constructor-arg>
<bean
class="org.springframework.batch.core.repository.d ao.MapJobInstanceDao" />
</constructor-arg>
<constructor-arg>
<bean
class="org.springframework.batch.core.repository.d ao.MapJobExecutionDao" />
</constructor-arg>
<constructor-arg>
<bean
class="org.springframework.batch.core.repository.d ao.MapStepExecutionDao" />
</constructor-arg>
</bean>
<bean id="transactionManager"
class="org.springframework.batch.support.transacti on.ResourcelessTransactionManager" />
<bean id="step"
class="org.springframework.batch.core.step.item.Si mpleStepFactoryBean">
<property name="transactionManager" ref="transactionManager" />
<property name="jobRepository" ref="jobRepository" />
<property name="itemReader" ref="itemReader" />
<property name="itemWriter" ref="itemWriter" />
</bean>
<bean id="lineAggregator"
class="org.springframework.batch.item.file.transfo rm.DelimitedLineAggregator">
<property name="delimiter" value="|" />
</bean>
<bean id="itemWriter"
class="org.springframework.batch.item.file.FlatFil eItemWriter">
<property name="fieldSetCreator" ref="fieldSetMapper" />
<property name="lineAggregator" ref="lineAggregator" />
<property name="resource" value="file:/hello2.txt" />
</bean>
<bean id="lineTokenizer"
class="org.springframework.batch.item.file.transfo rm.DelimitedLineTokenizer" />
<bean id="fieldSetMapper"
class="org.springframework.batch.item.file.mapping .PassThroughFieldSetMapper" />
<bean id="itemReader"
class="org.springframework.batch.item.file.FlatFil eItemReader">
<property name="resource" value="file:/hello1.txt" />
<property name="lineTokenizer" ref="lineTokenizer" />
<property name="fieldSetMapper" ref="fieldSetMapper" />
</bean>
</beans>
As i dont have maven, i tried launching this job from my java code like ,
ApplicationContext context = new FileSystemXmlApplicationContext("D:\\jboss-4.0.5.GA\\server\\default\\deploy\\test.war\\WEB-INF\\xml\\test-batch.xml");
Job job;
job = (Job) context.getBean("simpleJob");
JobParameters jobParameters = jobParametersConverter.getJobParameters(StringUtils
.splitArrayElementsIntoProperties(new String[]{"D:\\jboss-4.0.5.GA\\server\\default\\deploy\\venki.war\\WEB-INF\\xml\\venki-batch.xml","simpleJob"}, "="));
JobExecution jobExecution = launcher.run(job, jobParameters);
--But, some error is thrown in eclipse saying no class loaders found....below is the error i got....
org.apache.commons.lang.SerializationException: java.lang.ClassNotFoundException: No ClassLoaders found for: org.springframework.batch.core.JobExecution
at org.apache.commons.lang.SerializationUtils.deseria lize(SerializationUtils.java:164)
at org.apache.commons.lang.SerializationUtils.deseria lize(SerializationUtils.java:191)
at org.springframework.batch.core.repository.dao.MapJ obExecutionDao.copy(MapJobExecutionDao.java:33)
at org.springframework.batch.core.repository.dao.MapJ obExecutionDao.saveJobExecution(MapJobExecutionDao .java:56)
at org.springframework.batch.core.repository.support. SimpleJobRepository.saveOrUpdate(SimpleJobReposito ry.java:216)
at org.springframework.batch.core.repository.support. SimpleJobRepository.createJobExecution(SimpleJobRe pository.java:192)
at org.springframework.batch.core.launch.support.Simp leJobLauncher.run(SimpleJobLauncher.java:79)
at com.venki.handler.TestHandler.testSpringBatch(Test Handler.java:111)
--Please help me in this regard...i need to find out a code that reads from a flat file and inserts into a DB....i need to integrate this into a stand alone module and i should not go with using maven....
Why dont you run it by using org.springframework.batch.core.launch.support.CommandLineJobRunner
you can define it as your main class in Run Configuration of eclise.
In Arguments give
<your batch-job-xml-file><space><your Job (in your case its simpleJob)>
You dont need maven here too. Just set classpath of required libraries.
The sample Spring Batch football project reads from a CSV file and write into a database using JDBC.
You can find the details here, and the download links are at the top of that page.
As far as Maven goes, you don't need it, but you'll have to manually include all the required libraries on your classpath. If you're running it through Eclipse, trigger the CommandLineJobRunner.
Related
For config tow datasource, I config tow MapperScannerConfigurer in my .xml:
<bean id="hybirdMapperScannerConfigurer" class="org.mybatis.spring.mapper.MapperScannerConfigurer">
<property name="basePackage" value="com.XXX.A.hymapper" />
<property name="sqlSessionFactoryBeanName" value="hybirdSqlSessionFactory"/>
</bean>
<bean id="mysqlMapperScannerConfigurer" class="org.mybatis.spring.mapper.MapperScannerConfigurer">
<property name="basePackage" value="com.XXX.B.mysmapper" />
<property name="sqlSessionFactoryBeanName" value="mysqlSessionFactory"/>
</bean>
<bean id="mysqlSessionFactory" name="mysqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
<property name="dataSource" ref="mysqlDataSource" />
<property name="configLocation" value="classpath:mybatis-config.xml" />
</bean>
<bean id="hybirdSqlSessionFactory" name="hybirdSqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
<property name="dataSource" ref="hybirdDataSource" />
<property name="configLocation" value="classpath:mybatis-config.xml" />
</bean>
But when I run my app, error comes out:
So I find the source code about sqlSessionTemplateBeanName in MapperScannerConfigurer, and it shows:
It seems that my usage is not wrong, so what can I do to sovle this problem when config multi datasource with xml file and where is my mistake?
thx~
I am using ignoreUnresolvablePlaceholders in my spring context.
as below.
<bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>classpath:messaging.properties</value>
</list>
</property>
<property name="ignoreResourceNotFound" value="false" />
<property name="ignoreUnresolvablePlaceholders" value="true" />
</bean>
event setting ignoreUnresolvablePlaceholders to true its not ignoring my spring bean being injecting.
i am passing amq.topic= in property file for
<bean id="messageTopic1" class="org.apache.activemq.command.ActiveMQTopic">
<constructor-arg value="${amq.topic}" />
</bean>
But its not ignoring this bean for being injecting.
below is my Spring-context.xml amd property file.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.1.xsd">
<bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>classpath:messaging.properties</value>
</list>
</property>
<property name="ignoreResourceNotFound" value="false" />
<property name="ignoreUnresolvablePlaceholders" value="true" />
</bean>
<!-- Active MQ Broker Configuration Details -->
<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="${amq.url}" />
</bean>
<bean id="messageQueue1" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="${amq.queue}" />
</bean>
<bean id="messageTopic1" class="org.apache.activemq.command.ActiveMQTopic">
<constructor-arg value="${amq.topic}" />
</bean>
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="connectionFactory" />
</bean>
<bean id="activeMQ" class="com.isc.common.messaging.AmqUtilityHelper">
<property name="destination" ref="messageQueue1" />
<property name="jmsTemplate" ref="jmsTemplate" />
</bean>
<!-- <bean id="activeMQ1" class="com.isc.common.messaging.AmqUtilityHelper">
<property name="destination" ref="messageTopic1" />
<property name="jmsTemplate" ref="jmsTemplate" />
</bean> -->
<bean id="messageBroker" class="com.isc.common.messaging.AmqUtilityHelper">
<property name="activeBroker" value="${active.broker}" />
</bean>
</beans>
And Below is the property file that i am loading.
#AMQ/Solace properties production:
# uncomment the borker to use
active.broker=activeMQ
#active.broker=solace
#active.broker=activeMQ
#AMQ Broker Properties
amq.url=failover:(tcp://localhost:61616)??initialReconnectDelay=2000&maxReconnectAttempts=5
amq.queue=messageQueue1
amq.topic=
#Solace Broker Properties
solace.url=smf://192.168.56.101:55555
solace.userName=spring_user#Solace_Spring_VPN
solace.passWord=spring_password
solace.jndiName=JNDI/CF/spring1
solace.queue=JNDI/Q/requests
#solace.topic=JNDI/topic1
By default, ApplicationContext implementations eagerly create and
configure all singleton beans as part of the initialization process.
Generally, this pre-instantiation is desirable, because errors in the
configuration or surrounding environment are discovered immediately,
as opposed to hours or even days later. When this behavior is not
desirable, you can prevent pre-instantiation of a singleton bean by
marking the bean definition as lazy-initialized. A lazy-initialized
bean tells the IoC container to create a bean instance when it is
first requested, rather than at startup.
Can you please try to lazily initialize your bean using below construct.
lazy-init="true"
<bean id="messageTopic1" class="org.apache.activemq.command.ActiveMQTopic"
lazy-init="true">
<constructor-arg value="${amq.topic}" />
</bean>
My Requirement is I need to have two datasource connected to Spring Batch Application.
1) One for Spring Batch Jobs and Executions storing
2) One for Business Data Stroing, Processing and Retreiving.
I know that there are lot of solutions for achieving this. But I have achieved by setting the second datasource as primary. The problem is the second datasource is not coming under transaction scope instead it is committing for each sql statement executing expecially through jdbctemplate.
As I can't able to edit my question. I am writing another Post in detail
My Requirement is I need to have two datasource connected to Spring Batch Application.
1) One for Spring Batch Jobs and Executions storing
2) One for Business Data Stroing, Processing and Retreiving.
In env-context.xml I have following configuration
<!-- Enable annotations-->
<context:annotation-config/>
<bean primary="true" id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:/DB2XADS"/>
</bean>
<!-- Creating TransactionManager Bean, since JDBC we are creating of type
DataSourceTransactionManager -->
<bean id="transactionManager" primary="true"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<!-- jdbcTemplate uses dataSource -->
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="batchTransactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager"/>
<bean id="transactionTemplate"
class="org.springframework.transaction.support.TransactionTemplate">
<property name="transactionManager" ref="transactionManager" />
</bean>
In override-context.xml I have the following code
<tx:annotation-driven transaction-manager="transactionManager" />
<!-- jdbcTemplate uses dataSource -->
<bean id="batchDataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:/MySqlDS"/>
</bean>
<bean class="com.honda.pddabulk.utility.MyBatchConfigurer">
<property name="dataSource" ref="batchDataSource" />
</bean>
<!-- Use this to set additional properties on beans at run time -->
<bean id="placeholderProperties"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>classpath:/org/springframework/batch/admin/bootstrap/batch.properties
</value>
<value>classpath:/batch/batch-mysql.properties</value>
<value>classpath:log4j.properties</value>
</list>
</property>
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/>
<property name="ignoreResourceNotFound" value="true"/>
<property name="ignoreUnresolvablePlaceholders" value="true"/>
<property name="order" value="1"/>
</bean>
<!-- Overrider job repository -->
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
<property name="databaseType" value="mysql"/>
<property name="dataSource" ref="batchDataSource"/>
<property name="tablePrefix" value="${batch.table.prefix}"/>
<property name="maxVarCharLength" value="2000"/>
<property name="isolationLevelForCreate" value="ISOLATION_SERIALIZABLE"/>
<property name="transactionManager" ref="batchTransactionManager"/>
</bean>
<!-- Override job service -->
<bean id="jobService" class="org.springframework.batch.admin.service.SimpleJobServiceFactoryBean">
<property name="tablePrefix" value="${batch.table.prefix}"/>
<property name="jobRepository" ref="jobRepository"/>
<property name="jobLauncher" ref="jobLauncher"/>
<property name="jobLocator" ref="jobRegistry"/>
<property name="dataSource" ref="batchDataSource"/>
</bean>
<!-- Override job launcher -->
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor" ref="jobLauncherTaskExecutor" />
</bean>
<task:executor id="jobLauncherTaskExecutor" pool-size="21" rejection-policy="ABORT" />
<!-- Override job explorer -->
<bean id="jobExplorer"
class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean">
<property name="tablePrefix" value="${batch.table.prefix}"/>
<property name="dataSource" ref="batchDataSource"/>
</bean>
In job-config.xml I have the following code
<context:component-scan base-package="com.honda.*">
<context:exclude-filter type="regex"
expression="com.honda.pddabulk.utility.MyBatch*" />
</context:component-scan>
I have the custom Batch configurer set. Now the problem is when I try to execute queries with jdbctemplate for update and insert it is not under transaction which means #Transactional is not working.
Rather commit is happening for each method call. The example is
#Transactional
public void checkInsertion() throws Exception{
try{
jdbcTemplate.update("INSERT INTO TABLE_NAME(COLUMN1, COLUMN2) VALUES( 'A','AF' );
throw new PddaException("custom error");
}catch(Exception ex){
int count=jdbcTemplate.update("ROLLBACK");
log.info("DATA HAS BEEN ROLLBACKED SUCCESSFULLY... "+count);
throw ex;
}
}
In the above code I am trying to insert data and immediately I am also throwing a exception which means the insert should happen but commit will not. So we will not be able to see any data but unfortunately the commit is happening. Please some one help
Is there a way to prevent ResourcePropertySource from throwing java.io.FileNotFoundException
<bean class="org.springframework.context.support.PropertySourcesPlaceholderConfigurer">
<property name="ignoreResourceNotFound" value="true" />
<property name="ignoreUnresolvablePlaceholders" value="true" />
<property name="propertySources">
<list>
<bean class="org.springframework.core.io.support.ResourcePropertySource">
<constructor-arg value="file:#{systemProperties['user.home']}/user.properties"/>
</bean>
<bean class="org.springframework.core.io.support.ResourcePropertySource">
<constructor-arg value="classpath:project.properties"/>
</bean>
</list>
</property>
</bean>
I'd like t to fallback to the project.properties if the user.properties file if not found
Why so complex.
<context:property-placeholder location="${user.home}/user.properties,classpath:project.properties" ignore-resource-not-found="true" />
That should be all you need. No need for EL or directly use a ResourcePropertySource.
Problem: I want to make the below bean definitions (specified in aaplicationContext.xml) optional for "org.springframework.web.context.ContextLoaderListener". If i am not providing the "emsPropLocation" context parameter correctly, tomcat web container is not able to initialized properly and it is obvious reason. Is there any way to make it optional?
appicationContext.xml:
<bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreResourceNotFound" value="false"/>
<property name="location" value="file:/#{contextParameters.emsPropLocation}" />
</bean>
<!-- TIBCO Connection Factory Bean -->
<bean id="tibcoConnectionFactory" class="com.tibco.tibjms.TibjmsConnectionFactory">
<constructor-arg value="${emsServerURL}"/>
<property name="userName" value="${emsUserName}"/>
<property name="userPassword" value="${emsPassword}"/>
<property name="connAttemptCount" value="${connAttemptCount}"/>
<property name="connAttemptDelay" value="${connAttemptDelay}"/>
<property name="connAttemptTimeout" value="${connAttemptTimeout}"/>
<property name="reconnAttemptCount" value="${reconnAttemptCount}"/>
<property name="reconnAttemptDelay" value="${reconnAttemptDelay}"/>
<property name="reconnAttemptTimeout" value="${reconnAttemptTimeout}"/>
</bean>
<!-- bean id="tibcoUtil" class="com.nr.ns.upload.TibcoUtil" scope="singleton">
<constructor-arg value="true"/>
</bean-->
<bean id="jmsExceptionListener" class="com.nr.ns.upload.LogMsgExceptionListener"/>
<!-- Spring CachingConnectionFactory Bean -->
<bean id="connectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<constructor-arg ref="tibcoConnectionFactory"/>
<property name="reconnectOnException" value="${reconnectOnException}"/>
<property name="sessionCacheSize" value="${sessionCacheSize}"/>
<property name="exceptionListener" ref="jmsExceptionListener"/>
</bean>
<!-- JMSTemplate Bean -->
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<constructor-arg ref="connectionFactory"/>
<property name="receiveTimeout" value="${receiveTimeout}"/>
<property name="deliveryMode" value="${deliveryMode}"/>
</bean>
We are keeping WAR file outside tomcat and to make it happen we have "app.xml" file inside TOMCAT_HOME/conf/Catalina/localhost.
app.xml:
<Context path="/app"
docBase="/abc/ccp/app.war"
reloadable="true"
unpackWAR="false">
<Parameter name="emsPropLocation"
value="/xyz/config/EMSServerConf.properties"
override="false"/>
</Context>
have a try to change the ignoreResourceNotFound property of your propertyConfigurer to true.
If contextParameters.emsPropLocation is not set, this will default to what is afer the colon.
<property name="location" value="file:/#{contextParameters.emsPropLocation:/xyz/config/EMSServerConf.properties}" />