Need to get the schema name from properties file for spring batch application.
Where the schema name is different for dev and prod for MSSQL database.
Job configuration in xml as below
<bean id="dataItemWriter" class="org.springframework.batch.item.database.JdbcBatchItemWriter">
<property name="assertUpdates" value="true" />
<property name="itemPreparedStatementSetter">
<bean class="org.test.batch.model.ItemStatementMapper" />
</property>
<property name="sql" >
<value>
<![CDATA[
INSERT INTO dbo.EMPLOYEE
(PROJECT_NAME
,APP_NAME
,EMPLOYEE_NAME)
values (?,?,?)
]]>
</value>
</property>
<property name="dataSource" ref="dataDataSource" />
</bean>
The schema name "dbo" should be retrieved form proprieties file so that DEV and PROD this can be changed in configuration
I don't see the need to put the value in a CDATA block, there are no special xml characters in your query. Here is an example: https://github.com/spring-projects/spring-batch/blob/8762e3411557aaf887867f8d8594b01127538cb1/spring-batch-core/src/test/resources/org/springframework/batch/core/resource/ListPreparedStatementSetterTests-context.xml#L25. So in your case, it should be something like:
<bean id="dataItemWriter" class="org.springframework.batch.item.database.JdbcBatchItemWriter">
<property name="assertUpdates" value="true" />
<property name="itemPreparedStatementSetter">
<bean class="org.test.batch.model.ItemStatementMapper" />
</property>
<property name="sql" >
<value>
INSERT INTO dbo.EMPLOYEE (PROJECT_NAME ,APP_NAME ,EMPLOYEE_NAME) values (?,?,?)
</value>
</property>
<property name="dataSource" ref="dataDataSource" />
</bean>
Related
can you please help me figure out what is wrong with the DB_VENDOR in MyBatis? I'm using mybatis 3.4.2, mybatis-spring 1.3.1, spring 4.3.6.RELEASE.
Everything, but the vendor dependent queries, works fine. The example query is not vendor dependent, this is only for the sake of this example. If I remove the databaseId attribute from it and remove the duplicated one, this query works.
I need to support Oracle and PostgreSQL and have already checked the metadata getDatabaseProductName() directly within my test scope, getting "Oracle" and "PostgreSQL". So if I got the documentation right, it should work, shouldn't it?
All I get with the databaseId in place is:
org.apache.ibatis.binding.BindingException: Invalid bound statement (not found): mappers.ConfigurationMapper.containsKey
test-mybatis-config.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE configuration PUBLIC "-//mybatis.org//DTD Config 3.0//EN" "Http://mybatis.org/dtd/mybatis-3-config.dtd">
<configuration>
<databaseIdProvider type="DB_VENDOR">
<property name="PostgreSQL" value="postgres"/>
<property name="Oracle" value="oracle"/>
</databaseIdProvider>
<mappers>
<mapper resource="mybatis/ConfigurationMapper.xml"/>
</mappers>
</configuration>
test-context.xml (having another one with PostgreSQL datasource)
<mybatis:scan base-package="mappers" annotation="org.apache.ibatis.annotations.Mapper" factory-ref="sqlSessionFactory"/>
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="oracle.jdbc.OracleDriver"/>
<property name="url" value="jdbc:oracle:thin:#localhost:1521:XE"/>
<property name="username" value="test"/>
<property name="password" value="test"/>
</bean>
<bean class="org.mybatis.spring.SqlSessionFactoryBean" id="sqlSessionFactory">
<property name="dataSource" ref="dataSource"/>
<property name="configLocation" value="classpath:test-mybatis-config.xml"/>
</bean>
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource"/>
</bean>
ConfigurationMapper.xml
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="mappers.ConfigurationMapper">
<select id="containsKey" resultType="string" parameterType="string" databaseId="oracle">
SELECT '1' AS test FROM config_keys WHERE key = #{configKey}
</select>
<select id="containsKey" resultType="string" parameterType="string" databaseId="postgres">
SELECT '1' AS test FROM config_keys WHERE key = #{configKey}
</select>
</mapper>
mapper interface:
#Mapper
public interface ConfigurationMapper {
String containsKey(#Param("configKey") String configKey);
}
Found it. The answer is in the XMLConfigBuilder:
Environment environment = configuration.getEnvironment();
if (environment != null && databaseIdProvider != null) {
String databaseId = databaseIdProvider.getDatabaseId(environment.getDataSource());
configuration.setDatabaseId(databaseId);
}
The environment must be configured in order to make the databaseId work.
Here is how to use multi-db support in spring environment: http://www.mybatis.org/spring/factorybean.html
<bean id="vendorProperties" class="org.springframework.beans.factory.config.PropertiesFactoryBean">
<property name="properties">
<props>
<prop key="SQL Server">sqlserver</prop>
<prop key="DB2">db2</prop>
<prop key="Oracle">oracle</prop>
<prop key="MySQL">mysql</prop>
</props>
</property>
</bean>
<bean id="databaseIdProvider" class="org.apache.ibatis.mapping.VendorDatabaseIdProvider">
<property name="properties" ref="vendorProperties"/>
</bean>
<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="mapperLocations" value="classpath*:sample/config/mappers/**/*.xml" />
<property name="databaseIdProvider" ref="databaseIdProvider"/>
</bean>
can someone please let me know how to use oracle sequence in spring batch writer?.
I tried using custseq.nextval in the insert statement, but its failing.
<bean id="testSSRReader"
class="org.springframework.batch.item.database.JdbcCursorItemReader">
<property name="dataSource" ref="bconnectedDataSource" />
<property name="sql"
value="select CUST_USA_ID , CUST_FIRST_NAME , CUST_LAST_NAME from BL_CUSTOMER fetch first 100 rows only" />
<property name="rowMapper">
<bean class="com.macys.batch.rowmapper.TestSSRRowMapper" />
</property>
</bean>
<bean id="testSSRProcessor" class="com.macys.batch.processor.TestSSRProcessor" />
<bean id="testSSRWriter"
class="org.springframework.batch.item.database.JdbcBatchItemWriter">
<property name="dataSource" ref="ocDataSource" />
<property name="sql">
<value>
<![CDATA[
insert into TESTTABLESSR(CUSTOMER_ID,CUSTOMER_NAME,CITY)
VALUES (custseq.nextval,:firstName,:lastName)
]]>
</value>
</property>
<property name="itemSqlParameterSourceProvider" ref="itemSqlParameterSourceProvider" />
</bean>
I am reading the article http://spring.io/guides/gs/batch-processing/ which explains reading a csv and writing it back to a DB. I want to know how can I read mutiple CSV files say A.csv, B.csv etc and write the content back in respective tables table_A, table_B etc. Please note the content of each csv file should go in a different table.
The basic use case here would be to create as much steps as you have CSV files (since there is no default MultiResourceItemReader implementation).
Each of your step would read a CSV (with a FlatFileItemReader) and write to your database (using JdbcBatchItemWriter or another one of the same kind). Although you will have multiple steps, if your CSV files have the same format (columns, separators), you can factorize the configuration using an AbstractStep. See documentation : http://docs.spring.io/spring-batch/trunk/reference/html/configureStep.html
If not, then you can at least share the common attributes such as LineMapper, ItemPreparedStatementSetter and DataSource.
UPDATE
Here are examples for your readers and writers :
<bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="resource" value="yourFile.csv" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="names" value="column1,column2,column3..." />
</bean>
</property>
<property name="fieldSetMapper">
<bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
<property name="prototypeBeanName" value="yourBeanClass" />
</bean>
</property>
</bean>
</property>
</bean>
<bean id="writer" class="org.springframework.batch.item.database.JdbcBatchItemWriter">
<property name="dataSource" ref="dataSource" />
<property name="sql">
<value>
<![CDATA[
insert into YOUR_TABLE(column1,column2,column3...)
values (:beanField1, :beanField2, :beanField3...)
]]>
</value>
</property>
<property name="itemSqlParameterSourceProvider">
<bean class="org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider" />
</property>
</bean>
UPDATE 2
Here's an example to chain the steps in the job (with Java-based configuration) :
#Bean
public Job job() {
return jobBuilderFactory().get("job").incrementer(new RunIdIncrementer())
.start(step1()).next(step2()).build();
}
I am very new to spring batch. I have requirement in which i have to read a file having a header(Field Names) record and data records
i have to validate 1st record (check the field names matching against set of predefined names)- note that this record need to be skipped- i mean should not be part of items in processor)
read and store rest of the field values to a POJO
if the field 'date' is empty , i need to set the default value as 'xxxx-yy-zz'
i am unable to 1st and 3rd requirement with batch
here is the sample reader XML. please help
<bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="resource" value="classpath:input/import" />
<property name="encoding" value="UTF-8" />
<property name="linesToSkip" value="1" />
<property name="lineMapper" ref="line.mapper"/>
</bean>
<bean id="line.mapper" class="org.springframework.batch.item.file.mapping .DefaultLineMapper">
<property name="lineTokenizer" ref="line.tokenizer"/>
<property name="fieldSetMapper" ref="fieldSet.enity.mapper"/>
</bean>
<bean id="line.tokenizer" class="org.springframework.batch.item.file.transfo rm.DelimitedLineTokenizer">
<property name="delimiter">
<util:constant static-field="org.springframework.batch.item.file.transfo rm.DelimitedLineTokenizer.DELIMITER_TAB"/>
</property>
<property name="names" value="id,date,age " />
<property name="strict" value="false"/>
</bean>
<bean id="fieldSet.enity.mapper" class="org.springframework.batch.item.file.mapping .BeanWrapperFieldSetMapper">
<property name="targetType" value="a.b.myPOJO"/>
<property name="customEditors">
<map>
<entry key="java.util.Date">
<bean class="org.springframework.beans.propertyeditors.C ustomDateEditor">
<constructor-arg>
<bean class="java.text.SimpleDateFormat">
<constructor-arg value="yyyy-mm-dd" />
</bean>
</constructor-arg>
<constructor-arg value="true" />
</bean>
</entry>
</map>
</property>
Create your own custom FieldSetMapper like below
CustomeFieldSetMapper implements FieldSetMapper<a.b.myPOJO> {
#Override
public a.b.myPOJO mapFieldSet(FieldSet fs) {
a.b.myPOJO myPOJO = new a.b.myPOJO();
if(fs.readString("date").isEmpty()){
myPOJO.setDate("xxxx-yy-zz");
}
return a.b.myPOJO;
}
}
You think you should do date set in ItemProcessor.
Also, if <property name="linesToSkip" value="1" /> not fill your requirements - extend FlatFileItemReader and validate first line manually in it.
Within alfresco activiti, could I call a spring bean using the servicetask like :
<serviceTask id="assignApplicationId" name="Assign Application Id"
activiti:expression="${sequenceUtil.getOutboundId(task.id)}"
activiti:resultVariable="OutboundWF_ApplicationNumber"/>
however, in my custom context I declared the sequenceUtil as the following:
<bean id="sequenceUtil" name="sequenceUtil" class="com.tts.mersal.presentation.bean.dialog.util.SequenceUtil">
<property name="searchService">
<ref bean="searchService" />
</property>
<property name="nodeService">
<ref bean="nodeService" />
</property>
<property name="workflowService">
<ref bean="WorkflowService" />
</property>
</bean>
Actually I got the following exception
org.activiti.engine.impl.javax.el.PropertyNotFoundException: Cannot resolve identifier 'sequenceUtil'
at org.activiti.engine.impl.juel.AstIdentifier.eval(AstIdentifier.java:83)
at org.activiti.engine.impl.juel.AstMethod.invoke(AstMethod.java:79)
at org.activiti.engine.impl.juel.AstMethod.eval(AstMethod.java:75)
at org.activiti.engine.impl.juel.AstEval.eval(AstEval.java:50)
at org.activiti.engine.impl.juel.AstNode.getValue(AstNode.java:26)
at org.activiti.engine.impl.juel.TreeValueExpression.getValue(TreeValueExpression.java:114)
at org.activiti.engine.impl.el.JuelExpression.getValue(JuelExpression.java:46)
I got it :)
I have to override the activitiProcessEngineConfiguration bean to include my custom bean within beans property
<!-- -->
<!-- Activiti Process Engine -->
<!-- -->
<bean id="activitiProcessEngineConfiguration"
class="org.alfresco.repo.workflow.activiti.AlfrescoProcessEngineConfiguration">
<property name="dataSource" ref="wrappedDataSource" />
<property name="transactionManager" ref="transactionManager" />
<property name="databaseSchemaUpdate" value="${db.schema.update}" />
<property name="history" value="full" />
<property name="jobExecutorActivate" value="true" />
<!-- Limit the visible beans in expressions -->
<property name="beans">
<map>
<entry key="services" value-ref="ServiceRegistry" />
<entry key="sequenceUtil" value-ref="sequenceUtil" />
</map>
</property>
<property name="customTypes">
<list>
<ref bean="activitiScriptNodeType" />
<ref bean="activitiScriptNodeListType" />
</list>
</property>
<property name="customPreBPMNParseListeners">
<list>
<ref bean="activitiParseListener" />
</list>
</property>
</bean>
There is a much better way to map bean names to el epressions. At least for alfresco 5.2.
Originally activitiProcessEngineConfiguration defined like that:
<bean id="activitiProcessEngineConfiguration" class="org.alfresco.repo.workflow.activiti.AlfrescoProcessEngineConfiguration">
<!-- Limit the visible beans in expressions -->
<property name="beans" ref="activitiBeanRegistry" />
</bean>
Where activitiBeanRegistry defined like that:
<util:map id="activitiBeanRegistry" map-class="java.util.HashMap">
<entry key="services" value-ref="ServiceRegistry" />
</util:map>
So you can easy add your beans with names without touching original activitiProcessEngineConfiguration. Like that:
<bean id="my.activitiBeanRegistry" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean" depends-on="activitiBeanRegistry">
<property name="targetObject">
<ref bean="activitiBeanRegistry" />
</property>
<property name="targetMethod" value="put" />
<property name="arguments">
<list>
<value>sequenceUtil</value>
<ref bean="sequenceUtil" />
</list>
</property>
Open for extension closed for modification :)
Full source can be found here
See also activiti-context.xml