org.springframework.batch.item.file.transform.DelimitedLineTokenizer, by default, "de-escapes" the fields. That means, it replaces two double quotes into one in the value. How can I stop this behavior? I need this because there is a next DelimitedLineTokenizer in the program misinterpreting this single double quote.
How can I do it? Could not get an answer from other posts.
I do have feature one value on multiple lines, because of that I cannot return false from isQuoteCharacter, as mentioned in some other posts. Here is the code.
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="delimiter" ref="delimiter-#{jobExecutionContext['DELIMITER']}" />
<property name="names"
value="#{jobExecutionContext['COLUMNS_NAME_LOOKUP']}" />
</bean>
</property>
<property name="fieldSetMapper">
<bean class="com.tsys.enterprise.converters.flexible.delimited.file.FlexibleDelimiterBasedFileParserFieldSetMapper"
scope="step">
<aop:scoped-proxy/>
<property name="targetType" value="com.tsys.enterprise.converters.flexible.delimited.file.vo.DataHolder"/>
<property name="udfLabel1Label" value="#{jobExecutionContext['UDF1_LABEL']}"/>
</bean>
Related
I started doing some examples using Jasypt and Spring and unfortunately, could not test the decryption successfully.
I'm using Spring 4.3.15 and Jasypt 1.9.3. I think I followed the examples given in the internet carefully, but still see the password is sent encrypted only.
My spring configuration looks like below
<bean id="mydatasource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
</bean>
<bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreUnresolvablePlaceholders" value="true"/>
<property name="ignoreResourceNotFound" value="true" />
<property name="locations">
<list>
<value>classpath:jdbc.properties</value>
</list>
</property>
</bean>
<bean id="encryptedPropertyPlaceHolder" class=**"org.jasypt.spring4.properties.EncryptablePropertyPlaceholderConfigurer"**>
<property name="ignoreUnresolvablePlaceholders" value="true"/>
<property name="ignoreResourceNotFound" value="true" />
<constructor-arg>
<bean class="org.jasypt.encryption.pbe.StandardPBEStringEncryptor">
<property name="config">
<bean class="org.jasypt.encryption.pbe.config.EnvironmentStringPBEConfig">
<property name="algorithm" value="PBEWithMD5AndDES" />
***<!-- <property name="passwordEnvName" value="APP_ENCRYPTION_PASSWORD" /> -->***
<property name="password" value="jasypt" />
</bean>
</property>
</bean>
</constructor-arg>
<property name="locations">
<list>
<value>classpath:jdbc.properties</value>
</list>
</property>
</bean>
and jdbc.properties has values
jdbc.url=<<some url>>
jdbc.username=<<some username>>
jdbc.password=ENC(Y3EEFmt5lEAC96GJlzK9VcETH)
When I ran my testcase, which uses mydatasource as Autowired, I see that password field is set as ENC(Y3EEFmt5lEAC96GJlzK9VcETH). Other fields are properly assigned using propertyConfigurer. But the encrypted password is not decrypted before assigning to datasource.
I'm not sure, what is that I'm missing here ?
Also, as you noticed I used hard coded password in encryptedPropertyPlaceHolder. I have read about APP_ENCRYPTION_PASSWORD, but when trying to set the value using VM arguments in my eclipse as -DAPP_ENCRYPTION_PASSWORD=jasypt, it is not working.
I have a CSV file that doesn't have a fixed number of columns, like this:
col1,col2,col3,col4,col5
val1,val2,val3,val4,val5
column1,column2,column3
value1,value2,value3
Is there any way to read this kind of CSV file with Spring Batch?
I tried to do this:
<bean id="ItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
<!-- Read a csv file -->
<property name="resource" value="classpath:file.csv" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<!-- split it -->
<property name="lineTokenizer">
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="names"
value="col1,col2,col3,col4,col5,column1,column2,column3" />
</bean>
</property>
<property name="fieldSetMapper">
<bean
class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
<property name="prototypeBeanName" value="myBean" />
</bean>
</property>
</bean>
</property>
</bean>
But the result was this error:
You can use the PatternMatchingCompositeLineMapper to delegate to the appropriate LineMapper implementation per line based on a pattern. From there, each of your delegates would use a DelimtedLineTokenizer and a FieldSetMapper to map the line accordingly.
You can read more about this in the documentation here: http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapper.html
AbstractLineTokenizer#setStrict(boolean) in your DelimitedLineTokenizer should do the job.
From the javadoc :
Public setter for the strict flag. If true (the default) then number
of tokens in line must match the number of tokens defined (by Range,
columns, etc.) in LineTokenizer. If false then lines with less tokens
will be tolerated and padded with empty columns, and lines with more
tokens will simply be truncated.
You should change this part of your configuration to:
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="names" value="col1,col2,col3,col4,col5,column1,column2,column3" />
<property name="strict" value="false" />
</bean>
Is there a way to prevent ResourcePropertySource from throwing java.io.FileNotFoundException
<bean class="org.springframework.context.support.PropertySourcesPlaceholderConfigurer">
<property name="ignoreResourceNotFound" value="true" />
<property name="ignoreUnresolvablePlaceholders" value="true" />
<property name="propertySources">
<list>
<bean class="org.springframework.core.io.support.ResourcePropertySource">
<constructor-arg value="file:#{systemProperties['user.home']}/user.properties"/>
</bean>
<bean class="org.springframework.core.io.support.ResourcePropertySource">
<constructor-arg value="classpath:project.properties"/>
</bean>
</list>
</property>
</bean>
I'd like t to fallback to the project.properties if the user.properties file if not found
Why so complex.
<context:property-placeholder location="${user.home}/user.properties,classpath:project.properties" ignore-resource-not-found="true" />
That should be all you need. No need for EL or directly use a ResourcePropertySource.
I wanna make java.util.Date bean. Date must be specified in human readable format (for example as string "02/25/1990").
How do it easiest (without using deprecated Date() constructors)?
<bean id="date" class="java.util.Date" >
<!-- Need 02/25/1990 -->
</bean>
I found easier solution:
<bean id="dateFormat" class="java.text.SimpleDateFormat">
<constructor-arg value="MM/dd/yyyy"/>
</bean>
<bean id="date" factory-bean="dateFormat" factory-method="parse">
<constructor-arg value="02/25/1990"/>
</bean>
Using the non-deprecated Date(long) constructor:
<bean id="date" class="java.util.Date">
<constructor-arg type="java.lang.Long" value="635904000000"/>
</bean>
If you want to use a String instead of the milliseconds you could do the following (uses SimpleDateFormat to produce a Date, while taking advantage of MethodInvokingFactoryBean to invoke it with the specified arguments):
<bean id="dateFormat" class="java.text.SimpleDateFormat">
<constructor-arg value="MM/dd/yyyy" />
</bean>
<bean id="date" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject" ref="dateFormat"/>
<property name="targetMethod" value="parse"/>
<property name="arguments">
<list>
<value type="java.lang.String">02/25/1990</value>
</list>
</property>
</bean>
Of course you need to use the correct pattern for SimpleDateFormat or could end up with unexpected results
I am very new to spring batch. I have requirement in which i have to read a file having a header(Field Names) record and data records
i have to validate 1st record (check the field names matching against set of predefined names)- note that this record need to be skipped- i mean should not be part of items in processor)
read and store rest of the field values to a POJO
if the field 'date' is empty , i need to set the default value as 'xxxx-yy-zz'
i am unable to 1st and 3rd requirement with batch
here is the sample reader XML. please help
<bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="resource" value="classpath:input/import" />
<property name="encoding" value="UTF-8" />
<property name="linesToSkip" value="1" />
<property name="lineMapper" ref="line.mapper"/>
</bean>
<bean id="line.mapper" class="org.springframework.batch.item.file.mapping .DefaultLineMapper">
<property name="lineTokenizer" ref="line.tokenizer"/>
<property name="fieldSetMapper" ref="fieldSet.enity.mapper"/>
</bean>
<bean id="line.tokenizer" class="org.springframework.batch.item.file.transfo rm.DelimitedLineTokenizer">
<property name="delimiter">
<util:constant static-field="org.springframework.batch.item.file.transfo rm.DelimitedLineTokenizer.DELIMITER_TAB"/>
</property>
<property name="names" value="id,date,age " />
<property name="strict" value="false"/>
</bean>
<bean id="fieldSet.enity.mapper" class="org.springframework.batch.item.file.mapping .BeanWrapperFieldSetMapper">
<property name="targetType" value="a.b.myPOJO"/>
<property name="customEditors">
<map>
<entry key="java.util.Date">
<bean class="org.springframework.beans.propertyeditors.C ustomDateEditor">
<constructor-arg>
<bean class="java.text.SimpleDateFormat">
<constructor-arg value="yyyy-mm-dd" />
</bean>
</constructor-arg>
<constructor-arg value="true" />
</bean>
</entry>
</map>
</property>
Create your own custom FieldSetMapper like below
CustomeFieldSetMapper implements FieldSetMapper<a.b.myPOJO> {
#Override
public a.b.myPOJO mapFieldSet(FieldSet fs) {
a.b.myPOJO myPOJO = new a.b.myPOJO();
if(fs.readString("date").isEmpty()){
myPOJO.setDate("xxxx-yy-zz");
}
return a.b.myPOJO;
}
}
You think you should do date set in ItemProcessor.
Also, if <property name="linesToSkip" value="1" /> not fill your requirements - extend FlatFileItemReader and validate first line manually in it.