Spring Batch: Reading a File : if field is empty setting the default value - spring

I am very new to spring batch. I have requirement in which i have to read a file having a header(Field Names) record and data records
i have to validate 1st record (check the field names matching against set of predefined names)- note that this record need to be skipped- i mean should not be part of items in processor)
read and store rest of the field values to a POJO
if the field 'date' is empty , i need to set the default value as 'xxxx-yy-zz'
i am unable to 1st and 3rd requirement with batch
here is the sample reader XML. please help
<bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="resource" value="classpath:input/import" />
<property name="encoding" value="UTF-8" />
<property name="linesToSkip" value="1" />
<property name="lineMapper" ref="line.mapper"/>
</bean>
<bean id="line.mapper" class="org.springframework.batch.item.file.mapping .DefaultLineMapper">
<property name="lineTokenizer" ref="line.tokenizer"/>
<property name="fieldSetMapper" ref="fieldSet.enity.mapper"/>
</bean>
<bean id="line.tokenizer" class="org.springframework.batch.item.file.transfo rm.DelimitedLineTokenizer">
<property name="delimiter">
<util:constant static-field="org.springframework.batch.item.file.transfo rm.DelimitedLineTokenizer.DELIMITER_TAB"/>
</property>
<property name="names" value="id,date,age " />
<property name="strict" value="false"/>
</bean>
<bean id="fieldSet.enity.mapper" class="org.springframework.batch.item.file.mapping .BeanWrapperFieldSetMapper">
<property name="targetType" value="a.b.myPOJO"/>
<property name="customEditors">
<map>
<entry key="java.util.Date">
<bean class="org.springframework.beans.propertyeditors.C ustomDateEditor">
<constructor-arg>
<bean class="java.text.SimpleDateFormat">
<constructor-arg value="yyyy-mm-dd" />
</bean>
</constructor-arg>
<constructor-arg value="true" />
</bean>
</entry>
</map>
</property>

Create your own custom FieldSetMapper like below
CustomeFieldSetMapper implements FieldSetMapper<a.b.myPOJO> {
#Override
public a.b.myPOJO mapFieldSet(FieldSet fs) {
a.b.myPOJO myPOJO = new a.b.myPOJO();
if(fs.readString("date").isEmpty()){
myPOJO.setDate("xxxx-yy-zz");
}
return a.b.myPOJO;
}
}

You think you should do date set in ItemProcessor.
Also, if <property name="linesToSkip" value="1" /> not fill your requirements - extend FlatFileItemReader and validate first line manually in it.

Related

jasypt EncryptablePropertyPlaceholderConfigurer not helping in decrypting the password

I started doing some examples using Jasypt and Spring and unfortunately, could not test the decryption successfully.
I'm using Spring 4.3.15 and Jasypt 1.9.3. I think I followed the examples given in the internet carefully, but still see the password is sent encrypted only.
My spring configuration looks like below
<bean id="mydatasource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
</bean>
<bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreUnresolvablePlaceholders" value="true"/>
<property name="ignoreResourceNotFound" value="true" />
<property name="locations">
<list>
<value>classpath:jdbc.properties</value>
</list>
</property>
</bean>
<bean id="encryptedPropertyPlaceHolder" class=**"org.jasypt.spring4.properties.EncryptablePropertyPlaceholderConfigurer"**>
<property name="ignoreUnresolvablePlaceholders" value="true"/>
<property name="ignoreResourceNotFound" value="true" />
<constructor-arg>
<bean class="org.jasypt.encryption.pbe.StandardPBEStringEncryptor">
<property name="config">
<bean class="org.jasypt.encryption.pbe.config.EnvironmentStringPBEConfig">
<property name="algorithm" value="PBEWithMD5AndDES" />
***<!-- <property name="passwordEnvName" value="APP_ENCRYPTION_PASSWORD" /> -->***
<property name="password" value="jasypt" />
</bean>
</property>
</bean>
</constructor-arg>
<property name="locations">
<list>
<value>classpath:jdbc.properties</value>
</list>
</property>
</bean>
and jdbc.properties has values
jdbc.url=<<some url>>
jdbc.username=<<some username>>
jdbc.password=ENC(Y3EEFmt5lEAC96GJlzK9VcETH)
When I ran my testcase, which uses mydatasource as Autowired, I see that password field is set as ENC(Y3EEFmt5lEAC96GJlzK9VcETH). Other fields are properly assigned using propertyConfigurer. But the encrypted password is not decrypted before assigning to datasource.
I'm not sure, what is that I'm missing here ?
Also, as you noticed I used hard coded password in encryptedPropertyPlaceHolder. I have read about APP_ENCRYPTION_PASSWORD, but when trying to set the value using VM arguments in my eclipse as -DAPP_ENCRYPTION_PASSWORD=jasypt, it is not working.

Call SOAP service asynchronously using Spring

I've to call a SOAP web service asynchronously. Currently, I'm calling it in a synchronous way using Spring webservicetemplate.
Current config is like:
<bean id="interceptedHttpClientBuilder" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject" ref="httpClientBuilder" />
<property name="targetMethod" value="addInterceptorFirst"> </property>
<property name="arguments">
<list>
<bean class="org.springframework.ws.transport.http.HttpComponentsMessageSender.RemoveSoapHeadersInterceptor"/>
</list>
</property>
</bean>
<bean id="requestConfigBuilder" class="org.apache.http.client.config.RequestConfig" factory-method="custom">
<property name="socketTimeout" value="120000" />
</bean>
<bean id="requestConfig" factory-bean="requestConfigBuilder" factory-method="build" />
<bean id="httpClient" factory-bean="httpClientBuilder" factory-method="build" />
<bean id="httpClientBuilder" class="org.apache.http.impl.client.HttpClientBuilder" factory-method="create">
<property name="defaultRequestConfig" ref="requestConfig" />
</bean>
<bean id="messageSender" class="org.springframework.ws.transport.http.HttpComponentsMessageSender">
<constructor-arg ref="httpClient"></constructor-arg>
</bean>
<bean id="jaxb2Marshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
<property name="packagesToScan">
<list><value>...</value></list>
</property>
</bean>
<bean id="wsClientSecurityInterceptor"
class="org.springframework.ws.soap.security.wss4j.Wss4jSecurityInterceptor">
<property name="securementActions" value="UsernameToken" />
<property name="securementUsername"><value>${username}</value></property>
<property name="securementPassword"><value>${password}</value></property>
<property name="securementPasswordType" value="PasswordText" />
</bean>
<bean id="webServiceTemplate" class="org.springframework.ws.client.core.WebServiceTemplate">
<property name="marshaller" ref="jaxb2Marshaller"></property>
<property name="unmarshaller" ref="jaxb2Marshaller"></property>
<property name="defaultUri"><value>${ws.url}</value></property>
<property name="interceptors">
<list>
<ref local="wsClientSecurityInterceptor"/>
</list>
</property>
<property name="messageSender" ref="messageSender"></property>
</bean>
Java call looks like:
MyResponse response = (MyResponse) webServiceTemplate.marshalSendAndReceive(req, new WebServiceMessageCallback() {
public void doWithMessage(WebServiceMessage message) {
((SoapMessage) message).setSoapAction("test");
}
});
May I know how can I change it to call the service asynchronously? Or Do I need to use something else in spring to achieve this?
Not sure why you use spring-integration tag in your question, but if we are here, please, definitely take a look into the #MessagingGateway with the Future<> as a return type: https://docs.spring.io/spring-integration/docs/5.0.1.RELEASE/reference/html/messaging-endpoints-chapter.html#async-gateway
The SOAP WebService can be called via Spring Integration <int-ws:outbound-gateway>: https://docs.spring.io/spring-integration/docs/5.0.1.RELEASE/reference/html/ws.html
The samples are here: https://github.com/spring-projects/spring-integration-samples
To be more clear the code might looks like this:
<int:gateway id="mathService"
service-interface="org.springframework.integration.samples.async.gateway.MathServiceGateway"
default-request-channel="requestChannel"
async-executor="executor"/>
Where that MathServiceGateway is like this:
public interface MathServiceGateway {
Future<Integer> multiplyByTwo(int i);
}
The WS call is simple as well:
<int-ws:outbound-gateway request-channel="requestChannel" uri="http://www.w3schools.com/xml/tempconvert.asmx"/>

How to custom Spring Batch DelimitedLineTokenizer

I have two file types to insert in database.
Format are : aa;bb;cc and aa;bb;cc;dd;ee
This is my FlatFileItemReader :
<bean name="readerContractToAddIntoPRV" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="comments" value="#" />
<property name="linesToSkip" value="1" />
<property name="strict" value="false" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="fieldSetMapper">
<bean class="net.wl.batchs.fieldSetMapper.LineToCreateIntoPrvFieldSetMapper" />
</property>
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="delimiter" value=";"/>
<property name="names" value="aa,bb,cc,dd,ee" />
</bean>
</property>
</bean>
</property>
</bean>
I want a setup that works for both types of files.
For the moment, I have this :
org.springframework.batch.item.file.transform.IncorrectTokenCountException:
Incorrect number of tokens found in record: expected 3 actual 5
Do you have any ideas?
Thank you.
Edit : After correction :
<bean name="readerContractToAddIntoPRV" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="comments" value="#" />
<property name="linesToSkip" value="1" />
<property name="strict" value="false" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper" p:lineTokenizer-ref="multilineFileTokenizer">
<property name="fieldSetMapper">
<bean class="net.wl.batchs.fieldSetMapper.LineToCreateIntoPrvFieldSetMapper" />
</property>
</bean>
</property>
</bean>
<bean id="multilineFileTokenizer" class="org.springframework.batch.item.file.transform.PatternMatchingCompositeLineTokenizer">
<property name="tokenizers">
<map>
<entry key="*;*;*;*;*" value-ref="NSCE_ICCID_MSISDN_LOGIN_PWD"/>
<entry key="*;*;*" value-ref="NSCE_ICCID_MSISDN"/>
<entry key="*" value-ref="headerDefault"/>
</map>
</property>
</bean>
<bean id="parentLineTokenizer" class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer" abstract="true">
<property name="delimiter" value=";"/>
</bean>
<bean id="NSCE_ICCID_MSISDN_LOGIN_PWD" parent="parentLineTokenizer">
<property name="names" value="nsce,iccid,msisdn,login,pwd" />
</bean>
<bean id="NSCE_ICCID_MSISDN" parent="parentLineTokenizer">
<property name="names" value="nsce,iccid,msisdn" />
</bean>
<bean id="headerDefault" parent="parentLineTokenizer">
<property name="names" value="nsce,iccid,msisdn" />
</bean>
The issue isn't your tokenizer. What you'll have to do is use the PatternMatchingCompositeLineMapper (http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapper.html). This will allow you to create a pattern for each line type you have and associate it with the appropriate LineTokenizer.
You can see this LineMapper in action in our samples here: https://github.com/spring-projects/spring-batch/blob/master/spring-batch-samples/src/main/resources/jobs/multilineOrderInputTokenizers.xml

Jackson+Spring3.0.5 custom object mapper

I am having a hard time configuring Jackson on my Spring application. I can get it to work, but it does not seem to accept any kind of configurations. Basically what i'm trying to achieve is have an ObjectMapper that scans for Spring format annotations.
What i am trying is this:
<bean class="org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter">
<property name="messageConverters">
<list>
<bean
class="org.springframework.http.converter.json.MappingJacksonHttpMessageConverter">
<property name="objectMapper" ref="jacksonObjectMapper" />
</bean>
</list>
</property>
</bean>
<bean id="jacksonObjectMapper" class="ro.softwin.cnfp.ConversionServiceAwareObjectMapper" />
<bean
class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
<property name="mediaTypes">
<map>
<entry key="json" value="application/json" />
<entry key="html" value="text/html" />
</map>
</property>
<property name="viewResolvers">
<list>
<bean
class="org.springframework.web.servlet.view.InternalResourceViewResolver">
<property name="viewClass"
value="org.springframework.web.servlet.view.JstlView" />
<property name="prefix" value="/jsp/" />
<property name="suffix" value=".jsp" />
</bean>
</list>
</property>
<property name="defaultViews">
<list>
<bean
class="org.springframework.web.servlet.view.json.MappingJacksonJsonView">
<property name="prefixJson" value="false" />
<property name="objectMapper" value="jacksonObjectMapper" />
</bean>
</list>
</property>
</bean>
The code for the mapper is :
public class ConversionServiceAwareObjectMapper extends ObjectMapper {
#Autowired
public ConversionServiceAwareObjectMapper(
ConversionService conversionService) {
AnnotationIntrospector introspector = AnnotationIntrospector.pair(
new FormatAnnotationIntrospector(conversionService),
DEFAULT_ANNOTATION_INTROSPECTOR);
this.setDeserializationConfig(this.getDeserializationConfig()
.withAnnotationIntrospector(introspector));
this.setSerializationConfig(this.getSerializationConfig()
.withAnnotationIntrospector(introspector).withSerializationInclusion(JsonSerialize.Inclusion.NON_NULL));
}
}
When initializing the server the following error occurs:
java.lang.IllegalStateException: Cannot convert value of type [java.lang.String] to required type [org.codehaus.jackson.map.ObjectMapper] for property 'objectMapper': no matching editors or conversion strategy found
at org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:241)
at org.springframework.beans.BeanWrapperImpl.convertIfNecessary(BeanWrapperImpl.java:470)
... 55 more
I tried switching to a normal view resolver and just having the object mapper in the AnnotationMethodHandler, and that works regarding server start-up, but it just completely ignores any configuration.
I am completely at a loss as to what to do next.
Thanks for any help.
Update :
<property name="objectMapper" value="jacksonObjectMapper" />
with:
<property name="objectMapper" ref="jacksonObjectMapper" />
ref not value.

Spring ldap multiple group bases in authoritiespopulator

Is there any way in Spring to set it up such that an authoritiespopulator will look in more than one location?
<bean id="authoritiesPopulator" class="org.springframework.security.ldap.populator.DefaultLdapAuthoritiesPopulator">
<constructor-arg ref="contextSource" />
<constructor-arg value="CN=Users" />
<property name="groupRoleAttribute" value="CN" />
<property name="searchSubtree" value="true" />
<property name="rolePrefix" value="" />
<property name="convertToUpperCase" value="false" />
</bean>
This is the general idea, but there are also groups in CN=OtherGroups, and right now they don't get loaded (obviously). However, setting the groupsearchbase (the 2nd constructor arg) to value="" results in an error:
Unprocessed Continuation Reference(s); remaining name ''
Any ideas?
Answer: set referral to follow.
<bean id="contextSource" class="org.springframework.security.ldap.DefaultSpringSecurityContextSource">
....
<property name="baseEnvironmentProperties">
<map>
<entry key="java.naming.referral" value="follow" />
</map>
</property>
</bean>

Resources