How to custom Spring Batch DelimitedLineTokenizer - spring

Question

I have two file types to insert in database.
Format are : aa;bb;cc and aa;bb;cc;dd;ee
This is my FlatFileItemReader :
<bean name="readerContractToAddIntoPRV" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="comments" value="#" />
<property name="linesToSkip" value="1" />
<property name="strict" value="false" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="fieldSetMapper">
<bean class="net.wl.batchs.fieldSetMapper.LineToCreateIntoPrvFieldSetMapper" />
</property>
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="delimiter" value=";"/>
<property name="names" value="aa,bb,cc,dd,ee" />
</bean>
</property>
</bean>
</property>
</bean>
I want a setup that works for both types of files.
For the moment, I have this :
org.springframework.batch.item.file.transform.IncorrectTokenCountException:
Incorrect number of tokens found in record: expected 3 actual 5
Do you have any ideas?
Thank you.
Edit : After correction :
<bean name="readerContractToAddIntoPRV" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="comments" value="#" />
<property name="linesToSkip" value="1" />
<property name="strict" value="false" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper" p:lineTokenizer-ref="multilineFileTokenizer">
<property name="fieldSetMapper">
<bean class="net.wl.batchs.fieldSetMapper.LineToCreateIntoPrvFieldSetMapper" />
</property>
</bean>
</property>
</bean>
<bean id="multilineFileTokenizer" class="org.springframework.batch.item.file.transform.PatternMatchingCompositeLineTokenizer">
<property name="tokenizers">
<map>
<entry key="*;*;*;*;*" value-ref="NSCE_ICCID_MSISDN_LOGIN_PWD"/>
<entry key="*;*;*" value-ref="NSCE_ICCID_MSISDN"/>
<entry key="*" value-ref="headerDefault"/>
</map>
</property>
</bean>
<bean id="parentLineTokenizer" class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer" abstract="true">
<property name="delimiter" value=";"/>
</bean>
<bean id="NSCE_ICCID_MSISDN_LOGIN_PWD" parent="parentLineTokenizer">
<property name="names" value="nsce,iccid,msisdn,login,pwd" />
</bean>
<bean id="NSCE_ICCID_MSISDN" parent="parentLineTokenizer">
<property name="names" value="nsce,iccid,msisdn" />
</bean>
<bean id="headerDefault" parent="parentLineTokenizer">
<property name="names" value="nsce,iccid,msisdn" />
</bean>

Answer 1

The issue isn't your tokenizer. What you'll have to do is use the PatternMatchingCompositeLineMapper (http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapper.html). This will allow you to create a pattern for each line type you have and associate it with the appropriate LineTokenizer.
You can see this LineMapper in action in our samples here: https://github.com/spring-projects/spring-batch/blob/master/spring-batch-samples/src/main/resources/jobs/multilineOrderInputTokenizers.xml