Failed to convert property value of type 'java.lang.String' to required type in Spring Batch - spring

Question

I am working on a Spring Batch requirement. I am writing a converter class as a utility class in a separate util package in my project. I am reading the CSV file writing to MySQL database.
But I am facing issues like rejected value for field?
Domain Class
public class Customer implements Serializable {
private Integer id_type;
private String id_number;
private String customer_name;
private String email_address;
private LocalDate birthday;
private String citizenship;
private String address;
private Long msisdn;
private LocalDateTime kyc_date;
private String kyc_level;
private String goalscore;
private String mobile_network;
}
Utility Class
public final class StringToLocalDateConversion {
private StringToLocalDateConversion() {
}
static ConversionService createLocalDateConversionServicve() {
DefaultConversionService stringToLocalDateconversionService = new DefaultConversionService();
DefaultConversionService.addDefaultConverters(stringToLocalDateconversionService);
stringToLocalDateconversionService.addConverter(new Converter<String, LocalDate>() {
#Override
public LocalDate convert(String text) {
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-mm-dd");
return LocalDate.parse(text, formatter);
}
});
return stringToLocalDateconversionService;
}
}
public final class StringToLocalDateTimeConversion {
public StringToLocalDateTimeConversion() {
}
static ConversionService stringToLocalDateTimeConversionService() {
DefaultConversionService stringToLocalDateTimeConversion = new DefaultConversionService();
DefaultConversionService.addDefaultConverters(stringToLocalDateTimeConversion);
stringToLocalDateTimeConversion.addConverter(new Converter<String, LocalDateTime>() {
#Override
public LocalDateTime convert(String source) {
DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("yyyy-mm-dd hh:mm:ss");
return LocalDateTime.parse(source, dateTimeFormatter);
}
});
return stringToLocalDateTimeConversion;
}
}
BatchConfiguration Class
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
public StepBuilderFactory stepBuilderFactory;
#Value("classPath:/data/gcash.csv")
private Resource inputResource;
#Autowired
public DataSource dataSource;
#Bean
public Job readCSVFilesJob() {
return jobBuilderFactory.get("readCSVFilesJob").incrementer(new RunIdIncrementer()).start(step1()).build();
}
#Bean
public Step step1() {
return stepBuilderFactory.get("step1").<Customer, Customer>chunk(10).reader(itemReader()).processor(processor())
.writer(writer()).build();
}
/*
* #Bean public DataSource dataSource() { final DriverManagerDataSource
* dataSource = new DriverManagerDataSource();
* dataSource.setDriverClassName("com.mysql.jdbc.Driver");
* dataSource.setUrl("jdbc:mysql://localhost:3306/springbatch");
* dataSource.setUsername("root"); dataSource.setPassword("123456");
*
* return dataSource; }
*/
#Bean
public ItemReader<Customer> itemReader() {
FlatFileItemReader<Customer> customerItemReader = new FlatFileItemReader<>();
customerItemReader.setName("CUSTOMER_READER");
customerItemReader.setLineMapper(linemapper());
customerItemReader.setLinesToSkip(1);
customerItemReader.setResource(inputResource);
return customerItemReader;
}
#Bean
public LineMapper<Customer> linemapper() {
DefaultLineMapper<Customer> linemapper = new DefaultLineMapper<>();
final DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
tokenizer.setDelimiter(";");
tokenizer.setStrict(false);
tokenizer.setNames(new String[] { "id_type", "id_number", "customer_name", "email_address", "birthday",
"citizenship", "address", "msisdn", "kyc_date", "kyc_level", "goalscore", "mobile_network" });
linemapper.setLineTokenizer(tokenizer);
BeanWrapperFieldSetMapper<Customer> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
fieldSetMapper.setTargetType(Customer.class);
ConversionService localDateConversionService = StringToLocalDateConversion.createLocalDateConversionServicve();
ConversionService localDateTimeConversionService = StringToLocalDateTimeConversion
.stringToLocalDateTimeConversionService();
fieldSetMapper.setConversionService(localDateConversionService);
fieldSetMapper.setConversionService(localDateTimeConversionService);
linemapper.setFieldSetMapper(fieldSetMapper);
return linemapper;
}
#Bean
public CustomerItemProcessor processor() {
return new CustomerItemProcessor();
}
#Bean
public JdbcBatchItemWriter<Customer> writer() {
JdbcBatchItemWriter<Customer> writer = new JdbcBatchItemWriter<>();
writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<Customer>());
writer.setSql(
"INSERT INTO people (id_type, id_number,customer_name,email_address,birthday,citizenship,address,msisdn,kyc_date,kyc_level,goalscore,mobile_network) VALUES (:id_type, :id_number, :customer_name, :email_address, :birthday, :citizenship, :address, :msisdn, :kyc_date, :goalscore, :mobile_network)");
writer.setDataSource(this.dataSource);
return writer;
}
}
StackTrace
org.springframework.batch.item.file.FlatFileParseException: Parsing error at line: 2 in resource=[URL [classpath:/data/gcash.csv]], input=["LISONDRA, MARIA MINI, JUCOM",MJLISONDRA71#GMAIL.COM,1971-02-12,FILIPINO,2,06-1401967-8,"M L QUEZON CABANCALAN, QUEZON, MANDAUE",9052100646,2019-10-10 11:45:18,FULL KYC,525_549,Globe Prepaid]
at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:189) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:93) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
at org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:99) ~[spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.step.item.SimpleChunkProvider.read(SimpleChunkProvider.java:180) ~[spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:126) ~[spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:375) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:145) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
at org.springframework.batch.core.step.item.SimpleChunkProvider.provide(SimpleChunkProvider.java:118) ~[spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:71) ~[spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:407) ~[spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:331) ~[spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.3.4.jar:5.3.4]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:273) ~[spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:82) ~[spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:375) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:145) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:258) ~[spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:208) ~[spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:411) [spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:136) [spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) [spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:147) [spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.3.4.jar:5.3.4]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140) [spring-batch-core-4.3.1.jar:4.3.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_271]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_271]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_271]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_271]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) [spring-aop-5.3.4.jar:5.3.4]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) [spring-aop-5.3.4.jar:5.3.4]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.3.4.jar:5.3.4]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:128) [spring-batch-core-4.3.1.jar:4.3.1]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.3.4.jar:5.3.4]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) [spring-aop-5.3.4.jar:5.3.4]
at com.sun.proxy.$Proxy104.run(Unknown Source) [na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:199) [spring-boot-autoconfigure-2.4.3.jar:2.4.3]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:173) [spring-boot-autoconfigure-2.4.3.jar:2.4.3]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:160) [spring-boot-autoconfigure-2.4.3.jar:2.4.3]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:155) [spring-boot-autoconfigure-2.4.3.jar:2.4.3]
at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:150) [spring-boot-autoconfigure-2.4.3.jar:2.4.3]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:795) [spring-boot-2.4.3.jar:2.4.3]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:785) [spring-boot-2.4.3.jar:2.4.3]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:333) [spring-boot-2.4.3.jar:2.4.3]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1311) [spring-boot-2.4.3.jar:2.4.3]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1300) [spring-boot-2.4.3.jar:2.4.3]
at com.gcash.milo.GCashMiloApplication.main(GCashMiloApplication.java:10) [classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_271]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_271]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_271]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_271]
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49) [spring-boot-devtools-2.4.3.jar:2.4.3]
Caused by: org.springframework.validation.BindException: org.springframework.validation.BeanPropertyBindingResult: 3 errors
Field error in object 'target' on field 'id_type': rejected value ["LISONDRA, MARIA MINI, JUCOM",MJLISONDRA71#GMAIL.COM,1971-02-12,FILIPINO,2,06-1401967-8,"M L QUEZON CABANCALAN, QUEZON, MANDAUE",9052100646,2019-10-10 11:45:18,FULL KYC,525_549,Globe Prepaid]; codes [typeMismatch.target.id_type,typeMismatch.id_type,typeMismatch.java.lang.Integer,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [target.id_type,id_type]; arguments []; default message [id_type]]; default message [Failed to convert property value of type 'java.lang.String' to required type 'java.lang.Integer' for property 'id_type'; nested exception is java.lang.NumberFormatException: For input string: ""LISONDRA,MARIAMINI,JUCOM",MJLISONDRA71#GMAIL.COM,1971-02-12,FILIPINO,2,06-1401967-8,"MLQUEZONCABANCALAN,QUEZON,MANDAUE",9052100646,2019-10-1011:45:18,FULLKYC,525_549,GlobePrepaid"]
Field error in object 'target' on field 'kyc_date': rejected value []; codes [typeMismatch.target.kyc_date,typeMismatch.kyc_date,typeMismatch.java.time.LocalDateTime,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [target.kyc_date,kyc_date]; arguments []; default message [kyc_date]]; default message [Failed to convert property value of type 'java.lang.String' to required type 'java.time.LocalDateTime' for property 'kyc_date'; nested exception is org.springframework.core.convert.ConversionFailedException: Failed to convert from type [java.lang.String] to type [java.time.LocalDateTime] for value ''; nested exception is java.time.format.DateTimeParseException: Text '' could not be parsed at index 0]
Field error in object 'target' on field 'birthday': rejected value []; codes [typeMismatch.target.birthday,typeMismatch.birthday,typeMismatch.java.time.LocalDate,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [target.birthday,birthday]; arguments []; default message [birthday]]; default message [Failed to convert property value of type 'java.lang.String' to required type 'java.time.LocalDate' for property 'birthday'; nested exception is java.lang.IllegalStateException: Cannot convert value of type 'java.lang.String' to required type 'java.time.LocalDate' for property 'birthday': no matching editors or conversion strategy found]
at org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper.mapFieldSet(BeanWrapperFieldSetMapper.java:201) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
at org.springframework.batch.item.file.mapping.DefaultLineMapper.mapLine(DefaultLineMapper.java:43) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:185) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
... 53 common frames omitted
[2m2021-04-27 22:45:46.085[0;39m [32m INFO[0;39m [35m7456[0;39m [2m---[0;39m [2m[ restartedMain][0;39m [36mo.s.batch.core.step.AbstractStep [0;39m [2m:[0;39m Step: [step1] executed in 25ms
[2m2021-04-27 22:45:46.097[0;39m [32m INFO[0;39m [35m7456[0;39m [2m---[0;39m [2m[ restartedMain][0;39m [36mo.s.b.c.l.support.SimpleJobLauncher [0;39m [2m:[0;39m Job: [SimpleJob: [name=readCSVFilesJob]] completed with the following parameters: [{run.id=17}] and the following status: [FAILED] in 49ms
[2m2021-04-27 22:45:46.097[0;39m [32m INFO[0;39m [35m7456[0;39m [2m---[0;39m [2m[ restartedMain][0;39m [36m.ConditionEvaluationDeltaLoggingListener[0;39m [2m:[0;39m Condition evaluation unchanged
Well if you see my domain class three fields id_type,birthday,kyc_date, have taken them as Integer, LocalDate, LocalDateTime. For this i have also written Converter class. but i am getting three different errors.
Caused by: org.springframework.validation.BindException: org.springframework.validation.BeanPropertyBindingResult: 3 errors
Field error in object 'target' on field 'id_type': rejected value ["LISONDRA, MARIA MINI, JUCOM",MJLISONDRA71#GMAIL.COM,1971-02-12,FILIPINO,2,06-1401967-8,"M L QUEZON CABANCALAN, QUEZON, MANDAUE",9052100646,2019-10-10 11:45:18,FULL KYC,525_549,Globe Prepaid]; codes [typeMismatch.target.id_type,typeMismatch.id_type,typeMismatch.java.lang.Integer,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [target.id_type,id_type]; arguments []; default message [id_type]]; default message [Failed to convert property value of type 'java.lang.String' to required type 'java.lang.Integer' for property 'id_type'; nested exception is java.lang.NumberFormatException: For input string: ""LISONDRA,MARIAMINI,JUCOM",MJLISONDRA71#GMAIL.COM,1971-02-12,FILIPINO,2,06-1401967-8,"MLQUEZONCABANCALAN,QUEZON,MANDAUE",9052100646,2019-10-1011:45:18,FULLKYC,525_549,GlobePrepaid"]
Field error in object 'target' on field 'kyc_date': rejected value []; codes [typeMismatch.target.kyc_date,typeMismatch.kyc_date,typeMismatch.java.time.LocalDateTime,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [target.kyc_date,kyc_date]; arguments []; default message [kyc_date]]; default message [Failed to convert property value of type 'java.lang.String' to required type 'java.time.LocalDateTime' for property 'kyc_date'; nested exception is org.springframework.core.convert.ConversionFailedException: Failed to convert from type [java.lang.String] to type [java.time.LocalDateTime] for value ''; nested exception is java.time.format.DateTimeParseException: Text '' could not be parsed at index 0]
Field error in object 'target' on field 'birthday': rejected value []; codes [typeMismatch.target.birthday,typeMismatch.birthday,typeMismatch.java.time.LocalDate,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [target.birthday,birthday]; arguments []; default message [birthday]]; default message [Failed to convert property value of type 'java.lang.String' to required type 'java.time.LocalDate' for property 'birthday'; nested exception is java.lang.IllegalStateException: Cannot convert value of type 'java.lang.String' to required type 'java.time.LocalDate' for property 'birthday': no matching editors or conversion strategy found]
I am having trouble understanding this error:
[Failed to convert property value of type 'java.lang.String' to required type 'java.lang.Integer' for property 'id_type'
In my model class , id_type is of type Integer, then why it is saying "Failed to convert property value string to Integer?"

Answer 1

This is because you have line where some fields contain the delimiter (,) inside a quoted field:
"LISONDRA, MARIA MINI, JUCOM",MJLISONDRA71#GMAIL.COM,1971-02-12,FILIPINO,2,06-1401967-8,"M L QUEZON CABANCALAN, QUEZON, MANDAUE",9052100646,2019-10-10 11:45:18,FULL KYC,525_549,Globe Prepaid
Since you are using the DelimitedLineTokenizer in which the default quote character is ", the fields "LISONDRA, MARIA MINI, JUCOM" and "M L QUEZON CABANCALAN, QUEZON, MANDAUE" are not considered as a single value. Hence, the entire line as a String is attempted to be converted to the first field which type_id of type Integer. There is an open issue for that here: https://github.com/spring-projects/spring-batch/issues/1822.
You need to use a different quote character or create a custom FieldSetMapper that is able to handle this case.
EDIT: add example for conversion service
conversionService.addConverter(new Converter<String, LocalDate>() { .. });
conversionService.addConverter(new Converter<String, LocalDatTime>() { .. });
fieldSetMapper.setConversionService(conversionService);