Spring batch keeps writing record after exception - spring-boot

Question

I have a spring batch job where I have to check if the id is equal in all the file lines and should skip the lines that contains a different id . What I did is save the first record and then compare the id of each line, if the id is different then throw a Runtime exception, but for some reason spring batch works until it gets the line "to be excluded" and then repeats the writing process by writing all the records on exception .
here's what i mean :
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
2021-06-03 11:12:28.466 ERROR 41416 --- [ scheduling-1] tn.itserv.batch.SkipLinesListener : An error occured while writing the input Force rollback on skippable exception so that skipped item can be located.
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=5598661]
sms [ Id_Campaign=7798661]
My step:
#Bean
public Step loadFiles() throws IOException {
return stepBuilderFactory
.get("step1")
.<FileModelIn, FileModelOut>chunk(100)
.reader(multiResourceItemReader())
.processor(batchProcessor())
.writer(batchWriter())
.faultTolerant()
.skipPolicy(skipLinesListener())
.noRetry(RuntimeException.class)
.noRollback(RuntimeException.class)
.listener(MyStepListner())
.build();
}
SkipPolicy:
public class SkipLinesListener implements SkipPolicy {
private static final int MAX_SKIP_COUNT = 10;
private static final Logger logger = LoggerFactory.getLogger(SkipLinesListener.class);
#Override
public boolean shouldSkip(Throwable t, int skipCount) throws SkipLimitExceededException {
if (t instanceof RuntimeException && skipCount < MAX_SKIP_COUNT )
{ RuntimeException ex=(RuntimeException)t;
logger.error("An error occured while writing the input "+ ex.getMessage());
return true;
}
if (t instanceof FlatFileParseException && skipCount < MAX_SKIP_COUNT ) {
FlatFileParseException ex = (FlatFileParseException) t;
logger.error("An error occured while processing the "+ ex.getInput());
return true;
}
return false;
}
}
I don't know why am getting this behaviour, am I missing something?
am throwing the exception in the itemwriter class
#Override
public void write(List<? extends FileModelOut> items) throws Exception {
List<FCCampaignModel> campaigns=new ArrayList<FCCampaignModel>();
List<sms> smsList=new ArrayList<>();
FCCampaignModel firstLine=cmsDaoProxy.addCampaign(items.get(0).getFcCampaignModel());
for (FileModelOut fileContent : items) {
if (fileContent.getFcCampaignModel().getId_Campaign().equals(firstLine.getId_Campaign()))
{
smsRepository.save(fileContent.getSms());
}
else throw new RuntimeException("different id campaign detected : "+fileContent.getFcCampaignModel().getId_Campaign());
}

Answer 1

Your exception is declared as a skippable exception, so when it is thrown from the item writer, Spring Batch will scan the chunk item by item, ie re-process items one by one, each one in its own transaction.
This is because items are written in chunks (ie in bulk mode), and if an exception occurs during that bulk-write operation, Spring Batch cannot know which item caused the issue, so it will retry them one by one. You can find an example in the samples module: Chunk Scanning Sample.