Spring batch - Using mybatis pagination on union operation - spring

Question

Is there any better way of handling mybatis pagination while using union query?
Do not consider this as a strong use case but just representing my actual problem.
I need to do union in order to get records from 2 different tables.
Here the problem is if I set the pageSize as 100 for example, if 10 students have 20 records each, then I get only 100 records, even though there are 200 records. And in below example class, when I print the number of records that each student has, I will not see all records.
For Example -
<code>
with student AS (
select * from std (
select studentId, name, class, ROW_NUMBER() over (order by studentId) as paginationRank from Student
)std
where paginationRank > #{ _skipRows} and paginationRank <= ( #{_pageSize} *
(#{_page}+1))
)
select student.studentId, attendanceRegfields......Creditsfields....
from student left outer join
( select ..... from attendanceReg
union all
select .... from Credits ) all_records
on all_records.studentId = student.studentId
</code>
In my item writer, if i get all student records
<code>
class MyItemWriter extends ItemWriter<Student>
{
write(List<student> studentRecords){
Map<String, List<Student>> studentRecordsMap =
studentRecords.stream().collect(groupby(e-> e.getStudentId()));
studentRecordsMap .forEach((key, studentRecords) -> process(stuedntRecords);
}
process(List<Student> studentRecords){
// here I am processing all records
}
}
</code>

Answer 1

Each record of the result of your query should represent an item (a Student in your case). Your item reader should be able to return a complete student item with all its child records (apparently Credit records from your query). Otherwise pagination will not return the correct results for obvious reasons.
What you need in your case is the Driving Query Pattern: You reader can read only students (without child records) and then a processor would complete each student with child records (basically the result of the union query for the current item). With this approach, pagination will work only on students regardless of how many child records each item has.
Hope this helps.