doubly array from a stream of integers - spring-boot

Question

how to convert Flux<List> into Flux<int[][]>.
I have a Flux<List> -> {1,2,3,.....100} I want to group them by 30 numbers -> [[1,2,3,.....30], [31,32....60],[61.....100]]
I have tried the below approach but was not successful. elements are getting grouped in batches of 5 [ [1,2,3,4,5], [6,7,8,9,10],.....]
Flux<int[][]> groupedData = fluxData.map(x -> {
int outerArraySize = (int) Math.ceil(x.size() / 30) +1;
System.out.println(outerArraySize);
int[][] boxedData = new int[30][outerArraySize];
AtomicInteger innerArray = new AtomicInteger(0);
AtomicInteger outerArray = new AtomicInteger(0);
x.forEach(ids -> {
boxedData[innerArray.get()][outerArray.get()] = ids;
innerArray.getAndIncrement();
if (innerArray.get() == 30) {
innerArray.set(0);
outerArray.getAndIncrement();
}
});

Answer 1

Flux has a useful operator called 'buffer', we can use it to batch the List of Integers.
I have created my 'fluxData' like this, just so I can test my code:
List<Integer> list1 = IntStream.rangeClosed(1, 100).boxed().collect(Collectors.toList());
List<Integer> list2 = IntStream.rangeClosed(1, 40).boxed().collect(Collectors.toList());
List<Integer> list3 = IntStream.rangeClosed(1, 70).boxed().collect(Collectors.toList());
Flux<List<Integer>> fluxData = Flux.just(list1, list2, list3);
Now, we can do the following:
fluxData.map(integersList -> {
List<List<Integer>> batchesList = Flux.fromStream(integersList.stream())
.buffer(30) // This the magic.
.collectList()
.block();
// List<List<Integer>> --> int[][]
int[][] batchesArray = new int[batchesList.size()][];
for(int i = 0;i < batchesArray.length;i++){
batchesArray[i] = new int[batchesList.get(i).size()];
for (int j = 0; j < batchesArray[i].length; j++) {
batchesArray[i][j] = batchesList.get(i).get(j);
}
}
return batchesArray;
})
.subscribe(batchesArray -> System.out.println(Arrays.deepToString(batchesArray)));
Output:
[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], [31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60], [61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90], [91, 92, 93, 94, 95, 96, 97, 98, 99, 100]]
[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], [31, 32, 33, 34, 35, 36, 37, 38, 39, 40]]
[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], [31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60], [61, 62, 63, 64, 65, 66, 67, 68, 69, 70]]