How can I adjust the reading speed for the mongodb? - reactor

Question

Flux.range(1,100).delayElements(Duration.ofSeconds(10)).subscribe(i-> System.out.println(i));
When the Flux emmiter to 10 , I want to delayElement to 1 minute to this subscribe
The background was when I read some data from mongoDB and write to Elasticsearch,
but I want to dynamic to control the reading speed and don't want to exhaust the mongodb resource.
Flux<List<Document>> readFromMongoDB = getFromMongoDB();
product_2014.subscribe(new BaseSubscriber<List<Document>>() {
int counter;
#Override
protected void hookOnSubscribe(Subscription subscription) {
subscription.request(1);
}
#SneakyThrows
#Override
protected void hookOnNext(List<Document> value) {
Thread.sleep(1000);
if (counter == 1) {
counter = 0;
}
else {
counter++;
}
upstream().request(0);
upstream().cancel();
log.info("aaaaaaaaaaa");
}
});
ParallelFlux<List<Document>> getFromMongoDB(String product,
int size,
MongoDatabase mongoDatabase,
int parallel,
Duration duration) {
Publisher<Long> publisher = mongoDatabase.getCollection(product)
.countDocuments();
Mono<Long> count = Mono.from(publisher);
return count.flatMapMany(l -> {
log.info("split counter");
return Flux.range(0, (int) (l / size) + 1);
})
.log().doOnSubscribe(subscription -> {
log.info("doOnSubscribe");
})
.parallel(parallel,1)
.runOn(scheduler)
.doOnNext(integer -> log.info("get page = {}", integer))
.concatMap(page -> {
log.info("page in {}", page);
FindPublisher<Document> limit =
mongoDatabase.getCollection(product)
.find(Document.class)
.skip(page * size)
.limit(size);
Mono<List<Document>> listDocument = Flux.from(limit)
.publishOn(scheduler)
.collectList()
.doOnNext(list -> {
log.info("{} in list",
page);
});
return listDocument;
});
}
Why I can cacel the subscrition.And the Flux still emit the page element?
And how should I do?
I want to paralle to read from mongodb and dynamic control the reading speed.

Answer 1

I only see it like this:
Flux.range(1, 100)
.flatMap(el -> {
Mono<Integer> elMono = Mono.just(el);
if (el <= 10) {
return elMono.delayElement(Duration.ofMinutes(1));
}
else {
return elMono.delayElement(Duration.ofSeconds(10));
}
})
.subscribe(i-> System.out.println(i));
You examine each element and decide how to delay exactly this one.