Lazy `SignalProducer` that fetches more data asynchronously when all data has been consumed - frp

Question

Let's imagine that we can fetch a fixed number of messages asynchronously (one request, containing N elements)
func fetchMessages(max: UInt, from: Offset) -> SignalProducer<Message,NoError>
Now, I'd like to turn this into an unbounded SignalProducer that will lazily call fetchMessages when the previous stream completes.
func stream(from: Offset) -> SignalProducer<Message, NoError> {
// challenge is to implement this function
}
An initial idea that could work, but that would still require pre-computing all the ranges would be to genericize the following code
func lazyFetchFrom(from: Offset) -> SignalProducer<Message,NoError> {
return SignalProducer<Message,NoError> { (observer, disposable) in
fetchMessages(from).start(observer)
}
}
let lazyStream =
fetchMessages(1000, from)
.concat(lazyFetchFrom(from + 1000))
.concat(lazyFetchFrom(from + 2000))
.... // could probably be done generically using a flatMap
Now, I'd like to go one step further and evaluate the next call to lazyFetchFrom once the previous values have been consumed. Is that possible?
Thanks
PS: to be clear, my main concern is to provide some sort of backpressure so that the producer doesn't produce too quickly compared to the consumer
Edit: here is my latest attempt at implementing some backpressure. However, when we observeOn the signal, the backpressure disappears, and everything is queued in memory

Answer 1

- (RACSignal *)allContentFromId:(NSInteger)contentId afterDate:(NSDate *)date fetchSize:(NSInteger)fetchSize {
RACSignal *signalNextPagination = [self nextPaginationAllContentFromId:contentId afterDate:date fetchSize:fetchSize];
//in signalNextPagination will be send next fetch size data and send complete only after downloaded all data
//so we used aggregate
return [signalNextPagination aggregateWithStart:#[] reduce:^id(NSArray *before, NSArray *next) {
return [before arrayByAddingObjectsFromArray:next];
}];
}
- (RACSignal *)nextPaginationAllContentFromId:(NSInteger)contentId afterDate:(NSDate *)date fetchSize:(NSInteger)fetchSize {
//command will be send one fetch request
//after recv data in command need try size fetch
//if size eq fetch size, so need repeat command with new offset
RACCommand *command = [[RACCommand alloc] initWithSignalBlock:^RACSignal *(NSDate *date) {
return [self requestContentFromId:contentId afterDate:date limit:fetchSize];
}];
command.allowsConcurrentExecution = YES;
RACSignal *download = [RACSignal createSignal:^RACDisposable *(id<RACSubscriber> subscriber) {
[[command.executionSignals flattenMap:^RACStream *(id value) {
return value;
}] subscribeNext:^(NSArray *datas) {
[subscriber sendNext:datas];
if ([datas count] == fetchSize) {
NSDate *date = [[datas firstObject] pubDate];
[command execute:date];
} else {
[subscriber sendCompleted];
}
} error:^(NSError *error) {
[subscriber sendError:error];
[subscriber sendCompleted];
}];
[command execute:date];
return nil;
}];
return download;
}