How do I sequentially and nonparallel loop through an array in RxSwift? - rx-swift

I have a list of objects i need to send to a server and i would like to do this one after the other (not in parallel). After all objects have been sent and there was no error i want to run additional Observables which do different things.
let objects = [1, 2, 3]
let _ = Observable.from(objects).flatMap { object -> Observable<Void> in
return Observable.create { observer in
print("Starting request \(object)")
DispatchQueue.main.asyncAfter(deadline: .now() + 2) { // one request takes ~2sec
print("Request \(object) finished")
observer.onNext(Void())
observer.onCompleted()
}
return Disposables.create()
}
}.flatMap { result -> Observable<Void> in
print("Do something else (but only once)")
return Observable.just(Void())
}.subscribe(
onNext: {
print("Next")
},
onCompleted: {
print("Done")
}
)
What i get is
Starting request 1
Starting request 2
Starting request 3
Request 1 finished
Do something else (but only once)
Next
Request 2 finished
Do something else (but only once)
Next
Request 3 finished
Do something else (but only once)
Next
Done
The whole process ends after 2 sec. What i want is
Starting request 1
Request 1 finished
Starting request 2
Request 2 finished
Starting request 3
Request 3 finished
Do something else (but only once)
Next
Done
The whole sequence should end after 6 seconds (because it's not executed parallel).
I got this to work with a recursive function. But with lots of requests this ends in a deep recursion stack which i would like to avoid.

Use concatMap instead of flatMap in order to send them one at a time instead of all at once. Learn more here:
RxSwift’s Many Faces of FlatMap
Then to do something just once afterwards, use toArray(). Here is a complete example:
let objects = [1, 2, 3]
_ = Observable.from(objects)
.concatMap { object -> Observable<Void> in
return Observable.just(())
.debug("Starting Request \(object)")
.delay(.seconds(2), scheduler: MainScheduler.instance)
.debug("Request \(object) finished")
}
.toArray()
.flatMap { results -> Single<Void> in
print("Do something else (but only once)")
return Single.just(())
}
.subscribe(
onSuccess: { print("done") },
onError: { print("error", $0) }
)

Related

How can I do `onErrorContinue` in rxjs

I am trying to resume main stream same like onErrorContinue in java reactor core
Java example
Flux.range(1, 5)
.flatMap(n -> (n == 3) ? Mono.error(new Throwable("StoppedError")) : Mono.just(n))
.onErrorContinue((throwable, o) -> { System.out.println("error with " + o); })
.subscribe(System.out::println, System.out::println, System.out::println)
// 1
// 2
// error with 3
// 4
// 5
How can i do it in RXJS ? Thank you
Until Now already tried
Observable.range(1, 5)
.flatMap(v => v == 3 ? Observable.throwError(new Error("Stopped")): Observable.of(v))
.subscribe(...logs)
You will need to place the catchError operator on the inner observable. throwing the error and provide an error value, like -1
import { of, throwError, range } from "rxjs";
import { map, flatMap, catchError } from "rxjs/operators";
range(1, 5)
.pipe(
flatMap(v =>
(v == 3 ? throwError(new Error("Stopped")) : of(v)).pipe(
catchError(err => of(-1))
)
)
)
.subscribe(console.log);
Stackblitz: https://stackblitz.com/edit/rxjs-dnz4kx?devtoolsheight=60
In RxJS continuing after an error is like continuing after completing. It doesn't make semantic sense. Once an RxJS stream errors, it is done. That is the observable contract. complete() and error() emissions signal the end of an observable.
Without this in the observable contract, operators like retry could be disastrous. Also, RxJS streams aren't just dataprocessing and there's no standard for errors after transforming a stream with an operator.
Catching errors without streams
consider this non-stream error example:
for(let i = 0; i < 5, i++){
if(i !== 3) console.log(i);
else throw(new Error("Failed on " + i));
}
Here are two ways to handle this error:
1: try-catch outside the loop
try{
for(let i = 0; i < 5, i++){
if(i !== 3) console.log(i);
else throw(new Error("Failed on " + i));
}
}catch(e){
console.log(e);
}
2: try-catch inside the loop
for(let i = 0; i < 5, i++){
try{
if(i !== 3) console.log(i);
else throw(new Error("Failed on " + i));
}catch(e){
console.log(e.message);
}
}
If you wanted to try-catch outside the loop and then continue to print 4 and 5, that would be difficult. You'd need to rework the way errors are handled. They'd need a standard that said "This error was thrown in a loop and caused as a direct result of the current value in the loop, therefore it might make sense to continue the loop with the next value."
When you start nesting function calls inside of loops and so on, it becomes an increasingly bizarre tangle to even understand what "continue" after an error even means.
Consider that number 2 handles the error inside the loop and then lets the loop continue on. For number 2, we can throw a new error. Print a different number. Do nothing. We can even break out of the loop early without throwing an error. We have much more control because of where we are in the control flow of the program.
In number 1 above, once the loop failed, it was done. The control flow went outside of the loop. If you wanted to mimic a continue, you could start a new loop with the next number as its starting value.
function loopToFive(start = 0){
for(let i = start; i < 5, i++){
if(i !== 3) console.log(i);
else throw(new Error("Failed on " + i));
}
}
try{
loopToFive()
}catch(e){
console.log(e.message)
loopToFive(4)
}
This looks like catch and continue, but it's really just catch and recreate.
Catching Errors with Streams
Streams are just loops done asynchronously so that we can abstract away the time between values. Just like the example above, where you handle an error dramatically changes your ability to decide what happens next.
If you want to catch an error and continue, you can either catch it near the source and manage the control flow or you can catch it later and decide if/what part of the stream to re-create to continue.
Streams have built-in operators to retry, so the options are there. You can re-create onErrorContinue() for your case.

rxjs custom retryWhen strategy with auto incremented delay not working properly

I'm trying to create a custom retryWhen strategy which attempts to retry N times with X delay in-between and fail afterwards. To some extent the learnrxjs.io example is exactly what I'm looking for.
Unfortunately there is an issue with this code which I can't seem to figure how to resolve.
In my case, the observable can fail randomly - you can have 2 successful attempts and then 2 unsuccessful attempts. After a while the subscription will automatically complete, because the retryAttempts will exceed the maximum although that has not happened in practice.
To better understand the issue I've created a StackBlitz
The response will be:
Attempt 1: retrying in 1000ms
0
1
Attempt 2: retrying in 2000ms
Attempt 3: retrying in 3000ms
0
1
We are done!
But it should actually be
Attempt 1: retrying in 1000ms
0
1
Attempt 1: retrying in 1000ms <-- notice counter starts from 1
Attempt 2: retrying in 2000ms
0
1
Attempt 1: retrying in 1000ms <-- notice counter starts from 1
0
1
Attempt 1: retrying in 1000ms <-- notice counter starts from 1
Attempt 2: retrying in 2000ms
0
1
... forever
I feel like I'm missing something here.
I think the example given in the docs is written for an Observable that only emits once and then completes, such as an http get. It is assumed that if you want to get more data then you will subscribe again which will reset the counter inside genericRetryStrategy. If, however, you now want to apply this same strategy to a long-running observable whose stream won't complete unless it gives an error (such as you have with interval()), then you'll need to modify genericRetryStrategy() to be told when the counter needs to be reset.
This could be done a number of ways, I have given a simple example in this StackBlitz based off of what you said you were trying to accomplish. Note that I also changed your logic slightly to more match what you said you were trying to do which is have '2 successful attempts and then 2 unsuccessful attempts'. The important bits though are modifying the error object that is thrown into genericRetryStrategy() to communicate the current count of failed attempts so it can react appropriately.
Here is the code copied here for completeness:
import { timer, interval, Observable, throwError } from 'rxjs';
import { map, switchMap, tap, retryWhen, delayWhen, mergeMap, shareReplay, finalize, catchError } from 'rxjs/operators';
console.clear();
interface Err {
status?: number;
msg?: string;
int: number;
}
export const genericRetryStrategy = ({
maxRetryAttempts = 3,
scalingDuration = 1000,
excludedStatusCodes = []
}: {
maxRetryAttempts?: number,
scalingDuration?: number,
excludedStatusCodes?: number[]
} = {}) => (attempts: Observable<any>) => {
return attempts.pipe(
mergeMap((error: Err) => {
// i here does not reset and continues to increment?
const retryAttempt = error.int;
// if maximum number of retries have been met
// or response is a status code we don't wish to retry, throw error
if (
retryAttempt > maxRetryAttempts ||
excludedStatusCodes.find(e => e === error.status)
) {
return throwError(error);
}
console.log(
`Attempt ${retryAttempt}: retrying in ${retryAttempt *
scalingDuration}ms`
);
// retry after 1s, 2s, etc...
return timer(retryAttempt * scalingDuration);
}),
finalize(() => console.log('We are done!'))
);
};
let int = 0;
let err: Err = {int: 0};
//emit value every 1s
interval(1000).pipe(
map((val) => {
if (val > 1) {
//error will be picked up by retryWhen
int++;
err.msg = "equals 1";
err.int = int;
throw err;
}
if (val === 0 && int === 1) {
err.msg = "greater than 2";
err.int = 2;
int=0;
throw err;
}
return val;
}),
retryWhen(genericRetryStrategy({
maxRetryAttempts: 3,
scalingDuration: 1000,
excludedStatusCodes: [],
}))
).subscribe(val => {
console.log(val)
});
To me this is still very imperative, but without understanding the problem you are trying to solve more deeply, I can't currently think of a more declarative approach...

Reset SignalProducer state based on timer and result

I have a SignalProducer producer that asynchronously sends Ints. I can sum the values with
producer.scan(0, +)
Suppose I want to reset the sum to 0 if it is > 10 and no other values have been sent for 1 second. My first attempt looked like this:
producer
.scan(0, +)
.flatMap(.latest) { n -> SignalProducer<Int, NoError> in
if n <= 10 {
return SignalProducer(value: n)
} else {
return SignalProducer.merge([
SignalProducer(value: n),
SignalProducer(value: 0).delay(1, on: QueueScheduler.main)
])
}
}
While this correctly sends 0, it doesn't reset the state in scan. That is, a sequence of 9, 8, long pause, 7 sends 9, 17, 0, 24.
Is there a way to combine these two concept in a way that correctly resets the state?
I would use startWithSignal to gain access to the produced Signal in order to create a reset trigger to merge with the incoming values. I'm using an enum as my value to indicate whether the value coming into the scan is a reset trigger or an Int to accumulate.
enum ValOrReset {
case val(Int)
case reset
}
producer.startWithSignal { signal, _ in
resetTrigger = signal
.debounce(TimeInterval(1.0), on: QueueScheduler.main)
.map { _ in .reset }
signal
.map { val in .val(val) }
.merge(with: resetTrigger)
.scan(0) { (state, next) in
switch next {
case .val(let val):
return state + val
case .reset:
if state > 10 {
return 0
}
else {
return state
}
}
}
.observeValues { val in
print(val)
}
}
The way startWithSignal works is that it ensures no values will come until the closure is finished, which means you can create multiple downstream signals and wire them together without worrying about missing any values, even if the producer sends values synchronously.

RxJs - why Rx.Observable.fromNodeCallack(...)(...).retry() does not retry on error?

I was wondering why the following code (in coffeescript) will not retry as expected.
Rx = require 'rx'
count = 0
functToTest = (cb) ->
console.log "count is", count
count++
if count is 1
cb(new Error('some error'))
else if count is 2
cb(null,2)
else if count is 3
cb(null,3)
else
cb(null,4)
source = Rx.Observable.fromNodeCallback(functToTest)()
onNext = (value) ->
console.log value
onError = (err) ->
console.log err
onCompleted = ->
console.log "done"
retryableSrc = source.retry(3)
retryableSrc.subscribe(onNext, onError, onCompleted)
It will output following messages and quit
count is 0
[Error: some error]
I had thought this is might because fromNodeCallback() return a hot observable. But a test as below show it is NOT.
Rx = require 'rx'
count = 0
functToTest = (cb) ->
console.log "count is", count
count++
if count is 1
cb(new Error('some error'))
else if count is 2
cb(null,2)
else if count is 3
cb(null,3)
else
cb(null,4)
source = Rx.Observable.fromNodeCallback(functToTest)()
onNext = (value) ->
console.log value
onError = (err) ->
console.log err
onCompleted = ->
console.log "done"
retryableSrc = source.retry(3)
setTimeout ( -> ), 1000
If it was a hot observable, the program above should have printed some "count is 0" message. But in reality the program just waits 1 second and quits.
It actually is hot, or goes hot when you first subscribe to it.
Inside of fromNodeCallback is Rx.Observable.create(...).publishLast().refCount() meaning that when you first subscribe it will execute the method, print count then emit an error. The error will be caught downstream by retry, which will resubscribe thrice only to received the cached error, which it will finally emit itself.
You can fix it by using flatMap
ncb = Rx.Observable.fromNodeCallback(functToTest);
source = Rx.Observable.just(ncb).flatMap((fn) -> fn());

Scala stateful actor, recursive calling faster than using vars?

Sample code below. I'm a little curious why MyActor is faster than MyActor2. MyActor recursively calls process/react and keeps state in the function parameters whereas MyActor2 keeps state in vars. MyActor even has the extra overhead of tupling the state but still runs faster. I'm wondering if there is a good explanation for this or if maybe I'm doing something "wrong".
I realize the performance difference is not significant but the fact that it is there and consistent makes me curious what's going on here.
Ignoring the first two runs as warmup, I get:
MyActor:
559
511
544
529
vs.
MyActor2:
647
613
654
610
import scala.actors._
object Const {
val NUM = 100000
val NM1 = NUM - 1
}
trait Send[MessageType] {
def send(msg: MessageType)
}
// Test 1 using recursive calls to maintain state
abstract class StatefulTypedActor[MessageType, StateType](val initialState: StateType) extends Actor with Send[MessageType] {
def process(state: StateType, message: MessageType): StateType
def act = proc(initialState)
def send(message: MessageType) = {
this ! message
}
private def proc(state: StateType) {
react {
case msg: MessageType => proc(process(state, msg))
}
}
}
object MyActor extends StatefulTypedActor[Int, (Int, Long)]((0, 0)) {
override def process(state: (Int, Long), input: Int) = input match {
case 0 =>
(1, System.currentTimeMillis())
case input: Int =>
state match {
case (Const.NM1, start) =>
println((System.currentTimeMillis() - start))
(Const.NUM, start)
case (s, start) =>
(s + 1, start)
}
}
}
// Test 2 using vars to maintain state
object MyActor2 extends Actor with Send[Int] {
private var state = 0
private var strt = 0: Long
def send(message: Int) = {
this ! message
}
def act =
loop {
react {
case 0 =>
state = 1
strt = System.currentTimeMillis()
case input: Int =>
state match {
case Const.NM1 =>
println((System.currentTimeMillis() - strt))
state += 1
case s =>
state += 1
}
}
}
}
// main: Run testing
object TestActors {
def main(args: Array[String]): Unit = {
val a = MyActor
// val a = MyActor2
a.start()
testIt(a)
}
def testIt(a: Send[Int]) {
for (_ <- 0 to 5) {
for (i <- 0 to Const.NUM) {
a send i
}
}
}
}
EDIT: Based on Vasil's response, I removed the loop and tried it again. And then MyActor2 based on vars leapfrogged and now might be around 10% or so faster. So... lesson is: if you are confident that you won't end up with a stack overflowing backlog of messages, and you care to squeeze every little performance out... don't use loop and just call the act() method recursively.
Change for MyActor2:
override def act() =
react {
case 0 =>
state = 1
strt = System.currentTimeMillis()
act()
case input: Int =>
state match {
case Const.NM1 =>
println((System.currentTimeMillis() - strt))
state += 1
case s =>
state += 1
}
act()
}
Such results are caused with the specifics of your benchmark (a lot of small messages that fill the actor's mailbox quicker than it can handle them).
Generally, the workflow of react is following:
Actor scans the mailbox;
If it finds a message, it schedules the execution;
When the scheduling completes, or, when there're no messages in the mailbox, actor suspends (Actor.suspendException is thrown);
In the first case, when the handler finishes to process the message, execution proceeds straight to react method, and, as long as there're lots of messages in the mailbox, actor immediately schedules the next message to execute, and only after that suspends.
In the second case, loop schedules the execution of react in order to prevent a stack overflow (which might be your case with Actor #1, because tail recursion in process is not optimized), and thus, execution doesn't proceed to react immediately, as in the first case. That's where the millis are lost.
UPDATE (taken from here):
Using loop instead of recursive react
effectively doubles the number of
tasks that the thread pool has to
execute in order to accomplish the
same amount of work, which in turn
makes it so any overhead in the
scheduler is far more pronounced when
using loop.
Just a wild stab in the dark. It might be due to the exception thrown by react in order to evacuate the loop. Exception creation is quite heavy. However I don't know how often it do that, but that should be possible to check with a catch and a counter.
The overhead on your test depends heavily on the number of threads that are present (try using only one thread with scala -Dactors.corePoolSize=1!). I'm finding it difficult to figure out exactly where the difference arises; the only real difference is that in one case you use loop and in the other you do not. Loop does do fair bit of work, since it repeatedly creates function objects using "andThen" rather than iterating. I'm not sure whether this is enough to explain the difference, especially in light of the heavy usage by scala.actors.Scheduler$.impl and ExceptionBlob.

Resources