Using RxJS to remove nested callbacks when things must be done in sequence - rxjs

I need to do one HTTP request after another, but the second one can't start until after the first one has finished because the second needs as a parameter a value returned from the first.
Here it is with a nested callback which is great because it works and it's fairly clear from reading the code what is happening.
this.isLoading = true;
this.firstService.get(this.id)
.subscribe((response: FirstReturnType) => {
this.firstThing = response;
this.secondService.get(this.firstThing.secondId)
.subscribe(
(response: SecondReturnType) => {
this.secondThing = response;
this.isLoading = false;
}
}
The claim I see people making is that nested callbacks are bad and that one should use RxJS to make it better.
However, nobody making these claims has been able to produce a working example. Can you?

Your Code Re-written
Here is some code that has a 1-1 correspondence with your code, but it is flattened
this.isLoading = true;
this.firstService.get(this.id).pipe(
mergeMap((response: FirstReturnType) => {
this.firstThing = response;
return this.secondService.get(response.secondId);
})
).subscribe((response: SecondReturnType) => {
this.secondThing = response;
this.isLoading = false;
});
What this gets right: you're using a higher-order observable operator to map a value emitted by one observable into a new observable that you subscribe to. In this case, mergeMap is subscribing for you and getting rid of your nesting.
For Your Consideration
Consider this. The following is about as clean looking at six service calls (each giving some value to the next one) in a row can get if you're not using a higher-order operator:
this.firstService.getThing("First").subscribe(result1 => {
this.secondService.getThing(result1.value).subscribe(result2 => {
this.thirdService.getThing(result2.value).subscribe(result3 => {
this.fourthService.getThing(result3.value).subscribe(result4 => {
this.fifthService.getThing(result4.value).subscribe(result5 => {
this.sixthService.getThing(result5.value).subscribe(result6 => {
console.log("Result Six is: " + result6.value);
});
});
});
});
});
});
Here's the exact same thing with mergeMap:
this.firstService.getThing("First").pipe(
mergeMap(result1 => this.secondService.getThing(result1.value)),
mergeMap(result2 => this.thirdService.getThing(result2.value)),
mergeMap(result3 => this.fourthService.getThing(result3.value)),
mergeMap(result4 => this.fifthService.getThing(result4.value)),
mergeMap(result5 => this.sixthService.getThing(result5.value)),
).subscribe(result6 => {
console.log("Result Six is: " + result6.value);
});
If that's not enough to convince you, you can lean a little bit more into some functional programming to make this even cleaner (without repetitively naming each result)
const passValueToService = service => result => service.getThing(result.value);
passValueToService(this.firstService)("First").pipe(
mergeMap(passValueToService(this.secondService)),
mergeMap(passValueToService(this.thirdService)),
mergeMap(passValueToService(this.fourthService)),
mergeMap(passValueToService(this.fifthService)),
mergeMap(passValueToService(this.sixthService)),
).subscribe(finalResult => {
console.log("Result Six is: " + finalResult.value);
});
Or why not lean EVEN harder and keep our list of services in an array?
const [firstS, ...restS] = [this.firstService, this.secondService, this.thirdService, this.fourthService, this.fifthService, this.sixthService];
const passValueToService = service => result => service.getThing(result.value);
passValueToService(firstS)("first").pipe(
...restS.map(service => mergeMap(passValueToService(service)))
).subscribe(finalResult => {
console.log("Result Six is: " + finalResult.value);
});
None of these simplifications are very easily done while nesting subscribe calls. But with the help of some functional currying (and the handy RxJS pipe to compose with), you can begin to see that your options expand dramatically.
Understanding concatMap, mergeMap, & switchMap
The Setup
We'll have 3 helper functions as described here:
/****
* Operator: intervalArray
* -----------------------
* Takes arrays emitted by the source and spaces out their
* values by the given interval time in milliseconds
****/
function intervalArray<T>(intervalTime = 1000): OperatorFunction<T[], T> {
return s => s.pipe(
concatMap((v: T[]) => concat(
...v.map((value: T) => EMPTY.pipe(
delay(intervalTime),
startWith(value)
))
))
);
}
/****
* Emit 1, 2, 3, then complete: each 0.5 seconds apart
****/
function n123Stream(): Observable<number> {
return of([1,2,3]).pipe(
intervalArray(500)
);
}
/****
* maps:
* 1 => 10, 11, 12, then complete: each 1 second apart
* 2 => 20, 21, 22, then complete: each 1 second apart
* 3 => 30, 31, 32, then complete: each 1 second apart
****/
function numberToStream(num): Observable<number>{
return of([num*10, num*10+1, num*10+2]).pipe(
intervalArray(1000)
);
}
The above mapping function (numberToStream), takes care of the map part of concatMap, mergeMap, and switchMap
Subscribing to each operator
The following three snippits of code will all have different outputs:
n123Stream().pipe(
concatMap(numberToStream)
).subscribe(console.log);
n123Stream().pipe(
mergeMap(numberToStream)
).subscribe(console.log);
n123Stream().pipe(
switchMap(numberToStream)
).subscribe(console.log);
If you want to run these back-to-back:
concat(
...[concatMap, mergeMap, switchMap].map(
op => n123Stream().pipe(
op(numberToStream),
startWith(`${op.name}: `)
)
)
).subscribe(console.log);
concatMap:
concatMap will not subscribe to the second inner observable until the first one is complete. That means that the number 13 will be emitted before the second observable (starting with the number 20) will be subscribed to.
The output:
10 11 12 20 21 22 30 31 32
All the 10s are before the 20s and all the 20s are before the 30s
mergeMap:
mergeMap will subscribe to the second observable the moment the second value arrives and then to the third observable the moment the third value arrives. It doesn't care about the order of output or anything like that.
The output
10 20 11 30 21 12 31 22 32
The 10s are earlier because they started earlier and the 30s are later because they start later, but there's some interleaving in the middle.
switchMap
switchMap will subscribe to the first observable the moment the first value arrives. It will unsubscribe to the first observable and subscribe to the second observable the moment the second value arrives (and so on).
The output
10 20 30 31 32
Only the final observable ran to completion in this case. The first two only had time to emit their first value before being unsubscribed. Just like concatMap, there is no interleaving and only one inner observable is running at a time, but some emissions are effectively dropped.

You can use switchMap.
this.firstService.get(this.id)
.pipe(
tap((response: FirstReturnType) => this.firstThing = response),
switchMap((response: FirstReturnType) => this.secondService.get(response.secondId)),
tap((response: SecondReturnType) => {
this.secondThing = response;
this.isLoading = false;
})
).subscribe();

Related

How do I create an observable which emits a value from an ngrx selector, then another value after a delay?

Given an ngrx selector:
store.select('count')
I want to create an observable that will emit values emitted by the selector, then emit another specific value after a delay.
Using concat doesn't work as (I assume) the selector doesn't complete, so the 0 is never emitted:
this.count$ = concat(store.select('count'), of(0).pipe(delay(2000)));
Stackblitz: https://stackblitz.com/edit/so-selector-delay?file=src/app/my-counter/my-counter.component.ts
- click 'Increment' button - Current Count should change to 1 then back to 0 after 2 seconds.
If you want to emit the store.select('count') value, then essentially reset it to 0 after not receiving an emission for 2 seconds, you can use a switchMap to create a source that emits two values:
The emitted count
The "default" value of 0 after 2000ms
The trick here is that the second value (0) will NOT be emitted if another store.select('count') value is received, because switchMap will create a new source and "switch" to that:
this.count$ = store.select('count').pipe(
switchMap(count => concat(
of(count),
of(0).pipe(delay(2000))
))
);
Here's a working StackBlitz demo.
It might even be worth creating a custom operator:
this.count$ = this.store.select('count').pipe(
resetAfterDelay(0, 2000)
);
export function resetAfterDelay<T>(defaultValue: T, delayMs: number) {
return switchMap((value: T) => concat(
of(value),
of(defaultValue).pipe(delay(delayMs))
));
}
StackBlitz
Below is an approach using combineLatest and BehaviorSubject
We are hold a value in a subject and create a timer that emits value 0 after 2s. So we have two Observables, one emits immediately and the other after 2s. We combine this two and the effect is a single observable as desired
valueHolderSubject$ = new BehaviorSubject(0);
...
this.count$ = combineLatest([
store.select("count").pipe(
tap(x => this.valueHolderSubject$.next(x)),
tap(() =>
timer(2000).subscribe({
next: () => {
this.valueHolderSubject$.next(0);
}
})
),
map(() => this.valueHolderSubject$.value),
distinctUntilChanged()
),
this.valueHolderSubject$
]).pipe(map(([, x]) => x));
Demo Here
A per my comments on the answer from BizzyBob, I was getting unreliable results. But I refined it to work using:
this.count$ = merge(
store.select('count'),
store.select('count').pipe(delay(2000), map(_ => 0))
);
See stackblitz: https://stackblitz.com/edit/so-selector-delay-merge-working?file=src/app/my-counter/my-counter.component.ts

Observable make request multiple times and collect response together

In database I have 19 users.
In my API, I can get only 5 results in one call.
If I want to get all them, I need to do request 4 times, each time to get 5 users. With start query I will change from which position I want new users to get.
I'm trying to do it in RxJS together with redux-observable.
I have some idea, but maybe my approach is imperative, and RxJS is opposite ideology.
// get users from API and `pipe` them helps me to see actual data and to count length of array
function getUsers(position = 0) {
return ajax.getJSON(`${API}/users?_start=${position}&_limit=5`).
pipe(map(({data}) => ({responseLength: data.length, data})))
}
// here when I got response if array.lenght is equal to 5, I know that I need to do fetch of data again.
// Problem is encountered here: if I do recursion after doing I will get only last result, not both of them,
// if I put my previous result into array, and then recursion result again push in array it become too complicated after
// in userFetchEpic to manipulate with this data
function count(data) {
return data.pipe(
map(item => {
if (item.responseLength === 5) {
count(getUsers(5));
}
return {type: "TEST" , item}
})
)
}
function userFetchEpic(action$) {
return action$
.pipe(
ofType(USER_FETCH),
mergeMap(() => {
return count(getUsers()).pipe(
map(i => i)
)
})
);
}
My code is here just to show what was my way of thinking.
Main problem is in recursion how to save all values together, if I save values in array.
Then I need to loop through array of observables and that sounds complicated in my head. :)
Probably this problem have much easier and better solution.
19 Users with 4 concurrent calls
I've re-arranged your get-users function to generate 4 Ajax calls, run them all concurrently, then flatten the result into one array. This should get you all 19 users in a single array.
function getUsers() {
return forkJoin(
[0,5,10,15].map(position =>
ajax.getJSON(`${API}/users?_start=${position}&_limit=5`)
)
).pipe(
map(resArray => resArray.flatMap(res => res.data))
)
}
function userFetchEpic(action$) {
return action$.pipe(
ofType(USER_FETCH),
mergeMap(_ => getUsers())
);
}
Generalize: Recursively get users
This, again, will return all 19 users, but this time you don't need to know that you have 19 users ahead of time. On the other hand, this makes all its calls in sequence, so I would expect it to be slower.
You'll notice this one is done recursively. You are creating a call stack this way, but so long as you don't have millions of users, it shouldn't be a problem.
function getUsers(position = 0) {
return ajax.getJSON(`${API}/users?_start=${position}&_limit=5`).pipe(
switchMap(({data}) =>
data.length < 5 ?
of(data) :
getUsers(position + 5).pipe(
map(recursiveData => data.concat(recursiveData))
)
})
);
}

Delay for every element with RXJS

I'm using RxViz to simulate different actions that comes every 1 sec. When I try
Rx.Observable.create(obs => {
obs.next([1, 2, 3]); // or could be ['aaa', 'bbbb', 'ccc']
obs.complete();
}).delay(1000);
on https://rxviz.com
or on my own with a console.log
it keeps displaying the three number 1, 2, 3 at the same time
There's a post about this same problem, but none of the answer works for me. I'm using Rx last version 6
How can I create an observable with a delay
[EDIT] The array can contains anything like number, string or any object
If you want to delay each value (by 1 sec for example), you may do something like the following:
Rx.Observable.create(obs => {
obs.next([1, 2, 3]);
obs.complete();
})
.pipe(
// make observable to emit each element of the array (not the whole array)
mergeMap((x: [any]) => from(x)),
// delay each element by 1 sec
concatMap(x => of(x).pipe(delay(1000)))
)
.subscribe(x => console.log(x));
}
Here I did not modify the internals of the observable created by you. Instead, I just take your observable and apply appropriate operations to achieve what you seem to be expecting.
Here is my solution (very clean)
const fakeData = [1,2,3]
loadData$() {
return from(fakeData).pipe(
concatMap(item => of(item).pipe(
delay(1000)
)),
);
}
This one works by modifying a little bit #siva636's answer
Rx.Observable.create(obs => {
obs.next(1);
obs.next(2);
obs.next(3);
obs.complete();
}.concatMap(x=>Rx.Observable.of(x) .delay(1000) )
Here is a succinct way that builds on the other responses.
from([...Array(10).keys()]).pipe(
concatMap(x => of(x).pipe(delay(1000)))
).subscribe(y => console.log(y))
A more RxJs native version would be as follows.
const myInterval = rxjs.interval(1000);
myInterval.pipe(rxjs.operators.take(10)).subscribe(x => console.log(x));
Here, you emit in one observable emission the all array. [1,2,3].
You only delay that one emission by 1000 ms. But the emission is still one.
Even if we emit each value on its own, the delay function will only apply to the first emission. The others will come immediately after:
Rx.Observable.create(obs => {
var arr = [1, 2, 3];
arr.forEach(item => obs.next(item));
obs.complete();
}).delay(1000);
There is no magic in the create constructing function. If we want an emission to come every x time:
We could make an interval that emits those values (taken from learnrxjs)
import { Observable } from 'rxjs/Observable';
/*
Increment value every 1s, emit even numbers.
*/
const evenNumbers = Observable.create(function(observer) {
let value = 0;
const interval = setInterval(() => {
observer.next(value);
value++;
}, 1000);
return () => clearInterval(interval);
});
RxJS v7 supports the operator delayWhen [1], so you could write a simpler code as
import { delayWhen, interval, of } from 'rxjs';
of("John", "von", "Neumman", "János Neumann").pipe(
delayWhen((_, index) => interval(index*1000))
).subscribe(console.log);
Check out a demo on https://stackblitz.com/edit/vgibzv?file=index.ts
It works because it delays the emission of items by 0 seconds, 1000 seconds, 2000 seconds, 3000 seconds, and so on.
Another choice is the operator scan, you make the series from an interval [2].
[1] "RxJS - delayWhen." 16 Dec. 2022, https://rxjs.dev/api/operators/delayWhen
[2] "RxJS - scan." 16 Dec. 2022, rxjs.dev/api/index/function/scan

concatMap() equivalent but async like in mergeMap()

I have an observable myObservable :
let myObservable = Observable.of(2000, 1000)
With concatMap() : TOTAL TIME = 3000 millis, results in the original order.
myObservable.concatMap(v => Rx.Observable.of(v).delay(v))
// concatMap: 2000, concatMap: 1000
With mergeMap() : TOTAL TIME = 2000 millis, results not in the original order.
myObservable.mergeMap(v => Rx.Observable.of(v).delay(v))
// mergeMap: 1000, mergeMap: 2000
I want a way to get the results in the original order like in concatMap, but calling each nested observable asynchronously instead of waiting for the next nested observable to complete :
// --- The behavior that I want ---
myObservable.myCustomMap(v => Rx.Observable.of(v).delay(v))
// myCustomMap: 2000, myCustomMap: 1000
// TOTAL TIME = 2000 millis
Is there an elegant solution?
Edit : I am looking for a solution that also works if the source (myObservable) is asynchronous, not only for this particular synchronous case.
You should use forkJoin for firing all the observables at the same time.
Here's an example without comments:
const { Observable } = Rx;
const obs$ = Observable
.of(3000, 3000, 1000)
.map(x => Observable.of(x).delay(x));
const allObsWithDelay$ = obs$.toArray();
const result$ = allObsWithDelay$
.switchMap(arr => Observable.forkJoin(arr));
result$
.do(console.log)
.subscribe();
And the same with explanation:
const { Observable } = Rx;
// source observable, emitting simple values...
const obs$ = Observable
.of(3000, 3000, 1000)
// ... which are wrapped into a different observable and delayed
.map(x => Observable.of(x).delay(x));
// use a reduce to build an array containing all the observables
const allObsWithDelay$ = obs$.toArray();
const result$ = allObsWithDelay$
// when we receive the array with all the observable
// (so we get one event, with an array of multiple observables)
.switchMap(arr =>
// launch every observable into this array at the same time
Observable.forkJoin(arr)
);
// display the result
result$
.do(console.log)
.subscribe();
With those values: 3000, 3000, 1000 the whole process is taking 3 seconds (the maximum of them as they're fired at the same time)
Working Plunkr: https://plnkr.co/edit/IRgEhdjCmZSTc6hSaVeF?p=preview
Edit 1: Thanks to #PierreCitror for pointing out toArray which is better than scan :)
I would do it like this:
myObservable
.mergeMap((val, i) => Observable.forkJoin(
Observable.of(i),
Observable.of(v).delay(v)
))
.scan((acc, ([i, result])) => {
acc[i] = result;
return acc;
}, {})
.filter(allResults => {
// Whatever goes here
Object.keys(allResults) // list indices of all finished responses
})
This will accumulate all responses in a single object where each response is assigned an index at which it arrived into mergeMap.
Then in filter you can write whatever logic you want that decides whether the current state should be propagated further (eg. you can wait until a certain number of responses arrived or whatever).

Have withLatestFrom wait until all sources have produced one value

I'm making use of the withLatestFrom operator in RxJS in the normal way:
var combined = source1.withLatestFrom(source2, source3);
...to actively collect the most recent emission from source2 and source3 and to emit all three value only when source1 emits.
But I cannot guarantee that source2 or source3 will have produced values before source1 produces a value. Instead I need to wait until all three sources produce at least one value each before letting withLatestFrom do its thing.
The contract needs to be: if source1 emits then combined will always eventually emit when the other sources finally produce. If source1 emits multiple times while waiting for the other sources we can use the latest value and discard the previous values. Edit: as a marble diagram:
--1------------2---- (source)
----a-----b--------- (other1)
------x-----y------- (other2)
------1ax------2by--
--1------------2---- (source)
------a---b--------- (other1)
--x---------y------- (other2)
------1ax------2by--
------1--------2---- (source)
----a-----b--------- (other1)
--x---------y------- (other2)
------1ax------2by--
I can make a custom operator for this, but I want to make sure I'm not missing an obvious way to do this using the vanilla operators. It feels almost like I want combineLatest for the initial emit and then to switch to withLatestFrom from then on but I haven't been able to figure out how to do that.
Edit: Full code example from final solution:
var Dispatcher = new Rx.Subject();
var source1 = Dispatcher.filter(x => x === 'foo');
var source2 = Dispatcher.filter(x => x === 'bar');
var source3 = Dispatcher.filter(x => x === 'baz');
var combined = source1.publish(function(s1) {
return source2.publish(function(s2) {
return source3.publish(function(s3) {
var cL = s1.combineLatest(s2, s3).take(1).do(() => console.log('cL'));
var wLF = s1.skip(1).withLatestFrom(s2, s3).do(() => console.log('wLF'));
return Rx.Observable.merge(cL, wLF);
});
});
});
var sub1 = combined.subscribe(x => console.log('x', x));
// These can arrive in any order
// and we can get multiple values from any one.
Dispatcher.onNext('foo');
Dispatcher.onNext('bar');
Dispatcher.onNext('foo');
Dispatcher.onNext('baz');
// combineLatest triggers once we have all values.
// cL
// x ["foo", "bar", "baz"]
// withLatestFrom takes over from there.
Dispatcher.onNext('foo');
Dispatcher.onNext('bar');
Dispatcher.onNext('foo');
// wLF
// x ["foo", "bar", "baz"]
// wLF
// x ["foo", "bar", "baz"]
I think the answer is more or less as you described, let the first value be a combineLatest, then switch to withLatestFrom. My JS is hazy, but I think it would look something like this:
var selector = function(x,y,z) {};
var combined = Rx.Observable.concat(
source1.combineLatest(source2, source3, selector).take(1),
source1.withLatestFrom(source2, source3, selector)
);
You should probably use publish to avoid multiple subscriptions, so that would look like this:
var combined = source1.publish(function(s1)
{
return source2.publish(function(s2)
{
return source3.publish(function(s3)
{
return Rx.Observable.concat(
s1.combineLatest(s2, s3, selector).take(1),
s1.withLatestFrom(s2, s3, selector)
);
});
});
});
or using arrow functions...
var combined = source1.publish(s1 => source2.publish(s2 => source3.publish(s3 =>
Rx.Observable.concat(
s1.combineLatest(s2, s3, selector).take(1),
s1.withLatestFrom(s2, s3, selector)
)
)));
EDIT:
I see the problem with concat, the withLatestFrom isn't getting the values. I think the following would work:
var combined = source1.publish(s1 => source2.publish(s2 => source3.publish(s3 =>
Rx.Observable.merge(
s1.combineLatest(s2, s3, selector).take(1),
s1.skip(1).withLatestFrom(s2, s3, selector)
)
)));
...so take one value using combineLatest, then get the rest using withLatestFrom.
I wasn't quite satisfied with the accepted answer, so I ended up finding another solution. Many ways to skin a cat!
My use-case involves just two streams - a "requests" stream and a "tokens" stream. I want requests to fire as soon as they are received, using the whatever the latest token is. If there is no token yet, then it should wait until the first token appears, and then fire off all the pending requests.
I wasn't quite satisfied with the accepted answer, so I ended up finding another solution. Essentially I split the request stream into two parts - before and after first token arrives. I buffer the first part, and then re-release everything in one go once I know that the token stream is non-empty.
const first = token$.first()
Rx.Observable.merge(
request$.buffer(first).mergeAll(),
request$.skipUntil(first)
)
.withLatestFrom(token$)
See it live here: https://rxviz.com/v/VOK2GEoX
For RxJs 7:
const first = token$.first()
merge(
request$.pipe(
buffer(first),
mergeAll()
),
request$.pipe(
skipUntil(first)
)
).pipe(
withLatestFrom(token$)
)
I had similar requirements but for just 2 observables.
I ended up using switchMap+first:
observable1
.switchMap(() => observable2.first(), (a, b) => [a, b])
.subscribe(([a, b]) => {...}));
So it:
waits until both observables emit some value
pulls the value from second observable only if the first one has changed (unlike combineLatest)
doesn't hang subscribed on second observable (because of .first())
In my case, second observable is a ReplaySubject. I'm not sure if it will work with other observable types.
I think that:
flatMap would probably work too
it might be possible to extend this approach to handle more than 2 observables
I was surprised that withLatestFrom will not wait on second observable.
In my mind, the most elegant way to achieve the different behavior of an existing RxJS operator is to wrap it into a custom operator. So that from the outside it looks just like any regular operator and doesn't require you to restructure your code each time you need this behavior.
Here is how you can create your own operator which behaves just like withLatestFrom, except that at the very beginning it will emit as soon as the first value of the target observable is emitted (unlike standard withLatestFrom, which will ignore the first emission of the source if the target hasn't yet emitted once). Let's call it delayedWithLatestFrom.
Note that it's written in TypeScript, but you can easily transform it to plain JS. Also, it's a simple version that supports only one target observable and no selector function - you can extend it as needed from here.
export function delayedWithLatestFrom<T, N>(
target$: Observable<N>
): OperatorFunction<T, [T, N]> {
// special value to avoid accidental match with values that could originate from target$
const uniqueSymbol = Symbol('withLatestFromIgnore');
return pipe(
// emit as soon target observable emits the first value
combineLatestWith<T, [N]>(target$.pipe(first())),
// skip the first emission because it's handled above, and then continue like a normal `withLatestFrom` operator
withLatestFrom(target$.pipe(skip(1), startWith(uniqueSymbol))),
map(([[rest, combineLatestValue], withLatestValue]) => {
// take combineLatestValue for the first time, and then always take withLatestValue
const appendedValue =
withLatestValue === uniqueSymbol ? combineLatestValue : withLatestValue;
return [rest, appendedValue];
})
);
}
// SAMPLE USAGE
source$.pipe(
delayedWithLatestFrom(target$)
).subscribe(console.log);
So if you compare it with the original marble diagram for withLatestFrom, it will differ only in one fact: while withLatestFrom ignores the first emissions and produces b1 as the first value, the delayedWithlatestFrom operator will emit one more value a1 at the beginning, as soon as the second observable emits 1.
a) Standard withLatestFrom:
b) Custom delayedWithLatestFrom:
Use combineLatest and filter to remove tuples before first full set is found then set a variable to stop filtering. The variable can be within the scope of a wrapping defer to do things properly (support resubscription). Here it is in java (but the same operators exist in RxJs):
Observable.defer(
boolean emittedOne = false;
return Observable.combineLatest(s1, s2, s3, selector)
.filter(x -> {
if (emittedOne)
return true;
else {
if (hasAll(x)) {
emittedOne = true;
return true;
} else
return false;
}
});
)
I wanted a version where tokens are fetched regularly - and where I want to retry the main data post on (network) failure. I found shareReplay to be the key. The first mergeWith creates a "muted" stream, which causes the first token to be fetched immediately, not when the first action arrives. In the unlikely event that the first token will still not be available in time, the logic also has a startWith with an invalid value. This causes the retry logic to pause and try again. (Some/map is just a Maybe-monad):
Some(fetchToken$.pipe(shareReplay({refCount: false, bufferSize: 1})))
.map(fetchToken$ =>
actions$.pipe(
// This line is just for starting the loadToken loop immediately, not waiting until first write arrives.
mergeWith(fetchToken$.pipe(map(() => true), catchError(() => of(false)), tap(x => loggers.info(`New token received, success: ${x}`)), mergeMap(() => of()))),
concatMap(action =>
of(action).pipe(
withLatestFrom(fetchToken$.pipe(startWith(""))),
mergeMap(([x, token]) => (!token ? throwError(() => "Token not ready") : of([x, token] as const))),
mergeMap(([{sessionId, visitId, events, eventIds}, token]) => writer(sessionId, visitId, events, token).pipe(map(() => <ISessionEventIdPair>{sessionId, eventIds}))),
retryWhen(errors =>
errors.pipe(
tap(err => loggers.warn(`Error writing data to WG; ${err?.message || err}`)),
mergeMap((_error: any, attemptIdx) => (attemptIdx >= retryPolicy.retryCount ? throwError(() => Error("It's enough now, already")) : of(attemptIdx))), // error?.response?.status (int, response code) error.code === "ENOTFOUND" / isAxiosError: true / response === undefined
delayWhen(attempt => timer(attempt < 2 ? retryPolicy.shortRetry : retryPolicy.longRetry, scheduler))
)
)
)
),
)
)
Thanks to everyone on this question-page for good inputs.
Based on the answer from #cjol
Here's a RxJs 7 implementation of a waitFor operator that will buffer the source stream until all input observables have emitted values, then emit all buffered events on the source stream. Any subsequent events on the source stream are emitted immediately.
// Copied from the definition of withLatestFrom() operator.
export function waitFor<T, O extends unknown[]>(
inputs: [...ObservableInputTuple<O>]
): OperatorFunction<T, [T, ...O]>;
/**
* Buffers the source until every observable in "from" have emitted a value. Then
* emit all buffered source values with the latest values of the "from" array.
* Any source events are emitted immediately after that.
* #param from Array of observables to wait for.
* #returns Observable that emits an array that concatenates the source and the observables to wait.
*/
export function waitFor(
from: Observable<unknown>[]
): (source$: Observable<unknown>) => Observable<unknown> {
const combined$ = combineLatest(from);
// This served as a conditional that switched on and off the streams that
// wait for the the other observables, or emits the source right away because
// the other observables have emitted.
const firstCombined$ = combined$.pipe(first());
return function (source$: Observable<unknown>): Observable<unknown> {
return merge(
// This stream will buffer the source until the other observables have all emitted.
source$.pipe(
takeUntil(firstCombined$), // without this it continues to buffer new values forever
buffer(firstCombined$),
mergeAll()
),
// This stream emits the source straight away and will take over when the other
// observables have emitted.
source$.pipe(skipUntil(firstCombined$))
).pipe(
withLatestFrom(combined$),
// Flatten it to behave like withLatestFrom() operator.
map(([source, combined]) => [source, ...combined])
);
};
}
All of the above solutions are not really on the point, therefore I made my own. Hope it helps someone out.
import {
combineLatest,
take,
map,
ObservableInputTuple,
OperatorFunction,
pipe,
switchMap
} from 'rxjs';
/**
* ### Description
* Works similar to {#link withLatestFrom} with the main difference that it awaits the observables.
* When all observables can emit at least one value, then takes the latest state of all observables and proceeds execution of the pipe.
* Will execute this pipe only once and will only retrigger pipe execution if source observable emits a new value.
*
* ### Example
* ```ts
* import { BehaviorSubject } from 'rxjs';
* import { awaitLatestFrom } from './await-latest-from.ts';
*
* const myNumber$ = new BehaviorSubject<number>(1);
* const myString$ = new BehaviorSubject<string>("Some text.");
* const myBoolean$ = new BehaviorSubject<boolean>(true);
*
* myNumber$.pipe(
* awaitLatestFrom([myString$, myBoolean$])
* ).subscribe(([myNumber, myString, myBoolean]) => {});
* ```
* ### Additional
* #param observables - the observables of which the latest value will be taken when all of them have a value.
* #returns a tuple which contains the source value as well as the values of the observables which are passed as input.
*/
export function awaitLatestFrom<T, O extends unknown[]>(
observables: [...ObservableInputTuple<O>]
): OperatorFunction<T, [T, ...O]> {
return pipe(
switchMap((sourceValue) =>
combineLatest(observables).pipe(
take(1),
map((values) => [sourceValue, ...values] as unknown as [T, ...O])
)
)
);
}
Actually withLatestFrom already
waits for every source
emits only when source1 emits
remembers only the last source1-message while the other sources are yet to start
// when source 1 emits the others have emitted already
var source1 = Rx.Observable.interval(500).take(7)
var source2 = Rx.Observable.interval(100, 300).take(10)
var source3 = Rx.Observable.interval(200).take(10)
var selector = (a,b,c) => [a,b,c]
source1
.withLatestFrom(source2, source3, selector)
.subscribe()
vs
// source1 emits first, withLatestFrom discards 1 value from source1
var source1 = Rx.Observable.interval(500).take(7)
var source2 = Rx.Observable.interval(1000, 300).take(10)
var source3 = Rx.Observable.interval(2000).take(10)
var selector = (a,b,c) => [a,b,c]
source1
.withLatestFrom(source2, source3, selector)
.subscribe()

Resources