I would like to combine/merge multiple Observables and when each of them is completed execute a finally function. The merge operator seems to execute each subscription in parallel, which is what I need, but if any of them throws an error the execution is halted.
RxJS version 4 has an operator mergeDelayError that should keep the all subscriptions executing till all of them are completed, but this operator isn't implemented in version 5.
Should I revert to a different operator?
var source1 = Rx.Observable.of(1,2,3).delay(3000);
var source2 = Rx.Observable.throw(new Error('woops'));
var source3 = Rx.Observable.of(4,5,6).delay(1000);
// Combine the 3 sources into 1
var source = Rx.Observable
.merge(source1, source2, source3)
.finally(() => {
// finally is executed before all
// subscriptions are completed.
console.log('finally');
});
var subscription = source.subscribe(
x => console.log('next:', x),
e => console.log('error:', e),
() => console.log('completed'));
JSBin
We can avoid blocking the stream by collecting the errors and emitting them at the end.
function mergeDelayError(...sources) {
const errors = [];
const catching = sources.map(obs => obs.catch(e => {
errors.push(e);
return Rx.Observable.empty();
}));
return Rx.Observable
.merge(...catching)
.concat(Rx.Observable.defer(
() => errors.length === 0 ? Rx.Observable.empty() : Rx.Observable.throw(errors)));
}
const source1 = Rx.Observable.of(1,2,3);
const source2 = Rx.Observable.throw(new Error('woops'));
const source3 = Rx.Observable.of(4,5,6);
mergeDelayError(source1, source2, source3).subscribe(
x => console.log('next:', x),
e => console.log('error:', e),
() => console.log('completed'));
I think you can simulate the same behavior by using catch(). You'll just need to append it to every source Observable:
const sources = [source1, source2, source3].map(obs =>
obs.catch(() => Observable.empty())
);
Rx.Observable
.merge(sources)
.finally(...)
...
If you don't want to swallow your errors, but want to delay them to the end, you can:
const mergeDelayErrors = [];
const sources = [source1, source2, source3].map(obs => obs.catch((error) => {
mergeDelayErrors.push(error);
return Rx.Observable.empty();
}));
return Rx.Observable
.merge(...sources)
.toArray()
.flatMap(allEmissions => {
let spreadObs = Rx.Observable.of(...allEmissions);
if (mergeDelayErrors.length) {
spreadObs = spreadObs.concat(Rx.Observable.throw(mergeDelayErrors));
}
return spreadObs;
})
You may want to throw only the first error, or create a CompositeError. I'm not sure how mergeDelayErrors originally behaved when multiple errors were thrown.
Unfortunately, because this implementation must wait until the all observables complete before emitting errors, it also waits until all observables complete before emitting next. This is likely not the original behavior of mergeDelayError, which is supposed to emit as a stream, rather than emitting them all at the end.
Related
What is the correct way in RXJS to remap an observable into a timer start value, without interrupting the original stream?
obs.pipe(take(1000), startTimer())
.subscribe(start => {
// show how long it took to finish streaming 1000 values:
const duration = Date.now() - start;
console.log(duration);
});
I want startTimer to remap into once-off subscription with start, but without interrupting the original stream, i.e. in this case subscribe is to be triggered only after all 1000 values have finished streaming.
How do I implement such startTimer? It's supposed to result into a once-off Date.now() value to help measure full stream duration.
Or is there maybe a standard solution for this already that I'm missing?
update-1
The expected result is like the one below, but without the need for creating start as an external variable, and instead make it part of the stream:
const start = Date.now();
obs.pipe(take(1000))
.subscribe({
complete() {
const duration = Date.now() - start;
console.log(duration);
}
});
The reason I want to make it part of a stream is because the original observable and subscribers are very much detached from each other, as in sitting in unrelated source files.
P.S. Alternatively, a solution that emits duration in the end would also be good, if that is at all possible.
update-2
In the end, I used a generic drain operand, designed to drain an observable stream, and then produce an observable at the end:
/**
* Drains the source observable till it completes, and then posts a new value-observable.
*/
function drain<T>(value: T | Observable<T> | (() => T | Observable<T>)) {
const v = () => {
const a = typeof value === 'function' ? value.call(null) : value;
return a instanceof Observable ? a : of(a);
}
return s => defer(() => s.pipe(filter(_ => false), c => concat(c, v()))) as Observable<T>;
}
Using this operand, I can rewrite startTimer like this:
const startTimer = () => drain(Date.now);
Some code that does what you describe pretty much exactly the way you describe it:
function logRunTime<T>(prefix: string): MonoTypeOperatorFunction<T> {
return s => defer(() => {
const start = Date.now();
return s.pipe(
tap({
complete: () => console.log(`${prefix}: ${Date.now() - start}ms`)
})
);
});
}
interval(1000).pipe(
take(10),
logRunTime("Ten Seconds of Interval")
).subscribe(console.log);
Output:
0
1
2
3
4
5
6
7
8
9
Ten Seconds of Interval: 10014ms
Update 1
do not make the original observable stop emitting values [...] we just do not want the source values
It seem to me that either you keep emitting the values or you don't.
Here is a version that drops the source emissions.
Is this what you're after?
function reduceRunTime<T>(prefix: string): OperatorFunction<T, string> {
return s => defer(() => {
const start = Date.now();
return s.pipe(
filter(_ => false),
c => concat(c, of(null)),
map(_ => `${prefix}: ${Date.now() - start}ms`)
);
}) as Observable<string>;
}
interval(1000).pipe(
take(10),
reduceRunTime("Ten Seconds of Interval")
).subscribe(console.log);
Output:
Ten Seconds of Interval: 10013ms
Update 2
If you don't want a string, this will emit the start time once the observable completes.
function startTimer() {
return s => s.pipe(
filter(_ => false),
c => concat(c, of(Date.now()))
) as Observable<number>;
}
Update 3
Two separate behaviours
I think update 2 may have been cleaned up too much. Consider this example:
const timed$ = interval(500).pipe(
take(5),
startTimer()
);
const logDiff = (start: number) => console.log(Date.now() - start);
timed$.subscribe(logDiff);
setTimeout(() => {
timed$.subscribe(logDiff);
}, 1000);
setTimeout(() => {
timed$.subscribe(logDiff);
}, 5000);
The output:
2521
3507
7511
Notably, because Observables are lazy (do nothing until subscribed), but Date.now is called when the observable is created. Your startTime may well be set long before the observable even starts. Making a 2.5s observable appear to require 7.5s.
Using defer fixes this problem as it doesn't create the observable until it is subscribed.
Updated startTimer
function startTimer() {
return s => defer(() => s.pipe(
filter(_ => false),
c => concat(c, of(Date.now()))
)) as Observable<number>;
}
New output for example above:
2521
2507
2511
Now you can do fun things like run the same observable 10 times and average out the runtime to get a better idea of how long it will take.
const average = arr => arr.reduce( ( p, c ) => p + c, 0 ) / arr.length;
concat(...Array.from(Array(10)).map(_ => timed$)).pipe(
map(start => Date.now() - start),
tap(console.log),
toArray()
).subscribe(runs => console.log("Average Runtime: ", average(runs)));
Output:
2515
2506
2506
2506
2507
2505
2506
2506
2507
2507
Average Runtime: 2507.1
I find myself wanting to do this, which feels like it ought to be wrong.
this.isLoading = true;
this.service.getFirstValue().subscribe((response: firstValueType) => {
this.firstValue = response;
this.service.getSecondValue(this.firstValue).subscribe((response: secondValueType) => {
this.secondValue = response;
this.isLoading = false
});
});
What are you supposed to do?
you can use switchMap. Also embedded subscribe is bad practice, supposed to be avoided
this.isLoading = true;
this.service
.getFirstValue()
.pipe(
switchMap(response => {
this.firstValue = response;
return this.service.getSecondValue(this.firstValue);
})
)
.subscribe(response => {
this.secondValue = response;
this.isLoading = false;
});
PS fix the code, subscribe should be outside of pipe
The solution I figured out works fine for me in my project was using the following parts of rxjs:
ConnectableObservable
switchMap
tap
const { Subject } = rxjs;
const { publishReplay, tap, switchMapTo } = rxjs.operators;
// Simulates your service.getFirstValue, ...
const source1$$ = new Subject();
const source2$$ = new Subject();
const source3$$ = new Subject();
// Publish and Replay the last value to be able to connect (make it hot) and always get the lates value
const source1$ = source1$$.pipe(publishReplay(1));
const source2$ = source2$$.pipe(publishReplay(1));
const source3$ = source3$$.pipe(publishReplay(1));
// Connect to make the observable hot
source1$.connect()
source2$.connect()
source3$.connect()
source1$.pipe(
tap(v => console.log('tap source1$: ', v)),
switchMapTo(source2$),
tap(v => console.log('tap source2$: ', v)),
switchMapTo(source3$),
tap(v => console.log('tap source3$: ', v))
).subscribe()
source3$$.next('value 1');
source2$$.next('value 2');
source1$$.next('value 3');
source2$$.next('value 4');
source2$$.next('value 5');
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.5.3/rxjs.umd.min.js"></script>
I am not sure if this fits your exactly requirements. But imo this is a at least close szenario like yours. If you want give me feedback and I try to adapt :)
Nested subscribe should be avoided.
In this case I would go like this
this.isLoading = true;
this.service.getFirstValue().pipe(
// this tap is necessary only if you use this.firstValue somewhere else in the code
// otherwise you can skip it and go directly to concatMap
tap((response_1: firstValueType) => this.firstValue = response_1),
// concatMap is usually the preferred operator to use when concatenating http calls
// read the article I have linked below
concatMap(response_1 => this.service.getSecondValue(response_1)),
tap(response_2 => {
this.secondValue = response_2;
this.isLoading = false;
})
).subscribe()
You can get some more inspiration on how to deal with http calls and rxJs reading this article.
I find myself puzzled trying to set a very simple rxjs flow of subscriptions. Having multiple non-related subscriptions nested into another.
I'm in an angular application and I need a subject to be filled with next before doing other subscriptions.
Here would be the nested version of what I want to achieve.
subject0.subscribe(a => {
this.a = a;
subject1.subscribe(x => {
// Do some stuff that require this.a to exists
});
subject2.subscribe(y => {
// Do some stuff that require this.a to exists
});
});
I know that nested subscriptions are not good practice, I tried using flatMap or concatMap but didn't really get how to realize this.
It's always a good idea to separate the data streams per Observable so you can easily combine them later on.
const first$ = this.http.get('one').pipe(
shareReplay(1)
)
The shareReplay is used to make the Observable hot so it won't call http.get('one') per each subscription.
const second$ = this.first$.pipe(
flatMap(firstCallResult => this.http.post('second', firstCallResult))
);
const third$ = this.first$.pipe(
flatMap(firstCallResult => this.http.post('third', firstCallResult))
);
After this you can perform subscriptions to the Observables you need:
second$.subscribe(()=>{}) // in this case two requests will be sent - the first one (if there were no subscribes before) and the second one
third$.subscribe(() => {}) // only one request is sent - the first$ already has the response cached
If you do not want to store the first$'s value anywhere, simply transform this to:
this.http.get('one').pipe(
flatMap(firstCallResult => combineLatest([
this.http.post('two', firstCallResult),
this.http.post('three', firstCallResult)
])
).subscribe(([secondCallResult, thirdCallResult]) => {})
Also you can use BehaviorSubject that stores the value in it:
const behaviorSubject = new BehaviorSubject<string>(null); // using BehaviorSubject does not require you to subscribe to it (because it's a hot Observable)
const first$ = behaviorSubject.pipe(
filter(Boolean), // to avoid emitting null at the beginning
flatMap(subjectValue => this.http.get('one?' + subjectValue))
)
const second$ = first$.pipe(
flatMap(firstRes => this.http.post('two', firstRes))
)
const third$ = first$.pipe(
flatMap(()=>{...})
)
behaviorSubject.next('1') // second$ and third$ will emit new values
behaviorSubject.next('2') // second$ and third$ will emit the updated values again
You can do that using the concat operator.
const first = of('first').pipe(tap((value) => { /* doSomething */ }));
const second = of('second').pipe(tap((value) => { /* doSomething */ }));
const third = of('third').pipe(tap((value) => { /* doSomething */ }));
concat(first, second, third).subscribe();
This way, everything is chained and executed in the same order as defined.
EDIT
const first = of('first').pipe(tap(value => {
// doSomething
combineLatest(second, third).subscribe();
}));
const second = of('second').pipe(tap(value => { /* doSomething */ }));
const third = of('third').pipe(tap(value => { /* doSomething */ }));
first.subscribe();
This way, second and third are running asynchronously as soon as first emits.
You could do something like this:
subject$: Subject<any> = new Subject();
this.subject$.pipe(
switchMap(() => subject0),
tap(a => {
this.a = a;
}),
switchMap(() => subject1),
tap(x => {
// Do some stuff that require this.a to exists
}),
switchMap(() => subject2),
tap(y => {
// Do some stuff that require this.a to exists
})
);
if you want to trigger this, simply call this.subject$.next();
EDIT:
Here is an possible approach with forkJoin, that shout call the subjects parallel.
subject$: Subject<any> = new Subject();
this.subject$.pipe(
switchMap(() => subject0),
tap(a => {
this.a = a;
}),
switchMap(
() => forkJoin(
subject1,
subject2
)),
tap([x,y] => {
// Do some stuff that require this.a to exists
})
);
Trying to have a event triggering multiple switchMap with the initial event data.
Each actions creates a promise to some transform that is then written in the file system. The actions are independent and unrelated, but uses the same data, just for different purpose, so they should not be merged.
Currently using taps instead of switchMap, that can lead to multiple event running the at the same time.
const SomeApiCall = () => {return {some: 'data'} }
const AllowsDoAction = () => {console.log('Parsing API and writting some things to FS -- PLACEHOLDER')}
const SomeTimeDoThisActionTo = () => {console.log('Parsing API and writting some other things to fs, by asking more data from the API and first, so it is long thing to do, so a new event can arrive first -- PLACEHOLDER')}
const deepEqual = (prev, cur) => prev === cur // normally a proper deepEqual...
const taps = [tap(AllowsDoAction)];
if (someCondition) taps.push(SomeTimeDoThisActionTo)
const observable = timer(0, 500).pipe(
exhaustMap(SomeApiCall),
distinctUntilChanged((prev, cur) => deepEqual(prev, cur))
...taps
);
I would return the filesystem write observables (edit 2)
const SomeApiCall = () => {
return of({ some: 'data' });
};
const AllowsDoAction = () => {
console.log('Parsing API and writting some things to FS -- PLACEHOLDER');
return timer(100).pipe(map(() => 'write 1 finished'));
};
const SomeTimeDoThisActionTo = () => {
console.log(
'Parsing API and writting some other things to fs, by asking more data from the API and first, so it is long thing to do, so a new event can arrive first -- PLACEHOLDER'
);
return timer(1000).pipe(map(() => 'write 2 finished'));
};
And then use concatMap to wait for all filesystem operation to complete.
const deepEqual = (prev, cur) => prev === cur; // normally a proper deepEqual...
const taps = [(AllowsDoAction)];
const someCondition = true;
if (someCondition) {
taps.push(SomeTimeDoThisActionTo);
}
const reduxStorageEvent$ = of('replace this with real event');
const observable = merge(timer(0, 500), reduxStorageEvent$).pipe(
exhaustMap(SomeApiCall),
distinctUntilChanged((prev, cur) => deepEqual(prev, cur)),
// use switchMap to cancel previous writes (edit 2)
// await latest write operations, before starting new writes
concatMap((someData) => {
const writes = taps.map((tapFx) => {
return tapFx(someData);
});
// wait for all writes
return forkJoin(...writes);
})
);
concatMap is like a queue. The first one in this queue has to finish before the second one can start.
I have two source observables.
I would like to merge the two source observables, but the merged observable sould complete as soon as one of the source observables completes.
Desired behavior:
Source 1: ---1--------3--4-----------------------------x
Source 2: -------2----------x
"merged" ---1---2----3--4--x
In case of an error on one of the sources, the error should propagate to the merged observable:
Source 1: ---1--------3--4-----------------------------x
Source 2: -------2----------e
"merged" ---1---2----3--4--ex
The "merge" operator only completes the merged stream when both sources have completed:
Source 1: ---1--------3--4-----------------------------x
Source 2: -------2----------x
"merged" ---1---2----3--4-----------------------------x
How can I achieve my desired behavior?
You need to work with the metadata, information about each observable. To do this, use the materialize() operator on each stream and the use dematerialize() on the merged stream to actually emit the data.
Observable.merge( observableA.materialize(),
observableB.materialize() )
.takeWhile( notification -> notification.hasValue() )
.dematerialize()
.subscribe( ... );
This will merge the two observables until either one of them completes or emits an error.
I sure hope someone else answers with more elegant method but this works.
I think you would have to use one of the take operators. You could complete all sources when one source completes like so:
const a = Rx.Observable.interval(1000).take(3).map(x => `a${x}`);
const b = Rx.Observable.interval(800).take(6).map(x => `b${x}`);
Rx.Observable.merge(a.takeUntil(b.last()), b.takeUntil(a.last()))
.subscribe(
x => { console.log('next', x); },
null,
() => { console.log('complete'); }
);
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.5/Rx.min.js"></script>
Or a less readable but more scaleable version:
function merge(...obs) {
return Rx.Observable.merge(...obs.map(x => x.takeUntil(Rx.Observable.race(obs.filter(y => y !== x).map(z => z.last())))));
}
const a = Rx.Observable.interval(1000).take(3).map(x => `a${x}`);
const b = Rx.Observable.interval(800).take(6).map(x => `b${x}`);
merge(a, b)
.subscribe(
x => { console.log('next', x); },
null,
() => { console.log('complete'); }
);
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.5/Rx.min.js"></script>
Here is an illustration with error propagation:
function merge(...obs) {
return Rx.Observable.merge(...obs.map(x => x.takeUntil(Rx.Observable.race(obs.filter(y => y !== x).map(z => z.last())))));
}
const a = Rx.Observable.interval(1000).take(3).map(x => `a${x}`);
const b = Rx.Observable.interval(800).take(6).map(x => `b${x}`);
const c = Rx.Observable.timer(2200).map(x => { throw 'oops!'; });
merge(a, b, c)
.subscribe(
x => { console.log('next', x); },
x => { console.log('error', x); },
() => { console.log('complete'); }
);
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.5/Rx.min.js"></script>
Using the takeUntil outside on the merge is tricky as you would loose the last emitted value.
When an an observable completes, it does not emit a value, but we can concat it with another 'signal' observable that emits a single value. We can then watch for the 'signal' observable's value with the takeWhile operator.
Of course you'd have to ensure that the 'signal' observable's emitted value is not a value that could be emitted by the observables that are being merged - an empty object will suffice if the takeWhile predicate compares by reference.
Here's an example:
const obs1$ = Rx.Observable.interval(1000)
.map(x => `obs1: ${x}`)
.take(5);
const obs2$ = Rx.Observable.interval(300)
.map(x => `obs2: ${x}`)
.take(9);
const signalFinishMessage = {};
const signalFinish$ = Rx.Observable.of(signalFinishMessage);
Rx.Observable.merge(obs1$.concat(signalFinish$), obs2$.concat(signalFinish$))
.takeWhile(x => x !== signalFinishMessage)
.subscribe(
x => console.log(x),
err => console.log('received error:', err),
() => console.log('complete')
);
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.5/Rx.min.js"></script>
Errors will also get propagated:
const obs1$ = Rx.Observable.interval(1000)
.map(x => `obs1: ${x}`)
.take(5);
const obs2$ = Rx.Observable.interval(300)
.map(x => `obs2: ${x}`)
.take(9)
.concat(Rx.Observable.throw(`the world's about to end`));
const signalFinishMessage = {};
const signalFinish$ = Rx.Observable.of(signalFinishMessage);
Rx.Observable.merge(obs1$.concat(signalFinish$), obs2$.concat(signalFinish$))
.takeWhile(x => x !== signalFinishMessage)
.subscribe(
x => console.log(x),
err => console.log('received error:', err),
() => console.log('complete')
);
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.5/Rx.min.js"></script>
I ended up rolling my own:
import { Observable } from 'rxjs';
export function whileAll<T>(...observables: Observable<T>[]): Observable<T> {
return new Observable<T>(function (observer) {
if (observables.length === 0)
observer.complete();
else {
const next = observer.next.bind(observer);
const error = observer.error.bind(observer);
const complete = observer.complete.bind(observer);
for (let i = 0; i < observables.length; i++)
observer.add(observables[i].subscribe(next, error, complete));
}
});
}