What's the best way to handle asynchronous updates in the middle of an Observable stream.
Let's say there are 3 observables:
Obs1 (gets data from API) -> pipes to Obs2
Obs2 (transforms data) -> pipes to Obs3
Obs3 (sends transformed data)
(The actual application is more complex, and there's reasons it's not done in a single Observable, this is just a simple example).
That all works well and good if it's a linear synchronous path.
But we also have async messages that will change the output of Obs2.
3 scenarios I'm asking about are:
- we fetch data, and go through Obs1, Obs2 & Obs3
- we get a message to make a change, go through Obs2 & Obs3
- we get a different message to make a change which also needs to apply the change from the previous message, through Obs2 & Obs3
The main problem here is that there are different types of asynchronous messages that will change the outcome of Obs2, but they all need to still know what the previous outcome of Obs2 was (so the any other changes from messages that happened before is still applied)
I have tried using switchMap in Obs2 with a scan in Obs1 like this:
obs1
const obs1$ = obs1$.pipe(
// this returns a function used in the reducer.
map((data) => (prevData) => 'modifiedData',
scan((data, reducer) => reducer(betsMap), {})
)
obs2
const obs2$ = obs1$.pipe(
switchMap(data =>
someChange$.pipe(map(reducer => reducer(data)))
)
)
where someChange$ is a BehaviorSubject applying a change using another reducer function.
This works fine for async message #1 that makes some change.
But when message #2 comes in and a different change is needed, the first change is lost.
the changes that should be in "prevData" in obs1$ is always undefined because it happens before the message is applied.
How can I take the output from obs2$ and apply asynchronous updates to it that remembers what all of the past updates was? (in a way where I can clear all changes if needed)
So if i got the question right, there are two problems that this question tackles:
First: How to cache the last 2 emitted values from stream.
scan definitely is the right way, if this cache logic is needed in more than one place/file, I would go for a custom pipe operator, like the following one
function cachePipe() {
return sourceObservable =>
sourceObservable.pipe(
scan((acc, cur) => {
return acc.length === 2 ? [...acc.slice(1), cur] : [...acc, cur];
}, [])
);
}
cachePipe will always return the latest 2 values passed trough the stream.
...
.pipe(
cachePipe()
)
Second: How to access data from multiple streams at the same time, upon stream event
Here rxjs's combineLatest creation operator might do the trick for you,
combineLatest(API$, async1$ ,async2$,async3$)
.pipe(
// here I have access to an array of the last emitted value of all streams
// and the data can be passed to `Obs2` in your case
)
In the pipe I can chain whatever number of observables, which resolves the second problem.
Note:
combineLatest needs for all streams, inside of it, to emit once, before the operator strats to emit their combined value, one workaround is to use startWith operator with your input streams, another way to do it is by passing the data trough BehaviorSubject-s.
Here is a demo at CodeSandbox , that uses the cachePipe() and startWith strategy to combine the source (Obs1) with the async observables that will change the data.
Related
I need a specific behavior that I can't get with the RxJS operators. The closest would be to use DebounceTime only for values entered after the first one, but I can't find a way to do it. I have also tried with ThrottleTime but it is not exactly what I am looking for, since it launches intermediate calls, and I only want one at the beginning that is instantaneous, and another at the end, nothing else.
ThrottleTime
throttleTime(12 ticks, { leading: true, trailing: true })
source: --0--1-----2--3----4--5-6---7------------8-------9---------
throttle interval: --[~~~~~~~~~~~I~~~~~~~~~~~I~~~~~~~~~~~I~~~~~~~~~~~]--------
output: --0-----------3-----------6-----------7-----------9--------
source_2: --0--------1------------------2--------------3---4---------
throttle interval: --[~~~~~~~~~~~I~~~~~~~~~~~]---[~~~~~~~~~~~]--[~~~~~~~~~~~I~
output_2: --0-----------1---------------2--------------3-----------4-
DebounceTime
debounceTime(500)
source: --0--1--------3------------4-5-6-7-8-9-10-11--13----------------
debounce_interval: -----[~~~~~]--[~~~~~]--------------------------[~~~~~]----------
output: -----------1--------3--------------------------------13---------
What I want
debounceTimeAfterFirst(500) (?)
source: --0--1--------3------------4-5-6-7-8-9-10-11--13----------------
debounce_interval: -----[~~~~~]--[~~~~~]--------------------------[~~~~~]----------
output: --0--------1--3------------4-------------------------13---------
As you see, the debounce time is activated when a new value is entered. If the debounce time passes and any new value has been entered, it stops the listening the debounceTime action and waits to start a new one.
Edit: I forgot to comment that this must be integrated with NgRx’s Effects, so it must be a continuous stream that mustn't be completed. Terminating it would probably cause it to stop listening for dispatched actions.
I would use a throttle combined with a debounceTime:
throttle: from Documentation Emit value on the leading edge of an interval, but suppress new values until durationSelector has completed.
debounceTime: from Documentation Discard emitted values that take less than the specified time between output.
I would use a throttle stream to get the raising edge (the first emission) and then the debounce stream would give us the falling edge.
const source = fromEvent(document.getElementsByTagName('input'), 'keyup').pipe(
pluck('target', 'value')
);
const debounced = source.pipe(
debounceTime(4000),
map((v) => `[d] ${v}`)
);
const effect = merge(
source.pipe(
throttle((val) => debounced),
map((v) => `[t] ${v}`)
),
debounced
);
effect.subscribe(console.log);
See RxJS StackBlitz with the console open to see the values changing.
I prepared the setup to adapt it to NgRx which you mention. The effect I got working is:
#Injectable({ providedIn: 'root' })
export class FooEffects {
switchLight$ = createEffect(() => {
const source = this.actions$.pipe(
ofType('[App] Switch Light'),
pluck('onOrOff'),
share()
);
const debounced = source.pipe(debounceTime(1000), share());
return merge(source.pipe(throttle((val) => debounced)), debounced).pipe(
map((onOrOff) => SetLightStatus({ onOrOff }))
);
});
constructor(private actions$: Actions) {}
}
See NgRx StackBlitz with the proposed solution working in the context of an Angular NgRx application.
share: This operator prevents the downstream paths to simultaneously fetch the data from all the way up the chain, instead they grab it from the point where you place share.
I also tried to adapt #martin's connect() approach. But I don't know how #martin would "reset" the system so that after a long time if a new source value is emitted would not debounce it just in the same manner as you first run it, #martin, feel free to fork it and tweak it to make it work, I'm curious about your approach, which is very smart. I didn't know about connect().
#avicarpio give it a go on your application and let us know how it goes :)
I think you could do it like the following, even though I can't think of any easier solution right now (I'm assuming you're using RxJS 7+ with connect() operator):
connect(shared$ => shared$.pipe(
exhaustMap(value => merge(
of(value),
shared$.pipe(debounceTime(1000)),
).pipe(
take(2),
)),
)),
Live demo: https://stackblitz.com/edit/rxjs-qwoesj?devtoolsheight=60&file=index.ts
connect() will share the source Observable and lets you reuse it in its project function multiple times. I'm using it only to use the source Observable inside another chain.
exhaustMap() will ignore all next notifications until its inner Observable completes. In this case the inner Observable will immediately reemit the current value (of(value)) and then use debounceTime(). Any subsequent emission from source is ignored by exhaustMap() because the inner Observable hasn't completed yet but is also passed to debounceTime(). Then take(2) is used to complete the chain after debounceTime() emits and the whole process can repeat when source emits because exhaustMap() won't ignore the next notification (its inner Observable has completed).
Here's a custom operator that (as far s I can tell) does what you're after.
The two key insights here are:
Use connect so that you can subscribe to the source twice, once to ignore emissions with exhaustMap and another to inspect and debounce emissions with switchMap
Create an internal token so that you know when to exit without a debounced emission. (Insures that from your example above, the 4 is still emitted).
function throttleDebounceTime<T>(interval: number): MonoTypeOperatorFunction<T> {
// Use this token's memory address as a nominal token
const resetToken = {};
return connect(s$ => s$.pipe(
exhaustMap(a => s$.pipe(
startWith(resetToken),
switchMap(b => timer(interval).pipe(mapTo(b))),
take(1),
filter<T>(c => c !== resetToken),
startWith(a)
))
));
}
example:
of(1,2,3,4).pipe(
throttleDebounceTime(500)
).subscribe(console.log);
// 1 [...0.5s wait] 4
I have a stream of emissions conforming to: Observable<Notice[]>. Each Notice has a property, isVisible$ (Observable<boolean>) that determines whether or not it is on screen in this particular moment. I want to filter this array of notices by whether the most recent value of isVisible$ is true. When a new array of notices occurs, I want to begin the process again. I know this entails using switchMap on the higher order observable stream.
Neither types of observable will ever complete, so using operators like toArray() will not work here. Each isVisible$ stream is guaranteed to emit at least once.
I want the output to also be of Observable<Notice[]>, emitting each time the isVisible$ stream of any of the inner observable predicates updates.
What I have so far does emit the proper values, but the inner pipeline just groups notices together and emits them (via scan, in lieu of toArray), it doesn't buffer to the length of from(notices) and then emit (if that makes sense). This makes the end result of the stream is too busy.
notices.pipe(
switchMap(notices => from(notices).pipe(
mergeMap(notice => notice.isVisible$.pipe(
map(isVisible => ({ notice, isVisible }))
)),
filter(({ isVisible }) => isVisible),
map(({ notice }) => notice),
scan((noticesArr, noticeBeingAddedOrRemoved) => {
if (!noticesArr.find(n => n.identifier === noticeBeingAddedOrRemoved.id)) {
noticesArr.push(noticeBeingAddedOrRemoved);
}
return noticesArr;
}, [])
))
);
Here's a reproducible sample of what I'm working with on StackBlitz.
I've changed it to use zip, which will only emit when each of the isVisible$ observables emit. You could also use combineLatest if you want to emit whenever any of the source observables emit, rathern than waiting for all of them.
I have the following operators:
const prepare = (value$: Observable<string>) =>
value$.pipe(tap((x) => console.log("remove: ", x)), share());
const performTaskA = (removed$: Observable<string>) =>
removed$.pipe(tap((x) => console.log("pathA: ", x)),);
const performTaskB = (removed$: Observable<string>) =>
removed$.pipe(tap((x) => console.log("pathB: ", x)));
and I call them like this:
const prepared$ = value$.pipe(prepare);
const taskADone$ = prepared$.pipe(performTaskA);
const taskBDone$ = prepared$.pipe(performTaskB);
merge(taskADone$, taskBDone$).subscribe();
Due to the share in prepare I would expect 'remove' to be logged only once, however it appears twice.
Why is this not working?
Codesandbox: https://codesandbox.io/s/so-remove-fires-twice-iyk12?file=/src/index.ts
This is happening because your source Observable is of() that just emits one next notification and then complete. Everything in RxJS in synchronous unless you work with time or you intentionally make your code asynchronous (eg. with Promise.resolve or with asyncScheduler).
In your demo, share() receives one next and one complete notification immediately which makes its internal state to reset. It will also unsubscribe from its source Obserable because there are no more observers (the second source taskBDone$ you're merging has not subscribed yet). Then taskBDone$ is merged into the chain and share() creates internally a new instance of Subject and the whole process repeats.
These are the relevant parts in share():
Dispose handler triggered after receiving complete from source https://github.com/ReactiveX/rxjs/blob/master/src/internal/operators/share.ts#L120
New Subject created: https://github.com/ReactiveX/rxjs/blob/master/src/internal/operators/share.ts#L113
share() resets its state https://github.com/ReactiveX/rxjs/blob/master/src/internal/operators/share.ts#L163
So if your sources are going to be synchronous you should better use shareReplay() (instead of share()) that will just replay the entire sequence of events to every new observer.
Your updated demo: https://stackblitz.com/edit/rxjs-jawajw?devtoolsheight=60
Notice, that in your demo if you used of("TEST").pipe(delay(0)) as your source Observable it would work as you expected because delay(0) would force asynchronous behavior and both source Observables would first subscribe and then in another JavaScript frame would emit their next and complete.
TLDR: Working example is in the last codeblock of this question. Check out #bryan60 answer for a working example using concat rather than mergeMap.
I'm trying to run a number of remote requests sequentially, but only the first observable is executed.
The number of request vary, so I can't do a dodgy solution where I nest observables within each other.
I'm using the following code:
const observables = [
observable1,
observable2,
...
];
from(observables).pipe(
mergeMap(ob=> {
return ob.pipe(map(res => res));
}, undefined, 1)
).subscribe(res => {
console.log('Huzzah!');
})
In the past (rxjs 5.5) Ive used the following:
let o = Observable.from(observables).mergeMap((ob) => {
return ob;
}, null, 1);
o.subscribe(res => {
console.log('Huzzah!');
})
I'm not sure what I'm doing wrong, can anybody shed some light?
An additional request would be to only print 'Huzzah!' once on completion of all requests rather than for each individual Observable.
EDIT:
Removing undefined from my original code will make it work, however there was another issue causing only the first observable to be executed.
I'm using Angular's HttpClient for remote requests. My observable code looked like this:
const observables = [];
// Only the first observable would be executed
observables.push(this.http.get(urla));
observables.push(this.http.get(urlb));
observables.push(this.http.get(urlc));
Adding .pipe(take(1)) to each observable results in each observable being executed:
const observables = [];
// All observables will now be executed
observables.push(this.http.get(urla).pipe(take(1));
observables.push(this.http.get(urlb).pipe(take(1));
observables.push(this.http.get(urlc).pipe(take(1));
The code I ended up using, which executes all observables in sequential order and only triggers Huzzah! once is:
const observables = [];
observables.push(this.http.get(urla).pipe(take(1));
observables.push(this.http.get(urlb).pipe(take(1));
observables.push(this.http.get(urlc).pipe(take(1));
from(observables).pipe(
mergeMap(ob=> {
return ob.pipe(map(res => res));
}, 1),
reduce((all: any, res: any) => all.concat(res), [])
).subscribe(res => {
console.log('Huzzah!');
})
Thanks to #bryan60 for helping me wit this issue.
if these are http requests that complete, I think your bug is caused by a change to the mergeMap signature that removed the result selector. it's hard to be sure without knowing exactly which version you're on as it was there, then removed, then added again, and they're removing it once more for good in v7.
if you want to run them sequentially... this is all you need...
// concat runs input observables sequentially
concat(...observables).subscribe(res => console.log(res))
if you want to wait till they're all done to emit, do this:
concat(...observables).pipe(
// this will gather all responses and emit them all when they're done
reduce((all, res) => all.concat([res]), [])
// if you don't care about the responses, just use last()
).subscribe(allRes => console.log(allRes))
In my personal utility rxjs lib, I always include a concatJoin operator that combines concat and reduce like this.
the only trick is that concat requires observables to complete till it moves on to the next one, but the same is true for mergeMap with concurrent subscriptions set to 1.. so that should be fine. things like http requests are fine, as they complete naturally after one emission.. websockets or subjects or event emitters will behave a bit differently and have to be manually completed, either with operators like first or take or at the source.
If you are not concerned about the sequence of execution and just want 'Huzzah!' to be printed once all the observable has been executed forkJoin can also be used.Try this.
forkJoin(...observables).subscribe(res => console.log('Huzzah');
I have at least two buttons that I want to dynamically listen for clicks on. listeningArray$ will emit an array (ar) of button #'s that I need to be listening to. When somebody clicks on one of these buttons I'm listening to, I need to console log that the button that was clicked and also log the value from a time interval.
If ar goes from [1,2] to [1], we need to stop listening to clicks on button #2. So the DOM click event needs to be removed for 2 and that should trigger the .finally() operator. But for 1, we should remain subscribed and the code inside the .finally() should not run, since nothing is being unsubscribed.
const obj$ = {};
Rx.Observable.combineLatest(
Rx.Observable.interval(2000),
listeningArray$ // Will randomly emit either [1] or [1,2]
)
.switchMap(([x, ar]) => {
const observables = [];
ar.forEach(n => {
let nEl = document.getElementById('el'+n);
obj$[n] = obj$[n] || Rx.Observable.fromEvent(nEl, 'click')
.map(()=>{
console.log(' el' + n);
})
.finally(() => {
console.log(' FINALLY_' + n);
});
observables.push(obj$[n]);
})
return Rx.Observable.combineLatest(...observables);
})
.subscribe()
But what's happening is every time the interval emits a value, the DOM events ALL get removed and then immediately get added on again, and the code inside the .finally operator runs for 1 and 2.
This is really frustrating me. What am I missing?
It's a bit of a complex situation, so I created this: https://jsfiddle.net/mfp22/xtca98vx/7/
I was actually really close, but I misunderstood the point of switchMap.
switchMap is designed to unsubscribe from the observable it returns whenever a new value is emitted from above. This is why it can be used to cancel old pending Http requests when a new request needs to be made instead.
The problem I was having is to be expected. switchMap will unsubscribe from the previously returned observable before subscribing to the current one. This was unacceptable, as I explained in the question. The reason this was unacceptable was that in my actual project, the fromEvent observables were listening to Firebase child_added events, so when these cold observables went from having no subscribers to having 1 subscriber, Firebase would subsequently fire the event for every child already existing, as well as for future ones added.
I played with mergeMap for a while, but it was really difficult and buggy to manually have to unsubscribe from previously returned observables.
So I added a subscriber for the inner observables while switchMap was doing its process of unsubscribe from old => subscribe to new so that there would always be a subscriber. I used takeUntil(Observable.timer(0)) to make sure the subscribers didn't build up and cause a memory leak.
There may be a better solution, but this was the best one I found.
const obj$ = {};
Rx.Observable.combineLatest(
Rx.Observable.interval(2000),
listeningArray$ // Will randomly emit either [1] or [1,2]
)
.switchMap(([x, ar]) => {
const observables = [];
ar.forEach(n => {
let nEl = document.getElementById('el'+n);
obj$[n] = obj$[n] || Rx.Observable.fromEvent(nEl, 'click')
.map(()=>{
console.log(' el' + n);
})
.finally(() => {
console.log(' FINALLY_' + n);
})
.share();
obj$[n].takeUntil(Rx.Observable.timer(0))
.subscribe();
observables.push(obj$[n]);
})
return Rx.Observable.combineLatest(...observables);
})
.subscribe()
I also had to add the .share() method. I was going to need it anyway. I'm using this pattern to let some Angular components declare what data they need, ignoring what other components might want, to achieve a better separation of concerns. So multiple components can subscribe to the same Firebase observables, but the .share() operator ensures that each message from Firebase is only handled once (I'm dispatching actions to a Redux store for each one).
Working solution: https://jsfiddle.net/mfp22/xtca98vx/8/
State in FRP is immutable. Thus when you switchMap to the second emission the previous observable combineLatest containing [1,2] will get unsubscribed and the finally operator invoked. Before subscribing to the next containing only [1]
If you only want to unsubscribe from one button you can store state in the DOM (add atr to button) and use filter to ignore button.
Or you can add a TakeWhile() to every button dictating when it should be unsubscribed so it can invoke it's own finally()