One of the core principles of the Observable contract is that all onNext notifications are emitted before an onComplete/onError notification. I'm wondering if I can ever rely on onNext notifications from an operator to always come before onComplete of the original Observable. There are obvious combination and timing operators like delay, debounce and zip where this cannot be true in general, but what about the likes of map, scan or concat?
This timing guarantee is important when retrying/repeating conditional on the stream items. For example, suppose I want to retry a streaming network request if a packet is dropped:
const num_packets$ = new BehaviorSubject();
cold_request$.scan((num, _) => num + 1, 0)
.repeatWhen(completions =>
completions.takeWhile(_ => num_packets$.getValue() < expected_num_packets)
)
.subscribe(num_packets$);
The above code would be susceptible to a race condition between the last increment[s] of num_packets$ and the completion of the request unless onNext notifications out of scan all come before the request stream completes.
Related
I have a hot observable emitting messages continuously. I need to pause it with an API REST endpoint like an actuator /messages/pause, and resume it with other API REST endpoint like /messages/resume. During the pause time I need to allow with other API REST endpoint to emit a message like the original observable /messages/custom.
It is possible to pause the main observable stream during that interval (pause-resume) but not stop observing the mocked messages events, and continue/restore the main observable stream after resume it?
I think this would be an approach:
// The 2 sources of events.
const main$ = /* ... */;
const second$ = /* ... */;
// Emits when the `/pause` endpoint is reached.
// `pause.next(true);`
// When the `/resume` endpoint is reached: `pause.next(false)`
const pause = new Subject<boolean>();
const decoratedMain$ = main$.pipe(
withLatestFrom(paused.pipe(startWith(false))),
filter(([mainValue, isPaused]) => !isPaused),
map(([mainValue]) => mainValue),
);
const decoratedSecond$ = second$.pipe(
withLatestFrom(paused),
filter(([secondValue, isPaused]) => isPaused),
map(([secondValue]) => secondValue),
);
merge(decoratedMain$, decoratedSecond$).subscribe();
The above snippet, unless I've missed something, should implement this logic:
while the main$ observable emits, the second$ observable's events won't be taken into consideration
main$ can be paused(with pause.next(true)) and resumed(with pause.next(false))
when main$ is paused, its events are ignored and the second$ observable's events are now considered
when main$ is resumed, the switch takes place again: second$'s events are ignored and main$'s events are considered
Let's now see what part of RxJS' magic has been used to achieve this.
As you may have noticed, the main logic revolves around withLatestFrom and the pause Subject.
The decoratedMain$ observable keeps on emitting, but whether its events are ignored or not depends on the latest value of pause. If the /pause endpoint is reached, then the events will be ignored, since pause would have emitted true.
The decoratedSecond$ is built symmetrically. Its events are ignored if the latest value of pause is false.
Lastly, I thought it would be helpful to share a small variation of the above approach, just for learning purposes:
/* ... */
const isMainPaused$ = pause.pipe(filter(Boolean));
const isMainResumed$ = pause.pipe(filter(v => !v));
const decoratedMain$ = main$.pipe(
share({ resetOnRefCountZero: false }),
takeUntil(isMainPaused$),
repeatWhen(completionsSubject => completionsSubject.pipe(mergeMapTo(isMainResumed$)))
);
/* ... */
The rest of the code is the same as in the first approach. Here, what's going on is essentially ignoring the main$'s events by not having any subscriber to the share's Subject instance. When pause emits false, a new subscriber is created for that Subject instance, without re-subscribing to the source and that's due to the resetOnRefCountZero: false option.
I'm trying to cache http calls in the service so all subsequent calls returns same response. This is fairly easy with shareReplay:
data = this.http.get(url).pipe(
shareReplay(1)
);
But it doesn't work in case of backend / network errors. ShareReplay spams the backend with requests in case of any error when this Observable is bound to the view through async pipe.
I tried with retryWhen etc but the solution I got is untestable:
data = this.http.get(url).pipe(
retryWhen(errors => errors.pipe(delay(10000))),
shareReplay(1)
);
fakeAsync tests fails with "1 timer(s) still in the queue" error because delay timer has no end condition. I also don't want to have some hanging endless timer in the background - it should stop with the last subscription.
The behavior I would like:
Multicast - make only one subscription to source even with many subscribers.
Do not count refs for successful queries - reuse same result when subscriber count goes to 0 and back to 1.
In case of error - retry every 10 seconds but only if there are any subscribers.
My 2 cents:
This code is for rxjs > 6.4 (here V6.6)
To use a shared observable, you need to return the same observable for all the subscribers (or you will create an observable which has nothing to share)
Multicasting can be done using shareReplay and you can replay the last emitted value (even after the last subscriber to have unsubscribed) using the {refCount: false} option.
As long as there is no subscription, the observable does nothing. You will not have any fetch on the server before the first subscriber.
beware:
If refCount is false, the source will not be
unsubscribed meaning that the inner ReplaySubject will still be
subscribed to the source (and potentially run for ever).
Also:
A successfully completed source will stay cached in the shareReplayed
observable forever, but an errored source can be retried.
The problem is using shareReplay, you have to choose between:
Always getting the last value even if the refCount went back to 0 and having possible never ending retries in case of error (remember shareReplay with refCount to false never unsubscribes)
Or keeping the default refCount:true which mean you won't have the second "first subscriber" cache benefit. Conversely the retry will also stop if no subscriber is there.
Here is a dummy example:
class MyServiceClass {
private data;
// assuming you are injecting the http service
constructor(private http: HttpService){
this.data = this.buildData("http://some/data")
}
// use this accessor to get the unique (shared) instance of data observable.
public getData(){
return this.data;
}
private buildData(url: string){
return this.http.get(url).pipe(
retryWhen(errors => errors.pipe(delay(10000))),
shareReplay({refCount: false})
);
}
}
Now in my opinion, to fix the flow you should prevent your retry to run forever, adding for instance a maximum number of retries
Hi I'm looking for an RxJs operator that behaves similar to withLatestFrom, with the exception that it would wait for the second stream to emit a value instead of skipping it. To be claer: I only want emissions when the first stream emits a new value.
So instead of:
---A-----B----C-----D-|
------1----------2----|
withLatestFrom
---------B1---C1----D2|
I want this behavior:
---A-----B----C-----D-|
------1----------2----|
?????????????
------A1-B1---C1----D2|
Is there an operator for this?
Smola came up witha nice and clean solution in the comments that simply uses a distinctUntilKeyChanged operator:
combineLatest(first$, second$)
.pipe(distinctUntilKeyChanged(0))
As you can see in the RxViz diagram, this produces the desired result:
I don't think there's an operator that does exactly this, but you can achieve those results by combining a high order mapping operator and a Subject:
second$ = second$.pipe(shareReplay({ bufferSize: 1, refCount: false }));
first$.pipe(
concatMap(
firstVal => second$.pipe(
map(secondVal => `${firstVal}${secondVal}`),
take(1),
)
)
)
shareReplay places a ReplaySubject in front on the data producer. This means that it will reply latest N(bufferSize) values to every new subscriber. refCount makes sure that if there are no more active subscribers, the ReplaySubject in use won't be destroyed.
I decided to use concatMap as I think it's safer for the ReplaySubject to have only one active susbcriber.
Considering this scheme:
---A-----B----C-----D-| first$
------1----------2----| second$
When A comes in, the ReplaySubject(from shareReplay) will receive a new subscriber and A it will wait until second$ emits. When this happens, you'd get A1 and the inner observable will complete(meaning that its subscriber will be removed from the ReplaySubject's subscribers list). 1 will be cached by the ReplaySubject.
Then B comes in, the newly created inner subscriber will subscribe to second$ and will receive 1 immediately, resulting into B1. Same with C.
Now comes an important part: the ReplaySubject should have no active subscribers when it receives a new value from its source, so that's why I opted for take(1). When 2 will come, the ReplaySubject will have no active subscribers, so nothing happens.
Then D arrives and will receive the latest stored value, 2, resulting into D2.
This is how I did this in kotlin using RxJava
Observable.merge(
Observable.combineLatest(streamOne(), streamTwo(), ::Pair).take(1),
streamOne().skip(1).withLatestFrom(streamTwo(), ::Pair)
).subscribe { // Do your thing }
Yeah, that's join. From your example it isn't clear if you're aming left/right join or inner join.
Oh, sorry. It's clear indeed. You need the semantics of inner join (emits iff data in both joinee is present and otherwise waits).
What's the best way to handle asynchronous updates in the middle of an Observable stream.
Let's say there are 3 observables:
Obs1 (gets data from API) -> pipes to Obs2
Obs2 (transforms data) -> pipes to Obs3
Obs3 (sends transformed data)
(The actual application is more complex, and there's reasons it's not done in a single Observable, this is just a simple example).
That all works well and good if it's a linear synchronous path.
But we also have async messages that will change the output of Obs2.
3 scenarios I'm asking about are:
- we fetch data, and go through Obs1, Obs2 & Obs3
- we get a message to make a change, go through Obs2 & Obs3
- we get a different message to make a change which also needs to apply the change from the previous message, through Obs2 & Obs3
The main problem here is that there are different types of asynchronous messages that will change the outcome of Obs2, but they all need to still know what the previous outcome of Obs2 was (so the any other changes from messages that happened before is still applied)
I have tried using switchMap in Obs2 with a scan in Obs1 like this:
obs1
const obs1$ = obs1$.pipe(
// this returns a function used in the reducer.
map((data) => (prevData) => 'modifiedData',
scan((data, reducer) => reducer(betsMap), {})
)
obs2
const obs2$ = obs1$.pipe(
switchMap(data =>
someChange$.pipe(map(reducer => reducer(data)))
)
)
where someChange$ is a BehaviorSubject applying a change using another reducer function.
This works fine for async message #1 that makes some change.
But when message #2 comes in and a different change is needed, the first change is lost.
the changes that should be in "prevData" in obs1$ is always undefined because it happens before the message is applied.
How can I take the output from obs2$ and apply asynchronous updates to it that remembers what all of the past updates was? (in a way where I can clear all changes if needed)
So if i got the question right, there are two problems that this question tackles:
First: How to cache the last 2 emitted values from stream.
scan definitely is the right way, if this cache logic is needed in more than one place/file, I would go for a custom pipe operator, like the following one
function cachePipe() {
return sourceObservable =>
sourceObservable.pipe(
scan((acc, cur) => {
return acc.length === 2 ? [...acc.slice(1), cur] : [...acc, cur];
}, [])
);
}
cachePipe will always return the latest 2 values passed trough the stream.
...
.pipe(
cachePipe()
)
Second: How to access data from multiple streams at the same time, upon stream event
Here rxjs's combineLatest creation operator might do the trick for you,
combineLatest(API$, async1$ ,async2$,async3$)
.pipe(
// here I have access to an array of the last emitted value of all streams
// and the data can be passed to `Obs2` in your case
)
In the pipe I can chain whatever number of observables, which resolves the second problem.
Note:
combineLatest needs for all streams, inside of it, to emit once, before the operator strats to emit their combined value, one workaround is to use startWith operator with your input streams, another way to do it is by passing the data trough BehaviorSubject-s.
Here is a demo at CodeSandbox , that uses the cachePipe() and startWith strategy to combine the source (Obs1) with the async observables that will change the data.
I have a question regarding multicasted observables and an unexpected (for me) behaviour I noticed.
const a = Observable.fromEvent(someDom, 'click')
.map(e => 1)
.startWith(-1)
.share();
const b = a.pairwise();
a.subscribe(a => {
console.log(`Sub 1: ${a}`);
});
a.subscribe(a => {
console.log(`Sub 2: ${a}`)
});
b.subscribe(([prevA, curA]) => {
console.log(`Pairwise Sub: (${prevA}, ${curA})`);
});
So, there is a shared observable a, which emits 1 on every click event. -1 is emitted due to the startWith operator.
The observable b just creates a new observable by pairing up latest two values from a.
My expectation was:
[-1, 1] // first click
[ 1, 1] // all other clicks
What I observed was:
[1, 1] // from second click on, and all other clicks
What I noticed is that the value -1 is emitted immediately and consumed by Sub 1, before even Sub 2 is subscribed to the observable and since a is multicasted, Sub 2 is too late for the party.
Now, I know that I could multicast via BehaviourSubject and not use the startWith operator, but I want to understand the use case of this scenario when I use startWith and multicast via share.
As far as I understand, whenever I use .share() and .startWith(x), only one subscriber will be notified about the startWith value, since all other subscribers are subscribed after emitting the value.
So is this a reason to multicast via some special subject (Behavior/Replay...) or am I missing something about this startWith/share scenario?
Thanks!
This is actually correct behavior.
The .startWith() emits its value to every new subscriber, not only the first one. The reason why b.subscribe(([prevA, curA]) never receives it is because you're using multicasting with .share() (aka .publish().refCount()).
This means that the first a.subscribe(...) makes the .refCount() to subscribe to its source and it'll stay subscribed (note that Observable .fromEvent(someDom, 'click') never completes).
Then when you finally call b.subscribe(...) it'll subscribe only to the Subject inside .share() and will never go through .startWith(-1) because it's multicasted and already subscribed in .share().