I need to subscribe to a result, but wait for intermediate operations to finish before getting the result. The trick is that I "visit" my result to populate it:
// a service that gets a model
service.getModel(): Observable<MyModel>;
// I need to enrich my model before consuming it
service.getModel()
.makeSureAllCodesAreFetched(data => visitModel(model))
.subscribe(data => console.log("data is ready: ", data));
// a visitor that visits the model tree and enriches the leaves
// recursively visit the branches
visitModel(model: MyModel) {
if (model.isLeaf) {
// on condition, call a service to fetch additional data
service.fetchCodes(model.codeKey).subscribe(codes => model.codes = codes);
} else {
model.properties.forEach(prop: MyModel => visit(prop));
}
}
I tried to play with merges and forkJoin() without success. I just want to make sure that all calls to fetchCodes(), regardless of the result, are done before my data is subscribed to.
I found a solution, but it's not the cleanest to my opinion.
// a service that gets a model
service.getModel(): Observable<MyModel>;
// I need to enrich my model before consuming it
service.getModel()
.pipe(
mergeMap(data => forkJoin(visitModel(model))))
.subscribe(data => console.log("data is ready: ", data[0]));
// a visitor that visits the model tree and enriches the leaves
// recursively visit the branches
visitModel(model: MyModel, obs?: Observable<MyModel>[]): Observable<MyModel>[] {
if (obs === undefined) {
obs = [];
obs.push(of(model)); // make sure the very first Observable is the root
}
if (model.isLeaf) {
// on condition, call a service to fetch additional data
// push Observable result in the array
obs.push(service.fetchCodes(model.codeKey).map(codes => {
model.codes = codes;
return model;
}));
} else {
model.properties.forEach(prop: MyModel => visit(prop, obs)); // recursive call
}
return obs;
}
My visitor will actually append all the calls to fetchCodes() into an array of Observables and return it. That way forkJoin will wait for all the calls to be completed. The trick (and dirty part) is that I must make sure the first Observable is actually the root element I am interested in.
Related
I want combineLatest functional but for variable count of observables.
Something like:
// init combineLatest of three observables
[1, 2, 3]
// first observable produced new value "2"
[2, 2, 3]
// third observable ended
[2, 2]
// first observable produced new value "1"
[1, 2]
// new observable added
[2, 2, 4]
Is it possible in RxJS?
If I understand the problem right, the solution is pretty tricky for something that looks innocent.
I try to go step by step to explain a potential solution.
First of all we need understand that there are 3 different events that we need to manage:
the fact that one Observable completes
the fact that one Observable is added to the array which is given to combineLatest
the fact that a new array of Observables has to be passed to combineLatest, either because we are at the beginning of the processing (i.e. with the initial array) or because we have added a new Observable or because one Observable has completed
The second thing that we need to recognize is that we need to store the array of Observables we pass to combineLatest in a variable, otherwise we are not able to add or remove Obaservables from there.
Once these things are clear, we can build a solution in the form of a function that returns 2 things:
the Observable that we want to subscribe to and that should have the behavior that we are looking for
a Subject that we can use to communicate that we want to add a new Observable to the combineLatest function
The last point we need to recognize is that any time we change the list of Observable, either because we add or because we remove an Observable (because it completed), we need to run the combineLatest function with the new fresh list of Observables.
Now that all this has been clarified, this is the code of the function that returns an Observable which behaves as described
function dynamicCombineLatest(startingObservables: Observable<any>[]) {
// this is the variable holding the array of Observables
let observables = startingObservables;
// this is the array that contains the list of Observables which have been, potentially, transformed to emit
// immediately the last value emitted - this happens when a new Observable is added to the array
let observablesPotentiallyWithLastValueImmediatelyEmitted =
startingObservables;
// this is the variable holding the array of values last notified by each Observable
// we will use it when we need to add a new Observable to the list
const lastValues = [];
// this are the Subjects used to notify the 3 different types of events
const start = new BehaviorSubject<Observable<any>[]>(observables);
const add = new Subject<Observable<any>>();
const remove = new Subject<Observable<any>>();
let skipFirst = false;
// this is the chain of operations which must happen when a new Observable is added
const addToObservables = add.pipe(
tap({
next: (obs) => {
console.log("add");
// we need to make sure that the Observables in the list will immediately start to emit
// the last value they emitted. In this way we are sure that, as soon as the new added Observable emits somthing,
// the last value emitted by the previous Observables will be considered
observablesPotentiallyWithLastValueImmediatelyEmitted = observables.map(
(o, i) => {
return startWith(lastValues[i])(o);
}
);
// the new Observable is added to the list
observables.push(obs);
observablesPotentiallyWithLastValueImmediatelyEmitted.push(obs);
},
})
);
// this is the chain of operations which must happen when an Observable is removed
const removeFromObservables = remove.pipe(
tap({
next: (obs) => {
const index =
observablesPotentiallyWithLastValueImmediatelyEmitted.indexOf(obs);
console.log("remove");
// we simply remove the Observable from the list and it "last value"
observablesPotentiallyWithLastValueImmediatelyEmitted.splice(index, 1);
observables.splice(index, 1);
lastValues.splice(index, 1);
// we make sure that the Observables in the list will immediately start to emit with the last value they emitted
observablesPotentiallyWithLastValueImmediatelyEmitted = observables.map(
(o, i) => {
return lastValues[i] ? startWith(lastValues[i])(o) : o;
}
);
// we set that the first value of the new combineLatest Observable will be skipped
skipFirst = true;
},
})
);
// here we merge the 2 chains of operations so that both add and remove logic will be executed
// when the relative Subjects emit
merge(addToObservables, removeFromObservables).subscribe({
next: () => {
console.log("new start");
// we notify that a change in the Observable list has occurred and therefore we need to unsubscribe the previous "combineLatest"
// and subscribe to the new one we are going to build
start.next(observablesPotentiallyWithLastValueImmediatelyEmitted);
},
});
// this is where we switch to a new Observable, result of the "combineLatest" operation,
// any time the start Subject emits a new Observable list
const dynamicObservables = start.pipe(
switchMap((_observables) => {
const _observablesSavingLastValueAndSignallingRemove = _observables.map(
(o, i) =>
o.pipe(
tap({
next: (v) => {
// here we save the last value emitted by each Observable
lastValues[i] = v;
},
complete: () => {
// here we notify that the Observable has completed and we need to remove it from the list
remove.next(o);
},
})
)
);
console.log("add or remove");
// eventually this is the Observable created by combineLatest with the expected array of Observables
const _combineLatest = combineLatest(
_observablesSavingLastValueAndSignallingRemove
);
const ret = skipFirst ? _combineLatest.pipe(skip(1)) : _combineLatest;
skipFirst = false;
return ret;
})
);
// here we return the Observable which will be subscribed to and the add Subject to be used to add new Observables
return { dynamicObservables, add };
}
You can look at this stackblitz for an example.
Buffer and combine based on a key
Here's a slight variant of what you're asking for. It works just like mergeAll, only it keeps a buffer and emits the latest for any observable that have emitted so far.
The varient here is that you need to supply string keys for your values to get attached to. You should be able to see how to turn this into array indices if you so choose.
The reason I haven't done this with an array is because there's no much undefined behavior. For example, if the first observable completes and the second observable emits, your elements are all opaquely re-ordered.
Using keys returns control back to the caller, who can just use Object.keys() if they don't care about indices/labels for their data.
Here you are:
interface LabeledObservable<T> {
label: string,
stream: Observable<T>
}
interface CombinedLatest<T> {
[key:string]: T
}
function combineLatestAll<T>():
OperatorFunction<
LabeledObservable<T>,
CombinedLatest<T>
>
{
return source$ => defer(() => {
const buffer = {};
return source$.pipe(
mergeMap(({label, stream}) => stream.pipe(
map(v => {
buffer[label] = v;
return {...buffer};
}),
finalize(() => {
delete buffer[label];
})
))
);
});
}
Subject for new observables
If you like the idea of a subject you can use to inject new observables into your combineLatest operator, this still allows that. The only alteration needed is that you must supply unique labels for your observables. If you don't care about the labels, you can just use any ID generator pattern (Like incrementing a global id counter or something).
const startingObservables: Observable<any>[] = /*some observables */;
const add = new Subject<LabeledObservable<any>>();
add.pipe(
combineLatestAll()
).subscribe(console.log);
startingObservables.forEach((stream,i) => {
add.next({label: "" + i, stream});
});
I asked a question
Is Observable from chained promises equivalent of observables created with from and chained with concatMap?
on totally false premises. It seems that neither of my solutions had nothing to do with my intention.
I created a method that returns Observable and calls 2 methods returning Promise. I tried 2 ways:
public setItemInfos(itemInfos: IItemInfo[]): Observable<number> {
return from(this.db.selectionItemInfos.clear().then(() => {
return this.db.selectionItemInfos.bulkAdd(itemInfos);
}));
}
public setItemInfos(itemInfos: IItemInfo[]): Observable<number> {
const clear$ = from(this.db.selectionItemInfos.clear());
const bulkAdd$ = from(this.db.selectionItemInfos.bulkAdd(itemInfos));
return clear$.pipe(concatMap(() => bulkAdd$))
}
the use will be:
myService.setItemInfos(itemInfos).subsribe(count => {
console.log(`Cleared the table 1st and then added ${count} new items`);
});
I thought from both versions that:
table clear is execution is finished when bulkAdd starts
when bulkAdd is finished i get the count from that in subscribe
How this should really be done? Or can it be done?
This is (from what I can tell here), how I would do it.
In general, defer (or any higher-order operator) is a better way to create an observable from a promise. Defer lets you take the eager evaluation semantics of promises and turn them into the lazy evaluation semantics of observables.
Then all the usual observable operators and such will function as expected.
public setItemInfos(itemInfos: IItemInfo[]): Observable<number> {
const clear$ = defer(() => this.db.selectionItemInfos.clear());
const bulkAdd$ = defer(() => this.db.selectionItemInfos.bulkAdd(itemInfos));
return concat(clear$, bulkAdd$);
}
Update 1:
So I think I might know what you're after. This isn't really idiomatic RxJS since it's such an interleaving mix of declarative, imperative style of code. Even so, this should work? I haven't tested it fully, but some tinkering and I think this should do what you're after.
There's most assuredly a better way to accomplish the same thing, but without seeing the bigger picture of what you're after, it's hard to say.
interface Tagged<T> {
payload: T,
tag: number
}
class abitraryClass{
private setItemInfoSub: Subject<Tagged<IItemInfo[]>>;
private processItemInfo: Observable<Tagged<number>>;
private itemInfoTag = 0;
constructor(){
this.setItemInfoSub = new Subject<Tagged<IItemInfo[]>>();
this.processItemInfo = this.setItemInfoSub.pipe(
concatMap(({tag, payload: itemInfos}) => this.db.selectionItemInfos.clear().pipe(
ignoreElements(),
concatWith(defer(() => this.db.selectionItemInfos.bulkAdd(itemInfos))),
map(response => ({
payload: response,
tag
}))
)),
shareReplay(1)
);
// Make the processing pipeline live at all times.
this.processItemInfo.subscribe();
}
public setItemInfos(itemInfos: IItemInfo[]): Observable<number> {
const myTag = this.itemInfoTag++;
this.setItemInfoSub.next({
payload: itemInfos,
tag: myTag
});
return this.processItemInfo.pipe(
filter(({tag}) => tag == myTag),
map(({payload}) => payload)
);
}
}
RxJS provides the sequenceEqual operator to compare two streams in order. How would one go about testing equality of two streams regardless of order?
Pseudocode:
//how do we implement sequenceEqualUnordered?
from([1,2,3]).pipe(sequenceEqualUnordered(from([3,2,1]))).subscribe((eq) =>
console.log("Eq should be true because both observables contain the same values")
)
In my particular use case I need to wait until a certain set of values has been emitted or error but I don't care what order they're emitted in. I just care that each value of interest is emitted once.
Here's my solution:
import { Observable, OperatorFunction, Subscription } from 'rxjs';
export function sequenceEqualUnordered<T>(compareTo: Observable<T>, comparator?: (a: T, b: T) => number): OperatorFunction<T, boolean> {
return (source: Observable<T>) => new Observable<boolean>(observer => {
const sourceValues: T[] = [];
const destinationValues: T[] = [];
let sourceCompleted = false;
let destinationCompleted = false;
function onComplete() {
if (sourceCompleted && destinationCompleted) {
if (sourceValues.length !== destinationValues.length) {
emit(false);
return;
}
sourceValues.sort(comparator);
destinationValues.sort(comparator);
emit(JSON.stringify(sourceValues) === JSON.stringify(destinationValues));
}
}
function emit(value: boolean) {
observer.next(value);
observer.complete();
}
const subscriptions = new Subscription();
subscriptions.add(source.subscribe({
next: next => sourceValues.push(next),
error: error => observer.error(error),
complete: () => {
sourceCompleted = true;
onComplete();
}
}));
subscriptions.add(compareTo.subscribe({
next: next => destinationValues.push(next),
error: error => observer.error(error),
complete: () => {
destinationCompleted = true;
onComplete();
}
}));
return () => subscriptions.unsubscribe();
});
}
As many of RxJS operators have some input parameters and as all of them return functions, sequenceEqualUnordered also has some input parameter (mostly the same as Rx's sequenceEqual operator) and it returns a function. And this returned function has the Observable<T> as the source type, and has Observable<boolean> as the return type.
Creating a new Observable that will emit boolean values is exactly what you need. You'd basically want to collect all the values from both source and compareTo Observables (and store them to sourceValues and destinationValues arrays). To do this, you need to subscribe to both source and compareTo Observables. But, to be able to handle subscriptions, a subscriptions object has to be created. When creating a new subscriptions to source and compareTo, just add those subscriptions to subscriptions object.
When subscribing to any of them, collect emitted values to appropriate sourceValues or destinationValues arrays in next handlers. Should any errors happen, propagate them to the observer in error handlers. And in complete handlers, set the appropriate sourceCompleted or destinationCompleted flags to indicate which Observable has completed.
Then, in onComplete check if both of them have completed, and if they all are, compare the emitted values and emit appropriate boolean value. If sourceValues and destinationValues arrays don't have the same lengths, they can't equal the same, so emit false. After that, basically sort the arrays and then compare the two.
When emitting, emit both the value and complete notification.
Also, the return value of function passed to the new Observable<boolean> should be the unsubscribe function. Basically, when someone unsubscribes from new Observable<boolean>, it should also unsubscribe from both source and compareTo Observables and this is done by calling () => subscriptions.unsubscribe(). subscriptions.unsubscribe() will unsubscribe from all subscriptions that are added to it.
TBH, I haven't wrote any tests for this operator, so I'm not entirely sure that I have covered all edge cases.
My first idea. Use toArray on both then zip them together finally sort and compare results?
Hi i have a global service for several applications and i wish to make a method with several subscribes in order to stock and initialize all my datas and i wish make a subscribe to this method in my appComponent but i don't know how to make that
In my service
private initData(isLogged: boolean) {
this.http.get('/api/conf').subscribe(
conf => {
this.http.get('api/param').subscribe(
tokResp => {
this.appParams.token = tkResp.queoval;
this.appParams.culture = tkResp.culture;
this.appParams.GMT = tkResp.gmt;
this.http.get('/api/trad').subscribe(
trad => {
this.label = trad
// return an Observable
}
)
}
)
}
)
}
In my AppComponent
this.service.initData().subscribe(
result => {
this.test = result
}
How can i make that? I can't find the information in the documentation. Thank you for your help. It's important for my work, i used so much time to research for nothing :(
So since you want to make multiple async requests one after the other, you should use the observable function ".flatMap" (this is very similar to a Promises ".then"). The ".flatMap" function allows you to wait until the first request is completed before you continue.
So in your case, you would want your service to look something like this:
private initData(isLogged: boolean) {
return this.http.get('/api/conf').flatMap(
conf => {
return this.http.get('api/param');
}
).flatMap(
tokResp => {
this.appParams.token = tkResp.queoval;
this.appParams.culture = tkResp.culture;
this.appParams.GMT = tkResp.gmt;
return this.http.get('/api/trad');
}
).flatMap(
trad => {
this.label = trad;
return trad;
}
);
}
This function has all of the async requests chained together through ".flatMap" so that they are only called after the previous request completes.
The component file looks fine and should work with this new service.
As a general note, you should never subscribe to the observable inside
of the service. You should instead use functions like map, flatMap,
forkJoin ...
I need to create an observable, which I can "pull" data from, to work with a pageable api. I can only fetch 100 items per request, I want to be able to use observable as a generator function (on which I can call .next() to issue a request to get next 100 items.
I can't unfortunately find a way to do it with Rx. I suppose it's possible using controlled observable or a subject. Can you guys show me an example.
this is what I've gotten so far:
function list(entityType, viewName, fetchAll = false) {
var skip = 0,
total = 0;
const subject = new Rx.Subject(),
response$ = subject
.takeWhile(() => skip <= total)
.startWith(skip)
.flatMap((skip) => fetchPagePromise(skip)),
next = () => subject.onNext(skip);
if (fetchAll) {
Rx.Observable.timer(100, 100).subscribe(() => next());
}
return {
data$: response$.map(response => response),
next: fetchAll === true ? undefined : next
};
function fetchPagePromise() {
let limit = 100,
obj = {
viewName, limit, skip
},
qs = objectToQueryString(obj);
return $http.get(`${apiBase}/api/data/${entityType}${qs}`).then((res) => {
total = res.data.Total;
skip += limit;
return res.data.Rows;
});
}
}
this kinda works like a generator. it returns an Observable and next handler. Whenever next is called it pulls next 100 items from api and pushes into the Observable. Also if there’s a third parameter fetchAll passed, then it will keep fetching data until there’s no more. What scares me though that there are 2 mutating vars in function's closure - skip and total, and I don't know if managing them like this in asynchronous/unpredictable environment is ok.
One of the things you generally want to avoid is trying to make Rx into a plain old event emitter. Usually it is an indicator when you try and just trigger Observables manually by passing around a Subjects observer interface.
You should ask yourself, where is my data coming from? What calls next(), what calls that, etc. After enough of these you will generally find that this will lead you to something that can be wrapped by an Observable directly rather than explicitly calling next(). Also, I think the fetchAll flag should really be kept externally. You are only making the interface confusing by essentially turning it into a void method just by passing in a flag.
So I would recommend refactoring like so:
Rx.Observable.prototype.lazyRequest = function(entityType, viewName, limit = 100) {
var source = this;
return Rx.Observable.create(obs => {
var response = source
//Skip is really just the (limit * index)
.map((x, i) => i * limit)
.flatMap((skip) => {
let obj = {viewName, skip, limit},
qs = objectToQueryString(obj);
//Handle promises implicitly
return $http.get(`${apiBase}/api/data/${entityType}${qs}`);
},
//Return this with our skip information
(skip, res) => {skip, res})
//Publish it so the stream get shared.
.publish();
//This will emit once once you are out of data
var stop = response.first(x => x.skip >= x.res.data.Total);
return new CompositeDisposable(
//Complete this stream when stop emits
response.takeUntil(stop)
//Downstream only cares about the data rows
.map(x => x.res.data.Rows)
.subscribe(obs),
//Hook everything up
response.connect());
});
}
Then you can use it like so:
//An example of a "starting point", a button click
//Update the rows every time a new event comes through
Rx.Observable.fromEvent($button, 'click')
.startWith(0) //Inject some data into the pipeline
.lazyRequest(entityType, viewName)
.subscribe(/*Do something with the returned rows*/);
//Get all of the rows, will keep hitting the endpoint until it completes
Rx.Observable.interval(100)
.lazyRequest(entityType, viewName)
//Gather all the values into an array and emit that.
.toArray()
.subscribe();