In my case there are multiple requests could be performed in parallel at first, after those requests complete, another request will be sent with previous result, the pseudo code would look like
let uploads$ = [obs1$, obs2$, obs3$];
Observable.forkJoin(uploads$).mergeMap(
res => {
// never get called if uploads$ = []
let data = someCalculation(res);
return this.http.post('http://endpoint/api/resource', data);
}
).subscribe(
res => {
}
);
If uploads$ = [], the inner mergeMap never got called.
Can someone help? I'm on RxJS 5.4
It's not called b/c there is no emission on the source observable. To create one on, if observables is empty you can use the defaultIfEmpty or toArray operators.
const observables = [];
Rx.Observable.forkJoin(observables)
.defaultIfEmpty([]) // or .toArray()
.mergeMap(results => Rx.Observable.of(results.length))
.subscribe(console.log);
Related
My situation is as follows: I am performing sequential HTTP requests, where one HTTP request depends on the response of the previous. I would like to combine the response data of all these HTTP requests into one observable. I have implemented this before using an async generator. The code for this was relatively simple:
async function* AsyncGeneratorVersion() {
let moreItems = true; // whether there is a next page
let lastAssetId: string | undefined = undefined; // used for pagination
while (moreItems) {
// fetch current batch (this performs the HTTP request)
const batch = await this.getBatch(/* arguments */, lastAssetId);
moreItems = batch.more_items;
lastAssetId = batch.last_assetid;
yield* batch.getSteamItemsWithDescription();
}
}
I am trying to move away from async generators, and towards RxJs Observables. My best (and working) attempt is as follows:
const observerVersion = new Observable<SteamItem>((subscriber) => {
(async () => {
let moreItems = true;
let lastAssetId: string | undefined = undefined;
while (moreItems) {
// fetch current batch (this performs the HTTP request)
const batch = await this.getBatch(/* arguments */, lastAssetId);
moreItems = batch.more_items;
lastAssetId = batch.last_assetid;
const items = batch.getSteamItemsWithDescription();
for (const item of items) subscriber.next(item);
}
subscriber.complete();
})();
});
Now, I believe that there must be some way of improving this Observer variant - this code does not seem very reactive to me. I have tried several things using pipe, however unfortunately these were all unsuccessful.
I found concatMap to come close to a solution. This allowed me to concat the next HTTP request as an observable (done with the this.getBatch method), however I could not find a good way to also not abandon the response of the current HTTP request.
How can this be achieved? In short I believe this problem could be described as appending data to an observable inside the observable itself. (But perhaps this is not a good way of handling this situation)
TLDR;
Here's a working StackBlitz demo.
Explanation
Here would be my approach:
// Faking an actual request
const makeReq = (prevArg, response) =>
new Promise((r) => {
console.log(`Running promise with the prev arg as: ${prevArg}!`);
setTimeout(r, 1000, { prevArg, response });
});
// Preparing the sequential requests.
const args = [1, 2, 3, 4, 5];
from(args)
.pipe(
// Running the reuqests sequantially.
mergeScan(
(acc, crtVal) => {
// `acc?.response` will refer to the previous response
// and we're using it for the next request.
return makeReq(acc?.response, crtVal);
},
// The seed(works the same as `reduce`).
null,
// Making sure that only one request is run at a time.
1
),
// Combining all the responses into one object
// and emitting it after all the requests are done.
reduce((acc, val, idx) => ({ ...acc, [`request${idx + 1}`]: val }), {})
)
.subscribe(console.warn);
Firstly, from(array) will emit each item from the array, synchronously and one by one.
Then, there is mergeScan. It is exactly the result of combining scan and merge. With scan, we can accumulate values(in this case we're using it to have access to the response of the previous request) and what merge does is to allow us to use observables.
To make things a bit easier to understand, think of the Array.prototype.reduce function. It looks something like this:
[].reduce((acc, value) => { return { ...acc }}, /* Seed value */{});
What merge does in mergeScan is to allow us to use the accumulator something like (acc, value) => new Observable(...) instead of return { ...acc }. The latter indicates a synchronous behavior, whereas with the former we can have asynchronous behavior.
Let's go a bit step by step:
when 1 is emitted, makeReq(undefined, 1) will be invoked
after the first makeReq(from above) resolves, makeReq(1, 2) will be invoked
after makeReq(1, 2) resolves, makeReq(2, 3) will be invoked and so on...
Somebody I consulted regarding this matter came up with this solution, I think it's quite elegant:
defer(() => this.getBatch(options)).pipe(
expand(({ more_items, last_assetid }) =>
more_items
? this.getBatch({ ...options, startAssetId: last_assetid })
: EMPTY,
),
concatMap((batch) => batch.getSteamItemsWithDescription()),
);
From my understanding the use of expand here is very similar to the use of mergeScan in #Andrei's answer
To be honest I am a total noob at NGRX and only limited experience in rxjs. But essentially I have code similar to this:
#Effect()
applyFilters = this.actions$.pipe(
ofType<ApplyFilters>(MarketplaceActions.ApplyFilters),
withLatestFrom(this.marketplaceStore.select(appliedFilters),
this.marketplaceStore.select(catalogCourses)),
withLatestFrom(([action, filters, courses]) => {
return [courses,
this.combineFilters([
this.getCourseIdsFromFiltersByFilterType(filters, CatalogFilterType.TRAINING_TYPE),
this.getCourseIdsFromFiltersByFilterType(filters, CatalogFilterType.INDUSTRIES)
])
];
}),
map(([courses, filters]) => {
console.log('[applyFilters effect] currently applied filters =>', filters);
console.log('courseFilters', filters);
const filteredCourses = (courses as ShareableCourse[]).filter(x => (filters as number[]).includes(+x.id));
console.log('all', courses);
console.log('filtered', filteredCourses);
return new SetCatalogCourses(filteredCourses);
})
);
Helper method:
private combineFilters(observables: Observable<number[]>[]): number[] {
if (!observables.some(x => x)) {
return [];
} else {
let collection$ = (observables[0]);
const result: number[] = [];
for (let i = 0; i < observables.length; i++) {
if (i >= 1) {
collection$ = concat(collection$, observables[i]) as Observable<number[]>;
}
}
collection$.subscribe((x: number[]) => x.forEach(y => result.push(y)));
return result;
}
}
So essentially the store objects gets populated, I can get them. I know that the observables of 'this.getCourseIdsFromFiltersByFilterType(args)' do work as on the console log of the 'filters' they are there. But the timing of the operation is wrong. I have been reading up and am just lost after trying SwitchMap, MergeMap, Fork. Everything seems to look okay but when I am trying to actually traverse the collections for the result of the observables from the service they are not realized yet. I am willing to try anything but in the simplest form the problem is this:
Two observables need to be called either in similar order or pretty close. Their 'results' are of type number[]. A complex class collection that has a property of 'id' that this number[] should be able to include. This works just fine when all the results are not async or in a component.(I event dummied static values with variables to check my 'filter' then 'includes' logic and it works) But in NGRX I am kind of lost as it needs a return method and I am simply not good enough at rxjs to formulate a way to make it happy and ensure the observables are fully realized for their values from services to be used appropriately. Again I can see that my console log of 'filters' is there. Yet when I do a 'length' of it, it's always zero so I know somewhere there is a timing problem. Any help is much appreciated.
If I understand the problem, you may want to try to substitute this
withLatestFrom(([action, filters, courses]) => {
return [courses,
this.combineFilters([
this.getCourseIdsFromFiltersByFilterType(filters, CatalogFilterType.TRAINING_TYPE),
this.getCourseIdsFromFiltersByFilterType(filters, CatalogFilterType.INDUSTRIES)
])
];
}),
with something like this
switchMap(([action, filters, courses]) => {
return forkJoin(
this.getCourseIdsFromFiltersByFilterType(filters, CatalogFilterType.TRAINING_TYPE),
this.getCourseIdsFromFiltersByFilterType(filters, CatalogFilterType.INDUSTRIES
).pipe(
map(([trainingFilters, industryFilters]) => {
return [courses, [...trainingFilters, ...industryFilters]]
})
}),
Now some explanations.
When you exit this
withLatestFrom(this.marketplaceStore.select(appliedFilters),
this.marketplaceStore.select(catalogCourses)),
you pass to the next operator this array [action, filters, courses].
The next operator has to call some remote APIs and therefore has to create a new Observable. So you are in a situation when an upstream Observable notifies something which is taken by an operator which create a new Observable. Similar situations are where operators such as switchMap, mergeMap (aka flatMap), concatMap and exhastMap have to be used. Such operators flatten the inner Observable and return its result. This is the reason why I would use one of these flattening operators. Why switchMap in your case? It is not really a short story. Maybe reading this can cast some light.
Now let's look at the function passed to switchMap
return forkJoin(
this.getCourseIdsFromFiltersByFilterType(filters, CatalogFilterType.TRAINING_TYPE),
this.getCourseIdsFromFiltersByFilterType(filters, CatalogFilterType.INDUSTRIES
).pipe(
map(([trainingFilters, industryFilters]) => {
return [courses, [...trainingFilters, ...industryFilters]]
})
This function first executes 2 remote API calls in parallel via forkJoin, then take the result of these 2 calls and map it to a new Array containing both courses and the concatenation of trainingFilters and industryFilters
I have a class, QueueManager, which manages some queues.
QueueManager offers 3 APIs
deleteQueue(queueName: string): Observable<void>
createQueue(queueName: string): Observable<string>
listQueues(): Observable<string>: Observable`
deleteQueue is a fire-and-forget API, in the sense that it does not return any signal when it has completed its work and deleted the queue. At the same time createQueue fails if a queue with the same name already exists.
listQueues() returns the names of the queues managed by QueueManager.
I need to create a piece of logic which deletes a queue and recreates it. So my idea is to do something like
call the deleteQueue(queueName) method
start a loop calling the listQueues method until the result returned shows that queueName is not there any more
call createQueue(queueName)
I do not think I can use retry or repeat operators since they resubscribe to the source, which in this case would mean to issue the deleteQueue command more than once, which is something I need to avoid.
So what I have thought to do is something like
deleteQueue(queueName).pipe(
map(() => [queueName]),
expand(queuesToDelete => {
return listQueues().pipe(delay(100)) // 100 ms of delay between checks
}),
filter(queues => !queues.includes(queueName)),
first() // to close the stream when the queue to cancel is not present any more in the list
)
This logic seems actually to work, but looks to me a bit clumsy. Is there a more elegant way to address this problem?
The line map(() => [queueName]) is needed because expand also emits values from its source observable, but I don't think that's obvious from just looking at it.
You can use repeat, you just need to subscribe to the listQueues observable, rather than deleteQueue.
I've also put the delay before listQueues, otherwise you're waiting to emit a value that's already returned from the API.
const { timer, concat, operators } = rxjs;
const { tap, delay, filter, first, mapTo, concatMap, repeat } = operators;
const queueName = 'A';
const deleteQueue = (queueName) => timer(100);
const listQueues = () => concat(
timer(1000).pipe(mapTo(['A', 'B'])),
timer(1000).pipe(mapTo(['A', 'B'])),
timer(1000).pipe(mapTo(['B'])),
);
const source = deleteQueue(queueName).pipe(
tap(() => console.log('queue deleted')),
concatMap(() =>
timer(100).pipe(
concatMap(listQueues),
tap(queues => console.log('queues', queues)),
repeat(),
filter(queues => !queues.includes(queueName)),
first()
)
)
);
source.subscribe(x => console.log('next', x), e => console.error(e), () => console.log('complete'));
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.5.4/rxjs.umd.js"></script>
How to invoke a method when all the observers have unsubscribed from a subject.
Update
const alphaStore = new BehaviourSubject(0);
observer1 = alphaStore.subscribe(console.log);
observer2 = alphaStore.subscribe(console.log);
And when all of these observers unsubscribe. I want a method to be invoked. Like...
Observer1 unsubscribed
Observer2 unsubscribed
All observers left
What you describe already does the finalize() operator. Better said finalize() calls its callback when the chain disposes which means it's called when all observers unsubscribes, the chain completes or errors.
const subject = new Subject();
const shared = subject.pipe(
finalize(() => console.log('finalize')),
share(),
);
https://stackblitz.com/edit/rxjs-rebfba
When all observers unsubscribe share() unsubscribes from its source which triggers finalize().
Currently there's no way to distinguish why finalize() was invoked. See this issue https://github.com/ReactiveX/rxjs/issues/2823 and examples there on how to do it.
You can create a custom Observable, that will track the subscription count.
Heres a simple example:
let count = 0;
const tracked$ = new Observable(() => {
count++;
return ()=>{
count--;
if (count === 0) {
console.log('I am empty');
}
};
})
And then merge it with Observable that does actual work.
For simplicity sake, lets imagine its just a timer
// const tracked$ = ...
const data$ = Observable.timer(0, 5);
const result$ = data$
.merge(tracked$)
.take(5)
.subscribe(value => console.log('v:', value));
After 5 values were emitted -- it will log I am empty.
Heres a live example (with some rewrite and two subscriptions):
https://observable-playground.github.io/gist/4a7415f3528aa125fb686204041138cb
NOTE: this code uses rxjs-compat notation, which is easier to read. Above linked example uses .pipe notation, which is more common now.
Hope this helps.
The goal is to iterate through a collection of IDs, making an HTTP call for each ID. For each ID, I'm using a service with a get() method that returns an Observable. Each time the get() method is called, I'm subscribing to the returning Observable and trying to push the result into an array, which will eventually get passed on to a different method for a new operation.
Relevant service method:
public get(departmentId: number): Observable<IDepartmentModel> {
return super.get<IDepartmentModel>(departmentId);
}
note: the super class is leveraging Angular Http, which is well tested and confirmed to be working correctly. The problem with the logic isn't here...
Relevant component methods:
note the departmentService.get() call that's being called several times within the forEach.
setInitialDepartmentsAssignedGridData(): void {
this.settingsForDropdownSelectedCompanyId = this.userForm.get('defaultCompany').get('defaultCompanyId').value;
let departments: IDepartmentModel[] = [];
this.userService.user.getValue() //confirmed: valid user is being pulled back from the userService (logic is fine here..)
.userCompanies.find(comp => comp.companyId === this.settingsForDropdownSelectedCompanyId) // getting a valid match here (logic is fine here..)
.departmentIds.forEach(deptId => this.departmentService.get(deptId).first().subscribe(dept => { // getting a valid department back here (logic is fine here...)
departments.push(dept); // HERE LIES THE PROBLEM
}));
this.setDepartmentsAssignedRowData(departments);
}
setDepartmentsAssignedRowData(departments: IDepartmentModel[]): void {
console.log('setDeptAssignedRowData called'); // confirmed: method is getting called...
console.log(departments); // confirmed: fully-composed collection of departments logged to the console...
departments.forEach(dept => {
console.log(dept);
}); // Y U NO WORK!?
departments.map((department) => {
console.log(department); // Y U NO WORK?
this.departmentAssignedRowData.push({
departmentId: department.id,
departmentName: department.name
});
});
this.departmentAssignedGridOptions.api.setRowData(this.departmentAssignedRowData);
}
The problem is, although what's getting logged to the console is a fully-composed department-objects array, it's not TRULY "there"; what's getting passed to setDepartmentsAssignedRowData is an empty array.
I'm sure what's happening is that the async operations are not complete before the departments array gets passed to the second method. Some of what I've read online says to use forkJoin, but I can't see how that will look in this context. I've also read concatMap may work, but again, in this context, I'm not sure how to make that work...
In this context, how do I leverage RxJS to make sure the intended, fully-composed departments array is truly ready to be passed?
thanks for any insight you can provide. help is much appreciated!
You are correct, you need forkJoin
let observableArray = this.userService.user.getValue()
.userCompanies.find(comp => comp.companyId === this.settingsForDropdownSelectedCompanyId)
.departmentIds.map(deptId => this.departmentService.get(deptId)) // map is building out an array of observables
This will be an array of http request observables that you want to make in parallel. Now you can pass this array to forkJoin.
Observable.forkJoin(...observableArray)
The return of forkJoin will be an array of results from observableArray. forkJoin will not emit to the next operator in the sequence until all of the observables in observableArray have completed (so when all of the http requests have finished)
So altogether the code will be
let observableArray = this.userService.user.getValue()
.userCompanies.find(comp => comp.companyId === this.settingsForDropdownSelectedCompanyId)
.departmentIds.map(deptId => this.departmentService.get(deptId));
Observable.forkJoin(...observableArray).subscribe(res => {
// res = [resId0, resId1, resId2, ..., resIdX];
});
You mentioned passing the result to another operator. If that operator is another http request where you pass an array of data (from forkJoin), then you can use the flatMap operator.
Observable.forkJoin(...observableArray)
.flatMap(res => {
return this.otherApi(res);
})
.subscribe(res => {
// res is the result of the otherApi call
});
flatMap will chain your api requests together. So altogether what is happening is
run array of observables in parallel
once complete, run second api (otherApi)