how to make concatMap nested - rxjs

I need to obtain a series of data for each of the dates of origin that I have in an array
and I need to obtain them in order so I use concatMap to go through my observable of dates and when I obtain the first group of values everything goes fine
of(...this.etiquetasEjeX)
.pipe(
concatMap(item=>
this.dataService.getGastadoRealizadoEnMesYAño(this.proyectoId,
getMonthNumber(item.slice(0,item.length-4)),
+item.slice(-4),
this.acumular)
),
toArray()
)
.subscribe(item=>{
this.gastadoRealizado=item;
console.log('this.gastadoRealizado: ', this.gastadoRealizado);
});
I have my this.gastoRealizado array
but I need to do 3 More calls to the backend to obtain a total of 4 arrays that then feed a graph and I don't know how to add more calls in this established order
Any idea, please?
Thanks

Not sure if this answers your question (couldn't post a comment to ask for specifics). But assuming the other back end calls also depend only on an element from this.etiquetasEjeX you could use the zip operator. However regarding the usage of concatMap the four request would be performed simultaneously per item. If you were restricted to one api call at a time this solution would need some adjustments.
import { of, zip } from 'rxjs';
import { concatMap, toArray } from 'rxjs/operators';
//...
of(...this.etiquetasEjeX)
.pipe(
concatMap(item=>
zip(
this.dataService.getGastadoRealizadoEnMesYAño(...),
this.dataService.getB...,
this.dataService.getC...,
this.dataService.getD...,
),
toArray(),
)
.subscribe((arr: [ItemA, ItemB, ItemC, ItemD][])=> {
//...
});
Edit:
okay in your comment you mentioned that a subsequent request depends on the result of a prior request. Nesting concatMap operations like your initially requested would be a little messy as you can see in the following example:
of(...this.etiquetasEjeX)
.pipe(
concatMap(item=>
this.dataService.getGastadoRealizadoEnMesYAño(...).pipe(
concatMap(itemA =>
this.dataService.getB(itemA).pipe(
concatMap(itemB =>
this.dataService.getC(itemB).pipe(
concatMap(itemC =>
this.dataService.getD(itemC).pipe(
map(itemD =>
// every combination of items would be available here
this.getGraph(item, itemA, itemB, itemC, itemD)
)
)
)
)
)
)
)
)
),
toArray(),
)
.subscribe(graphs => {
// graphs array contains elements that were returned by this.getGraph
});
But the same operations could also be called sequentially without losing the intermediate results that you will need for feeding your graph:
of(...this.etiquetasEjeX)
.pipe(
concatMap(item=> combineLatest(
of(item),
this.dataService.getA(item, ...),
)),
concatMap(([item, itemA]) =>
this.dataService.getB(itemA, ...)
combineLatest(
of([item, itemA]), // it's important that the elements are enclosed inside an array here otherwise the of operator fires twice and doesn't have the proper effect
,
)),
concatMap(([prevItems, itemB]) => combineLatest(
of([...prevItems, itemB]),
this.dataService.getC(itemB, ...),
)),
concatMap(([prevItems, itemC]) => combineLatest(
of([...prevItems, itemC]),
this.dataService.getD(itemC, ...),
)),
map(([item, itemA, itemB, itemC, itemD]) =>
this.getGraph(item, itemA, itemB, itemC, itemD)
),
toArray(),
)
.subscribe(graphs => {
// update the state of your object eg.
this.myGraphs = graphs;
});
And I noticed that you are using toArray. That means your observable won't provide you with any intermediate results until all your api calls have been finished. Depending on the size of the resulting arrays provided by your api the given solution might be quite time and memory consuming.

Related

RxJS: execute concatMap i parallel

Is it possible to execute a high-order observable in parallel, but still preserve the order when merging the results?
I have something looking like this:
invoker$: Observable<void>;
fetch: (index: number) => Observable<T[]>;
invoker$
.pipe(
concatMap((_, index) => fetch(index)),
scan((acc: T[], values) => [...acc, ...values], [])
)
.subscribe(/* Do something with the array */);
The idea is having an observable that invokes a callback (e.g. backend call that takes a considerable amount of time) generating a new observable that emits a single value (array of some generic type). The returned values should be concatenated in another array while preserve their original fetch order.
I would, however, like the requests to be fired in parallel. So if the invoker$ is called rapidly, the requests are made in parallel and the results are merged as they complete.
My understanding is that the concatMap will wait for one observable to complete, before starting the next one. mergeMap will do it parallel, but won't do anything to preserve the order.
You can do it using mergeMap.
First, you need to pass the index together with the async response down the stream.
Then you can sort based on the index from the previous step.
Then you have two choices:
if the stream needs to end once all the requests are made and handle only once all the responses you can use reduce https://rxmarbles.com/#reduce
if the stream needs to continue for another batch of requests you need to use scan and later filter until you reach the needed event count. https://rxmarbles.com/#scan and https://rxmarbles.com/#filter
I am going to give you some pseudo-code for both examples:
In the reduce case, the stream ends once all requests are sent:
invoker$
.pipe(
mergeMap((_, index) => fetch(index).then(value => {value, index})),
reduce((acc: T[], singleValue) => [...acc, ...singleValue], []),
map(array => array.sort(/*Sort on index here*/).map(valueWithIndex => valueWithIndex.value))
)
.subscribe(/* Do something with the array */);
In the multiple-use case, I am assuming the size of the batch to be constant:
invoker$
.pipe(
mergeMap((_, index) => fetch(index).then(value => {value, index})),
scan((acc: T[], singleValue) => {
let resp = [...acc, ...singleValue];
// The scan can accumulate more than the batch size,
// so we need to limit it and restart for the new batch
if(resp.length > BATCH_SIZE) {
resp = [singleValue];
}
return resp;
}, []),
filter(array => array.length == BATCH_SIZE),
map(array =>
array
.sort(/*Sort on index here*/)
.map(valueWithIndex => valueWithIndex.value))
)
.subscribe(/* Do something with the array */);
2.1. In case the batch size is dynamic:
invoker$
.pipe(
mergeMap((_, index) => fetch(index).then(value => {value, index})),
withLatestFrom(batchSizeStream),
scan((acc: [T[], number], [singleValue, batchSize]) => {
let resp = [[...acc[0], ...singleValue], batchSize];
// The scan can accumulate more than the batch size,
// so we need to limit it and restart for the new batch
// NOTE: the batch size is dynamic and we do not want to drop data
// once the buffer size changes, so we need to drop the buffer
// only if the batch size did not change
if(resp[0].length > batchSize && acc[1] == batchSize) {
resp = [[singleValue], batchSize];
}
return resp;
}, [[],0]),
filter(arrayWithBatchSize =>
arrayWithBatchSize[0].length >= arrayWithBatchSize[1]),
map(arrayWithBatchSize =>
arrayWithBatchSize[0]
.sort(/*Sort on index here*/)
.map(valueWithIndex => valueWithIndex.value))
)
.subscribe(/* Do something with the array */);
EDIT: optimized sorting, added dynamic batch size case
I believe that the operator you are looking for is forkJoin.
This operator will take as input a list of observables, fire them in parallel and will return a list of the last emitted value of each observable once they all complete.
forkJoin({
invoker: invoker$,
fetch: fetch$,
})
.subscribe(({invoker, fetch}) => {
console.log(invoker, fetch);
});
Seems like this behavior is provided by the concatMapEager operator from the cartant/rxjs-etc library - written by Nicholas Jamieson
(cartant) who's a developer on the core RxJS team.

Rxjs, Combine last query result for multiple group (combine switchMap with groupBy)

I have an issue using the operator. Basically I have multiple payslips and I want to keep a debounce for each payslip and trigger a query. I want to subscribe to only the last query that succeed for a specific payslip and if a new request is triggered for this payslip before the previous one finished, I want to cancel the previous one.
Here's a sample marble diagram for what i'm looking for:
-----1----1-----1----3----3----3----3-----1---3---1---3---1------>
(magic operators for which I'm unclear)
-------------------1-------------------3-----1---3---1---3---1--->
I have debounce and query, which I like, but it does this:
-----1----1-----1----3----3----3----3-----1---3---1---3---1------>
debounce
-------------------1-------------------3--------------------1---->
.pipe(
groupBy(payslip => payslip._id),
map(group =>
group.pipe(
debounceTime(200),
switchMap(payslip => httpQuery)
)
),
mergeAll()
)
With the current solution, the merge all is grouping the switch map thus canceling even for other group. Is there a way to do what I want ?
Thanks !
I am struggling to find anything wrong with your code or it's outcomes.
I have put the following test code in the ThinkRx playground
const { rxObserver } = require('api/v0.3');
const { zip, timer, from, of } = require('rxjs');
const { take, map, groupBy, mergeAll, debounceTime, delay, switchMap } =
require('rxjs/operators');
zip(
timer(0, 50),
from([1,1,1,3,3,3,3,1,3,1,3,1]),
).pipe(
map(([_,i])=>i),
groupBy(i=>i),
map(group => group.pipe(
debounceTime(0),
switchMap(i=>of(i).pipe(delay(0))), // simulates http request with know duration
)),
mergeAll(),
).subscribe(rxObserver());
With debounce and time-for-http-request both 0 the result is:
With time-for-http-request = 60 (note the time scale is longer):
And then with debounce = 10:
Is it possible in your code that the http request simply takes longer than the time between the requests, so the switchMap is correctly cancelling the earlier one?

What is the difference between merge and mergeAll?

What is the difference between merge and mergeAll? They both seem identical to me:
http://reactivex.io/rxjs/class/es6/Observable.js~Observable.html#instance-method-mergeAll
http://reactivex.io/rxjs/class/es6/Observable.js~Observable.html#static-method-merge
merge is a static creation method that flattens group of observable.
according to the docs
Flattens multiple Observables together by blending their values into one Observable.
simply it will take a group of observables, and flattens them within one, so whenever any observable emits a value, the output will emit a value.
mergeAll However is different, it is an instance method that works with higher order observables (an observable that emits observables), according to docs
Converts a higher-order Observable into a first-order Observable which concurrently delivers all values that are emitted on the inner Observables.
I think that sums it up, but mergeAll can be confusing, so let's look at this example provided by rxjs docs
import { fromEvent, interval } from 'rxjs';
import { take, map, mergeAll } from 'rxjs/operators';
const higherOrder = fromEvent(document, 'click').pipe(
map((ev) => interval(1000).pipe(take(10))),
);
const firstOrder = higherOrder.pipe(mergeAll(2));
firstOrder.subscribe(x => console.log(x));
you have a document click observable (higher order) which return an interval observable (inner observable) that emits a value every second, it will complete after 10 intervals emits, which means every time you click on the document, a new interval will be returned, here where merge all comes in, it will subscribe to these intervals returned by the higher order observable, and flattens them into one observable, the first order observable, the argument 2, is to limit to 2 concurrent intervals at a time, so if you clicked 3 times, only 2 will run, but since these 2 intervals will complete after 10 seconds, then you can click again and mergeAll will subscribe to the new intervals.
Both merge and mergeAll inherit from mergeMap !
mergeAll
mergeAll is the same as calling mergeMap with an identity function(const identity = x => x)
mergeAll() === mergeMap(obs$ => obs$)
Example:
of(a$, b$, c$)
.pipe(
mergeAll(),
)
.subscribe()
// Same as
of(a$, b$, c$)
.pipe(
mergeMap(obs$ => obs$)
)
.subscribe()
Both will subscribe to the incoming observables(a$, b$ and c$) and will pass along to their values to the data consumer. Thus, a$, b$ and c$ are considered inner observables.
merge
Armed with the knowledge from the previous section, understanding merge should not be difficult.
merge(a$, b$, c$).subscribe() is essentially the same as
const observables = [a$, b$, c$];
new Observable(subscriber => {
for (let i = 0; i < observables.length; i++) {
subscriber.next(observables[i]);
}
subscriber.complete();
}).pipe(
mergeAll()
).subscribe();

Initialize observable with the result of other observable

I have 2 requests.
getCurrentBook(): Observable<Book>
getDetailedInfo(bookId): Observable <BookDetailed>
They both return observables with information, however to use second request I have to make sure that I received the information from the first one since bookId is in the response.
I understand that I could subscribe inside other subscribe, however this solution doesn't seem appealing to me. There must be a much more elegant way.
The existing solution
getCurrentBook().subscribe(res => {
getDetailedInfo(res.id).subscribe(...);
})
I get that it should look something like:
booksSubs = getCurrentBook().pipe(
map(res =>
{this.currentBook = res}
)
)
detailedSubs = getDetailedInfo(this.currentBook.id).pipe(
map(res =>
{this.detailed = res}
)
)
this.subscriptions.push(SOME OPERATOR(booksSubs, detailedSubs).subscribe();
But the option higher won't work since I need result of first observable to initialize second.
You can achieve it using some of "flattening" operators, for example mergeMap:
const currentBookDetails$ = getCurrentBook().pipe(
mergeMap(book => getDetailedInfo(book.id))
);

RxJS mergeMap doesn't behave as expected

I have this piece of RxJS code
this.listItems$ = this.store.select(EntityState.relationshipItems).pipe(
map(fn => fn(12)),
mergeMap(items => items),
map(this.toListItem),
toArray<ListItem>(),
tap(x => console.log(x))
);
Using mergeMap(items => items) I'm trying to "flatten" the array, then map each item to another object, and then convert it back to an array.
However, the flow doesn't even reach the last tap. I can see the toListItem function is called, but I don't understand why it stops there.
Transforming it to
this.listItems$ = this.store.select(EntityState.relationshipItems).pipe(
map(fn => fn(12)),
map(items => items.map(this.toListItem)),
tap(x => console.log(x))
);
makes it work, but I'd like to understand why the above one doesn't work.
That's because this.store.select(...) is a Subject that never completes (if it did then you could select data just once which doesn't make sense).
However, toArray collects all emissions from its source and when its source completes it emits a single array. But the source is this.store.select(...) that never completes so toArray never emits anything.
So probably the easiest workaround would be just restructuring your chain:
this.listItems$ = this.store.select(EntityState.relationshipItems).pipe(
map(fn => fn(12)),
mergeMap(items => from(items).pipe(
map(this.toListItem),
toArray<ListItem>(),
tap(x => console.log(x))
)),
);
Now the source is from that completes after iterating items so toArray will receive complete notification and emit its content as well.

Resources