concatMap() equivalent but async like in mergeMap() - rxjs

I have an observable myObservable :
let myObservable = Observable.of(2000, 1000)
With concatMap() : TOTAL TIME = 3000 millis, results in the original order.
myObservable.concatMap(v => Rx.Observable.of(v).delay(v))
// concatMap: 2000, concatMap: 1000
With mergeMap() : TOTAL TIME = 2000 millis, results not in the original order.
myObservable.mergeMap(v => Rx.Observable.of(v).delay(v))
// mergeMap: 1000, mergeMap: 2000
I want a way to get the results in the original order like in concatMap, but calling each nested observable asynchronously instead of waiting for the next nested observable to complete :
// --- The behavior that I want ---
myObservable.myCustomMap(v => Rx.Observable.of(v).delay(v))
// myCustomMap: 2000, myCustomMap: 1000
// TOTAL TIME = 2000 millis
Is there an elegant solution?
Edit : I am looking for a solution that also works if the source (myObservable) is asynchronous, not only for this particular synchronous case.

You should use forkJoin for firing all the observables at the same time.
Here's an example without comments:
const { Observable } = Rx;
const obs$ = Observable
.of(3000, 3000, 1000)
.map(x => Observable.of(x).delay(x));
const allObsWithDelay$ = obs$.toArray();
const result$ = allObsWithDelay$
.switchMap(arr => Observable.forkJoin(arr));
result$
.do(console.log)
.subscribe();
And the same with explanation:
const { Observable } = Rx;
// source observable, emitting simple values...
const obs$ = Observable
.of(3000, 3000, 1000)
// ... which are wrapped into a different observable and delayed
.map(x => Observable.of(x).delay(x));
// use a reduce to build an array containing all the observables
const allObsWithDelay$ = obs$.toArray();
const result$ = allObsWithDelay$
// when we receive the array with all the observable
// (so we get one event, with an array of multiple observables)
.switchMap(arr =>
// launch every observable into this array at the same time
Observable.forkJoin(arr)
);
// display the result
result$
.do(console.log)
.subscribe();
With those values: 3000, 3000, 1000 the whole process is taking 3 seconds (the maximum of them as they're fired at the same time)
Working Plunkr: https://plnkr.co/edit/IRgEhdjCmZSTc6hSaVeF?p=preview
Edit 1: Thanks to #PierreCitror for pointing out toArray which is better than scan :)

I would do it like this:
myObservable
.mergeMap((val, i) => Observable.forkJoin(
Observable.of(i),
Observable.of(v).delay(v)
))
.scan((acc, ([i, result])) => {
acc[i] = result;
return acc;
}, {})
.filter(allResults => {
// Whatever goes here
Object.keys(allResults) // list indices of all finished responses
})
This will accumulate all responses in a single object where each response is assigned an index at which it arrived into mergeMap.
Then in filter you can write whatever logic you want that decides whether the current state should be propagated further (eg. you can wait until a certain number of responses arrived or whatever).

Related

Using RxJS to remove nested callbacks when things must be done in sequence

I need to do one HTTP request after another, but the second one can't start until after the first one has finished because the second needs as a parameter a value returned from the first.
Here it is with a nested callback which is great because it works and it's fairly clear from reading the code what is happening.
this.isLoading = true;
this.firstService.get(this.id)
.subscribe((response: FirstReturnType) => {
this.firstThing = response;
this.secondService.get(this.firstThing.secondId)
.subscribe(
(response: SecondReturnType) => {
this.secondThing = response;
this.isLoading = false;
}
}
The claim I see people making is that nested callbacks are bad and that one should use RxJS to make it better.
However, nobody making these claims has been able to produce a working example. Can you?
Your Code Re-written
Here is some code that has a 1-1 correspondence with your code, but it is flattened
this.isLoading = true;
this.firstService.get(this.id).pipe(
mergeMap((response: FirstReturnType) => {
this.firstThing = response;
return this.secondService.get(response.secondId);
})
).subscribe((response: SecondReturnType) => {
this.secondThing = response;
this.isLoading = false;
});
What this gets right: you're using a higher-order observable operator to map a value emitted by one observable into a new observable that you subscribe to. In this case, mergeMap is subscribing for you and getting rid of your nesting.
For Your Consideration
Consider this. The following is about as clean looking at six service calls (each giving some value to the next one) in a row can get if you're not using a higher-order operator:
this.firstService.getThing("First").subscribe(result1 => {
this.secondService.getThing(result1.value).subscribe(result2 => {
this.thirdService.getThing(result2.value).subscribe(result3 => {
this.fourthService.getThing(result3.value).subscribe(result4 => {
this.fifthService.getThing(result4.value).subscribe(result5 => {
this.sixthService.getThing(result5.value).subscribe(result6 => {
console.log("Result Six is: " + result6.value);
});
});
});
});
});
});
Here's the exact same thing with mergeMap:
this.firstService.getThing("First").pipe(
mergeMap(result1 => this.secondService.getThing(result1.value)),
mergeMap(result2 => this.thirdService.getThing(result2.value)),
mergeMap(result3 => this.fourthService.getThing(result3.value)),
mergeMap(result4 => this.fifthService.getThing(result4.value)),
mergeMap(result5 => this.sixthService.getThing(result5.value)),
).subscribe(result6 => {
console.log("Result Six is: " + result6.value);
});
If that's not enough to convince you, you can lean a little bit more into some functional programming to make this even cleaner (without repetitively naming each result)
const passValueToService = service => result => service.getThing(result.value);
passValueToService(this.firstService)("First").pipe(
mergeMap(passValueToService(this.secondService)),
mergeMap(passValueToService(this.thirdService)),
mergeMap(passValueToService(this.fourthService)),
mergeMap(passValueToService(this.fifthService)),
mergeMap(passValueToService(this.sixthService)),
).subscribe(finalResult => {
console.log("Result Six is: " + finalResult.value);
});
Or why not lean EVEN harder and keep our list of services in an array?
const [firstS, ...restS] = [this.firstService, this.secondService, this.thirdService, this.fourthService, this.fifthService, this.sixthService];
const passValueToService = service => result => service.getThing(result.value);
passValueToService(firstS)("first").pipe(
...restS.map(service => mergeMap(passValueToService(service)))
).subscribe(finalResult => {
console.log("Result Six is: " + finalResult.value);
});
None of these simplifications are very easily done while nesting subscribe calls. But with the help of some functional currying (and the handy RxJS pipe to compose with), you can begin to see that your options expand dramatically.
Understanding concatMap, mergeMap, & switchMap
The Setup
We'll have 3 helper functions as described here:
/****
* Operator: intervalArray
* -----------------------
* Takes arrays emitted by the source and spaces out their
* values by the given interval time in milliseconds
****/
function intervalArray<T>(intervalTime = 1000): OperatorFunction<T[], T> {
return s => s.pipe(
concatMap((v: T[]) => concat(
...v.map((value: T) => EMPTY.pipe(
delay(intervalTime),
startWith(value)
))
))
);
}
/****
* Emit 1, 2, 3, then complete: each 0.5 seconds apart
****/
function n123Stream(): Observable<number> {
return of([1,2,3]).pipe(
intervalArray(500)
);
}
/****
* maps:
* 1 => 10, 11, 12, then complete: each 1 second apart
* 2 => 20, 21, 22, then complete: each 1 second apart
* 3 => 30, 31, 32, then complete: each 1 second apart
****/
function numberToStream(num): Observable<number>{
return of([num*10, num*10+1, num*10+2]).pipe(
intervalArray(1000)
);
}
The above mapping function (numberToStream), takes care of the map part of concatMap, mergeMap, and switchMap
Subscribing to each operator
The following three snippits of code will all have different outputs:
n123Stream().pipe(
concatMap(numberToStream)
).subscribe(console.log);
n123Stream().pipe(
mergeMap(numberToStream)
).subscribe(console.log);
n123Stream().pipe(
switchMap(numberToStream)
).subscribe(console.log);
If you want to run these back-to-back:
concat(
...[concatMap, mergeMap, switchMap].map(
op => n123Stream().pipe(
op(numberToStream),
startWith(`${op.name}: `)
)
)
).subscribe(console.log);
concatMap:
concatMap will not subscribe to the second inner observable until the first one is complete. That means that the number 13 will be emitted before the second observable (starting with the number 20) will be subscribed to.
The output:
10 11 12 20 21 22 30 31 32
All the 10s are before the 20s and all the 20s are before the 30s
mergeMap:
mergeMap will subscribe to the second observable the moment the second value arrives and then to the third observable the moment the third value arrives. It doesn't care about the order of output or anything like that.
The output
10 20 11 30 21 12 31 22 32
The 10s are earlier because they started earlier and the 30s are later because they start later, but there's some interleaving in the middle.
switchMap
switchMap will subscribe to the first observable the moment the first value arrives. It will unsubscribe to the first observable and subscribe to the second observable the moment the second value arrives (and so on).
The output
10 20 30 31 32
Only the final observable ran to completion in this case. The first two only had time to emit their first value before being unsubscribed. Just like concatMap, there is no interleaving and only one inner observable is running at a time, but some emissions are effectively dropped.
You can use switchMap.
this.firstService.get(this.id)
.pipe(
tap((response: FirstReturnType) => this.firstThing = response),
switchMap((response: FirstReturnType) => this.secondService.get(response.secondId)),
tap((response: SecondReturnType) => {
this.secondThing = response;
this.isLoading = false;
})
).subscribe();

What is the difference between tap and map in RxJS?

I read the difference from the article but the main points look like this.
so with tap I can change the variables such as that if I put x=3+4 then it changes the values of variable then I can say there is one side effect.
But with map I can change the value looping each value, isn't it?
Can you pinpoint what outstanding differences they have?
tap
RxJS tap performs side effects for every value emitted by source Observable and returns an Observable identical to the source Observable until there is no error.
map
map is a RxJS pipeable operator. map applies a given function to each element emitted by the source Observable and emits the resulting values as an Observable
A mapping function takes a thing and returns another thing. e.g. I can build a function that takes 10 and returns 11, that takes 11 and returns 12, etc.
const inc = n => n + 1;
Array#map applies such mapping function to all elements of an array but "map" doesn't mean "iteration".
In RxJS, when a data is sent to the stream it goes through a series of operators:
The map operator will simply apply a function to that data and return the result.
The tap operator however takes a data, apply a function to that data but returns the original data, if the function bothered to return a result, tap just ignores it.
Here's an example:
We push 10 to stream a$, tap just log the value. We know that console.log always return undefined but that's fine because tap simply returns its parameter.
We push 10 to stream b$, it goes through map(inc) which applies inc to 10 returning 11.
const a$ = of(10).pipe(tap(n => console.log(`tap: ${n}`)));
const b$ = of(10).pipe(map(inc));
a$.subscribe(n => console.log(`n from a$: ${n}`));
b$.subscribe(n => console.log(`n from b$: ${n}`));
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.5.5/rxjs.umd.min.js"></script>
<script>
const {of} = rxjs;
const {map, tap} = rxjs.operators;
const inc = n => n + 1;
</script>
Tap should be Used for Notification, logging non-contextual/critical side effects.
It's like a "peek" into the "pipe". The data stays the same, You can do something with it. Some data goes in, you look, same data comes out.
Map is for transformation/mapping of the Data in the "pipe". Some data comes in, different/transformed data comes out.
The purpose of tap is to execute an action keeping the same value
of the observable
The purpose of map is to transform the emitted values of the
observable
const messagesCount$ = newMessages$
.pipe(tap(messages => notificationService.notify('You have ' + message.length + ' message(s)')))
.pipe(map(messages => messages.length))
The tap and map are both RxJS operators, RxJS operators are just function that performs some manipulation over the data.
Both of them are pipeable operators which takes input as Observable, perform some action and return an output observable.
Difference between map and tap:
The map is a pipeable operator that takes an input observable, performs some manipulation on it and returns a new manipulated observable. For example
const source$ = of(1,2,3) // observable which will emit 1,2,3
// It take an input observable and return a new observable which will emit square of input values.
// So, the output observable will emit 1,4,9
const mapSource$ = of(1,2,3)
.pipe(map(value => value * value))
The tap operator on another hand takes an input observable perform some action and returns the same input observable.
const source$ = of(1,2,3) // observable which will emit 1,2,3
// It take an input observable and return a same observable after console value.
// So, the output observable will emit 1,2,3
const tapSource$ = of(1,2,3)
.pipe(tap(value => console.log(value)))
you can think of tap operator as a void function that whatever it does to the input value it does not change the original value
const source = of(1, 2, 3, 4, 5);
// here we are manipulating the input value but the output value of the observable still the same
const example = source.pipe(
tap(val => val + 100),
);
// output: 1, 2, 3, 4, 5
const subscribe = example.subscribe(val => console.log(val));
in the other hand if we made any manipulation of the input values of the observable using the map operator it will change the output values
const example = source.pipe(
map(val => val + 100)
);
// output: 101, 102, 103, 104, 105
const subscribe = example.subscribe(val => console.log(val));
I addition to what the others are saying, in Rxjs 7.4 tap now has three more subscribe handlers, so you can use it to get notified on subscribe, unsubscribe and finalize:
https://github.com/ReactiveX/rxjs/commit/eb26cbc4488c9953cdde565b598b1dbdeeeee9ea#diff-93cd3ac7329d72ed4ded62c6cbae17b6bdceb643fa7c1faa6f389729773364cc
This is great for debugging purposes, so you can use tap to find out much more about what is happening with your stream.
Example:
const subscription = subject
.pipe(
tap({
subscribe: () => console.log('subscribe'),
next: (value) => console.log(`next ${value}`),
error: (err) => console.log(`error: ${err.message}`),
complete: () => console.log('complete'),
unsubscribe: () => console.log('unsubscribe'),
finalize: () => console.log('finalize'),
})
)
.subscribe();
TAP, can NOT transform:
interval(1000).pipe(tap(el=> el*2)).subscribe(console.log); // 0,1,2,3
MAP, CAN transform:
interval(1000).pipe(map(el=> el*2)).subscribe(console.log); // 0,2,4,6
If you do NOT need to transform the value, just console.log it or run external function to pass the Original value = TAP is good.
If you NEED TO TRANSFORM/CHANGE the value = MAP is the way to go.

Resolve array of observables and append in final array

I have an endpoint url like http://site/api/myquery?start=&limit= which returns an array of strings.
If I call this endpoint in this way, the server hangs since the array of strings length is huge.
I need to generate an an array of observables with incremental "start" and "limit" parameters, resolve all of then either in sequence or in parallel, and then get a final observable which at the end yields the true array of strings, obtained merging all the subarray of strings returned by the inner observables.
How should I do that?
i.e. the array of observables would be something like
[
httpClient.get(http://site/api/myquery?start=0&limit=1000),
httpClient.get(http://site/api/myquery?start=1000&limit=1000),
httpClient.get(http://site/api/myquery?start=2000&limit=1000),
....
]
If you know the length before making all these queries — then you can create as many http-get Observables as you need, and then forkJoin them using projection fn.
forkJoin will let you make parallel queries and then merge results of those queries. Heres an example:
import { forkJoin } from 'rxjs';
// given we know the length:
const LENGTH = 500;
// we can pick arbitrary page size
const PAGE_SIZE = 50;
// calculate requests count
const requestsCount = Math.ceil(LENGTH / 50);
// generate calculated number of requests
const requests = (new Array(requestsCount))
.fill(void 0)
.map((_,i) => {
const start = i * PAGE_SIZE;
return http.get(`http://site/api/myquery?start=${start}&limit=${PAGE_SIZE}`);
});
forkJoin(
requests,
// projecting fn
// merge all arrays into one
// suboptimal merging, just for example
(...results) => results.reduce(((acc, curr)=> [...acc, ...curr]) , [])
).subscribe(array => {
console.log(array);
})
Check this forkJoin example for reference.
Hope this helps
In the case that you do not know the total number of items, you can do this using expand.
The following article gives a good introduction to expand and an explanation of how to use it for pagination.
https://ncjamieson.com/understanding-expand/
Something along the lines of the code below would work in your case, making the requests for each page in series.
const limit = 1000;
let currentStart = 0;
let getUrl = (start, limit) => `http://site/api/myquery?start=${start}&limit=${limit}`;
httpClient.get(getUrl(currentStart, limit)).pipe(
expand(itemsArray => {
if (itemsArray.length) {
currentStart += limit;
return httpClient.get(getUrl(currentStart, limit));
}
return empty();
}),
reduce((acc, value) => [...acc, ...value]),
).subscribe(itemsArray => {
console.log(itemsArray);
})
This will log out the final array of items once the entire series of requests has been resolved.

Delay first 3 items by 1 second and the fourth by 4 seconds

I'm trying to display the first 3 numbers each one delayed by 1 second and the fourth one by 4 seconds. Unfortunately, my code displays the four numbers by 1 second
import { from, of, race, timer, interval } from 'rxjs';
import { groupBy, mergeMap, toArray, map,merge, reduce, concatMap, delay, concat, timeout, catchError, take } from 'rxjs/operators';
const obs$ = from([1,2,3]).pipe(concatMap(a => of(a).pipe(delay(1000))));
const obs2$ = of(4).pipe(delay(4000));
const result$ = obs$.pipe(merge(obs2$));
const subscribe = result$.subscribe(val => console.log(val));
It displays
1234|
instead of
123----4|
this question is entirely for learning rxjs as beginner and has been tested on https://stackblitz.com
METHOD 1
The merge operator subscribes to both observable (obs$ and obs2$) at the same time. Therefore the result you get from your code may be explained as follows:
obs$ -----1-----2-----3
obs2$ -----------------------4
result$ -----1-----2-----3-----4
You may achieve your goal by forcing merge to subscribe only one observable at a time by providing the second argument as 1 (which is Number.POSITIVE_INFINITY by default) like the following:
const obs$ = from([1,2,3]).pipe(concatMap(a => of(a).pipe(delay(1000))));
const obs2$ = of(4).pipe(delay(4000));
// Provide the concurrency (second) argument as 1
const result$ = obs$.pipe(merge(obs2$, 1));
const subscribe = result$.subscribe(val => console.log(val));
METHOD 2
Use concat instead of merge:
const obs$ = from([1, 2, 3]).pipe(concatMap(a => of(a).pipe(delay(1000))));
const obs2$ = of(4).pipe(delay(4000));
const result$ = concat(obs$, obs2$);
const subscribe = result$.subscribe(val => console.log(val));
METHOD 3
Otherwise you simply make use of the second parameter of concatMap which is the index (starts from 0) of emitted items.
const obs$ = from([1, 2, 3, 4]);
const delayed$ = obs$.pipe(
concatMap((value, index) => {
if (index <= 2) {
// Delay the first 3 items by 1 sec
return of(value).pipe(delay(1000));
} else {
// Delay other items (here the 4th item) by 4 sec
return of(value).pipe(delay(4000));
}
})
);
delayed$.subscribe(x => console.log(x));

Delay for every element with RXJS

I'm using RxViz to simulate different actions that comes every 1 sec. When I try
Rx.Observable.create(obs => {
obs.next([1, 2, 3]); // or could be ['aaa', 'bbbb', 'ccc']
obs.complete();
}).delay(1000);
on https://rxviz.com
or on my own with a console.log
it keeps displaying the three number 1, 2, 3 at the same time
There's a post about this same problem, but none of the answer works for me. I'm using Rx last version 6
How can I create an observable with a delay
[EDIT] The array can contains anything like number, string or any object
If you want to delay each value (by 1 sec for example), you may do something like the following:
Rx.Observable.create(obs => {
obs.next([1, 2, 3]);
obs.complete();
})
.pipe(
// make observable to emit each element of the array (not the whole array)
mergeMap((x: [any]) => from(x)),
// delay each element by 1 sec
concatMap(x => of(x).pipe(delay(1000)))
)
.subscribe(x => console.log(x));
}
Here I did not modify the internals of the observable created by you. Instead, I just take your observable and apply appropriate operations to achieve what you seem to be expecting.
Here is my solution (very clean)
const fakeData = [1,2,3]
loadData$() {
return from(fakeData).pipe(
concatMap(item => of(item).pipe(
delay(1000)
)),
);
}
This one works by modifying a little bit #siva636's answer
Rx.Observable.create(obs => {
obs.next(1);
obs.next(2);
obs.next(3);
obs.complete();
}.concatMap(x=>Rx.Observable.of(x) .delay(1000) )
Here is a succinct way that builds on the other responses.
from([...Array(10).keys()]).pipe(
concatMap(x => of(x).pipe(delay(1000)))
).subscribe(y => console.log(y))
A more RxJs native version would be as follows.
const myInterval = rxjs.interval(1000);
myInterval.pipe(rxjs.operators.take(10)).subscribe(x => console.log(x));
Here, you emit in one observable emission the all array. [1,2,3].
You only delay that one emission by 1000 ms. But the emission is still one.
Even if we emit each value on its own, the delay function will only apply to the first emission. The others will come immediately after:
Rx.Observable.create(obs => {
var arr = [1, 2, 3];
arr.forEach(item => obs.next(item));
obs.complete();
}).delay(1000);
There is no magic in the create constructing function. If we want an emission to come every x time:
We could make an interval that emits those values (taken from learnrxjs)
import { Observable } from 'rxjs/Observable';
/*
Increment value every 1s, emit even numbers.
*/
const evenNumbers = Observable.create(function(observer) {
let value = 0;
const interval = setInterval(() => {
observer.next(value);
value++;
}, 1000);
return () => clearInterval(interval);
});
RxJS v7 supports the operator delayWhen [1], so you could write a simpler code as
import { delayWhen, interval, of } from 'rxjs';
of("John", "von", "Neumman", "János Neumann").pipe(
delayWhen((_, index) => interval(index*1000))
).subscribe(console.log);
Check out a demo on https://stackblitz.com/edit/vgibzv?file=index.ts
It works because it delays the emission of items by 0 seconds, 1000 seconds, 2000 seconds, 3000 seconds, and so on.
Another choice is the operator scan, you make the series from an interval [2].
[1] "RxJS - delayWhen." 16 Dec. 2022, https://rxjs.dev/api/operators/delayWhen
[2] "RxJS - scan." 16 Dec. 2022, rxjs.dev/api/index/function/scan

Resources