How is it possible with RXJS to make a cascaded forEach loop? Currently, I have 4 observables containing simple string lists, called x1 - x4. What I want to achieve now is to run over all variation and to call a REST-Api with an object of variation data. Usually, I would do something like that with a forEach, but how to do with RXJS? Please see the abstracted code:
let x1$ = of([1,2]);
let x2$ = of([a,b,c,d,e,f]);
let x3$ = of([A,B,C,D,E,F]);
let x4$ = of([M,N,O,P]);
x1$.forEach(x1 => {
x2$.forEach(x2 => {
x3$.forEach(x3 => {
x4$.forEach(x4 => {
let data = {
a: x1,
b: x2,
c: x3,
d: x4
}
return this.restService.post('/xxxx', data)
})
})
})
})
Is something like that possible with RXJS in an elegant way?
Let's assume you have a function combineLists which represent the plain-array version of the logic to turn static lists into an array of request observables:
function combineLists(lists: unknown[][]) {
const [x1s, x2s, x3s, x4s] = lists;
// Calculate combinations, you can also use your forEach instead
const combinations = x1s
.flatMap(a => x2s
.flatMap(b => x3s
.flatMap(c => x4s
.flatMap(d => ({a, b, c, d})))));
return combinations.map(combination => this.restService.post('/xxxx', combination));
}
Since your input observables are one-offs as well, we can use e.g. forkJoin. This waits for all of them to complete and then runs with their respective plain values. At this point you're back to computing the combinations with your preferred method.
forkJoin([x1$, x2$, x3$, x4$]).pipe(
map(combineLists),
);
Assuming your REST call is typed to return T, the above produces Observable<Observable<T>[]>. How you proceed from here depends on what data structure you're looking for / how you want to continue working with this. This didn't seem to be part of your question anymore, but I'll give a couple hints nonetheless:
If you want a Observable<T>, you can just add e.g. a mergeAll() operator. This observable will just emit the results of all individual requests after another in whichever order they arrive.
forkJoin([x1$, x2$, x3$, x4$]).pipe(
map(combineLists),
mergeAll(),
);
If you want an Observable<T[]> instead, which collects the results into a single emission, you could once again forkJoin the produced array of requests. This also preserves the order.
forkJoin([x1$, x2$, x3$, x4$]).pipe(
map(combineLists),
switchMap(forkJoin),
);
Some words of caution:
Don't forget to subscribe to make it actually do something.
You should make sure to handle errors on all your REST calls. This must happen right at the call itself, not after this entire pipeline, unless you want one single failed request to break the entire pipe.
Keep in mind that forkJoin([]) over an empty array doesn't emit anything.
Triggering a lot of requests like this probably means the API should be changed (if possible) as the number of requests grows exponentially.
Related
Trying to learn RxJs, and I found what looks like a nice tutorial on that topic at https://www.learnrxjs.io.
I'm going through their primer section, and I am not clear on what the return statement in the pipe() function actually does or what it means. These are a couple of screen shots from their tutorial:
In traditional programming, I've always understood a return statement to be an exit - if function A calls function B, and function B has the line return 1, then control goes back to function A.
Is that what happens here? If so, in either of these two examples, where am I returning to???
Or what if I don't want to return anywhere but act on the data immediately? For example, in the error handling example, instead of return makeRequest..., I want to do something like const result = makeRequest.... Can I do that?
In general I'm having some conceptual difficulties around all the returns I've seen used with observables, and any help in explaining what they do/are would be appreciated. So would any other tutorial sites on RxJs.
These are all very similar constructs in javascript
function adder0(a,b){
return a + b
}
const adder1 = (a,b) => {
return a + b
}
const adder2 = (a,b) => a + b
console.log(adder0(3,7)) // 10
console.log(adder1(3,7)) // 10
console.log(adder2(3,7)) // 10
lets re-write the code from that example but with explicit function definitions (instead of arrow syntax).
function maker(value){
return makeRequest(value).pipe(
catchError(handleError)
);
}
function actioner(value){
// take action
}
source.pipe(
mergeMap(maker)
).subscribe(actioner)
The thing to notice is that you never actually call these functions. RxJS does that for you. You give them a function and they'll call it when they're good and ready based on the spec for a given operator.
I have tried to unsubscribe within the subscribe method. It seems like it works, I haven't found an example on the internet that you can do it this way.
I know that there are many other possibilities to unsubscribe the method or to limit it with pipes. Please do not suggest any other solution, but answer why you shouldn't do that or is it a possible way ?
example:
let localSubscription = someObservable.subscribe(result => {
this.result = result;
if (localSubscription && someStatement) {
localSubscription.unsubscribe();
}
});
The problem
Sometimes the pattern you used above will work and sometimes it won't. Here are two examples, you can try to run them yourself. One will throw an error and the other will not.
const subscription = of(1,2,3,4,5).pipe(
tap(console.log)
).subscribe(v => {
if(v === 4) subscription.unsubscribe();
});
The output:
1
2
3
4
Error: Cannot access 'subscription' before initialization
Something similar:
const subscription = of(1,2,3,4,5).pipe(
tap(console.log),
delay(0)
).subscribe(v => {
if (v === 4) subscription.unsubscribe();
});
The output:
1
2
3
4
This time you don't get an error, but you also unsubscribed before the 5 was emitted from the source observable of(1,2,3,4,5)
Hidden Constraints
If you're familiar with Schedulers in RxJS, you might immediately be able to spot the extra hidden information that allows one example to work while the other doesn't.
delay (Even a delay of 0 milliseconds) returns an Observable that uses an asynchronous scheduler. This means, in effect, that the current block of code will finish execution before the delayed observable has a chance to emit.
This guarantees that in a single-threaded environment (like the Javascript runtime found in browsers currently) your subscription has been initialized.
The Solutions
1. Keep a fragile codebase
One possible solution is to just ignore common wisdom and continue to use this pattern for unsubscribing. To do so, you and anyone on your team that might use your code for reference or might someday need to maintain your code must take on the extra cognitive load of remembering which observable use the correct scheduler.
Changing how an observable transforms data in one part of your application may cause unexpected errors in every part of the application that relies on this data being supplied by an asynchronous scheduler.
For example: code that runs fine when querying a server may break when synchronously returned a cashed result. What seems like an optimization, now wreaks havoc in your codebase. When this sort of error appears, the source can be rather difficult to track down.
Finally, if ever browsers (or you're running code in Node.js) start to support multi-threaded environments, your code will either have to make do without that enhancement or be re-written.
2. Making "unsubscribe inside subscription callback" a safe pattern
Idiomatic RxJS code tries to be schedular agnostic wherever possible.
Here is how you might use the pattern above without worrying about which scheduler an observable is using. This is effectively scheduler agnostic, though it likely complicates a rather simple task much more than it needs to.
const stream = publish()(of(1,2,3,4,5));
const subscription = stream.pipe(
tap(console.log)
).subscribe(x => {
if(x === 4) subscription.unsubscribe();
});
stream.connect();
This lets you use a "unsubscribe inside a subscription" pattern safely. This will always work regardless of the scheduler and would continue to work if (for example) you put your code in a multi-threaded environment (The delay example above may break, but this will not).
3. RxJS Operators
The best solutions will be those that use operators that handle subscription/unsubscription on your behalf. They require no extra cognitive load in the best circumstances and manage to contain/manage errors relatively well (less spooky action at a distance) in the more exotic circumstances.
Most higher-order operators do this (concat, merge, concatMap, switchMap, mergeMap, ect). Other operators like take, takeUntil, takeWhile, ect let you use a more declarative style to manage subscriptions.
Where possible, these are preferable as they're all less likely to cause strange errors or confusion within a team that is using them.
The examples above re-written:
of(1,2,3,4,5).pipe(
tap(console.log)
first(v => v === 4)
).subscribe();
It's working method, but RxJS mainly recommend use async pipe in Angular. That's the perfect solution. In your example you assign result to the object property and that's not a good practice.
If you use your variable in the template, then just use async pipe. If you don't, just make it observable in that way:
private readonly result$ = someObservable.pipe(/...get exactly what you need here.../)
And then you can use your result$ in cases when you need it: in other observable or template.
Also you can use pipe(take(1)) or pipe(first()) for unsubscribing. There are also some other pipe methods allowing you unsubscribe without additional code.
There are various ways of unsubscribing data:
Method 1: Unsubscribe after subscription; (Not preferred)
let localSubscription = someObservable.subscribe(result => {
this.result = result;
}).unsubscribe();
---------------------
Method 2: If you want only first one or 2 values, use take operator or first operator
a) let localSubscription =
someObservable.pipe(take(1)).subscribe(result => {
this.result = result;
});
b) let localSubscription =
someObservable.pipe(first()).subscribe(result => {
this.result = result;
});
---------------------
Method 3: Use Subscription and unsubscribe in your ngOnDestroy();
let localSubscription =
someObservable.subscribe(result => {
this.result = result;
});
ngOnDestroy() { this.localSubscription.unsubscribe() }
----------------------
Method 4: Use Subject and takeUntil Operator and destroy in ngOnDestroy
let destroySubject: Subject<any> = new Subject();
let localSubscription =
someObservable.pipe(takeUntil(this.destroySubject)).subscribe(result => {
this.result = result;
});
ngOnDestroy() {
this.destroySubject.next();
this.destroySubject.complete();
}
I would personally prefer method 4, because you can use the same destroy subject for multiple subscriptions if you have in a single page.
Q: can RxJs operators be used to flatten an array, transform items, then unflatten it, whilst maintaining a continuous stream (not completing)?
For the simplified example here: https://stackblitz.com/edit/rxjs-a1791p?file=index.ts
If following the approach:
mergeMap(next => next),
switchMap(next => of(***transforming logic***)),
toArray()
then the observable does not complete, and the values do not come through. A take(1) could be added but this is intended to be a continuous stream.
If using:
mergeMap(next => next),
switchMap(next => of(***transforming logic***)),
scan()
then this works great. However, then each time the source observable emits, the accumulator never resets, so the scan() which is intended to accumulate the values back into an array ends up combining multiple arrays from each pass. Can the accumulator be reset?
Obviously it can be accomplished with:
switchMap(next => of(next.map(***transforming logic***)))
But my real-world example is an awful lot more complicated than this, and is tied into NgRx.
Here would be one approach:
src$.pipe(
mergeMap(
arr => from(arr)
.pipe(
switchMap(item => /* ... */),
toArray(),
)
)
)
For each emitted array, mergeMap will create an inner observable(from(..)). There, from(array) will emit each item separately, allowing you to perform some logic in switchMap. Attaching toArray() at the end will give you an array with the results from switchMap's inner observable.
You don't need to use mergeMap or switchMap here. You would only need those if you are doing something asynchronously. Like if you were taking the input value and creating an observable (ex: to make an http call).
By using of inside of mergeMap, you are essentially starting with an Observable, taking the unpacked value (an array), then turning it back into an Observable.
From your stack blitz:
The reason your first strategy doesn't complete is because toArray() is happening on the level of the source (clicksFromToArrayButton), and that is never going to complete.
If you really wanted to, you could nest it up a level, so that toArray() happens on the level of your array (created with from(), which will complete after all values are emitted).
const transformedMaleNames = maleNames.pipe(
mergeMap(next => from(next).pipe(
map(next => {
const parts = next.name.split(' ');
return { firstName: parts[0], lastName: parts[1] };
}),
toArray()
)
),
);
But... we don't really need to use from to create an observable, just so it can complete, just so toArray() can put it back together for you. We can use the regular map operator instead of mergeMap, along with Array.map():
const transformedMaleNames = maleNames.pipe(
map(nextArray => {
return nextArray.map(next => {
const parts = next.name.split(' ');
return { firstName: parts[0], lastName: parts[1] };
})
})
);
this works, but isn't necessarily utilizing RxJS operators fully?
Well, ya gotta use the right tool for the right job! In this case, you are simply transforming array elements, so Array.map() is perfect for this.
But my real-world example is an awful lot more complicated than this
If you are concerned about the code getting messy, you can just break the transformation logic out into it's own function:
const transformedMaleNames = maleNames.pipe(
map(next => next.map(transformName))
);
function transformName(next) {
const parts = next.name.split(' ');
return { firstName: parts[0], lastName: parts[1] };
}
Here's a working StackBlitz.
TLDR: Working example is in the last codeblock of this question. Check out #bryan60 answer for a working example using concat rather than mergeMap.
I'm trying to run a number of remote requests sequentially, but only the first observable is executed.
The number of request vary, so I can't do a dodgy solution where I nest observables within each other.
I'm using the following code:
const observables = [
observable1,
observable2,
...
];
from(observables).pipe(
mergeMap(ob=> {
return ob.pipe(map(res => res));
}, undefined, 1)
).subscribe(res => {
console.log('Huzzah!');
})
In the past (rxjs 5.5) Ive used the following:
let o = Observable.from(observables).mergeMap((ob) => {
return ob;
}, null, 1);
o.subscribe(res => {
console.log('Huzzah!');
})
I'm not sure what I'm doing wrong, can anybody shed some light?
An additional request would be to only print 'Huzzah!' once on completion of all requests rather than for each individual Observable.
EDIT:
Removing undefined from my original code will make it work, however there was another issue causing only the first observable to be executed.
I'm using Angular's HttpClient for remote requests. My observable code looked like this:
const observables = [];
// Only the first observable would be executed
observables.push(this.http.get(urla));
observables.push(this.http.get(urlb));
observables.push(this.http.get(urlc));
Adding .pipe(take(1)) to each observable results in each observable being executed:
const observables = [];
// All observables will now be executed
observables.push(this.http.get(urla).pipe(take(1));
observables.push(this.http.get(urlb).pipe(take(1));
observables.push(this.http.get(urlc).pipe(take(1));
The code I ended up using, which executes all observables in sequential order and only triggers Huzzah! once is:
const observables = [];
observables.push(this.http.get(urla).pipe(take(1));
observables.push(this.http.get(urlb).pipe(take(1));
observables.push(this.http.get(urlc).pipe(take(1));
from(observables).pipe(
mergeMap(ob=> {
return ob.pipe(map(res => res));
}, 1),
reduce((all: any, res: any) => all.concat(res), [])
).subscribe(res => {
console.log('Huzzah!');
})
Thanks to #bryan60 for helping me wit this issue.
if these are http requests that complete, I think your bug is caused by a change to the mergeMap signature that removed the result selector. it's hard to be sure without knowing exactly which version you're on as it was there, then removed, then added again, and they're removing it once more for good in v7.
if you want to run them sequentially... this is all you need...
// concat runs input observables sequentially
concat(...observables).subscribe(res => console.log(res))
if you want to wait till they're all done to emit, do this:
concat(...observables).pipe(
// this will gather all responses and emit them all when they're done
reduce((all, res) => all.concat([res]), [])
// if you don't care about the responses, just use last()
).subscribe(allRes => console.log(allRes))
In my personal utility rxjs lib, I always include a concatJoin operator that combines concat and reduce like this.
the only trick is that concat requires observables to complete till it moves on to the next one, but the same is true for mergeMap with concurrent subscriptions set to 1.. so that should be fine. things like http requests are fine, as they complete naturally after one emission.. websockets or subjects or event emitters will behave a bit differently and have to be manually completed, either with operators like first or take or at the source.
If you are not concerned about the sequence of execution and just want 'Huzzah!' to be printed once all the observable has been executed forkJoin can also be used.Try this.
forkJoin(...observables).subscribe(res => console.log('Huzzah');
I have an array of observables which I'm executing in parallel using:
let observables: Observable<any>[]
Observable.forkJoin(observables)
This works perfectly, however, I need to execute the array of observables sequentially, and only emit one result if the last observable has been completed. That's when I tried to use
Observable.concat(observables)
But this returns multiple results, and not only one - combined - result which I get when using forkJoin. So I actually need a combination of the two.
I've tried to use the reduce functionality to execute them sequentially, like this:
return observables.reduce((previous, current) => {
return previous.flatMap(() => current);
}, Observable.empty());
But with this solution the observables are not executed at all.
Assuming that your observables emit singular values, not arrays, you could rework your current approach to something like:
return Observable.concat(...observables).reduce((acc, current) => [...acc, current], []);
or even shorter:
return Observable.concat(...observables).toArray();
In the case that they emit array values, you could do the following:
const source = Observable.concat(...observables).flatMap(list => list).toArray();
As Jota.Toledo or Mateusz Witkowski showed in their answers, with the new syntax of RxJS you can do:
return concat(...observables).pipe(toArray());
You can you use toArray() operator:
Observable.concat(observables).toArray().subscribe()
As stated in RxJS documentation: it creates "an observable sequence containing a single element with a list containing all the elements of the source sequence".