I have been working with Observables a bit, but can't figure out how to do the following:
I have an observable that emits an array of IDs.
I then want to retrieve these IDs, again using Observables.
Finally I want to put everything together in an Observable that
emits an array of the retrieved items.
I could put my code attempts here, but I don't think it would help a lot.
I'm using RxJS 5.5
You can use for that flattening operators for that (like mergeMap or concatMap depending on requirements). For example:
const { Observable, combineLatest, of, from } = rxjs; // = require("rxjs")
const { mergeMap, toArray, map } = rxjs.operators; // = require("rxjs/operators")
const ids = [0,1,2,3,4,5,6,7,8,9];
const transform = id => of(`${id}-transformed`);
of(ids).pipe(
mergeMap(ids => ids),
mergeMap(id => transform(id)),
toArray()
).subscribe(e => console.log(e))
<script src="https://unpkg.com/rxjs#6.2.2/bundles/rxjs.umd.min.js"></script>
Related
I want combineLatest functional but for variable count of observables.
Something like:
// init combineLatest of three observables
[1, 2, 3]
// first observable produced new value "2"
[2, 2, 3]
// third observable ended
[2, 2]
// first observable produced new value "1"
[1, 2]
// new observable added
[2, 2, 4]
Is it possible in RxJS?
If I understand the problem right, the solution is pretty tricky for something that looks innocent.
I try to go step by step to explain a potential solution.
First of all we need understand that there are 3 different events that we need to manage:
the fact that one Observable completes
the fact that one Observable is added to the array which is given to combineLatest
the fact that a new array of Observables has to be passed to combineLatest, either because we are at the beginning of the processing (i.e. with the initial array) or because we have added a new Observable or because one Observable has completed
The second thing that we need to recognize is that we need to store the array of Observables we pass to combineLatest in a variable, otherwise we are not able to add or remove Obaservables from there.
Once these things are clear, we can build a solution in the form of a function that returns 2 things:
the Observable that we want to subscribe to and that should have the behavior that we are looking for
a Subject that we can use to communicate that we want to add a new Observable to the combineLatest function
The last point we need to recognize is that any time we change the list of Observable, either because we add or because we remove an Observable (because it completed), we need to run the combineLatest function with the new fresh list of Observables.
Now that all this has been clarified, this is the code of the function that returns an Observable which behaves as described
function dynamicCombineLatest(startingObservables: Observable<any>[]) {
// this is the variable holding the array of Observables
let observables = startingObservables;
// this is the array that contains the list of Observables which have been, potentially, transformed to emit
// immediately the last value emitted - this happens when a new Observable is added to the array
let observablesPotentiallyWithLastValueImmediatelyEmitted =
startingObservables;
// this is the variable holding the array of values last notified by each Observable
// we will use it when we need to add a new Observable to the list
const lastValues = [];
// this are the Subjects used to notify the 3 different types of events
const start = new BehaviorSubject<Observable<any>[]>(observables);
const add = new Subject<Observable<any>>();
const remove = new Subject<Observable<any>>();
let skipFirst = false;
// this is the chain of operations which must happen when a new Observable is added
const addToObservables = add.pipe(
tap({
next: (obs) => {
console.log("add");
// we need to make sure that the Observables in the list will immediately start to emit
// the last value they emitted. In this way we are sure that, as soon as the new added Observable emits somthing,
// the last value emitted by the previous Observables will be considered
observablesPotentiallyWithLastValueImmediatelyEmitted = observables.map(
(o, i) => {
return startWith(lastValues[i])(o);
}
);
// the new Observable is added to the list
observables.push(obs);
observablesPotentiallyWithLastValueImmediatelyEmitted.push(obs);
},
})
);
// this is the chain of operations which must happen when an Observable is removed
const removeFromObservables = remove.pipe(
tap({
next: (obs) => {
const index =
observablesPotentiallyWithLastValueImmediatelyEmitted.indexOf(obs);
console.log("remove");
// we simply remove the Observable from the list and it "last value"
observablesPotentiallyWithLastValueImmediatelyEmitted.splice(index, 1);
observables.splice(index, 1);
lastValues.splice(index, 1);
// we make sure that the Observables in the list will immediately start to emit with the last value they emitted
observablesPotentiallyWithLastValueImmediatelyEmitted = observables.map(
(o, i) => {
return lastValues[i] ? startWith(lastValues[i])(o) : o;
}
);
// we set that the first value of the new combineLatest Observable will be skipped
skipFirst = true;
},
})
);
// here we merge the 2 chains of operations so that both add and remove logic will be executed
// when the relative Subjects emit
merge(addToObservables, removeFromObservables).subscribe({
next: () => {
console.log("new start");
// we notify that a change in the Observable list has occurred and therefore we need to unsubscribe the previous "combineLatest"
// and subscribe to the new one we are going to build
start.next(observablesPotentiallyWithLastValueImmediatelyEmitted);
},
});
// this is where we switch to a new Observable, result of the "combineLatest" operation,
// any time the start Subject emits a new Observable list
const dynamicObservables = start.pipe(
switchMap((_observables) => {
const _observablesSavingLastValueAndSignallingRemove = _observables.map(
(o, i) =>
o.pipe(
tap({
next: (v) => {
// here we save the last value emitted by each Observable
lastValues[i] = v;
},
complete: () => {
// here we notify that the Observable has completed and we need to remove it from the list
remove.next(o);
},
})
)
);
console.log("add or remove");
// eventually this is the Observable created by combineLatest with the expected array of Observables
const _combineLatest = combineLatest(
_observablesSavingLastValueAndSignallingRemove
);
const ret = skipFirst ? _combineLatest.pipe(skip(1)) : _combineLatest;
skipFirst = false;
return ret;
})
);
// here we return the Observable which will be subscribed to and the add Subject to be used to add new Observables
return { dynamicObservables, add };
}
You can look at this stackblitz for an example.
Buffer and combine based on a key
Here's a slight variant of what you're asking for. It works just like mergeAll, only it keeps a buffer and emits the latest for any observable that have emitted so far.
The varient here is that you need to supply string keys for your values to get attached to. You should be able to see how to turn this into array indices if you so choose.
The reason I haven't done this with an array is because there's no much undefined behavior. For example, if the first observable completes and the second observable emits, your elements are all opaquely re-ordered.
Using keys returns control back to the caller, who can just use Object.keys() if they don't care about indices/labels for their data.
Here you are:
interface LabeledObservable<T> {
label: string,
stream: Observable<T>
}
interface CombinedLatest<T> {
[key:string]: T
}
function combineLatestAll<T>():
OperatorFunction<
LabeledObservable<T>,
CombinedLatest<T>
>
{
return source$ => defer(() => {
const buffer = {};
return source$.pipe(
mergeMap(({label, stream}) => stream.pipe(
map(v => {
buffer[label] = v;
return {...buffer};
}),
finalize(() => {
delete buffer[label];
})
))
);
});
}
Subject for new observables
If you like the idea of a subject you can use to inject new observables into your combineLatest operator, this still allows that. The only alteration needed is that you must supply unique labels for your observables. If you don't care about the labels, you can just use any ID generator pattern (Like incrementing a global id counter or something).
const startingObservables: Observable<any>[] = /*some observables */;
const add = new Subject<LabeledObservable<any>>();
add.pipe(
combineLatestAll()
).subscribe(console.log);
startingObservables.forEach((stream,i) => {
add.next({label: "" + i, stream});
});
rxjs collects the values sent by Subject and then sends the past values as an array
import { Subject } from "rxjs";
const list$ = new Subject<number>();
list$
.pipe(/* Which pipeline should I use to achieve the following log information */)
.subscribe(console.log);
list$.next(1); // log: [1]
list$.next(2); // log: [1,2]
In RxJS .pipe() is where you list a series of operators. Each operator is a pure functinon that receives the emitted value from the previous observable (or operator), and returns a new value for the next operator.
Because of the utility that operators provide, the subscription method can be kept rather simple, or completely empty.
Given your requirement to convert each emitted value to an array, I would recommend the scan() operator. Similar to Array.reduce(), the scan() operator gives you an aggregate value you can add to for each emission from the source observable (in this case, your subject).
const list = new Subject<number>();
list.pipe(
scan((numArray, number)=>
[...numArray, number], [] as number[]
)
).subscribe(numArray=>console.log(numArray));
list.next(1); // [1]
list.next(2); // [1,2]
The reason I didn't include $ in the variable name, is that this is typically reserved only for public observables (not subjects or subscriptions, which shouldn't be exposed to other classes/modules/files).
I customized Observable to solve the problem, but I want to know how to use the pipe that comes with rxjs
import { Observable, Subject } from "rxjs";
const list$ = new Subject<number>();
var observable = new Observable((subscribe) => {
const cache = [];
const sub = list$.subscribe((v) => {
cache.push(v);
subscribe.next(cache);
});
return function unsubscribe() {
sub.unsubscribe();
};
});
const sub = observable.subscribe(console.log);
list$.next(1); // log: [1]
list$.next(2); // log: [1,2]
Is there a better way to re-write this code and avoid chaining of subscriptions ?
Why am I chaining? because I need to the output of source1$ in child subscriptions
And also I have if conditions because I want to call child subscriptions conditionally
PS i checked solution in this post
Here is the stackblitz link and code
import { from } from 'rxjs';
//emit array as a sequence of values
const source1$ = from([1]);
const source2$ = from([2]);
const source3$ = from([3]);
const useCond1 = true; // this is dynamic can be false too
const useCond2 = true; // this is dynamic can be false too
source1$.subscribe(val => {
if (useCond1) {
source2$.subscribe(() => {
console.log('val from source1 in source2', val);
});
}
if (useCond2) {
source3$.subscribe(() => {
console.log('val from source1 in source3', val);
});
}
});
Not sure, but it seems that you need switchMap or mergeMap and iif
from rxjx doc:
import { fromEvent, iif, of } from 'rxjs';
import { mergeMap, map, throttleTime, filter } from 'rxjs/operators';
const r$ = of(`I'm saying R!!`);
const x$ = of(`X's always win!!`);
fromEvent(document, 'mousemove')
.pipe(
throttleTime(50),
filter((move: MouseEvent) => move.clientY < 210),
map((move: MouseEvent) => move.clientY),
mergeMap(yCoord => iif(() => yCoord < 110, r$, x$))
)
.subscribe(console.log);
Yes, there is a better way!
RxJS provides many different operators and static functions for combining, filtering, and transforming observables. When you use what the library provides, you do not need to have nested subscriptions.
In general, I find it simpler to not do any logic at all inside the subscribe, but rather design observables that emit the exact data that is needed.
A simplistic example could look like this:
someValue$ = source1$.pipe(
switchMap(val1 => useCond1 ? source2$ : of(val1))
);
someValue$.subscribe();
switchMap will subscribe to an "inner observable" whenever it receives an emission. The logic above says to either return the value emitted from source1$ (val1) or return whatever source2$ emits depending on the value of useCond1.
So source2$ will only get subscribed to when useCond1 is true;
Note: the function inside switchMap should return an observable (because switchMap subscribes to it), so of was used to turn the emitted value into an observable.
In your case, let's assume you want to emit some calculated value, based possibly on the other two sources.
We can use combineLatest to create a single observable based on 3 different sources. Since you only want to optionally call source2$ and source3$, we can define the sources based on your conditions. We can then use map to transform the array of values from the 3 sources, into the desired output:
someValue$ = source1$.pipe(
switchMap(val1 => {
const s1$ = of(val1);
const s2$ = useCond1 ? source2$ : of('default val2');
const s3$ = useCond2 ? source3$ : of('default val3');
return combineLatest([s1$, s2$, s3$]);
}),
map(([val1, val2, val3]) => {
return ... // your logic to return desired value
})
);
combineLatest will emit an array containing the latest emissions from each source whenever any source emits. This means someValue$ will emit the latest calculated value whenever any of the sources change.
Following use case: A user can join 0...* groups. Each group has an ID and contains 0...* posts.
I subscribe to an Observable (to get the groups of the user he joined) and this returns an array of strings (the group IDs).
const groupIds$ = of(['a', 'b', 'c']);
If I only had one I would now use switchMap and return this new observable and subscribe to it to get the posts from the group.
But so I have an array and so this isn't working. Does anyone has an idea which RxJS operator(s) can achieve this to get the posts from all groups?
Or does no operator for such use case exist and I have to do it separately at subscribe?
You can use ideally forkJoin if you know you'll have all source Observables as an array:
groupIds$.pipe(
concatMap(groups => {
const observables = groups.map(id => ...);
return forkJoin(...observables);
})
).subscribe(...);
forkJoin will emit a single array with all the results in the same order as in groupIds$.
Eventually if you don't care about the order and just want to get all the results in parallel you can use merge instead of forkJoin (I mean the "Observable creation method" called merge imported directly from rxjs. Not the merge operator from 'rxjs/operators').
Everyone who arrives at this question, here is the answer for "switchMap for Array of strings" (thanks to martin). You only have to use 'merge' from 'rxjs' (not the operator!). Inside switchMap return merge and you are done:
groupIds$.pipe(
switchMap(groups => {
const observables = groups.map(id => {
// your function that returns the observable
return something.getObservable(id);
});
return merge(...observables);
})
).subscribe(data => {
console.log('DATA', data);
});
Possible solution: https://www.learnrxjs.io/operators/transformation/mergemap.html.
const groupIds$ = of(['a', 'b', 'c']);
const myPromise = id =>
new Promise(resolve => resolve(something.getPostsByGroupId(id)));
const posts$ = groupIds$.pipe(
mergeMap(
id => myPromise(id),
(valueFromSource, valueFromPromise) => {
return valueFromPromise;
}));
posts$.subscribe(...);
Terser improvement on great existing answers. This emits an array of all posts each time the groupIds$ source emits.
groupIds$.pipe(
switchMap(ids => zip(...ids.map(id => service.getPosts(id)))),
map((posts: Array<Post[]>) => [].concat.apply([], posts))
).subscribe(posts => /*is an array of all posts*/);
I have a Observable<Array<Observable<T>>> which I want to map to Observable<Array<T>>.
When a new array is emitted, the inner observables should unsubscribe/subscribe as follows:
If Observable exists in previous array and the new/current array, retain pre-existing subscription
If Observable did not exist in previous array but does exist in new/current array, create new subscription
If Observable existed in previous array but does not exist in new/current array, unsubscribe from pre-existing subscription
I hoped to achieve this using switchMap on the outer observable and then passing Array<Observable<T>> into combineLatest. However, switchMap will unsubscribe from its previous inner Observable before subscribing to the new inner Observable, which means inner subscriptions are not retained as desired.
Example (https://stackblitz.com/edit/typescript-b4wgr1). Given code:
import 'rxjs/Rx';
import { Observable } from 'rxjs';
const debugObservable = <T>(t$: Observable<T>, name: string) =>
new Observable<T>(observer => {
console.log(name, 'subscribe');
const subscription = t$.subscribe(observer);
return () => {
console.log(name, 'unsubscribe');
return subscription.unsubscribe();
};
});
const ofSingle = <T>(t: T) =>
new Observable<T>(observer => {
observer.next(t);
});
const observableOfArrayOfObservablesOfNumber = new Observable<
Array<Observable<number>>
>(observe => {
const keep = debugObservable(ofSingle(1), 'keep');
const remove = debugObservable(ofSingle(2), 'remove');
const add = debugObservable(ofSingle(3), 'add');
observe.next([keep, remove]);
setTimeout(() => {
observe.next([keep, add]);
}, 2000);
return () => {};
});
// The `switchMap` will unsubscribe to the previous inner observable *before* subscribing to the new
// inner observable.
const final$ = observableOfArrayOfObservablesOfNumber.switchMap(
arrayOfObservablesOfNumber => {
const observableOfArrayOfNumbers = Observable.combineLatest(
arrayOfObservablesOfNumber,
);
return debugObservable(
observableOfArrayOfNumbers,
'observableOfArrayOfNumbers',
);
},
);
final$.subscribe(x => console.log('final', x));
This produces:
observableOfArrayOfNumbers subscribe
keep subscribe
remove subscribe
final [1, 2]
keep unsubscribe <--- bad!
remove unsubscribe
observableOfArrayOfNumbers unsubscribe
observableOfArrayOfNumbers subscribe
keep subscribe <--- bad!
add subscribe
final [1, 3]
However, this is what I desire:
observableOfArrayOfNumbers subscribe
keep subscribe
remove subscribe
final [1, 2]
remove unsubscribe
observableOfArrayOfNumbers unsubscribe
observableOfArrayOfNumbers subscribe
add subscribe
final [1, 3]
I ended up achieving this by publishing + replaying the inner observables with publishReplay(1) and then ref counting.
Note that refCount is not sufficient because the count will drop to 0 when switchMap unsubscribes from the previous inner observable (before it subscribes to the new inner observable), so I had to use a special refCountWithDelay operator which only unsubscribes by ref counting after a delay (i.e. within the same tick of the event loop but not synchronously). More on that here:
https://github.com/ReactiveX/rxjs/issues/171
https://medium.com/#volkeron/rxjs-unsubscribe-delay-218a9ab2672e
refCountWithDelay: https://gist.github.com/marinho/3637210b13c0f298e1692a0b7b104e64
https://stackblitz.com/edit/typescript-4xfwsh?file=index.ts
const createObservable = <T>(t: T, name: string) => {
return refCountWithDelay(debugObservable(ofSingle(t), name).publishReplay(1), 0, 0);
}
const observableOfArrayOfObservablesOfNumber = new Observable<
Array<Observable<number>>
>(observe => {
const keep = createObservable(1, 'keep');
const remove = createObservable(2, 'remove');
const add = createObservable(3, 'add');
observe.next([keep, remove]);
setTimeout(() => {
observe.next([keep, add]);
}, 2000);
return () => {};
});
Produces:
observableOfArrayOfNumbers subscribe
keep subscribe
remove subscribe
final [1, 2]
observableOfArrayOfNumbers unsubscribe
observableOfArrayOfNumbers subscribe
remove unsubscribe
add subscribe
final [1, 3]
Notice that keep is only subscribed to once.
I came up with a better solution, to use combineLatestHigherOrder from rxjs-etc: https://github.com/cartant/rxjs-etc
https://stackblitz.com/edit/typescript-hfze6m?file=index.ts
The closest to what you described is an xstream operator called pickCombine in Cycle.js Onionify.
There doesn't seem to be one single official RxJS operator that solves this, but it is possible to build your own operator that implements this behavior. You can use the xstream implementation of pickCombine as a reference.
Key parts are:
Stream did not exist in previous but exists in new: https://github.com/staltz/cycle-onionify/blob/8b6344a749fb5b0ad7c0c8f1dcf2db4e5e1cda03/src/pickCombine.ts#L128
Stream existed in previous but not in new: https://github.com/staltz/cycle-onionify/blob/8b6344a749fb5b0ad7c0c8f1dcf2db4e5e1cda03/src/pickCombine.ts#L107
Notice that it is easier and more efficient to create a custom data structure (which uses a Map and relies on keys to disambiguate array items) than to do it directly on an array. You can hide the custom data structure from the external API.