I have an Observable which emits keys. Some of values for the keys are available locally and some need to be retrieved from the backend. I split the observable with:
const [keysLocal, keysServer]
= keysObservable.pipe(partition(key => valueMap.has(key)));
I then retrieve both values and output them to the same Subject:
keysLocal.pipe(map(key => valueMap.get(key)))
.subscribe(value => valuesSubject.next(value));
keysServer.pipe(switchMap(key => server.slowRequest(key)))
.subscribe(value => valuesSubject.next(value));
This works fine for most cases. However, when keysObservable emits keyServer_524 and keyLocal_314 in quick succession then valueLocal_314 is retrieved and output before valueServer_524. This leads to values output from valuesSubject to not be in the same order as the keys output from keysObservable.
What is the most idiomatic way to make sure that values are not emitted out-of-order? Since for my use-case I only care about the latest value it is okay to drop previous requests (valueServer_524 in the example).
The problem is that after partition the order of keys is lost. There is no way to reconstruct the order of the source observable. As a result I settled for a solution in which I use switchMap to get the latest value without partitioning the observable.
getValueObservable(key) {
if (valueMap.has(key)) {
return of(valueMap.get(key));
} else {
return server.slowRequest(key);
}
}
keysObservable.pipe(switchMap(getValueObservable))
.subscribe(value => valuesSubject.next(value));
I think this is due to the order in which subscriptions are created.
I'd say a quick way to fix this is to use the observeOn(asyncScheduler) operator on keysLocal so that it will output after keysServer:
keysLocal.pipe(observeOn(asyncScheduler), map(key => valueMap.get(key)))
.subscribe(value => valuesSubject.next(value));
keysServer.pipe(switchMap(key => server.slowRequest(key)))
.subscribe(value => valuesSubject.next(value));
Related
I have this rxjs code where cobineLatest is used
cobineLatest(
this.controls.toArray().map(c => c.changeEvent.asObservable())
).subscribe(x => {
console.log(x);
});
The thing is that there won't be any result in subscription until all observables emits, I wonder how would you change that behavior so it will start emitting even if one single observable emits?
I suggest you to just pipe the single observables to start with null. This way you ensure that each observable has emitted at least one value:
cobineLatest(
this.controls.toArray().map(c => c.changeEvent.asObservable().pipe(startWith(null)))
).subscribe(x => {
console.log(x);
});
If your are interested in the emitted value only (not the array), then merge might be your friend.
merge(
this.controls.toArray().map(c => c.changeEvent.asObservable())
).subscribe(x => {
console.log(x); // x is not an array
});
Is it possible to execute a high-order observable in parallel, but still preserve the order when merging the results?
I have something looking like this:
invoker$: Observable<void>;
fetch: (index: number) => Observable<T[]>;
invoker$
.pipe(
concatMap((_, index) => fetch(index)),
scan((acc: T[], values) => [...acc, ...values], [])
)
.subscribe(/* Do something with the array */);
The idea is having an observable that invokes a callback (e.g. backend call that takes a considerable amount of time) generating a new observable that emits a single value (array of some generic type). The returned values should be concatenated in another array while preserve their original fetch order.
I would, however, like the requests to be fired in parallel. So if the invoker$ is called rapidly, the requests are made in parallel and the results are merged as they complete.
My understanding is that the concatMap will wait for one observable to complete, before starting the next one. mergeMap will do it parallel, but won't do anything to preserve the order.
You can do it using mergeMap.
First, you need to pass the index together with the async response down the stream.
Then you can sort based on the index from the previous step.
Then you have two choices:
if the stream needs to end once all the requests are made and handle only once all the responses you can use reduce https://rxmarbles.com/#reduce
if the stream needs to continue for another batch of requests you need to use scan and later filter until you reach the needed event count. https://rxmarbles.com/#scan and https://rxmarbles.com/#filter
I am going to give you some pseudo-code for both examples:
In the reduce case, the stream ends once all requests are sent:
invoker$
.pipe(
mergeMap((_, index) => fetch(index).then(value => {value, index})),
reduce((acc: T[], singleValue) => [...acc, ...singleValue], []),
map(array => array.sort(/*Sort on index here*/).map(valueWithIndex => valueWithIndex.value))
)
.subscribe(/* Do something with the array */);
In the multiple-use case, I am assuming the size of the batch to be constant:
invoker$
.pipe(
mergeMap((_, index) => fetch(index).then(value => {value, index})),
scan((acc: T[], singleValue) => {
let resp = [...acc, ...singleValue];
// The scan can accumulate more than the batch size,
// so we need to limit it and restart for the new batch
if(resp.length > BATCH_SIZE) {
resp = [singleValue];
}
return resp;
}, []),
filter(array => array.length == BATCH_SIZE),
map(array =>
array
.sort(/*Sort on index here*/)
.map(valueWithIndex => valueWithIndex.value))
)
.subscribe(/* Do something with the array */);
2.1. In case the batch size is dynamic:
invoker$
.pipe(
mergeMap((_, index) => fetch(index).then(value => {value, index})),
withLatestFrom(batchSizeStream),
scan((acc: [T[], number], [singleValue, batchSize]) => {
let resp = [[...acc[0], ...singleValue], batchSize];
// The scan can accumulate more than the batch size,
// so we need to limit it and restart for the new batch
// NOTE: the batch size is dynamic and we do not want to drop data
// once the buffer size changes, so we need to drop the buffer
// only if the batch size did not change
if(resp[0].length > batchSize && acc[1] == batchSize) {
resp = [[singleValue], batchSize];
}
return resp;
}, [[],0]),
filter(arrayWithBatchSize =>
arrayWithBatchSize[0].length >= arrayWithBatchSize[1]),
map(arrayWithBatchSize =>
arrayWithBatchSize[0]
.sort(/*Sort on index here*/)
.map(valueWithIndex => valueWithIndex.value))
)
.subscribe(/* Do something with the array */);
EDIT: optimized sorting, added dynamic batch size case
I believe that the operator you are looking for is forkJoin.
This operator will take as input a list of observables, fire them in parallel and will return a list of the last emitted value of each observable once they all complete.
forkJoin({
invoker: invoker$,
fetch: fetch$,
})
.subscribe(({invoker, fetch}) => {
console.log(invoker, fetch);
});
Seems like this behavior is provided by the concatMapEager operator from the cartant/rxjs-etc library - written by Nicholas Jamieson
(cartant) who's a developer on the core RxJS team.
I have 2 requests.
getCurrentBook(): Observable<Book>
getDetailedInfo(bookId): Observable <BookDetailed>
They both return observables with information, however to use second request I have to make sure that I received the information from the first one since bookId is in the response.
I understand that I could subscribe inside other subscribe, however this solution doesn't seem appealing to me. There must be a much more elegant way.
The existing solution
getCurrentBook().subscribe(res => {
getDetailedInfo(res.id).subscribe(...);
})
I get that it should look something like:
booksSubs = getCurrentBook().pipe(
map(res =>
{this.currentBook = res}
)
)
detailedSubs = getDetailedInfo(this.currentBook.id).pipe(
map(res =>
{this.detailed = res}
)
)
this.subscriptions.push(SOME OPERATOR(booksSubs, detailedSubs).subscribe();
But the option higher won't work since I need result of first observable to initialize second.
You can achieve it using some of "flattening" operators, for example mergeMap:
const currentBookDetails$ = getCurrentBook().pipe(
mergeMap(book => getDetailedInfo(book.id))
);
I have this piece of RxJS code
this.listItems$ = this.store.select(EntityState.relationshipItems).pipe(
map(fn => fn(12)),
mergeMap(items => items),
map(this.toListItem),
toArray<ListItem>(),
tap(x => console.log(x))
);
Using mergeMap(items => items) I'm trying to "flatten" the array, then map each item to another object, and then convert it back to an array.
However, the flow doesn't even reach the last tap. I can see the toListItem function is called, but I don't understand why it stops there.
Transforming it to
this.listItems$ = this.store.select(EntityState.relationshipItems).pipe(
map(fn => fn(12)),
map(items => items.map(this.toListItem)),
tap(x => console.log(x))
);
makes it work, but I'd like to understand why the above one doesn't work.
That's because this.store.select(...) is a Subject that never completes (if it did then you could select data just once which doesn't make sense).
However, toArray collects all emissions from its source and when its source completes it emits a single array. But the source is this.store.select(...) that never completes so toArray never emits anything.
So probably the easiest workaround would be just restructuring your chain:
this.listItems$ = this.store.select(EntityState.relationshipItems).pipe(
map(fn => fn(12)),
mergeMap(items => from(items).pipe(
map(this.toListItem),
toArray<ListItem>(),
tap(x => console.log(x))
)),
);
Now the source is from that completes after iterating items so toArray will receive complete notification and emit its content as well.
Within an observable chain, I need to perform some async work, then return the source value to the next observable so I had to pipe(mapTo(x)) after the async work.
A more complete example:
// fake async work with just 1 null value
someAsyncWork = () => of(null)
of('1', '2', '3').pipe(
// some async work
concatMap(no => someAsyncWork().pipe(mapTo(no))),
concatMap(no => `Some async work [${no}] done!`)
).subscribe(message => console.log(message))
I cannot use tap(no => someAsyncWork()) because that would cause the next observable to run before someAsyncWork() returns.
While my current approach works, it somewhat clutters the code...and I have this pattern repeated in many places within the codebase.
Question: Anyway to do this without pipe(mapTo(no)) - in a more concise/readable way?
Perhaps the simplest thing to do would be to write your own pipeable operator.
For example:
const concatTap = <T>(project: (value: T) => Observable<any>) =>
concatMap((value: T) => project(value).pipe(mapTo(value)));
However, that assumes the observable for the async operation emits only a single value. To guard against multiple values being emitted you could do something like this:
const concatTap = <T>(project: (value: T) => Observable<any>) =>
concatMap((value: T) => concat(project(value).pipe(ignoreElements()), of(value)));
You could use concatTap like this:
of('1', '2', '3').pipe(
concatTap(() => someAsyncWork()),
concatMap(no => `Some async work [${no}] done!`)
).subscribe(message => console.log(message));
I'm sure you could choose a better name than I did. concatTap was the first thing that popped into my head. Naming things is hard.