New to RxJS, but I'm trying to map a single element stream to another that produces an array after all internal/subsequent streams are finished/loaded. However, my inner observables don't seem to be executing. They just get returned cold.
High level, I need to execute http post to upload a list of files (in two different arrays to two different endpoints). Since they are large I emulate with a delay of 5 seconds. The requests need to be executed in parallel, but limited to concurrently executing X at a time (here 2). This all needs to be inside a pipe and the pipe should only allow the stream to continue after all posts are complete.
https://stackblitz.com/edit/rxjs-pnwa1b
import { map, mapTo, mergeMap, mergeAll, delay, tap, catchError, toArray } from 'rxjs/operators';
import { interval, merge, forkJoin, of, from, range, Observable } from 'rxjs';
const single = "name";
const first = ["abc", "def"];
const second = of("ghi", "jkl", "mno");
of(single)
.pipe(tap(val => console.log(`emit:${val}`)))
.pipe(
map(claim =>
merge(
from(first).pipe(map(photo => of(photo).pipe(delay(5000)))),
from(second).pipe(map(video => of(video).pipe(delay(5000))))
)
.pipe(
mergeAll(2)
)
.pipe(tap(val => console.log(`emit:${val}`)))
.pipe(toArray())
.pipe(tap(val => console.log(`emit:${val}`)))
)
)
.pipe(
catchError(error => {
console.log("error");
return Observable.throw(error);
})
)
.subscribe(val => console.log(`final:${val}`));
An inner subscribe would not wait until they are complete. Using forkJoin would not allow me to limit the concurrent uploads. How can I accomplish this?
Update:
Answer by #dmcgrandle was very helpful and led me to make the changes below that seem to be working:
import { map, mapTo, mergeMap, mergeAll, delay, tap, catchError, toArray } from 'rxjs/operators';
import { interval, merge, forkJoin, of, from, range, Observable, throwError } from 'rxjs';
const single = "name";
const first = ["abc", "def"];
const second = of("ghi", "jkl", "mno");
of(single)
.pipe(tap(val => console.log(`emit:${val}`)))
.pipe(
mergeMap(claim =>
merge(
from(first).pipe(map(photo => of(photo).pipe(delay(5000)).pipe(tap(val => console.log(`emit:${val}`))))),
from(second).pipe(map(video => of(video).pipe(delay(5000)).pipe(tap(val => console.log(`emit:${val}`)))))
)
),
mergeAll(2),
toArray()
)
.pipe(
catchError(error => {
console.log("error");
return throwError(error);
})
)
.subscribe(val => console.log(`final:${val}`));
If I am understanding you correctly, then I think this is a solution. Your issue was with the first map, which won't perform an inner subscribe, but rather just transform the stream into Observables of Observables, which didn't seem to be what you wanted. Instead I used mergeMap there.
Inside the from's I used concatMap to force each emission from first and second to happen in order and wait for one to complete before another started. I also set up postToEndpoint functions that return Observables to be closer to what your actual code will probably look like.
StackBlitz Demo
code:
import { mergeMap, concatMap, delay, tap, catchError, toArray } from 'rxjs/operators';
import { merge, of, from, concat, throwError } from 'rxjs';
const single = "name";
const first = ["abc", "def"];
const second = of("ghi", "jkl", "mno");
const postToEndpoint1$ = photo => of(photo).pipe(
tap(data => console.log('start of postTo1 for photo:', photo)),
delay(5000),
tap(data => console.log('end of postTo1 for photo:', photo))
);
const postToEndpoint2$ = video => of(video).pipe(
tap(data => console.log('start of postTo2 for video:', video)),
delay(5000),
tap(data => console.log('end of postTo2 for video:', video))
);
of(single).pipe(
tap(val => console.log(`initial emit:${val}`)),
mergeMap(claim =>
merge(
from(first).pipe(concatMap(postToEndpoint1$)),
from(second).pipe(concatMap(postToEndpoint2$))
)
),
toArray(),
catchError(error => {
console.log("error");
return throwError(error);
})
).subscribe(val => console.log(`final:`, val));
I hope this helps.
Related
We have a timer that ticks e.g. every 2 seconds, whenever there is a tick we make a http call. The response time varies, it might be 100 ms but it might be 4 seconds. The catch is that we only make a new http call if the previous one completed. So we have 2 cases:
#1 If each http call takes 10sec we should be only sending a request every 10 seconds.
#2 If each http call takes 100ms we should be sending a request every 2 seconds.
What is also difficult in this scenario is that the getHttp call relies on value from the interval$ stream.
The best solution I came up so far is this:
import { of, interval, timer, combineLatest, BehaviorSubject } from 'rxjs';
import { filter, map, switchMap, tap } from 'rxjs/operators';
const interval$ = interval(2000).pipe(tap(console.log))
const readyForNext$ = new BehaviorSubject(true);
// Mocked http request that (in reality) uses the value from interval$
function getHttp(value) {
return of(value).pipe(
tap(() => readyForNext$.next(false)),
tap(() => console.log('getHttp: starting ')),
switchMap(() => timer(3000)),
tap(() => console.log('getHttp: resolve')),
map(() => 'hello'),
);
}
const anotherMockedStream$ = of(1) // In real code we have 2 streams in combineLatest
combineLatest([interval$, readyForNext$])
.pipe(
filter(([_, isReady]) => isReady),
switchMap(([value]) => {
return combineLatest([getHttp(value), anotherMockedStream$])
}),
tap(() => readyForNext$.next(true)),
)
.subscribe(console.log);
Playground: https://playcode.io/1023332
Alternative playground: https://stackblitz.com/edit/typescript-jdjjsc?file=index.ts
There are however two problem with this solution, it only works for case #1 and it is not very elegant (the use of readyForNext$). How to improve this code?
Thanks to #martin for pointing to exhaustMap, this is the correct operator for the operation, it seems to me that it is similar to switchMap but only does the switch once the steam is completed (or the emitted value reaches the subscription).
The working code looks as follows:
import { of, interval, timer, combineLatest } from 'rxjs';
import { map, switchMap, tap, exhaustMap } from 'rxjs/operators';
const interval$ = interval(2000).pipe(tap(console.log))
// Mocked http request that (in reality) uses the value from interval$
function getHttp(value) {
return of(value).pipe(
tap(() => console.log('getHttp: starting ')),
switchMap(() => timer(3000)),
tap(() => console.log('getHttp: resolve')),
map(() => 'hello'),
);
}
const anotherMockedStream$ = of(1)
interval$.pipe(
exhaustMap(val => {
return combineLatest([getHttp(val), anotherMockedStream$])
})
).subscribe(console.log);
With it I was able to remove the unnecessary readyForNext$ and it also works for both case #1 and case #2, i.e. if timer's value is lower or higher than interval.
Im trying to update this code to version 6 but I cannot figure out how rework the flow, from reading and playing around I think I need to pipe results etc but I cant see how it can be done using the flow of merge, filter, timer, map, first, toPromise() that used to work. Any RxJS folks able to educate me or point me in the right direction ?
const chats: chats[] = <chats[]>await Observable.merge(
this.chatService.$chats.filter(chats => chats.length > 0),
Observable.timer(5000).map(x => {
throw 'Timeout'
})
).first().toPromise()
if(chats.find( chat => chat.id === chatId )) {
this.log.log(`Found active chat ${chatId}, navigating to it from push notification...`)
}
Try:
import { merge, timer } from 'rxjs';
import { filter, first, map, } from 'rxjs/operators';
const chats: chats[] = <chats[]> await merge(
this.chatService.$chat.pipe(
filter(chats => chats.length > 0),
),
timer(5000).pipe(
map(x => {
throw 'Timeout';
}),
),
).pipe(
first(),
).toPromise();
if(chats.find( chat => chat.id === chatId )) {
this.log.log(`Found active chat ${chatId}, navigating to it from push notification...`)
}
You have to pipe the operators instead of chaining them with ..
I can't seem to be able to turn this cold observable into a hot one:
const test = new BehaviorSubject('test').pipe(tap(() => console.log('I want this to be logged only once to the console!')))
const grr = test.pipe(
share(), // share() seems to not do anything
take(1), // The culprit is here, causes logging to take place 5 times (5 subscribers)
share() // share() seems to not do anything
)
grr.subscribe(() => console.log(1))
grr.subscribe(() => console.log(2))
grr.subscribe(() => console.log(3))
grr.subscribe(() => console.log(4))
grr.subscribe(() => console.log(5))
// Expected output:
// 'I want this to be logged only once to the console!'
// 1
// 2
// 3
// 4
// 5
How should I change this to produce the wanted output?
You can use publishReplay and refCount operators like this:
import { interval, BehaviorSubject } from 'rxjs';
import { publishReplay, tap, refCount } from 'rxjs/operators';
const test = new BehaviorSubject('test').
pipe(
tap(() => console.log('I want this to be logged only once to the console!')
),
publishReplay(1),
refCount()
);
test.subscribe(() => console.log(1));
test.subscribe(() => console.log(2));
test.subscribe(() => console.log(3));
test.subscribe(() => console.log(4));
Working Stackblitz: https://stackblitz.com/edit/typescript-cvcmq6?file=index.ts
I'm trying to make multiple http requests and get returned data in one object.
const pagesToFetch = [2,3]
const request$ = forkJoin(
from(pagesToFetch)
.pipe(
mergeMap(page => this.mockRemoteData(page)),
)
)
mockRemoteData() return a simple Promise.
After first Observable emits (the once created from first entry of pagesToFetch the request$ is completed, second value in not included. How can I fix this?
You can turn each value in pagesToFetch into an Observable and then wait until all of them complete:
const observables = pagesToFetch.map(page => this.mockRemoteData(page));
forkJoin(observables)
.subscribe(...);
Or in case it's not that simple and you need pagesToFetch to be an Observable to collect urls first you could use for example this:
from(pagesToFetch)
.pipe(
toArray(),
mergeMap(pages => {
const observables = pages.map(page => this.mockRemoteData(page));
return forkJoin(observables);
}),
)
.subscribe(...);
Try the below sample format...
Observable.forkJoin(
URL 1,
URL 2
).subscribe((responses) => {
console.log(responses[0]);
console.log(responses[1]);
},
error => {console.log(error)}
);
I call backend that respond with:
[
"https://some-url.com/someData1.json",
"https://some-url.com/someData2.json"
]
Each JSON can have following schema:
{
"isValid": boolean,
"data": string
}
I want to get array with all data, that have isValid is set to true
backend.get(url)
.pipe(
mergeMap((urls: []) =>
urls.map((url: string) =>
backend.get(url)
.pipe(
filter(response => response.isValid),
map(response => response.data)
)
)
),
combineAll()
)
When both .json have "isValid" set to true, I get array with both data.
But when one of them has "isValid" set to false observable never completes.
I could use mergeAll instead of combineAll, but then I receive stream of single data not collection of all data.
Is there any better way to filter out observable?
As you said, the inner observable never emits, because filter does not forward the only value that is ever emitted by the backend.get observable. In that case, the operator subscribing on that observable - in your case combineAll - will also never receive any value and cannot ever emit itself.
What I would do is just move the filtering and mapping to combineAll by providing a project function, like that:
backend.get(url)
.pipe(
mergeMap((urls: string[]) =>
urls.map((url: string) => backend.get(url))
),
combineAll(responses =>
responses
.filter(response => response.isValid)
.map(response => response.data)
)
)
See if that works for you ;)
import { forkJoin, Observable } from 'rxjs';
import { map } from 'rxjs/operators';
interface IRes {
isValid: boolean;
data: string;
}
interface IResValid {
isValid: true;
data: string;
}
function isValid(data: IRes): data is IResValid {
return data.isValid;
}
const res1$: Observable<IRes> = backend.get(url1);
const res2$: Observable<IRes> = backend.get(url2);
// When all observables complete, emit the last emitted value from each.
forkJoin([res1$, res2$])
.pipe(map((results: IRes[]) => results.filter(isValid)))
.subscribe((results: IResValid[]) => console.log(results));