I have an xhr request that is getting an array with which I execute subsequent xhr requests like so:
const Rx = require('rxjs/Rx');
const fetch = require('node-fetch');
const url = `url`;
// Get array of tables
const tables$ = Rx.Observable
.from(fetch(url).then((r) => r.json()));
// Get array of columns
const columns$ = (table) => {
return Rx.Observable
.from(fetch(`${url}/${table.TableName}/columns`).then(r => r.json()));
};
tables$
.mergeMap(tables => Rx.Observable.forkJoin(...tables.map(columns$)))
.subscribe(val => console.log(val));
I would like to execute the column requests in chuncks so that the requests are not being sent to the server at once.
This SO question is somewhat in the same direction but not completely: Rxjs: Chunk and delay stream?
Now I'm trying something like this:
tables$
.mergeMap(tables => Rx.Observable.forkJoin(...tables.map(columns$)))
.flatMap(e => e)
.bufferCount(4)
.executeTheChunksSerial(magic)
.flatMap(e => e)
.subscribe(val => console.log(val));
But I cannot wrap my head around how to execute the chunks in series...
You can utilize the concurrency argument of mergeMap to get max x requests concurrently to your server:
const getTables = Promise.resolve([{ tableName: 'foo' },{ tableName: 'bar' },{ tableName: 'baz' }]);
const getColumns = (table) => Rx.Observable.of('a,b,c')
.do(_ => console.log('getting columns for table: ' + table))
.delay(250);
Rx.Observable.from(getTables)
.mergeAll()
.mergeMap(
table => getColumns(table.tableName),
(table, columns) => ({ table, columns }),
2)
.subscribe(console.log)
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.4.3/Rx.js"></script>
Related
I have an unexpected behavior with the operator withLatestFrom.
Output
map a
a
map a <== why a is mapped again ?
map b
b
const { Subject, operators } = window.rxjs
const { map, withLatestFrom } = operators
const createA = new Subject()
const createB = new Subject()
const a = createA.pipe(
map(() => console.log('map a'))
)
const b = createB.pipe(
withLatestFrom(a),
map(() => console.log('map b'))
)
a.subscribe(() => { console.log('a') })
b.subscribe(() => { console.log('b') })
createA.next()
createB.next()
<script src="https://unpkg.com/#reactivex/rxjs#6.6.3/dist/global/rxjs.umd.js"></script>
I found that the operator share() allows multiple subscribers.
const a = createA.pipe(
map(() => console.log('map a')),
share()
)
The problem here isn't with withLatestFrom() but rather with how subscriptions work. Observables are lazy and don't run until you subscribe. Each new subscriptions will run the observable again.
const stream$ = from([1,2,3]);
stream$.subscribe(console.log) // output: 1 2 3
stream$.subscribe(console.log) // output: 1 2 3
In this case, `from([1,2,3)] ran twice. If I alter my stream, anything I do will happen for each subscriber.
const stream$ = from([1,2,3]).pipe(
tap(_ => console.log("hi"))
);
stream$.subscribe(console.log) // output: hi 1 hi 2 hi 3
stream$.subscribe(console.log) // output: hi 1 hi 2 hi 3
The final piece of the puzzle is this: internally withLatestFrom() subscribes to the stream that you give it. Just like an explicit .subscribe() runs the observable, so does withLatestFrom() once it's subscribed to.
You can use shareReplay to cache the latest values and replay them instead of running the observable again. It's one way to manage a multicasted stream:
const createA = new Subject()
const createB = new Subject()
const a = createA.pipe(
tap(() => console.log('tap a')),
shareReplay(1)
)
const b = createB.pipe(
withLatestFrom(a),
tap(() => console.log('tap b'))
)
a.subscribe(() => { console.log('a') })
b.subscribe(() => { console.log('b') })
createA.next()
createB.next()
Now a.subscribe() and withLatestFrom(a) are both getting a buffered value that only gets run when createA.next() is executed.
As an aside, mapping a value to nothing is bad habit to get into. Consider the following code:
from([1,2,3]).pipe(
map(val => console.log(val))
).subscribe(val => console.log(val));
This will output
1
undefined
2
undefined
3
undefined
because you're actually mapping each value to nothing. tap on the other hand doesn't change the source observable, so it's a much better tool for debugging and/or side effects that don't alter the stream
from([1,2,3]).pipe(
tap(val => console.log(val))
).subscribe(val => console.log(val));
This will output
1
1
2
2
3
3
I have to create a queue of ajax requests and group the result but i have no clue about to accomplish this.
Let's say i have an array like this:
const actors = [
"Al Pacino",
"Robert De Niro"
];
I have to iterate over it and for each values make an api call:
export const getMovies = action$ =>
action$.pipe(
ofType(LOAD_REQUEST),
// iterate over the array
// make api call with the actor name
// for each result, make a second api call with the id of the actor (get in the response.results.id)
// group the result in an array with all films of all actors of the array passed in the payload
);
Im stuck with switchMap, pipe ... and don't know the correct way to accomplish this.
Edit Tried your solution Valeriy but got this error:
export const getMovies = action$ =>
action$.pipe(
ofType(LOAD_REQUEST),
switchMap(({ payload }) =>
combineLatest(
payload.map(a => {
return ajax
.getJSON(actor(a))
.pipe(map(response => console.log(response)));
})
)
)
);
TypeError: You provided 'function (source) {
return source.lift.call(Object(_observable_from__WEBPACK_IMPORTED_MODULE_2__["from"])([source].concat(observables)), new _observable_combineLatest__WEBPACK_IMPORTED_MODULE_1__["CombineLatestOperator"](project));
}' where a stream was expected. You can provide an Observable, Promise, Array, or Iterable.
If I understand you correctly, you are trying to achieve something like this:
export const getMovies = action$ => action$.pipe(
ofType(LOAD_REQUEST),
switchMap(() => {
// group the result in an array with all films of all actors of the array passed in the payload
return combineLatest(
// iterate over the array
...actors.map(actorName => {
// make api call with the actor name
return loadActor(actorName).pipe(
// for each result, make a second api call with the id of the actor (get in the response.results.id)
switchMap(response => loadActorFilms(response.results.id))
);
})
);
})
);
I've used combineLatest to group multiple observables together.
I have 2 observables that both indicate if they're loading data or not. They come from #ngrx/data.
loadingA$: Observable<boolean>
loadingB$: Observable<boolean>
I'd like to "logical OR" combine the two to do whatever when either is true, using rxjs or more elegant method. Maybe ramdajs?, maybe a combined loading state? However different components need different combinations of loading streams.
Also, what if I have 20 streams, it shouldn't be limited to 2 streams only.
(!) I do not want to assign additional local variables.
combineLatest(loadingA$, loadingB$).pipe(map((a, b) => a || b));
or
const anyLoading = (...observables: Observable<boolean>[]) => combineLatest(observables).pipe(
map(bools => bools.some(loading => loading))
);
and use it
anyLoading(loadingA$, loadingB$);
const { combineLatest, BehaviorSubject } = rxjs;
const { map } = rxjs.operators;
const anyLoading = (...observables) => combineLatest(observables).pipe(
map(bools => bools.some(loading => loading))
);
const loadingA$ = new BehaviorSubject(false);
const loadingB$ = new BehaviorSubject(true);
anyLoading(loadingA$, loadingB$).subscribe(loading => { console.log(loading); });
loadingB$.next(false);
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.5.3/rxjs.umd.min.js"></script>
I'm trying to make multiple http requests and get returned data in one object.
const pagesToFetch = [2,3]
const request$ = forkJoin(
from(pagesToFetch)
.pipe(
mergeMap(page => this.mockRemoteData(page)),
)
)
mockRemoteData() return a simple Promise.
After first Observable emits (the once created from first entry of pagesToFetch the request$ is completed, second value in not included. How can I fix this?
You can turn each value in pagesToFetch into an Observable and then wait until all of them complete:
const observables = pagesToFetch.map(page => this.mockRemoteData(page));
forkJoin(observables)
.subscribe(...);
Or in case it's not that simple and you need pagesToFetch to be an Observable to collect urls first you could use for example this:
from(pagesToFetch)
.pipe(
toArray(),
mergeMap(pages => {
const observables = pages.map(page => this.mockRemoteData(page));
return forkJoin(observables);
}),
)
.subscribe(...);
Try the below sample format...
Observable.forkJoin(
URL 1,
URL 2
).subscribe((responses) => {
console.log(responses[0]);
console.log(responses[1]);
},
error => {console.log(error)}
);
This is for redux-observable but I think the general pattern is pretty generic to rxjs
I have a stream of events (from redux-observable, these are redux actions) and I'm specifically looking to pair up two differnt types of events for the same "resource" - "resource set active" and "resource loaded" - and emit a new event when these events "match up". The problem is these can come in in any order, for any resources, and can be fired multiple times. You might set something active before it is loaded, or load something before it is set active, and other resources might get loaded or set active in between.
What I want is a stream of "this resource, which is now loaded, is now active" - which also means that once a resource is loaded, it can be considered forever loaded.
If these events were not keyed by a resource id, then it would be very simple:
First I would split them up by type:
const setActive$ = action$.filter(a => a.type == 'set_active');
const load = action$.filter(a => a.type == 'loaded');
In a simple case where there is no keying, I'd say something like:
const readyevent$ = setActive$.withLatestFrom(loaded$)
then readyevent$ is just a stream of set_active events where there has been at least one loaded event.
But my problem is that the set_active and loaded events are each keyed by a resource id, and for reasons beyond my control, the property to identify the resource is different in the two events. So this becomes something like:
const setActive$ = action$.filter(a => a.type === 'set_active').groupBy(a => a.activeResourceId);
const loaded$ = action$.filter(a => a.type === 'loaded').groupBy(a => a.id);
but from this point I can't really figure out how to then "re-join
" these two streams-of-grouped-observables on the same key, so that I can emit a stream of withLatestFrom actions.
I believe this does what you are describing:
const action$ = Rx.Observable.from([
{ activeResourceId: 1, type: 'set_active' },
{ id: 2, type: 'loaded' },
{ id: 1, type: 'random' },
{ id: 1, type: 'loaded' },
{ activeResourceId: 2, type: 'set_active' }
]).zip(
Rx.Observable.interval(500),
(x) => x
).do((x) => { console.log('action', x); }).share();
const setActiveAction$ = action$.filter(a => a.type === 'set_active')
.map(a => a.activeResourceId)
.distinct();
const allActive$ = setActiveAction$.scan((acc, curr) => [...acc, curr], []);
const loadedAction$ = action$.filter(a => a.type === 'loaded')
.map(a => a.id)
.distinct();
const allLoaded$ = loadedAction$.scan((acc, curr) => [...acc, curr], []);
Rx.Observable.merge(
setActiveAction$
.withLatestFrom(allLoaded$)
.filter(([activeId, loaded]) => loaded.includes(activeId)),
loadedAction$
.withLatestFrom(allActive$)
.filter(([loadedId, active]) => active.includes(loadedId))
).map(([id]) => id)
.distinct()
.subscribe((id) => { console.log(`${id} is loaded and active`); });
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.10/Rx.min.js"></script>
The basic approach is to create a distinct stream for each action type and join it with the distinct aggregate of the other. Then merge the two streams. This will emit the value when there are matching setActive and loaded events. The distinct() on the end of the merge makes it so that you will only get notified once. If you want a notification on each setActive action after the initial one then just remove that operator.
groupBy looks somewhat complicated to do this with, there's a key value in there but you get an Observable of Observables - so maybe a little hard to get right.
I would map the id to a common property and then use scan to combine. I use this pattern for grouping in my app.
The accumulator in the scan is an object, which is used as an associative array - each property is an id and the property value is an array of actions accumulated so far.
After the scan, we convert to an observable stream of arrays of matching actions - sort of like withLatestFrom but some arrays may not yet be complete.
The next step is to filter for those arrays we consider complete.
Since you say
where there has been at least one loaded event
I'm going to assume that the presence of two or more actions, with at least one is type 'loaded' - but it's a bit tricky to tell from your question if that is sufficient.
Finally, reset that id in the accumulator as presumably it may occur again later in the stream.
const setActive$ = action$.filter(a => a.type === 'set_active')
.map(a => { return { id: a.activeResourceId, action: a } });
const loaded$ = action$.filter(a => a.type === 'loaded')
.map(a => { return { id: a.id, action: a } });
const accumulator = {};
const readyevent$: Observable<action[]> =
Observable.merge(setActive$, loaded$)
.scan((acc, curr) => {
acc[curr.id] = acc[curr.id] || [];
acc[curr.id].push(curr.action)
}, accumulator)
.mergeMap((grouped: {}) => Observable.from(
Object.keys(grouped).map(key => grouped[key])
))
.filter((actions: action[]) => {
return actions.length > 1 && actions.some(a => a.type === 'loaded')
})
.do(actions => {
const id = actions.find(a => a.type === 'loaded').id;
accumulator[id] = [];
});