Re-execute async RxJS stream after delay - rxjs

I'm using RxJS 6 to lazily step through iterable objects using code similar to example running below. This is working well but I'm having trouble solving my final use case.
Full code here
import { EMPTY, defer, from, of } from "rxjs";
import { delay, expand, mergeMap, repeat } from "rxjs/operators";
function stepIterator (iterator) {
return defer(() => of(iterator.next())).pipe(
mergeMap(result => result.done ? EMPTY : of(result.value))
);
}
function iterateValues ({ params }) {
const { values, delay: delayMilliseconds } = params;
const isIterable = typeof values[Symbol.iterator] === "function";
// Iterable values which are emitted over time are handled manually. Otherwise
// the values are provided to Rx for resolution.
if (isIterable && delayMilliseconds > 0) {
const iterator = values[Symbol.iterator]();
// The first value is emitted immediately, the rest are emitted after time.
return stepIterator(iterator).pipe(
expand(v => stepIterator(iterator).pipe(delay(delayMilliseconds)))
);
} else {
return from(values);
}
}
const options = {
params: {
// Any iterable object is walked manually. Otherwise delegate to `from()`.
values: ["Mary", "had", "a", "little", "lamb"],
// Delay _between_ values.
delay: 350,
// Delay before the stream restarts _after the last value_.
runAgainAfter: 1000,
}
};
iterateValues(options)
// Is not repeating?!
.pipe(repeat(3))
.subscribe(
v => {
console.log(v, Date.now());
},
console.error,
() => {
console.log('Complete');
}
);
I'd like to add in another option which will re-execute the stream, an indefinite number of times, after a delay (runAgainAfter). I'm having trouble composing this in cleanly without factoring the result.done case deeper. So far I've been unable to compose the run-again behavior around iterateValues.
What's the best approach to accomplish the use case?
Thanks!
Edit 1: repeat just hit me in the face. Perhaps it means to be friendly.
Edit 2: No, repeat isn't repeating but the observable is completing. Thanks for any help. I'm confused.
For posterity here is the full code sample for a revised edition is repeat-able and uses a consistent delay between items.
import { concat, EMPTY, defer, from, interval, of, throwError } from "rxjs";
import { delay, expand, mergeMap, repeat } from "rxjs/operators";
function stepIterator(iterator) {
return defer(() => of(iterator.next())).pipe(
mergeMap(result => (result.done ? EMPTY : of(result.value)))
);
}
function iterateValues({ params }) {
const { values, delay: delayMilliseconds, times = 1 } = params;
const isIterable =
values != null && typeof values[Symbol.iterator] === "function";
if (!isIterable) {
return throwError(new Error(`\`${values}\` is not iterable`));
}
// Iterable values which are emitted over time are handled manually. Otherwise
// the values are provided to Rx for resolution.
const observable =
delayMilliseconds > 0
? defer(() => of(values[Symbol.iterator]())).pipe(
mergeMap(iterator =>
stepIterator(iterator).pipe(
expand(v => stepIterator(iterator).pipe(delay(delayMilliseconds)))
)
)
)
: from(values);
return observable.pipe(repeat(times));
}

I'm gonna be honest, but there could be better solution for sure. In my solution, I ended up encapsulating delay logic in a custom runAgainAfter operator. Making it an independent part, that doesn't affect your code logic directly.
Full working code is here
And the code of runAgainAfter if anybody needs it:
import { Observable } from "rxjs";
export const runAgainAfter = delay => observable => {
return new Observable(observer => {
let timeout;
let subscription;
const subscribe = () => {
return observable.subscribe({
next(value) {
observer.next(value);
},
error(err) {
observer.error(err);
},
complete() {
timeout = setTimeout(() => {
subscription = subscribe();
}, delay);
}
});
};
subscription = subscribe();
return () => {
subscription.unsubscribe();
clearTimeout(timeout);
};
});
};
Hope it helps <3

Related

Delay batch of observables with RxJS

I perform http requests to my db and have noticed that if I send all the requests at once, some of them will get a timeout errors. I'd like to add a delay between calls so the server doesn't get overloaded. I'm trying to find the RxJS solution to this problem and don't want to add a setTimeout.
Here is what I currently do:
let observables = [];
for(let int = 0; int < 10000; int++){
observables.push(new Observable((observer) => {
db.add(doc[int], (err, result)=>{
observer.next();
observer.complete();
})
}))
}
forkJoin(observables).subscribe(
data => {
},
error => {
console.log(error);
},
() => {
db.close();
}
);
You can indeed achieve this with Rxjs quite nicely. You'll need higher order observables, which means you'll emit an observable into an observable, and the higher order observable will flatten this out for you.
The nice thing about this approach is that you can easily run X requests in // without having to manage the pool of requests yourself.
Here's the working code:
import { Observable, Subject } from "rxjs";
import { mergeAll, take, tap } from "rxjs/operators";
// this is just a mock to demonstrate how it'd behave if the API was
// taking 2s to reply for a call
const mockDbAddHtppCall = (id, cb) =>
setTimeout(() => {
cb(null, `some result for call "${id}"`);
}, 2000);
// I have no idea what your response type looks like so I'm assigning
// any but of course you should have your own type instead of this
type YourRequestType = any;
const NUMBER_OF_ITEMS_TO_FETCH = 10;
const calls$$ = new Subject<Observable<YourRequestType>>();
calls$$
.pipe(
mergeAll(3),
take(NUMBER_OF_ITEMS_TO_FETCH),
tap({ complete: () => console.log(`All calls are done`) })
)
.subscribe(console.log);
for (let id = 0; id < NUMBER_OF_ITEMS_TO_FETCH; id++) {
calls$$.next(
new Observable(observer => {
console.log(`Starting a request for ID "${id}""`);
mockDbAddHtppCall(id, (err, result) => {
if (err) {
observer.error(err);
} else {
observer.next(result);
observer.complete();
}
});
})
);
}
And a live demo on Stackblitz: https://stackblitz.com/edit/rxjs-z1x5m9
Please open the console of your browser and note that the console log showing when a call is being triggered starts straight away for 3 of them, and then wait for 1 to finish before picking up another one.
Looks like you could use an initial timer to trigger the http calls. e.g.
timer(delayTime).pipe(combineLatest(()=>sendHttpRequest()));
This would only trigger the sendHttpRequest() method after the timer observable had completed.
So with your solution. You could do the following...
observables.push(
timer(delay + int).pipe(combineLatest(new Observable((observer) => {
db.add(doc[int], (err, result)=>{
observer.next();
observer.complete();
}))
}))
Where delay could probably start off at 0 and you could increase it using the int index of your loop by some margin.
Timer docs: https://www.learnrxjs.io/learn-rxjs/operators/creation/timer
Combine latest docs: https://www.learnrxjs.io/learn-rxjs/operators/combination/combinelatest
merge with concurrent value:
mergeAll and mergeMap both allow you to define the max number of subscribed observables. mergeAll(1)/mergeMap(LAMBDA, 1) is basically concatAll()/concatMap(LAMBDA).
merge is basically just the static mergeAll
Here's how you might use that:
let observables = [...Array(10000).keys()].map(intV =>
new Observable(observer => {
db.add(doc[intV], (err, result) => {
observer.next();
observer.complete();
});
})
);
const MAX_CONCURRENT_REQUESTS = 10;
merge(...observables, MAX_CONCURRENT_REQUESTS).subscribe({
next: data => {},
error: err => console.log(err),
complete: () => db.close()
});
Of note: This doesn't batch your calls, but it should solve the problem described and it may be a bit faster than batching as well.
mergeMap with concurrent value:
Perhaps a slightly more RxJS way using range and mergeMap
const MAX_CONCURRENT_REQUESTS = 10;
range(0, 10000).pipe(
mergeMap(intV =>
new Observable(observer => {
db.add(doc[intV], (err, result) => {
observer.next();
observer.complete();
});
}),
MAX_CONCURRENT_REQUESTS
)
).subscribe({
next: data => {},
error: err => console.log(err),
complete: () => db.close()
});

Keeping error information and the outer observable alive

To ensure an error doesn't complete the outer observable, a common rxjs effects pattern I've adopted is:
public saySomething$: Observable<Action> = createEffect(() => {
return this.actions.pipe(
ofType<AppActions.SaySomething>(AppActions.SAY_SOMETHING),
// Switch to the result of the inner observable.
switchMap((action) => {
// This service could fail.
return this.service.saySomething(action.payload).pipe(
// Return `null` to keep the outer observable alive!
catchError((error) => {
// What can I do with error here?
return of(null);
})
)
}),
// The result could be null because something could go wrong.
tap((result: Result | null) => {
if (result) {
// Do something with the result!
}
}),
// Update the store state.
map((result: Result | null) => {
if (result) {
return new AppActions.SaySomethingSuccess(result);
}
// It would be nice if I had access the **error** here.
return new AppActions.SaySomethingFail();
}));
});
Notice that I'm using catchError on the inner observable to keep the outer observable alive if the underlying network call fails (service.saySomething(action.payload)):
catchError((error) => {
// What can I do with error here?
return of(null);
})
The subsequent tap and map operators accommodate this in their signatures by allowing null, i.e. (result: Result | null). However, I lose the error information. Ultimately when the final map method returns new AppActions.SaySomethingFail(); I have lost any information about the error.
How can I keep the error information throughout the pipe rather than losing it at the point it's caught?
As suggested in comments you should use Type guard function
Unfortunately I can't run typescript in snippet so I commented types
const { of, throwError, operators: {
switchMap,
tap,
map,
catchError
}
} = rxjs;
const actions = of({payload: 'data'});
const service = {
saySomething: () => throwError(new Error('test'))
}
const AppActions = {
}
AppActions.SaySomethingSuccess = function () {
}
AppActions.SaySomethingFail = function() {
}
/* Type guard */
function isError(value/*: Result | Error*/)/* value is Error*/ {
return value instanceof Error;
}
const observable = actions.pipe(
switchMap((action) => {
return service.saySomething(action.payload).pipe(
catchError((error) => {
return of(error);
})
)
}),
tap((result/*: Result | Error*/) => {
if (isError(result)) {
console.log('tap error')
return;
}
console.log('tap result');
}),
map((result/*: Result | Error*/) => {
if (isError(result)) {
console.log('map error')
return new AppActions.SaySomethingFail();
}
console.log('map result');
return new AppActions.SaySomethingSuccess(result);
}));
observable.subscribe(_ => {
})
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.5.5/rxjs.umd.js"></script>
I wouldn't try to keep the error information throughout the pipe. Instead you should separate your success pipeline (tap, map) from your error pipeline (catchError) by adding all operators to the observable whose result they should actually work with, i.e. your inner observable.
public saySomething$: Observable<Action> = createEffect(() => {
return this.actions.pipe(
ofType<AppActions.SaySomething>(AppActions.SAY_SOMETHING),
switchMap((action) => this.service.saySomething(action.payload).pipe(
tap((result: Result) => {
// Do something with the result!
}),
// Update the store state.
map((result: Result) => {
return new AppActions.SaySomethingSuccess(result);
}),
catchError((error) => {
// I can access the **error** here.
return of(new AppActions.SaySomethingFail());
})
)),
);
});
This way tap and map will only be executed on success results from this.service.saySomething. Move all your error side effects and error mapping into catchError.

How to return from within an observer?

I was trying to return filter function but return doesn't seem to work with callbacks. Here this.store.let(getIsPersonalized$) is an observable emitting boolean values and this.store.let(getPlayerSearchResults$) is an observable emiting objects of video class.
How do I run this synchronously, can I avoid asynchronus callback altogether given that I can't modify the observables received from store.
isPersonalized$ = this.store.let(getIsPersonalized$);
videos$ = this.store.let(getPlayerSearchResults$)
.map((vids) => this.myFilter(vids));
myFilter(vids) {
this.isPersonalized$.subscribe((x){
if(x){
return this.fileterX(vids);//Return from here
}
else {
return this.filterY(vids);//Or Return from here
}
});
}
fileterX(vids) {
return vids.filter((vid) => vids.views>100;);
}
fileterY(vids) {
return vids.filter((vid) => vids.views<20;);
}
I got it working this way, you don't need myFilter(vids) at all if you can get the branching out on isPersonalized$'s subscribe. Here is the updated code.
this.store.let(getIsPersonalized$);
videos$: Observable<any>;
ngOnInit() {
this.isPersonalized$.subscribe((x) => {
if (x) {
this.videos$ = this.store.let(getPlayerSearchResults$)
.map((vids) => this. fileterX(vids));
} else {
this.videos$ = this.store.let(getPlayerSearchResults$)
.map((vids) => this. fileterY(vids));
}
});
}
fileterX(vids) {
return vids.filter((vid) => vids.views>100;);
}
fileterY(vids) {
return vids.filter((vid) => vids.views<20;);
}
It looks like you want to evaluate the latest value of isPersonalized$ within the map function, i'd do that via withLatestFrom (Example: The first one toggles true/false every 5s, the second emits an increasing number every 1s):
const isPersonalized$ = Rx.Observable.interval(5000)
.map(value => value % 2 === 0);
const getPlayerSearchResults$ = Rx.Observable.interval(1000)
.withLatestFrom(isPersonalized$)
.map(bothValues => {
const searchResult = bothValues[0];
const isPersonalized = bothValues[1];
...
});

How to get an observable to return data immediately and every 5 seconds thereafter

I want to create an observable that returns data from a webapi. I'd like it to return the data immediately, and poll the API every 10 seconds. The code below shows I'm using the 'interval' method. But this delays the first set of data by 10 seconds. How do I get that first flush of data to come down with no initial delay?
export class EventService {
public events$: Observable<Event[]>;
private _eventsObserver: Observer<Event[]>;
private pollInterval: number = 5000;
private _dataStore: {
events: Event[];
};
constructor(private http: Http) {
this._dataStore = { events: [] };
this.events$ = new Observable(observer => this._eventsObserver = observer)
.startWith(this._dataStore.events)
.share();
}
pollEvents() {
return Observable.interval(10000)
.switchMap(() => {
return this.http.get('app/resources/data/scheduleevents.json')
.map((responseData) => {
return responseData.json();
});
})
.map((events: Array<any>) => {
let result: Array<Event> = [];
if (events["data"]) {
events["data"].forEach((event) => {
result.push(event);
});
}
return result;
});
}
}
Got it:
.interval(5000)
.startWith(0);
Use timer. I think the timer is what you need (see RxJS tab):
http://reactivex.io/documentation/operators/timer.html#collapseRxJS
Could be used like:
Observable.timer(0, 5000).flatMap(() => apiCall())
Where 0 - delay before emitting the first value, 5000 - emit value after each 5s
let timer = TimerObservable.create(0, 5000);
this.sub = timer.subscribe(t => {
this.yourMethod()
});
To unsubscribe run this.sub.unsubscribe()
I personnally use interval with startWith (need RxJs 6+), here is a complete example:
history: any;
historySubscription: Subscription;
constructor(private jobService: JobService) { }
ngOnInit() {
this.historySubscription = interval(10000).pipe(
startWith(0),
flatMap(() => this.jobService.getHistory())
).subscribe(data => {
this.history = data;
});
}
ngOnDestroy() {
this.historySubscription.unsubscribe();
}
This retrieves history on init and then every 10 seconds.
Another alternative is using timer as explained by #Alendorff.
For angualr2 below is the code i have written in my application and it is working as expected -
In service --
import { Observable } from 'rxjs/Observable';
import 'rxjs/Rx';
getList(): Observable<IVM> {
return Observable.interval(5000).startWith(0)
.switchMap(() =>
this._http.get(this._vmURL )
.map((response: Response) => <IVM>response.json().data)
.do(data => console.log('All: ' + JSON.stringify(data)))
.catch(this.handleError)
);
}
In component --
private getInstanceDetails(): void {
this._vmDataService.getList()
.subscribe(vmList => {
//Do whatever you want with the vmList here :)
},
error => this.errorMessage = <any>error);
}
Thanks,
Kindly let me know your thoughts.
Observable.interval(5L, TimeUnit.SECONDS)
.startWith(0)
.observeOn(AndroidSchedulers.mainThread())
.map { foobar() }
works fine for me.
Thanks

Create an Rx.Subject using Subject.create that allows onNext without subscription

When creating an Rx.Subject using Subject.create(observer, observable), the Subject is so lazy. When I try to use subject.onNext without having a subscription, it doesn't pass messages on. If I subject.subscribe() first, I can use onNext immediately after.
Let's say I have an Observer, created like so:
function createObserver(socket) {
return Observer.create(msg => {
socket.send(msg);
}, err => {
console.error(err);
}, () => {
socket.removeAllListeners();
socket.close();
});
}
Then, I create an Observable that accepts messages:
function createObservable(socket) {
return Observable.fromEvent(socket, 'message')
.map(msg => {
// Trim out unnecessary data for subscribers
delete msg.blobs;
// Deep freeze the message
Object.freeze(msg);
return msg;
})
.publish()
.refCount();
}
The subject is created using these two functions.
observer = createObserver(socket);
observable = createObservable(socket);
subject = Subject.create(observer, observable);
With this setup, I'm not able to subject.onNext immediately (even if I don't care about subscribing). Is this by design? What's a good workaround?
These are actually TCP sockets, which is why I haven't relied on the super slick websocket subjects.
The basic solution, caching nexts before subscription with ReplaySubject:
I think all you wanted to do is use a ReplaySubject as your observer.
const { Observable, Subject, ReplaySubject } = Rx;
const replay = new ReplaySubject();
const observable = Observable.create(observer => {
replay.subscribe(observer);
});
const mySubject = Subject.create(replay, observable);
mySubject.onNext(1);
mySubject.onNext(2);
mySubject.onNext(3);
mySubject.subscribe(x => console.log(x));
mySubject.onNext(4);
mySubject.onNext(5);
Results in:
1
2
3
4
5
A socket implementation (example, don't use)
... but if you're looking at doing a Socket implementation, it gets a lot more complicated. Here is a working socket implementation, but I don't recommend you use it. Rather, I'd suggest that you use one of the community supported implementations either in rxjs-dom (if you're an RxJS 4 or lower) or as part of RxJS 5, both of which I've helped work on.
function createSocketSubject(url) {
let replay = new ReplaySubject();
let socket;
const observable = Observable.create(observer => {
socket = new WebSocket(url);
socket.onmessage = (e) => {
observer.onNext(e);
};
socket.onerror = (e) => {
observer.onError(e);
};
socket.onclose = (e) => {
if (e.wasClean) {
observer.onCompleted();
} else {
observer.onError(e);
}
}
let sub;
socket.onopen = () => {
sub = replay.subscribe(x => socket.send(x));
};
return () => {
socket && socket.readyState === 1 && socket.close();
sub && sub.dispose();
}
});
return Subject.create(replay, observable);
}
const socket = createSocketSubject('ws://echo.websocket.org');
socket.onNext('one');
socket.onNext('two');
socket.subscribe(x => console.log('response: ' + x.data));
socket.onNext('three');
socket.onNext('four');
Here's the obligatory JsBin

Resources