In rxjs5, I'm trying to implement a Throttler class.
import Rx from 'rxjs/rx';
export default class Throttler {
constructor(interval) {
this.timeouts = [];
this.incomingActions = new Rx.Subject();
this.incomingActions
.concatMap(action => Rx.Observable.just(action).delay(interval / 2))
.subscribe(action => action());
}
clear() {
// How do I do this?
}
do(action) {
this.incomingActions.next(action);
}
}
The following invariants must hold:
every action passed to do gets added to an action queue
the action queue gets processed in order and at a fixed interval as determined by the constructor parameter
the action queue can be cleared using clear().
My current implementation, as seen above, handles the fixed interval, but I don't know how to clear the queue. It also has the problem that all actions are delayed by interval / 2ms even when the queue is empty.
P.S. The way I describe the invariants maps very easily to an implementation with setInterval and an array as a queue, but I'm wondering how I would do this with Rx.
This seems like not a good place for the default Subject class. Extending it with your own subclass would be better because of reasons you listed.
However, in your case I'd try to identify each action that comes to .do(action) method with some index and add .filter() operator before subscribe() to be able to cancel particular actions by checking some array for what indices are marked as canceled. Since you're using concatMap() you know that actions will be always called in the order they were added. Then clear() method that you want would just mark all actions to be canceled in the array.
You can also add .do() operator after concatMap() and keep track of how many action are scheduled at the moment with some accumulator. Adding action would cause scheduledAction++ while passing .do() right before .subscribe() would scheduledAction--. Then you can use this variable to decide whether you want to chain a new action with .delay(interval / 2) or not.
Related
I'm trying to cache http calls in the service so all subsequent calls returns same response. This is fairly easy with shareReplay:
data = this.http.get(url).pipe(
shareReplay(1)
);
But it doesn't work in case of backend / network errors. ShareReplay spams the backend with requests in case of any error when this Observable is bound to the view through async pipe.
I tried with retryWhen etc but the solution I got is untestable:
data = this.http.get(url).pipe(
retryWhen(errors => errors.pipe(delay(10000))),
shareReplay(1)
);
fakeAsync tests fails with "1 timer(s) still in the queue" error because delay timer has no end condition. I also don't want to have some hanging endless timer in the background - it should stop with the last subscription.
The behavior I would like:
Multicast - make only one subscription to source even with many subscribers.
Do not count refs for successful queries - reuse same result when subscriber count goes to 0 and back to 1.
In case of error - retry every 10 seconds but only if there are any subscribers.
My 2 cents:
This code is for rxjs > 6.4 (here V6.6)
To use a shared observable, you need to return the same observable for all the subscribers (or you will create an observable which has nothing to share)
Multicasting can be done using shareReplay and you can replay the last emitted value (even after the last subscriber to have unsubscribed) using the {refCount: false} option.
As long as there is no subscription, the observable does nothing. You will not have any fetch on the server before the first subscriber.
beware:
If refCount is false, the source will not be
unsubscribed meaning that the inner ReplaySubject will still be
subscribed to the source (and potentially run for ever).
Also:
A successfully completed source will stay cached in the shareReplayed
observable forever, but an errored source can be retried.
The problem is using shareReplay, you have to choose between:
Always getting the last value even if the refCount went back to 0 and having possible never ending retries in case of error (remember shareReplay with refCount to false never unsubscribes)
Or keeping the default refCount:true which mean you won't have the second "first subscriber" cache benefit. Conversely the retry will also stop if no subscriber is there.
Here is a dummy example:
class MyServiceClass {
private data;
// assuming you are injecting the http service
constructor(private http: HttpService){
this.data = this.buildData("http://some/data")
}
// use this accessor to get the unique (shared) instance of data observable.
public getData(){
return this.data;
}
private buildData(url: string){
return this.http.get(url).pipe(
retryWhen(errors => errors.pipe(delay(10000))),
shareReplay({refCount: false})
);
}
}
Now in my opinion, to fix the flow you should prevent your retry to run forever, adding for instance a maximum number of retries
I'm new in RxSwift, I don't understand what is difference between do(onNext:) and subscribe(onNext:).
I google it but did't found good resources to explain the difference.
At the beginning of a cold Observable chain there is a function that generates events, for e.g. the function that initiates a network request.
That generator function will not be called unless the Observable is subscribed to (and by default, it will be called each time the observable is subscribed to.) So if you add a do(onNext:) to your observable chain, the function will not be called and the action that generates events will not be initiated. You have to add a subscribe(onNext:) for that to happen.
(The actual internals are a bit more complex than the above description, but close enough for this explanation.)
The do operator allows you to insert side effects; that is, handlers to do things that will not change the emitted event in any way. do will just pass the event through to the next operator in the chain.
The method for using the do operator is here.
And you can provide handlers for any or all of these events.
Let's say We have an observable that never emits anything. Even though it emits nothing, it is still an observable and we can subscribe to it. do operator allows us to do something when a subscription was made to it.
So below example will print "Subscribed" when a subscription was made to that observable.
Feel free to include any of the other handlers if you’d like; they work just like subscribe’s handlers do
let observable = Observable<Any>.never()
let disposeBag = DisposeBag()
observable
.do(onSubscribe: {
print("Subscribed")
})
.subscribe(
onNext: { element in
print(element)
},
onCompleted: {
print("Completed")
},
onDisposed: {
print("Disposed")
}
)
.disposed(by: disposeBag)
I have the following epic I use in my application to handle api requests:
action$ => {
return action$.ofType(actions.requestType)
.do(() => console.log('handled epic ' + actions.requestType))
.switchMap((action) => (
Observable.create((obs) => {
obs.next({ type: type, value: action.value, form: action.form });
})
.debounceTime(250)
.switchMap((iea) => (
Observable.ajax(ajaxPost(url(iea.value), body ? body(iea.value) : action.form))
.mergeMap(payload => {
return Observable.merge(
Observable.of(actions.success(payload)),
/* some other stuff */
);
})
.catch(payload => {
return [actions.failure(payload)];
})
))
))
.takeUntil(action$.filter((a) => (a.type === masterCancelAction))
.repeat();
};
Basically, any time I perform an api request, I dispatch a request action. If I dispatch another request quickly, the previous one is ignored using debounceTime. Additionally, the request can be cancelled using the masterCancelAction and when cancelled repeat() restarts the epic. This epic works as intended in all cases expect one.
The failure case occurs when a user uses the browser back during a request. In this case I fire the masterCancelAction to the request. However, on the same execution context as a result from the masterCancelAction, another request action dispatches to perform a new request on the same epic, but the api request does not occur (the console.log does occur though) as if there was no repeat(). In other cases where cancels occur, the next request is not invoked from the same execution context and it works fine, so it seems in this case my code does not give repeat a chance to restart the epic?
A dirty workaround I found was to use setTimeout(dispatch(action), 0) on the request that dispatches after the cancellation. This seems to allow repeat() to execute. I tried passing different schedulers into repeat, but that didn't seem to help. Also, attaching takeUntil and repeat into my inner switchMap solves the problem, but then other cases where my next request does not execute in the same call stack fail.
Is there a way I can solve this problem without using setTimeout? Maybe it is not a repeat related problem, but it seems to be the case.
Using rxjs 5.0.3 and redux-observable 0.14.1.
The issue is not 100% clear without something like a jsbin to see what you mean, but I do see some general issues that might help:
Anonymous Observable never completes
When creating a custom anonymous Observable it's important to call observer.complete() if you do indeed want it to complete. In most cases, not doing so will cause the subscription to be a memory leak and might also other strange behaviors
Observable.create((observer) => {
observer.next({ type: type, value: action.value, form: action.form });
observer.complete();
})
Observable.of would have been equivalent:
Observable.of({ type: type, value: action.value, form: action.form })
However, it's not clear why this was done as the values it emits are in captured in scope.
debounceTime in this case does not debounce, it delays
Since the anonymous observable it's applied to only ever emits a single item, debounceTime will act just as a regular .delay(250). I'm betting you intended instead to debounce actions.requestType actions, in which case you'd need to apply your debouncing outside the switchMap, after the action$.ofType(actions.requestType).
Observable.of accepts any number of arguments to emit
This is more of a "did you know?" rather than an issue, but I noticed you're merging your of and /* some other actions */ I assume would be other of observables merged in. Instead, you can just return a single of and pass the actions as arguments.
Observable.of(
actions.success(payload),
/* some other actions */
actions.someOtherOne(),
actions.etc()
);
Also, when you find yourself emitting multiple actions synchronously like this, consider whether your reducers should be listening for the same, single action instead of having two or more. Sometimes this wouldn't make sense as you want them to have completely unrelated actions, just something to keep in mind that people often forget--that all reducers receive all actions and so multiple reducers can change their state from the same action.
.takeUntil will stop the epic from listening for future actions
Placing the takeUntil on the top-level observable chain causes the epic to stop listening for action$.ofType(actions.requestType), which is why you added the .repeat() after. This might work in some cases, but it's inefficient and can cause other hard to realize bugs. Epics should be thought of instead as sort of like sidecar processes that usually "start up" with the app and then continue listening for a particular action until the app "shuts down" aka the user leaves the app. They aren't actually processes, it's just helpful to conceptually think of them this way as an abstraction.
So each time it matches its particular action it then most often will switchMap, mergeMap, concatMap, or exhaustMap into some side effect, like an ajax call. That inner observable chain is what you want to make cancellable. So you'd place your .takeUntil on it, at the appropriate place in the chain.
Summary
As mentioned, it's not clear what you intended to do and what the issue is, without a more complete example like a jsbin. But strictly based on the code provided, this is my guesstimate:
const someRequestEpic = action$ => {
return action$.ofType(actions.requestType)
.debounceTime(250)
.do(() => console.log('handled epic ' + actions.requestType))
.switchMap((action) =>
Observable.ajax(ajaxPost(url(action.value), body ? body(action.value) : action.form))
.takeUntil(action$.ofType(masterCancelAction))
.mergeMap(payload => {
return Observable.of(
actions.success(payload),
/* some other actions */
...etc
);
})
.catch(payload => Observable.of(
actions.failure(payload)
))
);
};
Check out the Cancellation page in the redux-observable docs.
If this is a bit confusing, I'd recommend digging a bit deeper into what Observables are and what an "operator" is and does so that it doesn't feel magical and where you should place an operator makes more sense.
Ben's post on Learning Observable by Building Observable is a good start.
I'm working on something that is recording data coming from a queue. It was easy enough to process the queue into an Observable so that I can have multiple endpoints in my code receiving the information in the queue.
Furthermore, I can be sure that the information arrives in order. That bit works nicely as well since the Observables ensure that. But, one tricky bit is that I don't want the Observer to be notified of the next thing until it has completed processing the previous thing. But the processing done by the Observer is asynchronous.
As a more concrete example that is probably simple enough to follow. Imagine my queue contains URLs. I'm exposing those as an Observable in my code. The I subscribe an Observer whose job is to fetch the URLs and write the content to disk (this is a contrived example, so don't take issue with these specifics). The important point is that fetching and saving are async. My problem is that I don't want the observer to be given the "next" URL from the Observable until they have completed the previous processing.
But the call to next on the Observer interface returns void. So there is no way for the Observer to communicate back to me that has actually completed the async task.
Any suggestions? I suspect there is probably some kind of operator that could be coded up that would basically withhold future values (queue them up in memory?) until it somehow knew the Observer was ready for it. But I was hoping something like that already existed following some established pattern.
similar use case i ran into before
window.document.onkeydown=(e)=>{
return false
}
let count=0;
let asyncTask=(name,time)=>{
time=time || 2000
return Rx.Observable.create(function(obs) {
setTimeout(function() {
count++
obs.next('task:'+name+count);
console.log('Task:',count ,' ', time, 'task complete')
obs.complete();
}, time);
});
}
let subject=new Rx.Subject()
let queueExec$=new Rx.Subject()
Rx.Observable.fromEvent(btnA, 'click').subscribe(()=>{
queueExec$.next(asyncTask('A',4000))
})
Rx.Observable.fromEvent(btnB, 'click').subscribe(()=>{
queueExec$.next(asyncTask('B',4000))
})
Rx.Observable.fromEvent(btnC, 'click').subscribe(()=>{
queueExec$.next(asyncTask('C',4000))
})
queueExec$.concatMap(value=>value)
.subscribe(function(data) {
console.log('onNext', data);
},
function(error) {
console.log('onError', error);
},function(){
console.log('completed')
});
What you describe sounds like "backpressure". You can read about it in RxJS 4 documentation https://github.com/Reactive-Extensions/RxJS/blob/master/doc/gettingstarted/backpressure.md. However this is mentioning operators that don't exist in RxJS 5. For example have a look at "Controlled Observables" that should refer to what you need.
I think you could achieve the same with concatMap and an instance of Subject:
const asyncOperationEnd = new Subject();
source.concatMap(val => asyncOperationEnd
.mapTo(void 0)
.startWith(val)
.take(2) // that's `val` and the `void 0` that ends this inner Observable
)
.filter(Boolean) // Always ignore `void 0`
.subscribe(val => {
// do some async operation...
// call `asyncOperationEnd.next()` and let `concatMap` process another value
});
Fro your description it actually seems like the "observer" you're mentioning works like Subject so it would make maybe more sense to make a custom Subject class that you could use in any Observable chain.
Isn't this just concatMap?
// Requests are coming in a stream, with small intervals or without any.
const requests=Rx.Observable.of(2,1,16,8,16)
.concatMap(v=>Rx.Observable.timer(1000).mapTo(v));
// Fetch, it takes some time.
function fetch(query){
return Rx.Observable.timer(100*query)
.mapTo('!'+query).startWith('?'+query);
}
requests.concatMap(q=>fetch(q));
https://rxviz.com/v/Mog1rmGJ
If you want to allow multiple fetches simultaneously, use mergeMap with concurrency parameter.
I'm somewhat familiar with basic RxJS concepts like Observables, Observers and Subjects but RxJS Notifications concept is completely new to me.
What is it for? When should I use it?
The documentation you quoted mentions :
This class is particularly useful for operators that manage notifications, like materialize, dematerialize, observeOn, and others. Besides wrapping the actual delivered value, it also annotates it with metadata of, for instance, what type of push message it is (next, error, or complete).
So the question turns out to be about use cases for materialize and the like.
Basically, you use materialize to get meta-information about the dataflow without incurring into the associated side-effects (an error incurring in a stream for example propagates, a stream which completes can lead to the completion of other streams etc.). dematerialize allows to restore the side-effects.
Here are uses case from former SO questions :
Receiving done notifications from observables built using switch
RxJs - parse file, group lines by topics, but I miss the end
A use case: as errors or completions are propagated immediately, you can't for example delay them. To do so, you can try this approach:
// sample stream
interval(500).pipe(
mapTo('normal value'),
// sometimes value, sometimes throw
map(v => {
if (randomInt() > 50) {
throw new Error('boom!')
} else return v;
}),
materialize(),
// turns Observable<T> into Notification<Observable<T>>
// so we can delay or what you want
delay(500),
// and we need to do some magic and change Notification of error into
// Notification of value (error message)
map(n => n.hasValue? n : new Notification('N', n.error.message, null)),
// back to normal
dematerialize()
)
// now it never throw so in console we will have
// `normal value` or `boom!` but all as... normal values (next() emmision)
// and delay() works as expected
.subscribe(v => console.log(v))