I'm using rxjs with NodeJS in backend.
I have a Rest API which allow consumers to run remote yarn installation process. The install function returns an observable of the process. So when the module is installed successfully it emits a value in the observable and complete. At this point, the Rest API will returns a response to the user to say that the installation is successful. In case that the installation fails, the process will throw an Error in the stream and the Rest API returns another response with the error information.
My issue is:
The API is called multiple times in parallel by consumers, so there will be a parallel installations in the backend.
I tried throttle operator to create a queue but it keeps the first stream active. So if the first process is "completed", it returns "true" but the stream doesn't complete
export class MyService {
// the function called by the REST API
installGlobal(moduleName: string): Observable < boolean > {
// I think, there are something to do here to make it queuing
return this.run('yarn', ['global', 'add', moduleName]);
}
private run(cmd: string, args: string[]): Observable < boolean > {
const cmd$ = fromPromise(spawn(cmd, args)).pipe(
map(stdout => {
this.logger.info(`Install Module Successfully`);
this.logger.info(`stdout: ${stdout.toString()}`);
return true;
}),
catchError(error => {
const errorMessage: string = error.stderr.toString();
return _throw(errorMessage.substr(errorMessage.indexOf(' ') + 1));
})
);
return cmd$;
}
}
My expectation:
Either there are multiple request, they must be queued. So the first one will be treated and all parallel onces must be queued. When the first is processed, it must returns the response to the API consumers (like 200 completed) and resume the next stream from the queue.
[UPDATE-01 July 2019]: adding an example
You can fetch a demo of the code at stackblitz
I have reimplemented the existant code and i'm simulating my API call by subscribing multi time to the service which will call the queue
A simple queque in Rxjs can be done like below
const queque=new Subject()
// sequential processing
queue.pipe(concatMap(item=>yourObservableFunction(item)).subscribe()
// add stuff to the queue
queque.next(item)
Related
I am currently working on a project where I send UDP commands to a Tello drone.
The problem is that it uses UDP and when I send commands too fast before the previous one hasn't finished yet, the second command/action doesn't take place. I am using RxJS for this project and I want to create a mechanism to wait for the response ("ok" or "error") from the drone.
My Idea is to have 2 different observables. 1 observable that is the input stream from the responses from the drone and one observable of observables that I use as a commandQueue. This commandQueue has simple observables on it with 1 command I want to send. And I only want to send the next command when I received the "ok" message from the other observable. When I get the "ok" I would complete the simple command observable and it would automatically receive the next value on the commandQueue, being the next command.
My code works only when I send an array of commands, but I want to call the function multiple times, so sending them 1 by 1.
The following code is the function in question, testsubject is an observable to send the next command to the drone.
async send_command_with_return(msg) {
let parentobject = this;
let zeroTime = timestamp();
const now = () => numeral((timestamp() - zeroTime) / 10e3).format("0.0000");
const asyncTask = data =>
new Observable(obs => {
console.log(`${now()}: starting async task ${data}`);
parentobject.Client.pipe(take(1)).subscribe(
dataa => {
console.log("loool")
obs.next(data);
this.testSubject.next(data);
console.log(`${now()}: end of async task ${data}`);
obs.complete();
},
err => console.error("Observer got an error: " + err),
() => console.log("observer asynctask finished with " + data + "\n")
);
});
let p = this.commandQueue.pipe(concatMap(asyncTask)).toPromise(P); //commandQueue is a subject in the constructor
console.log("start filling queue with " + msg);
zeroTime = timestamp();
this.commandQueue.next(msg);
//["streamon", "streamoff", "height?", "temp?"].forEach(a => this.commandQueue.next(a));
await p;
// this.testSubject.next(msg);
}
streamon() {
this.send_command_with_return("streamon");
}
streamoff() {
this.send_command_with_return("streamoff");
}
get_speed() {
this.send_command_with_return("speed?");
}
get_battery() {
this.send_command_with_return("battery?");
}
}
let tello = new Tello();
tello.init();
tello.streamon();
tello.streamoff();
You can accomplish sending commands one at a time by using a simple subject to push commands through and those emissions through concatMap which will execute them one at a time.
Instead of trying to put all the logic in a single function, it will may be easier to make a simple class, maybe call it TelloService or something:
class TelloService {
private commandQueue$ = new Subject<Command>();
constructor(private telloClient: FakeTelloClient) {
this.commandQueue$
.pipe(
concatMap(command => this.telloClient.sendCommand(command))
)
.subscribe()
}
sendCommand(command: Command) {
this.commandQueue$.next(command);
}
}
When the service is instantiated, it subscribes to the commandQueue$ and for each command that is received, it will "do the work" of making your async call. concatMap is used to process commands one at a time.
Consumers would simply call service.sendCommand() to submit commands to the queue. Notice commands are submitted one at a time, it's not necessary to submit an array of commands.
Here is a working StackBlitz example.
To address your condition of waiting until you receive an ok or error response before continuing, you can use takeWhile(), this means it will not complete the observable until the condition is met.
To introduce a max wait time, you can use takeUntil() with timer() to end the stream if the timer emits:
this.commandQueue$
.pipe(
concatMap(command => this.telloClient.sendCommand(command).pipe(
takeWhile(status => !['ok', 'error'].includes(status), true),
takeUntil(timer(3000))
))
)
.subscribe()
Here's an updated StackBlitz.
I have the following requirement.
I have An Angular service with an BehaviorSubject.
A http request is done and when this is done the BehaviorSubject.next method is invoked with the value.
This value can change during the lifecycle of the single page.
Different subscribers are registered to it and get invoked whenever this changes.
The problem is that while the http request is pending the BehaviorSubject already contains a default value and subscribers are already immediately getting this value.
What I would want is that subscribers have to wait till the http request is done (deferred) and get the value when the http request is done and sets the value.
So what I need is some kind of deferred Behavior subject mechanism.
How would i implement this using rxjs?
Another requirement is that if I subscribe to the behaviorsubject in a method we want the subcriber to get the first non default value and that the subscription ends. We don't want local subscriptions in functions to be re-executed.
Use a filter on your behavior subject so your subscribers won't get the first default emitted value:
mySubject$: BehaviorSubject<any> = new BehaviorSubject<any>(null);
httpResponse$: Observable<any> = this.mySubject$.pipe(
filter(response => response)
map(response => {
const modifyResponse = response;
// modify response
return modifyResponse;
}),
take(1)
);
this.httpResponse$.subscribe(response => console.log(response));
this.myHttpCall().subscribe(response => this.mySubject$.next(response));
You can of course wrap the httpResponse$ observable in a method if you need to.
I think the fact that you want to defer the emitted default value, straight away brings into question why you want to use a BehaviorSubject. Let's remember: the primary reason to use a BehaviorSubject (instead of a Subject, or a plain Observable), is to emit a value immediately to any subscriber.
If you need an Observable type where you need control of the producer (via .next([value])) and/or you want multicasting of subscription out of the box, then Subject is appropriate.
If an additional requirement on top of this is that subscribers need a value immediately then you need to consider BehaviorSubject.
If you didn't say you need to update the value from other non-http events/sources, then I would've suggested using a shareReplay(1) pattern. Nevertheless...
private cookieData$: Subject<RelevantDataType> = new
Subject<RelevantDataType>(null);
// Function for triggering http request to update
// internal Subject.
// Consumers of the Subject can potentially invoke this
// themselves if they receive 'null' or no value on subscribe to subject
public loadCookieData(): Observable<RelevantDataType> {
this.http.get('http://someurl.com/api/endpoint')
.map(mapDataToRelevantDataType());
}
// Function for dealing with updating the service's
// internal cookieData$ Subject from external
// consumer which need to update this value
// via non-http events
public setCookieData(data: any): void {
const newCookieValue = this.mapToRelevantDataType(data); // <-- If necessary
this.cookieData$.next(newCookieValue); // <-- updates val for all subscribers
}
get cookieData(): Observable<RelevantDataType> {
return this.cookieData$.asObservable();
}
The solution is based on OPs comments etc.
- deals with subscribing to subject type.
- deals with external subscribers not being able to 'next' a new value directly
- deals with external producers being able to set a new value on the Subject type
- deals with not giving a default value whilst http request is pending
I'm looking for some guidance on the correct way to setup a WebSocket connection with RxJS 5. I am connecting to a WebSocket that uses JSON-RPC 2.0. I want to be able to execute a function which sends a request to the WS and returns an Observable of the associated response from the server.
I set up my initial WebSocketSubject like so:
const ws = Rx.Observable.webSocket("<URL>")
From this observable, I have been able to send requests using ws.next(myRequest), and I have been able to see responses coming back through the ws` observable.
I have struggled with creating functions that will filter the ws responses to the correct response and then complete. These seem to complete the source subject, stopping all future ws requests.
My intended output is something like:
function makeRequest(msg) {
// 1. send the message
// 2. return an Observable of the response from the message, and complete
}
I tried the following:
function makeRequest(msg) {
const id = msg.id;
ws.next(msg);
return ws
.filter(f => f.id === id)
.take(1);
}
When I do that however, only the first request will work. Subsequent requests won't work, I believe because I am completing with take(1)?
Any thoughts on the appropriate architecture for this type of situation?
There appears to be either a bug or a deliberate design decision to close the WebSocket on unsubscribe if there are no further subscribers. If you are interested here is the relevant source.
Essentially you need to guarantee that there is always a subscriber otherwise the WebSocket will be closed down. You can do this in two ways.
Route A is the more semantic way, essentially you create a published version of the Observable part of the Subject which you have more fine grained control over.
const ws = Rx.Observable.webSocket("<URL>");
const ws$ = ws.publish();
//When ready to start receiving messages
const totem = ws$.connect();
function makeRequest(msg) {
const { id } = msg;
ws.next(msg);
return ws$.first(f => f.id === id)
}
//When finished
totem.unsubscribe();
Route B is to create a token subscription that simply holds the socket, but depending on the actual life cycle of your application you would do well to attach to some sort of closing event just to make sure it always gets closed down. i.e.
const ws = Rx.Observable.webSocket("<URL>");
const totem = ws.subscribe();
//Later when closing:
totem.unsubscribe();
As you can see both approaches are fairly similar, since they both create a subscription. B's primary disadvantage is that you create an empty subscription which will get pumped all the events only to throw them away. They only advantage of B is that you can refer to the Subject for emission and subscription using the same variable whereas A you must be careful that you are using ws$ for subscription.
If you were really so inclined you could refine Route A using the Subject creation function:
const safeWS = Rx.Subject.create(ws, ws$);
The above would allow you to use the same variable, but you would still be responsible for shutting down ws$ and transitively, the WebSocket, when you are done with it.
I'm new to ReactiveX/RxJs and I'm wondering if my use-case is feasible smoothly with RxJs, preferably with a combination of built-in operators. Here's what I want to achieve:
I have an Angular2 application that communicates with a REST API. Different parts of the application need to access the same information at different times. To avoid hammering the servers by firing the same request over and over, I'd like to add client-side caching. The caching should happen in a service layer, where the network calls are actually made. This service layer then just hands out Observables. The caching must be transparent to the rest of the application: it should only be aware of Observables, not the caching.
So initially, a particular piece of information from the REST API should be retrieved only once per, let's say, 60 seconds, even if there's a dozen components requesting this information from the service within those 60 seconds. Each subscriber must be given the (single) last value from the Observable upon subscription.
Currently, I managed to achieve exactly that with an approach like this:
public getInformation(): Observable<Information> {
if (!this.information) {
this.information = this.restService.get('/information/')
.cache(1, 60000);
}
return this.information;
}
In this example, restService.get(...) performs the actual network call and returns an Observable, much like Angular's http Service.
The problem with this approach is refreshing the cache: While it makes sure the network call is executed exactly once, and that the cached value will no longer be pushed to new subscribers after 60 seconds, it doesn't re-execute the initial request after the cache expires. So subscriptions that occur after the 60sec cache will not be given any value from the Observable.
Would it be possible to re-execute the initial request if a new subscription happens after the cache timed out, and to re-cache the new value for 60sec again?
As a bonus: it would be even cooler if existing subscriptions (e.g. those who initiated the first network call) would get the refreshed value whose fetching had been initiated by the newer subscription, so that once the information is refreshed, it is immediately passed through the whole Observable-aware application.
I figured out a solution to achieve exactly what I was looking for. It might go against ReactiveX nomenclature and best practices, but technically, it does exactly what I want it to. That being said, if someone still finds a way to achieve the same with just built-in operators, I'll be happy to accept a better answer.
So basically since I need a way to re-trigger the network call upon subscription (no polling, no timer), I looked at how the ReplaySubject is implemented and even used it as my base class. I then created a callback-based class RefreshingReplaySubject (naming improvements welcome!). Here it is:
export class RefreshingReplaySubject<T> extends ReplaySubject<T> {
private providerCallback: () => Observable<T>;
private lastProviderTrigger: number;
private windowTime;
constructor(providerCallback: () => Observable<T>, windowTime?: number) {
// Cache exactly 1 item forever in the ReplaySubject
super(1);
this.windowTime = windowTime || 60000;
this.lastProviderTrigger = 0;
this.providerCallback = providerCallback;
}
protected _subscribe(subscriber: Subscriber<T>): Subscription {
// Hook into the subscribe method to trigger refreshing
this._triggerProviderIfRequired();
return super._subscribe(subscriber);
}
protected _triggerProviderIfRequired() {
let now = this._getNow();
if ((now - this.lastProviderTrigger) > this.windowTime) {
// Data considered stale, provider triggering required...
this.lastProviderTrigger = now;
this.providerCallback().first().subscribe((t: T) => this.next(t));
}
}
}
And here is the resulting usage:
public getInformation(): Observable<Information> {
if (!this.information) {
this.information = new RefreshingReplaySubject(
() => this.restService.get('/information/'),
60000
);
}
return this.information;
}
To implement this, you will need to create your own observable with custom logic on subscribtion:
function createTimedCache(doRequest, expireTime) {
let lastCallTime = 0;
let lastResult = null;
const result$ = new Rx.Subject();
return Rx.Observable.create(observer => {
const time = Date.now();
if (time - lastCallTime < expireTime) {
return (lastResult
// when result already received
? result$.startWith(lastResult)
// still waiting for result
: result$
).subscribe(observer);
}
const disposable = result$.subscribe(observer);
lastCallTime = time;
lastResult = null;
doRequest()
.do(result => {
lastResult = result;
})
.subscribe(v => result$.next(v), e => result$.error(e));
return disposable;
});
}
and resulting usage would be following:
this.information = createTimedCache(
() => this.restService.get('/information/'),
60000
);
usage example: https://jsbin.com/hutikesoqa/edit?js,console
I am trying to get time expiry cache to work for an observable that abstracts a "request-response", using postMessage and message events on the window.
The remote window expects a message getItemList and replies to it with a message of type {type: 'itemList', data: []}.
I would like to model the itemList$ observable in such a way that it caches the last result for 3 seconds, so that no new requests are made during that time, however, I cannot think of a way to achieve that in an elegant (read, one observable – no subjects) and succint manner.
Here is the example in code:
const remote = someIframe.contentWindow;
const getPayload = message => message.data;
const ofType = type => message => message.type === type;
// all messages coming in from the remote iframe
const messages$ = Observable.fromEvent(window, 'message')
.map(getPayload)
.map(JSON.parse);
// the observable of (cached) items
const itemList$ = Observable.defer(() => {
console.log('sending request');
// sending a request here, should happen once every 3 seconds at most
remote.postMessage('getItemList');
// listening to remote messages with the type `itemList`
return messages$
.filter(ofType('itemList'))
.map(getPayload);
})
.cache(1, 3000);
/**
* Always returns a promise of the list of items
* #returns {Promise<T>}
*/
function getItemList() {
return itemList$
.first()
.toPromise();
}
// poll every second
setInterval(() => {
getItemList()
.then(response => console.log('got response', response));
}, 1000);
I am aware of the (very similar) question, but I am wondering if anyone can come up with a solution without explicit subjects.
Thank you in advance!
I believe you are looking for the rxjs operator throttle:
Documentation on rxjs github repo
Returns an Observable that emits only the first item emitted by the
source Observable during sequential time windows of a specified
duration.
Basically, if you would like to wait until the inputs have quieted for a certain period of time before taking action, you want to debounce.
If you do not want to wait at all, but do not wish to make more than 1 query within a specific amount of time, you will want to throttle. From your use case, I think you want to throttle