I'm trying to to group the values from an observable into an array of n size, to be able to batch send these to a service to improve the overall performance.
The thing is that I want to make sure that when the items left are less then n, they will be still be passed down the chain after a certain timeout.
I'm trying to rewrite the C# solution from
https://stackoverflow.com/a/22873833/2157455
in Javascript.
The main problem is that in Rx.Js lots of methods have been deprecated and it's hard to find the new functions.
var people = new List<(string name, int age)>
{
("Sue", 25 ),
("Joe", 30 ),
("Frank", 25 ),
("Sarah", 35 ),
("John", 37)
}.ToObservable();
var buffers = people
.GroupByUntil(
// yes. yes. all items belong to the same group.
x => true,
g => Observable.Amb(
// close the group after 5 seconds of inactivity
g.Throttle(TimeSpan.FromSeconds(5)),
// close the group after 10 items
g.Skip(1)
))
// Turn those groups into buffers
.SelectMany(x => x.ToArray());
I could get this far, but I can't find the replacement for groupByUntil. And I'm not sure what's the selectMany operator in Rx.Js, probably toArray().
Most examples I find are using deprecated or non-exising functions.
I'm using rxjs 7.8.0
The syntax does not help as well, using the pipe all the time makes the code difficult to read in my opinion.
const people = [
{ name: 'Sue', age: 25 },
{ name: 'Joe', age: 30 },
{ name: 'Frank', age: 25 },
{ name: 'Sarah', age: 35 },
{ name: 'John', age: 37 }
];
const source = from(people);
const example = source.pipe(
groupBy(person => true),
mergeMap(group => group.pipe(
raceWith(
group.pipe(throttle(() => interval(1000))),
group.pipe(skip(2))
),
toArray()
)));
example.forEach(x => console.log(x.length));
I'm getting all 5, instead of two arrays, one with 3 the other with 2.
Perhaps there is a better way to write it in js, but I can;t see the replacement for groupByUntil.
Thanks.
bufferTime is probably what you are looking for
One of its signature is :
bufferTime(bufferTimeSpan: number, bufferCreationInterval: number, maxBufferSize: number, scheduler?: SchedulerLike): OperatorFunction<T, T[]>
so with bufferTime(1000, null, 2) you get a buffered of length=2 or every 1s.
Related
I have a costly server ajax request which has one input (full: boolean). If full is false, the server can return either a partial or a full response (response.isFull == true); but if full is true, the server will return a full response. Normally the partial response is good enough, but there are certain conditions that will require a full response. I need to avoid requesting a full response explicitly as much as possible, so I thought I'd start with a BehaviorSubject which I can eventually feed with true and combine it with distinctUntilChanged if I ever need to get the full response. This will give me an observable with false initially and that can give me true if I feed that into it:
const fullSubject = new BehaviorSubject<boolean>(false);
Then I've got a function that takes a boolean parameter and returns an observable with the server request (retried, transformed, etc.). As said, the answer can be partial or full, but it can be full even if the input parameter was false at the server's discretion. For example:
interface IdentityData {
...
isFull: boolean;
}
private getSimpleIdentity(full: boolean): Observable<IdentityData> {
return Axios.get(`/api/identity${full?"?full=true":""}`)
.pipe( ... retry logic ...,
... transformation logic ...,
shareReplay(1) );
}
I need to know how can I combine these so that the following is true:
The server needs to be queried at most twice.
If the first answer is a full answer, no further queries must be performed to the server.
If the first answer is a partial answer, and true is fed into fullSubject, a full answer must be requested.
The expected output from all this is an observable that emits either one full response, or a partial response and, when asked, a full response.
Environment: Vue 2.6.11, RxJS 6.5.5, Axios 0.19.2, TypeScript 3.7.5.
Thanks in advance
Here would be my approach:
const fullSubject = new BehaviorSubject(false);
const src$ = fullSubject.pipe(
switchMap(isFull => Axios.get('...')),
take(2), // Server required at most twice
takeWhile(response => !response.isFull, true), // When `isFull`, it will complete & unsubscribe -> no more requests to the server
shareReplay(1),
);
src$.subscribe(() => { /* ... */ });
function getFullAnswer () {
fullSubject.next(true);
}
takeWhile takes a second argument, inclusive. When set to true, when the predicate function evaluates to false(e.g isFull is true) it will send that value as well. –
if I've got it correctly
private getSimpleIdentity(): Observable<IdentityData> {
return fullSubject.pipe(
switchMap(full => Axios.get(`/api/identity${full ? "?full=true" : ""}`)),
shareReplay(1),
);
}
Uses the retryWhen() operator
const source = of("").pipe(map(() => Math.floor(Math.random() * 10 + 1)));
const example = source
.pipe(
tap((val) => console.log("tap", val)),
map((val) => {
//error will be picked up by retryWhen
if (val !== 5) throw val;
return val;
}),
retryWhen((errors) =>
errors.pipe(
tap(() => console.log("--Wait 1 seconds then repeat")),
delay(1000)
)
)
)
.subscribe((val) => console.log("subscription", val));
/*
output:
tap 3
--Wait 1 seconds then repeat
tap 8
--Wait 1 seconds then repeat
tap 1
--Wait 1 seconds then repeat
tap 4
--Wait 1 seconds then repeat
tap 7
--Wait 1 seconds then repeat
tap 5
subscription 5
*/
If I have an array of events that include a utc timestamp and event data like as follows:
[{utcts: , data: , ... ];
how would you use RxJS to "replay" those events with the correct time differentials between each item in the array? Assume the array is ordered by the utcts field so the first item has the lowest value.
here is a very basic set of data to get started:
var testdata = [
{utcts: 1, data: 'a'},
{utcts: 4, data: 'b'},
{utcts: 6, data: 'c'},
{utcts: 10, data: 'd'}
];
Assume the utcts is just the number of seconds from the start of replaying the event which starts at 0 seconds.
Use delayWhen to give you timed replay.
Since utcts given is relative (not absolute) time, don't need to refresh the timestamp inside the data object.
I have added a timestamp to the console log so we can see the elapsed output time.
Note the extra few milliseconds is typical of rxjs process time.
console.clear()
const testdata = [
{utcts: 1, data: 'a'},
{utcts: 4, data: 'b'},
{utcts: 6, data: 'c'},
{utcts: 10, data: 'd'}
];
const replayData = (data) => Rx.Observable.from(data)
.delayWhen(event => Rx.Observable.of(event).delay(event.utcts * 1000))
// Show replay items with output time (in milliseconds)
const start = new Date()
replayData(testdata)
.timestamp()
.subscribe(x => console.log(x.value, 'at', x.timestamp - start, 'ms'))
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.6/Rx.js"></script>
Ref delayWhen, timestamp
This also works, arguably simpler, not sure which is best.
mergeMap() flattens the inner observable, which is necessary to apply the delay.
const replayData = (data) => Rx.Observable.from(data)
.mergeMap(event => Rx.Observable.of(event).delay(event.utcts * 1000))
Rough pseudo (comes out of my head directly without running to verify) might be something similar to
Observable.scan((acc, value) => ({
delay: delay === NaN ? value.utcts - delay,
value
}), { delay: NaN, value: null })
.mergeMap(({delay, value}) => Observable.from(value).delay(delay))
scan operator is similar to reduce, but emits intermediate values. Using that compute diff between times to get delay those values, then emit values per given delayed time. There are couple of other approaches could work in same way.
This should work in https://rxviz.com (copy-paste there):
const { delay, mergeMap } = RxOperators;
const { from, Observable, of } = Rx;
const testdata = [
{utcts: 0.2, data: 'a'},
{utcts: 2.0, data: 'b'},
{utcts: 2.8, data: 'c'},
{utcts: 4.0, data: 'd'}
];
from(testdata).pipe(
mergeMap(event => of(event).pipe(
delay(event.utcts * 1000)
))
)
I am learning Rxjs and wanted to try out a few examples on my own
but I can't seem to get my head around to think reactively.
I am trying to calculate the time a user's mouse pointer spends inside and outside a div.
see fiddle - https://jsfiddle.net/ishansoni22/44af3n3k/
<div class = "space">
<div>
let $space = $(".space")
let in$ = Rx.Observable.fromEvent($space, "mouseenter")
.map((event) => "in")
let out$ = Rx.Observable.fromEvent($space, "mouseleave")
.map((event) => "out")
let inOut$ = Rx.Observable.merge(in$, out$)
let time$ = Rx.Observable.interval(1000)
.buffer(inOut$)
.map((list) => list.length)
time$.subscribe((value) => console.log(value));
I am able to calculate the time but how do I relate it to the respective in/ out streams? I want the output to look something like :
inside, in - 20, out - 30
outside, in - 20, out - 35
inside, in - 100, out - 35
Also, can someone point me to some examples I could do so that I can start thinking in the reactive paradigm?
There are some examples in the official documentation (http://reactivex.io/rxjs) but they are a little bit scarce indeed.
I think I would some your sample something like this:
let $space = $(".space")
let in$ = Rx.Observable.fromEvent($space, "mouseenter")
let out$ = Rx.Observable.fromEvent($space, "mouseleave")
let durations$ = in$
.map(_ => Date.now())
.switchMap(inTime => out$
.take(1)
.map(_ => Date.now())
.map(outTime => outTime - inTime)
)
durations$
.scan((sum, next) => sum + next, 0)
.subscribe(total => console.log(total))
This would start listening to in$, then upon a mouseenter-event it starts to listen to mouseleaves, takes 1 of those events and calculate the duration.
I have written multiple maps below each other for clarity, but of course you can compose that into a single function.
One of the things I found most challenging when starting out with Rx was using streams of streams, and becoming comfortable with flatMap and switchMap. The problem you describe is most easily solved using exactly this approach. With your streams defined as follows (I prefer const over let to make it clear no mutation is occuring):
const in$ = Rx.Observable.fromEvent($space, 'mouseenter');
const out$ = Rx.Observable.fromEvent($space, 'mouseleave');
you can describe entering and then leaving as follows:
const inThenOut$ = in$.switchMap(() => out$);
To understand exactly what this is doing I urge you to learn about flatMap, become comfortable with streams of streams, and then learn how switchMap works by only maintaining a subscription to the most recent inner stream. For this I found the official rxjs documentation the best source. The included marble diagrams often tell complex stories with just a few dots and lines.
From here it's a relatively small step to get the time spent inside. First, we map our original streams into timestamp values:
const timestamp = () => + new Date();
const in$ = Rx.Observable.fromEvent($space, 'mouseenter').map(() => timestamp());
const out$ = Rx.Observable.fromEvent($space, 'mouseleave').map(() => timestamp());
(note: there is a timestamp method in rxjs you could use instead of doing this manually, but I feel this better illustrates how you can map your stream elements into anything you please).
From there, we can adjust our switchMap usage to access both the in and out values, and return the difference between them:
const inThenOut$ = in$.switchMap(() => out$, (x, y) => y - x);
Here's the whole thing working:
https://jsbin.com/qoruyoluho/edit?js,console,output
You could use RXJS - Timestamp operator to attach timestamp to each item emitted by an Observable indicating when it was emitted.
const { fromEvent } = Rx;
const { map, switchMap, timestamp, take, tap } = RxOperators;
const in$ = fromEvent($space, 'mouseenter').pipe(
timestamp(),
tap(x => console.log(`In: ${x.timestamp}`))
)
const out$ = fromEvent($space, 'mouseleave').pipe(
timestamp(),
tap(x => console.log(`Out: ${x.timestamp}`))
)
const duration$ = in$.pipe(
switchMap(start => out$.pipe(
take(1),
map(finish => finish.timestamp - start.timestamp),
tap(value => console.log(`Duration ms: ${value}`))
)
)
)
/* output example
In: 1552295324302
Out: 1552295325158
Duration ms: 856
*/
Try it here: https://rxviz.com/v/rOW5g9x8
I have a stream of data from car auctions. Each car auction has n-number of lanes. I want to log the auction of each vehicle.
The stream looks something like this...
--{lane: 1, action: bid} --- { lane: 2, action: start} --- { lane:1, action: bid} --- {lane: 2, action: bid} --- {lane:1, action: sold} ---
I have the following to buffer each auction lane and close the buffer on sale...
const bufferOpen$= auctionWebSocketStream$
.filter(stream => stream.tag === 'CURITEM');
const bufferClose$ = () => auctionWebSocketStream$.filter(stream => stream.tag === 'SOLD');
auctionWebSocketStream$
.bufferToggle(bufferOpen$, bufferClose$)
.subscribe(x => console.log(x));
The above works fine so long as there is one auction and one lane. With multiple lanes, there's bid/sale information about multiple lanes.
How would I aggregate the stream by lanes into the buffer? Similar solutions always had known aggregation parameters. But I need to split the stream anytime there's a new lane.
Help is greatly appreciated.
UPDATE
I made a JSBin to show off my frustration and cluelessness. It gives an example input stream and explains the desired output.
http://jsbin.com/tuxitev/edit?js,console
(For bonus points, it only shows empty arrays under Babel. Not sure why Typescript is required)
If someone knows where to get RxJS questions answered, please let me know. I'll accept the answer. This is the third unanswered RxJS question I've had.
For anyone curious of the answer, here it is.
stream$
.groupBy(stream => stream.lane)
.mergeMap(stream =>
stream.scan((acc, cur) => {
if (cur.action === 'start') {
acc = [];
}
acc.push(cur)
return acc;
}, [])
.filter(stream => stream[stream.length-1].action === 'sold')
)
.subscribe(
x => console.log(x),
(e) => console.error(e),
() => console.log('complete')
)
http://jsbin.com/tuxitev/edit?js,console
I have some pre-defined events set to occur at specific times.
And I have a timer, like this:
const timer = Rx.Observable.interval(100).timeInterval()
.map(x => x.interval)
.scan((ms, total) => total + ms, 0)
The timer emits something close to 100,200,300,400,500 (although in reality it's more like 101,200,302,401,500...which is totally fine)
I also have some stuff I want to do at certain times. For example, let's say I want to do stuff at the following times:
const stuff = Rx.Observable.from([1000, 2000, 2250, 3000, 5000]);
What I'd like is to combine "stuff" and "timer" in such a way that the resulting stream emits a value once per time defined in "stuff" at that time (or ever so slightly later). in this case, that would be t=1000 ms, 2000 ms, 2250 ms, 3000 ms and 5000 ms. Note: the 2250 guy should emit around time 2300 because of the interval size. that's fine. they just can't come early or more than once.
I have one solution, but it's not very good. it re-starts "stuff" every single step (every single 100 ms in this case) and filters it and takes 1. I would prefer that, once an event is emitted from "stuff", that it be gone, so subsequent filters on it don't have those values.
In the real application, there will be stuff and stuff2 and maybe stuff3...(but I will call them something else!)
Thanks in advance! I hope that was clear.
If I've understood what you're after correctly, this should be achievable with a simple projection:
const times$ = stuff.flatMap(x => Rx.Observable.timer(x));
Here's a working sample: https://jsbin.com/negiyizibu/edit?html,js,console,output
Edit
For the second requirement, try something like this:
const times$ = Rx.Observable
.from([{"val":"jeff", "t": 1000}, {"val":"fred", "t": 2500}])
.flatMap(x => Rx.Observable.timer(x.t).map(y => x.val));
https://jsbin.com/cegijudoci/edit?js,console,output
Here's a typescript function I wrote based on Matt's solution.
import {from, timer} from 'rxjs';
import {flatMap, map} from 'rxjs/operators';
export interface ActionQueueEntry {
action: string;
payload?: any;
delay: number;
}
export function actionQueue(entries: ActionQueueEntry[]) {
return from(entries).pipe(flatMap((x: any) => {
return timer(x.delay).pipe(map(y => x));
}));
}
const q = actionQueue([
{action: 'say: hi', delay: 500},
{action: 'ask: how you are', delay: 2500},
{action: 'say: im fine', delay: 5000},
]);
q.subscribe(console.log);