Confusing behavior of rxjs operator `delay` - rxjs

I'm a bit confused about the rxjs operator delay.
When I test it with a fake observable created with from, then I only see an initial delay:
const { from } = Rx;
const { delay, tap } = RxOperators;
from([1, 2, 3, 4]).pipe(
tap(console.log),
delay(1000));
(You can copy & paste this code snippet into rxviz.)
I placed a tap in there to make sure from actually emits the array items as separate values instead of a single array value.
An initial delay is not what I expected, but at least that's what the docs say:
[...] this operator time shifts the source Observable by that amount of time expressed in milliseconds. The relative time intervals between the values are preserved.
However, when I test it with an observable created from an event, then I see a delay before each emitted value:
const { fromEvent } = Rx;
const { delay } = RxOperators;
fromEvent(document, 'click')
.pipe(delay(1000))
What's going on here? Why is delay behaving differently in both cases?

All delay does is what it says: whenever it receives a value, it holds on to that value for the delay period, then emits it. It does the same thing for each value it receives. delay does not change the relative timings between items in the stream.
So, when you do from([1,2,3,4]).pipe(delay(1000)), what happens is:
Time 0: from emits 1
Time 0: delay sees 1 and starts timer1
Time 0: from emits 2
Time 0: delay sees 2 and starts timer2
...
Time 1000: timer1 completes and delay emits 1
Time 1000: timer2 completes and delay emits 2
...
So because all 4 values were emitted in rapid succession, you really only see an initial delay and then all 4 values get emitted downstream. In reality, each value was delayed by 1 second from when it was originally emitted.
If you want to "spread apart" the items so that they are at least 1 second apart, then you could do something like:
const source = from([1, 2, 3, 4])
const spread = source.pipe(concatMap(value => of(value).pipe(delay(1000))));
spread.subscribe(value => console.log(value));
This converts each individual value into an observable that emits the value after a delay, then concatenates these observables. This means the timer for each item will not start ticking until the previous item's timer finishes.

You tap the stream and get the values that are emitted then you pipe them into delay which emits them one second later. Each function in the pipe returns a new observable which emits a value to the next function in the pipe. Tap returns the same observable that has not been delayed yet and delay returns an observable that emits one second later.
const { from } = rxjs;
const { delay, tap } = rxjs.operators;
from([1, 2, 3, 4]).pipe(
tap(val => { console.log(`Tap: ${val}`); }),
delay(1000)).subscribe(val => { console.log(`Sub: ${val}`); });
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.5.2/rxjs.umd.min.js"></script>
If you put the tap after the delay then you see them after the delay.
const { from } = rxjs;
const { delay, tap } = rxjs.operators;
from([1, 2, 3, 4]).pipe(
delay(1000),
tap(val => { console.log(`Tap: ${val}`); })).subscribe(val => { console.log(`Sub: ${val}`); });
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.5.2/rxjs.umd.min.js"></script>

In first code snippet you are emitting an array element by element. First delay, then array elements are handled.
'from' and 'pipe' make 'delay' perform once. Pipe sequences processing, first delay, then tap, tap, tap, tap.
In second code snippet you are emitting objects (they arrive), so delay happens once for each object.
'fromEvent' and 'pipe' make 'delay' per event. Pipe sequences processing of delay before each event.

Related

RxJS withLatestFrom skips initial Subject value

In the following example, for whatever reason, initial value is ignored.
const frameRateSubject: BehaviorSubject<number> = new BehaviorSubject(24);
const loadedMetadata$: Observable<Event> = fromEvent(this.videoElement, 'loadedmetadata');
frameRateSubject.asObservable()
.pipe(
withLatestFrom(loadedMetadata$), // by commenting out this line, both 24 and 20 values are received
tap(([frameRate]: [number, Event]) => {
// initial value of 24 is never received, why is it?
console.log('frameRateSubject', frameRate)
})
)
.subscribe();
setTimeout(() => {
frameRateSubject.next(20)
}, 10000)
Any ideas why?
withLatestFrom combines the source observable (here frameRateSubject$) with other streams (loadedMetadata$) and emits values calculated from the latest values of each, only when the source emits.
But in your case loadedMetadata$ hasn't emitted when frameRateSubject$ emits 24. So the value is skipped.
CombineLatest is most likely the operator you are looking for here.

How to get similar behavior to bufferCount whilst emitting if there are less items than the buffer count

I'm trying to achieve something very similar to a buffer count. As values come through the pipe, bufferCount of course buffers them and sends them down in batches. I'd like something similar to this that will emit all remaining items if there are currently fewer than the buffer size in the stream.
It's a little confusing to word, so I'll provide an example with what I'm trying to achieve.
I have something adding items individually to a subject. Sometimes it'll add 1 item a minute, sometimes it'll add 1000 items in 1 second. I wish to do a long running process (2 seconds~) on batches of these items as to not overload the server.
So for example, consider the timeline where P is processing
---A-----------B----------C---D--EFGHI------------------
|_( P(A) ) |_(P(B)) |_( P(C) ) |_(P([D, E, F, G, H, I]))
This way I can process the events in small or large batches depending on how many events are coming through, but i ensure the batches remain smaller than X.
I basically need to map all the individual emits into emits that contain chunks of 5 or fewer. As I pipe the events into a concatMap, events will start to stack up. I want to pick these stacked up events off in batches. How can I achieve this?
Here's a stackblitz with what I've got so far: https://stackblitz.com/edit/rxjs-iqwcbh?file=index.ts
Note how item 4 and 5 don't process until more come in and fill in the buffer. Ideally after 1,2,3 are processed, it'll pick off 4,5 the queue. Then when 6,7,8 come in, it'll process those.
EDIT: today I learned that bufferTime has a maxBufferSize parameter, that will emit when the buffer reaches that size. Therefore, the original answer below isn't necessary, we can simply do this:
const stream$ = subject$.pipe(
bufferTime(2000, null, 3), // <-- buffer emits # 2000ms OR when 3 items collected
filter(arr => !!arr.length)
);
StackBlitz
ORIGINAL:
It sounds like you want a combination of bufferCount and bufferTime. In other words: "release the buffer when it reaches size X or after Y time has passed".
We can use the race operator, along with those other two to create an observable that emits when the buffer reaches the desired size OR after the duration has passed. We'll also need a little help from take and repeat:
const chunk$ = subject$.pipe(bufferCount(3));
const partial$ = subject$.pipe(
bufferTime(2000),
filter(arr => !!arr.length) // don't emit empty array
);
const stream$ = race([chunk$, partial$]).pipe(
take(1),
repeat()
);
Here we define stream$ to be the first to emit between chunk$ and partial$. However, race will only use the first source that emits, so we use take(1) and repeat to sort of "reset the race".
Then you can do your work with concatMap like this:
stream$.pipe(
concatMap(chunk => this.doWorkWithChunk(chunk))
);
Here's a working StackBlitz demo.
You may want to roll it into a custom operator, so you can simply do something like this:
const stream$ = subject$.pipe(
bufferCountTime(5, 2000)
);
The definition of bufferCountTime() could look like this:
function bufferCountTime<T>(count: number, time: number) {
return (source$: Observable<T>) => {
const chunk$ = source$.pipe(bufferCount(count));
const partial$ = source$.pipe(
bufferTime(time),
filter((arr: T[]) => !!arr.length)
);
return race([chunk$, partial$]).pipe(
take(1),
repeat()
);
}
}
Another StackBlitz sample.
Since I noticed the use of forkJoin in your sample code, I can see you are sending a request to the server for each emission (I was originally under the impression that you were making only 1 call per batch with combined data).
In the case of sending one request per item the solution is much simpler!
There is no need to batch the emissions, you can simply use mergeMap and specify its concurrency parameter. This will limit the number of currently executing requests:
const stream$ = subject$.pipe(
mergeMap(val => doWork(val), 3), // 3 max concurrent requests
);
Here is a visual of what the output would look like when the subject rapidly emits:
Notice the work only starts for the first 3 items initially. Emissions after that are queued up and processed as the prior in flight items complete.
Here's a StackBlitz example of this behavior.
TLDR;
A StackBlitz app with the solution can be found here.
Explanation
Here would be an approach:
const bufferLen = 3;
const count$ = subject.pipe(filter((_, idx) => (idx + 1) % bufferLen === 0));
const timeout$ = subject.pipe(
filter((_, idx) => idx === 0),
switchMapTo(timer(0))
);
subject
.pipe(
buffer(
merge(count$, timeout$).pipe(
take(1),
repeat()
)
),
concatMap(buffer => forkJoin(buffer.map(doWork)))
)
.subscribe(/* console.warn */);
/* Output:
Processing 1
Processing 2
Processing 3
Processed 1
Processed 2
Processed 3
Processing 4
Processing 5
Processed 4
Processed 5
Processing 6 <- after the `setTimeout`'s timer expires
Processing 7
Processing 8
Processed 6
Processed 7
Processed 8
*/
The idea was to still use the bufferCount's behavior when items come in synchronously, but, at the same time, detect when fewer items than the chosen bufferLen are in the buffer. I thought that this detection could be done using a timer(0), because it internally schedules a macrotask, so it is ensured that items emitted synchronously will be considered first.
However, there is no operator that exactly combines the logic delineated above. But it's important to keep in mind that we certainly want a behavior similar to the one the buffer operator provides. As in, we will for sure have something like subject.pipe(buffer(...)).
Let's see how we can achieve something similar to what bufferTime does, but without using bufferTime:
const bufferLen = 3;
const count$ = subject.pipe(filter((_, idx) => (idx + 1) % bufferLen === 0));
Given the above snippet, using buffer(count$) and bufferTime(3), we should get the same behavior.
Let's move now onto the detection part:
const timeout$ = subject.pipe(
filter((_, idx) => idx === 0),
switchMapTo(timer(0))
);
What it essentially does is to start a timer after the subject has emitted its first item. This will make more sense when we have more context:
subject
.pipe(
buffer(
merge(count$, timeout$).pipe(
take(1),
repeat()
)
),
concatMap(buffer => forkJoin(buffer.map(doWork)))
)
.subscribe(/* console.warn */);
By using merge(count$, timeout$), this is what we'd be saying: when the subject emits, start adding items to the buffer and, at the same time, start the timer. The timer is started too because it is used to determine if fewer items will be in the buffer.
Let's walk through the example provided in the StackBlitz app:
from([1, 2, 3, 4, 5])
.pipe(tap(i => subject.next(i)))
.subscribe();
// Then mimic some more items coming through a while later
setTimeout(() => {
subject.next(6);
subject.next(7);
subject.next(8);
}, 10000);
When 1 is emitted, it will be added to the buffer and the timer will start. Then 2 and 3 arrive immediately, so the accumulated values will be emitted.
Because we're also using take(1) and repeat(), the process will restart. Now, when 4 is emitted, it will be added to the buffer and the timer will start again. 5 arrives immediately, but the number of the collected items until now is less than the given buffer length, meaning that until the 3rd value arrives, the timer will have time to finish. When the timer finishes, the [4,5] chunk will be emitted. What happens with [6, 7, 8] is the same as what happened with [1, 2, 3].

Is there an RXJS operator similar to throttleTime, but one that doesn't ignore inermediate events?

I have this sample code:
interval(500).pipe(
throttleTime(1000)
).subscribe(arg => {
console.log(arg);
});
Which emits:
0
3
6
...
I understand that it emits the latest value every 1000 milliseconds. My problem is that it ignores the values that aren't the latest. Is there an operator similar to throttleTime, but one that saves these ignored values? I'd like it to emit:
[0]
[1,2,3]
[4,5,6]
...
Edit: Ideally, I'd like something that listens to clicks of a button. When the first click happens, the code starts a timer for N milliseconds. The user can keep clicking during this time. Once N milliseconds are up, the operator fires with an array holding all of the events that happened during those N milliseconds.
Super-ideally, I'd like the timer to reset everytime the user clicks the button.
You can use bufferToggle. It collects values and returns in an array, just as per your requirement:
const click$ = fromEvent(document, 'click').pipe(
// count emitions
scan(acc => acc += 1, 0)
)
const timerInterval = () => timer(5000);
// buffer emitions to an array. When first click happens 'turn on' buffer
// and when timer ends turn it off.
// throttle used to avoid turning on buffer on every click
click$.pipe(
bufferToggle(
click$.pipe(throttle(timerInterval)),
timerInterval
),
)
.subscribe(console.log)
But to note - there is no clear separation between clicking intervals. For ex., user might be clicking longer than 5 secs and in result, two emitions will happen.
But this is more as an architectural task for you to solve.
DEMO

Guarantee `n` seconds between emit without waiting initially

Given an event stream like (each - is 10ms)
--A-B--C-D
With debounceTime(20) we get
-----------D
With throttleTime(20) we get
--A----C--
With throttleTime(20, undefined, {leading: true, trailing: true} we get
--A----CD
How can I instead guarantee that I have that much time between each emit, so for example with 20ms
--A-----C--D
In general the throttleTime with the trailing: true gets closest, but it can sometimes cause the trailing output to be too close to the leading output.
Sample code can be found on rxviz.com
1. Concat a delay
Concatenate an empty delay to each item, that doesn't emit anything and only completes after a given time.
const { EMTPY, of, concat } = Rx;
const { concatMap, delay } = RxOperators;
event$.pipe(
concatMap(item => concat(of(item), EMPTY.pipe(delay(20))))
);
2. ConcatMap to a timer
Map every item to a timer that starts with the given item and completes after a given amount of time. The next item will be emitted when the timer completes. Values emitted by the timer itself are ignored.
const { timer } = Rx;
const { concatMap, ignoreElements, startWith } = RxOperators;
event$.pipe(
concatMap(item => timer(20).pipe(ignoreElements(), startWith(item)))
);
3. Zip with an interval (not optimal)
If your event stream emits items faster than the desired delay you could use zip to emit events when an interval emits.
const { interval, zip } = Rx;
const { map } = RxOperators;
zip(event$, interval(20)).pipe(map(([item, i]) => item));
This method won't guarantee n seconds between every emitted item in all circumstances, e.g. when there is a gap larger than the desired delay followed by a small gap in the event stream.
E.g zip works in your example with emits at 20, 30, 50, 60 with min delay 20.
zip won't work perfectly with emits at 20, 30, 65, 70 with min delay 20.
When the interval emits faster than events are coming in, those interval items will just pile up inside zip. If this is the case zip will immediately zip any new event with an already present interval item from its stack causing events to be emitted without the intended delay.
Not sure if there's a ready-made operator available to achieve this (there might be!), but you can do it by timestamping each value and adding necessary delay in between:
Timestamp each value
Scan over the sequence and calculate relative delay based on previous value's effective timestamp
delay each value by appropriate amount
concat the resulting sequence
Here's an rxviz illustrating it. Code looks like this:
const minTimeBetween = 800
events.pipe(
timestamp(),
scan((a, x) => ({
...x,
delayBy: a === null
? 0
: Math.max(0, minTimeBetween - (x.timestamp - (a.timestamp + a.delayBy)))
}), null),
concatMap(x => of(x.value).pipe(
delay(x.delayBy)
))
);

What is the best way to implement a poller with timeout as a reactive stream.

What is the best way to model a poller with a timeout, where a certain condition causes an early-exit as 'reactive streams'?
e.g.
If I had an observable which produced a decreasing sequence of positive integers every second
9,8,7,6,5,4,3,2,1,0
What is the best way to write a consumer which takes the latest single event after 5 seconds OR the '0' event if it produced earlier than the timeout.
This is my code as it stands at the moment: (Example in Java)
int intialValue = 10;
AtomicInteger counter = new AtomicInteger(intialValue);
Integer val = Observable.interval(1, TimeUnit.SECONDS)
.map(tick -> counter.decrementAndGet())
.takeUntil(it -> it == 0)
.takeUntil(Observable.timer(5, TimeUnit.SECONDS))
.lastElement()
.blockingGet();
System.out.println(val);
if initialValue = 10, I expect 6 to print. if initialValue = 2, i expect 0 to print before the 5 second timeout expires.
I'm interested if there is a better way to do this.
I don't think there is really a much better way than what you did. You have to have the following:
An interval to emit on (interval)
An aggregator to decrement and store the last value (scan)
A termination condition on the value (takeWhile)
A termination condition on time (takeUntil(timer(...)))
Get the last value on completion (last)
Each one is represented by an operator. You can't do much to get around that. I used a few different operators (scan for aggregation and takeWhile for termination on value) but it is the same number of operators.
const { interval, timer } = rxjs;
const { scan, takeWhile, takeUntil, last, tap } = rxjs.operators;
function poll(start) {
console.log('start', start);
interval(1000).pipe(
scan((x) => x - 1, start),
takeWhile((x) => x >= 0),
takeUntil(timer(5000)),
tap((x) => { console.log('tap', x); }),
last()
).subscribe(
(x) => { console.log('next', x); },
(e) => { console.log('error', e); },
() => { console.log('complete'); }
);
}
poll(10);
setTimeout(() => { poll(2); }, 6000);
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.1.0/rxjs.umd.min.js"></script>
I'm not clear on how you expect it to function on the boundaries. In your example you always decrement before emiting so if your initial value is 10 then you emit 9, 8, 7, 6 (4 values). If you wanted to start with 10 then . you could do scan(..., start + 1) but that would end you at 7 because the timer in the takeUntil(...) aligns with the source interval so that 6 would be excluded. If you want to emit 5 values then you could do takeUntil(timer(5001)). Also, if you don't want to wait a second to emit the first value then you could put startWith(start) right after the scan(...). Or you could do timer(0, 1000) with scan(..., start + 1) instead of the source interval.
Also note that the termination on value (takeWhile) will not terminate till the invalid value is produced (-1). So it will continue for a second after receiving the termination value (0). It seems that most of the termination operators work that way where if they terminate on some value then they wont let the others through.
You could do a take(5) instead of takeUntil(timer(5000)) because you know it fires on a matching interval if that works for your scenario. That would also get around the issue of excluding the last value because of the timers lining up.

Resources