I'm looking for a way to make streams that are combined
Note: this is the simplest form of my problem, in reality I'm combining 8 different streams some are intertwined, some are async etc :(
import { BehaviorSubject, map, combineLatest } from 'rxjs';
const $A = new BehaviorSubject(1)
const $B = $A.pipe(map(val => `$B : ${val}`))
const $C = $A.pipe(map(val => `$C : ${val}`))
// prints out:
// (1) [1, "$B : 1", "$C : 1"]
combineLatest([$A,$B,$C]).subscribe(console.log)
$A.next(2)
// prints out:
// (2) [2, "$B : 1", "$C : 1"]
// (3) [2, "$B : 2", "$C : 1"]
// (4) [2, "$B : 2", "$C : 2"]
Code example
The print out (1) is great, all streams have a value of "1": [1, "$B : 1", "$C : 1"]
The print out (4) is great, all streams have a value of "2": [2, "$B : 2", "$C : 2"]
But the combine latest fires for (2) and (3) after each stream is updated individually meaning that you have a mixture of "1" and "2"
**What way can I modify the code to only get notified when a change has fully propgaged? **
My best solutions so far:
A) using debouceTime(100)
combineLatest([$A,$B,$C]).pipe(debounceTime(100)).subscribe(console.log)
But it's flaky because it can either swallow valid states if the are process to quickly or notify with invalid states if individual pipes are too slow
B) filter only valid state
combineLatest([$A,$B,$C]).pipe(
filter(([a,b,c])=>{
return b.indexOf(a) > -1 && c.indexOf(a) > -1
})
).subscribe(console.log)
works but adding a validation function seems like the wrong way to do it (and more work :))
C) Make B$ and C$ in which we push the latest and reset at every change"
A$.pipe(tap(val)=>{
B$.next(undefined);
B$.next(val);
C$.next(undefined)
C$.next(val);
})
...
combineLatest([$A,$B.pipe(filter(b => !!b)),$C.pipe(filter(c => !!c))]).pipe(
filter(([a,b,c])=>{
return b.indexOf(a) > -1 && c.indexOf(a) > -1
})
Works but quite a lot of extra code and vars
I have the feeling I'm missing a concept or not seeing how to achieve this in a clean/robust way, but I sure I'm not the first one :)
Thanks
As you've observed, the observable created by combineLatest will emit when any of its sources emit.
Your problem is occurring because you pass multiple observables into combineLatest that share a common source. So whenever that common source emits, it causes each derived observable to emit.
One way to "fix" this in a synchronous scenario is to simply apply debounceTime(0) which will mask the duplicate emission that happens in the same event loop. This approach is a bit naive, but works in simple scenarios:
combineLatest([$A,$B,$C]).pipe(
debounceTime(0)
)
But, since you have some async things going on, I think your solution is to not include duplicate sources inside combineLatest and handle the logic further down the chain:
combineLatest([$A]).pipe(
map(([val]) => [
val,
`$B : ${val}`,
`$C : ${val}`,
])
)
The code above produces the desired output. Obviously, you wouldn't need combineLatest with a single source, but the idea is the same if you had multiple sources.
Let's use a more concrete example that has the same issue:
const userId$ = new ReplaySubject<string>(1);
const maxMsgCount$ = new BehaviorSubject(2);
const details$ = userId$.pipe(switchMap(id => getDetails(id)));
const messages$ = combineLatest([userId$, maxMsgCount$]).pipe(
switchMap(([id, max]) => getMessages(id, max))
);
const user$ = combineLatest([userId$, details$, messages$]).pipe(
map(([id, details, messages]) => ({
id,
age: details.age,
name: details.name,
messages
}))
);
Notice when userId emits a new value, the user$ observable would end up emitting values that had the new userId, but the details from the old user!
We can prevent this by only including unique sources in our combineLatest:
const userId$ = new ReplaySubject<string>(1);
const maxMsgCount$ = new BehaviorSubject(2);
const user$ = combineLatest([userId$, maxMsgCount$]).pipe(
switchMap(([id, max]) => combineLatest([getDetails(id), getMessages(id, max)]).pipe(
map(([details, messages]) => ({
id,
age: details.age,
name: details.name,
messages
}))
))
);
You can see this behavior in action in the below stackblitz samples:
Problem
Solution
Related
I have a costly server ajax request which has one input (full: boolean). If full is false, the server can return either a partial or a full response (response.isFull == true); but if full is true, the server will return a full response. Normally the partial response is good enough, but there are certain conditions that will require a full response. I need to avoid requesting a full response explicitly as much as possible, so I thought I'd start with a BehaviorSubject which I can eventually feed with true and combine it with distinctUntilChanged if I ever need to get the full response. This will give me an observable with false initially and that can give me true if I feed that into it:
const fullSubject = new BehaviorSubject<boolean>(false);
Then I've got a function that takes a boolean parameter and returns an observable with the server request (retried, transformed, etc.). As said, the answer can be partial or full, but it can be full even if the input parameter was false at the server's discretion. For example:
interface IdentityData {
...
isFull: boolean;
}
private getSimpleIdentity(full: boolean): Observable<IdentityData> {
return Axios.get(`/api/identity${full?"?full=true":""}`)
.pipe( ... retry logic ...,
... transformation logic ...,
shareReplay(1) );
}
I need to know how can I combine these so that the following is true:
The server needs to be queried at most twice.
If the first answer is a full answer, no further queries must be performed to the server.
If the first answer is a partial answer, and true is fed into fullSubject, a full answer must be requested.
The expected output from all this is an observable that emits either one full response, or a partial response and, when asked, a full response.
Environment: Vue 2.6.11, RxJS 6.5.5, Axios 0.19.2, TypeScript 3.7.5.
Thanks in advance
Here would be my approach:
const fullSubject = new BehaviorSubject(false);
const src$ = fullSubject.pipe(
switchMap(isFull => Axios.get('...')),
take(2), // Server required at most twice
takeWhile(response => !response.isFull, true), // When `isFull`, it will complete & unsubscribe -> no more requests to the server
shareReplay(1),
);
src$.subscribe(() => { /* ... */ });
function getFullAnswer () {
fullSubject.next(true);
}
takeWhile takes a second argument, inclusive. When set to true, when the predicate function evaluates to false(e.g isFull is true) it will send that value as well. –
if I've got it correctly
private getSimpleIdentity(): Observable<IdentityData> {
return fullSubject.pipe(
switchMap(full => Axios.get(`/api/identity${full ? "?full=true" : ""}`)),
shareReplay(1),
);
}
Uses the retryWhen() operator
const source = of("").pipe(map(() => Math.floor(Math.random() * 10 + 1)));
const example = source
.pipe(
tap((val) => console.log("tap", val)),
map((val) => {
//error will be picked up by retryWhen
if (val !== 5) throw val;
return val;
}),
retryWhen((errors) =>
errors.pipe(
tap(() => console.log("--Wait 1 seconds then repeat")),
delay(1000)
)
)
)
.subscribe((val) => console.log("subscription", val));
/*
output:
tap 3
--Wait 1 seconds then repeat
tap 8
--Wait 1 seconds then repeat
tap 1
--Wait 1 seconds then repeat
tap 4
--Wait 1 seconds then repeat
tap 7
--Wait 1 seconds then repeat
tap 5
subscription 5
*/
Hope someone can help me with this problem.
I have 2 streams that I need to use the operator combineLatest on. After a while I need to add dynamically streams that also need to use combineLatest on.
Here is what I need to do:
stream a ---------a---------------b-------------c------------------------>
stream b ----1--------2-----3-------------------------4------------------>
stream c (not defined at start) -----z-----------------x------------>
stream d (not defined at start) ----------k------>
(combineLatest)
result ---------(a1)(a2)--(a3)--(b3)----(b3z)-(c3z)-(c4z)-(c4x)-(c4xk)->
UPDATE
To be more specific I want to turn this STREAM (link)
To this result:
A----B---B0-C0--D0--D1--E1--E1a--F1a--F2a---G2a---G3a--H3a-H3b--I3b
The idea it that everything is a stream. Even stream of streams :)
const onNew$ = new Rx.Subject();
const a$ = Rx.Observable.interval(1000).mapTo('a');
const b$ = Rx.Observable.interval(1000).mapTo('b');
const comb$ = Rx.Observable
.merge(
onNew$,
Rx.Observable.from([a$, b$]),
)
.scan((acc, v) => {
acc.push(v);
return acc;
}, [])
.switchMap(vs => Rx.Observable.combineLatest(vs))
comb$.take(4).subscribe(v => console.log(v));
setTimeout(
() => onNew$.next(Rx.Observable.interval(1000).mapTo('c')),
2000,
);
setTimeout(
() => onNew$.next(Rx.Observable.interval(1000).mapTo('d')),
4000,
);
<script src="https://unpkg.com/rxjs/bundles/Rx.min.js"></script>
Taking Oles' answer, simplifying a little and adding test data as given in question update
const Subject = Rx.ReplaySubject
const ReplaySubject = Rx.ReplaySubject
const newStream = new Subject()
// Set up output, no streams yet
const streamOfStreams = newStream
.scan( (acc, stream) => {
acc.push(stream);
return acc;
}, [])
.switchMap(vs => Observable.combineLatest(vs))
.map(arrayOfValues => arrayOfValues.join('')) // declutter
.subscribe(console.log)
// Add a stream
const s1 = new ReplaySubject()
newStream.next(s1)
// emit on streams
s1.next('A'); s1.next('B')
// Add a stream
const s2 = new ReplaySubject()
newStream.next(s2)
// emit on streams
s2.next('0'); s1.next('C')
s1.next('D'); s2.next('1'); s1.next('E');
// Add a stream
const s3 = new ReplaySubject()
newStream.next(s3)
// emit on streams
s3.next('a');
s1.next('F'); s2.next('2'); s1.next('G'); s2.next('3'); s1.next('H');
s3.next('b'); s1.next('I')
Working example: CodePen
Update
Christian has kindly supplied some test streams which are more 'real world' than the sequenced Subjects I've used above. Unfortunately, these highlight a bug in the solution as it stands.
For reference, the new test streams are
const streamA = Rx.Observable.timer(0,800).map(x => String.fromCharCode(x+ 65));
const streamB = Rx.Observable.timer(0,1300).map(x => x);
const streamC = Rx.Observable.timer(1100, 2000).map(x => String.fromCharCode(x+ 97));
setTimeout(() => newStream.next(streamA), 500);
setTimeout(() => newStream.next(streamB), 2000);
setTimeout(() => newStream.next(streamC), 3000);
Problem #1
The first problem stems from the core line in streamOfStreams,
.switchMap(vs => Observable.combineLatest(vs))
This essentially says, every time a new array of streams appears, map it to a combineLatest() of the new array and switch to the new observable. However, the test observables are cold, which means each re-subscription gets the full stream.
Ref: Introduction to Rx - Hot and Cold observables
Some observable sequences can appear to be hot when they are in fact
cold. A couple of examples that surprise many is Observable.Interval
and Observable.Timer
So we get
- expected A--B--B0...
- actual A--B--A0--B0...
The obvious solution is to turn the cold observables hot,
const asHot = (stream) => {
const hot = stream.multicast(() => new Rx.Subject())
hot.connect()
return hot
}
but this omits B0 from the sequence, A--B--C0..., so we want hot + 1 previous which can be had with a buffer size one
const asBuffered = (stream) => {
const bufferOne = new ReplaySubject(1)
stream.subscribe(value => bufferOne.next(value))
return bufferOne
}
Problem #2
The second problem comes from the fact that streamC delays it's first emit by 1100ms (good test Christian!).
This results is
- expected A--B--B0--C0--D0--D1--E1--E1a...
- actual A--B--B0--C0--D0--E1a...
which means we need to delay adding a stream until it's first emit
const addStreamOnFirstEmit = (stream) => {
const buffered = asBuffered(stream)
buffered.first().subscribe( _ => {
newStream.next(buffered)
})
}
Working example: CodePen
Notes on the CodePen
I've left in the various streamAdder functions for experimentation, and there are also _debug versions that emit the streams and the addStream events to show the sequence.
Also limited the source streams so that the console doesn't scroll too much.
Note on the expected output
The new solution diverges from the expected output given in the question after 'G3a'
expected A----B---B0-C0--D0--D1--E1--F1---F2---F2a---G2a---G3a--H3a--H3b--I3b
actual A----B---B0-C0--D0--D1--E1--E1a--F1a--F2a---G2a---G3a--G3b--H3b--I3b
which is due to the simultaneous emission of of 'H' and 'b'. Problem #3?
One more test
In order to see if the solution failed if streamC delayed first emission until after two emits of streamA & streamB, I changed the delay to 1800ms
const streamC = Rx.Observable.timer(1800, 2000).map(x => String.fromCharCode(x+ 97));
I believe the output for this test is correct.
Can be done if you can unsubscribe and re-subscribe for each new stream
// Start with two streams
const s1 = new ReplaySubject(1)
const s2 = new ReplaySubject(1)
let out = Observable.combineLatest(s1, s2)
let subscription = out.subscribe(console.log)
s2.next('1'); s1.next('a'); s2.next('2'); s2.next('3'); s1.next('b')
// Add a new stream
subscription.unsubscribe()
const s3 = new ReplaySubject(1)
out = Observable.combineLatest(s1, s2, s3)
subscription = out.subscribe(console.log)
s3.next('z'); s1.next('c'); s2.next('4'); s3.next('x')
// Add a new stream
subscription.unsubscribe()
const s4 = new ReplaySubject(1)
out = Observable.combineLatest(s1, s2, s3, s4)
subscription = out.subscribe(console.log)
s4.next('k')
Working example: CodePen
I am learning Rxjs and wanted to try out a few examples on my own
but I can't seem to get my head around to think reactively.
I am trying to calculate the time a user's mouse pointer spends inside and outside a div.
see fiddle - https://jsfiddle.net/ishansoni22/44af3n3k/
<div class = "space">
<div>
let $space = $(".space")
let in$ = Rx.Observable.fromEvent($space, "mouseenter")
.map((event) => "in")
let out$ = Rx.Observable.fromEvent($space, "mouseleave")
.map((event) => "out")
let inOut$ = Rx.Observable.merge(in$, out$)
let time$ = Rx.Observable.interval(1000)
.buffer(inOut$)
.map((list) => list.length)
time$.subscribe((value) => console.log(value));
I am able to calculate the time but how do I relate it to the respective in/ out streams? I want the output to look something like :
inside, in - 20, out - 30
outside, in - 20, out - 35
inside, in - 100, out - 35
Also, can someone point me to some examples I could do so that I can start thinking in the reactive paradigm?
There are some examples in the official documentation (http://reactivex.io/rxjs) but they are a little bit scarce indeed.
I think I would some your sample something like this:
let $space = $(".space")
let in$ = Rx.Observable.fromEvent($space, "mouseenter")
let out$ = Rx.Observable.fromEvent($space, "mouseleave")
let durations$ = in$
.map(_ => Date.now())
.switchMap(inTime => out$
.take(1)
.map(_ => Date.now())
.map(outTime => outTime - inTime)
)
durations$
.scan((sum, next) => sum + next, 0)
.subscribe(total => console.log(total))
This would start listening to in$, then upon a mouseenter-event it starts to listen to mouseleaves, takes 1 of those events and calculate the duration.
I have written multiple maps below each other for clarity, but of course you can compose that into a single function.
One of the things I found most challenging when starting out with Rx was using streams of streams, and becoming comfortable with flatMap and switchMap. The problem you describe is most easily solved using exactly this approach. With your streams defined as follows (I prefer const over let to make it clear no mutation is occuring):
const in$ = Rx.Observable.fromEvent($space, 'mouseenter');
const out$ = Rx.Observable.fromEvent($space, 'mouseleave');
you can describe entering and then leaving as follows:
const inThenOut$ = in$.switchMap(() => out$);
To understand exactly what this is doing I urge you to learn about flatMap, become comfortable with streams of streams, and then learn how switchMap works by only maintaining a subscription to the most recent inner stream. For this I found the official rxjs documentation the best source. The included marble diagrams often tell complex stories with just a few dots and lines.
From here it's a relatively small step to get the time spent inside. First, we map our original streams into timestamp values:
const timestamp = () => + new Date();
const in$ = Rx.Observable.fromEvent($space, 'mouseenter').map(() => timestamp());
const out$ = Rx.Observable.fromEvent($space, 'mouseleave').map(() => timestamp());
(note: there is a timestamp method in rxjs you could use instead of doing this manually, but I feel this better illustrates how you can map your stream elements into anything you please).
From there, we can adjust our switchMap usage to access both the in and out values, and return the difference between them:
const inThenOut$ = in$.switchMap(() => out$, (x, y) => y - x);
Here's the whole thing working:
https://jsbin.com/qoruyoluho/edit?js,console,output
You could use RXJS - Timestamp operator to attach timestamp to each item emitted by an Observable indicating when it was emitted.
const { fromEvent } = Rx;
const { map, switchMap, timestamp, take, tap } = RxOperators;
const in$ = fromEvent($space, 'mouseenter').pipe(
timestamp(),
tap(x => console.log(`In: ${x.timestamp}`))
)
const out$ = fromEvent($space, 'mouseleave').pipe(
timestamp(),
tap(x => console.log(`Out: ${x.timestamp}`))
)
const duration$ = in$.pipe(
switchMap(start => out$.pipe(
take(1),
map(finish => finish.timestamp - start.timestamp),
tap(value => console.log(`Duration ms: ${value}`))
)
)
)
/* output example
In: 1552295324302
Out: 1552295325158
Duration ms: 856
*/
Try it here: https://rxviz.com/v/rOW5g9x8
I have a stream of data from car auctions. Each car auction has n-number of lanes. I want to log the auction of each vehicle.
The stream looks something like this...
--{lane: 1, action: bid} --- { lane: 2, action: start} --- { lane:1, action: bid} --- {lane: 2, action: bid} --- {lane:1, action: sold} ---
I have the following to buffer each auction lane and close the buffer on sale...
const bufferOpen$= auctionWebSocketStream$
.filter(stream => stream.tag === 'CURITEM');
const bufferClose$ = () => auctionWebSocketStream$.filter(stream => stream.tag === 'SOLD');
auctionWebSocketStream$
.bufferToggle(bufferOpen$, bufferClose$)
.subscribe(x => console.log(x));
The above works fine so long as there is one auction and one lane. With multiple lanes, there's bid/sale information about multiple lanes.
How would I aggregate the stream by lanes into the buffer? Similar solutions always had known aggregation parameters. But I need to split the stream anytime there's a new lane.
Help is greatly appreciated.
UPDATE
I made a JSBin to show off my frustration and cluelessness. It gives an example input stream and explains the desired output.
http://jsbin.com/tuxitev/edit?js,console
(For bonus points, it only shows empty arrays under Babel. Not sure why Typescript is required)
If someone knows where to get RxJS questions answered, please let me know. I'll accept the answer. This is the third unanswered RxJS question I've had.
For anyone curious of the answer, here it is.
stream$
.groupBy(stream => stream.lane)
.mergeMap(stream =>
stream.scan((acc, cur) => {
if (cur.action === 'start') {
acc = [];
}
acc.push(cur)
return acc;
}, [])
.filter(stream => stream[stream.length-1].action === 'sold')
)
.subscribe(
x => console.log(x),
(e) => console.error(e),
() => console.log('complete')
)
http://jsbin.com/tuxitev/edit?js,console
I'm trying to slice observable stream by itself, eg.:
val source = Observable.from(1 to 10).share
val boundaries = source.filter(_ % 3 == 0)
val result = source.tumblingBuffer(boundaries)
result.subscribe((buf) => println(buf.toString))
Te output is:
Buffer()
Buffer()
Buffer()
Buffer()
source is probably iterated on boundaries line, before it reaches the result so it only create boundaries and resulting buffers but there's nothing to fill in.
My approach to this is using publish/connect:
val source2 = Observable.from(1 to 10).publish
val boundaries2 = source2.filter(_ % 3 == 0)
val result2 = source2.tumblingBuffer(boundaries2)
result2.subscribe((buf) => println(buf.toString))
source2.connect
This produces output alright:
Buffer(1, 2)
Buffer(3, 4, 5)
Buffer(6, 7, 8)
Buffer(9, 10)
Now I just need to hide connect from outer world and connect it when result gets subscribed (I am doing this inside a class and I don't want to expose it). Something like:
val source3 = Observable.from(1 to 10).publish
val boundaries3 = source3.filter(_ % 3 == 0)
val result3 = source3
.tumblingBuffer(boundaries3)
.doOnSubscribe(() => source3.connect)
result3.subscribe((buf) => println(buf.toString))
But now, doOnSubscribe action gets never called so published source gets never connected...
What's wrong?
You were on the right track with your publish solution. There is however an alternative publish operator that takes a lambda as its argument (see documentation) of type Observable[T] => Observable[R]. The argument of this lambda is the original stream, to which you can safely subscribe multiple times. Within the lambda you transform the original stream to your liking; in your case you filter the stream and buffer it on that filter.
Observable.from(1 to 10)
.publish(src => src.tumblingBuffer(src.filter(_ % 3 == 0)))
.subscribe(buf => println(buf.toString()))
The best thing of this operator is that you don't need to call anything like connect afterwards.