I have a stream of data from car auctions. Each car auction has n-number of lanes. I want to log the auction of each vehicle.
The stream looks something like this...
--{lane: 1, action: bid} --- { lane: 2, action: start} --- { lane:1, action: bid} --- {lane: 2, action: bid} --- {lane:1, action: sold} ---
I have the following to buffer each auction lane and close the buffer on sale...
const bufferOpen$= auctionWebSocketStream$
.filter(stream => stream.tag === 'CURITEM');
const bufferClose$ = () => auctionWebSocketStream$.filter(stream => stream.tag === 'SOLD');
auctionWebSocketStream$
.bufferToggle(bufferOpen$, bufferClose$)
.subscribe(x => console.log(x));
The above works fine so long as there is one auction and one lane. With multiple lanes, there's bid/sale information about multiple lanes.
How would I aggregate the stream by lanes into the buffer? Similar solutions always had known aggregation parameters. But I need to split the stream anytime there's a new lane.
Help is greatly appreciated.
UPDATE
I made a JSBin to show off my frustration and cluelessness. It gives an example input stream and explains the desired output.
http://jsbin.com/tuxitev/edit?js,console
(For bonus points, it only shows empty arrays under Babel. Not sure why Typescript is required)
If someone knows where to get RxJS questions answered, please let me know. I'll accept the answer. This is the third unanswered RxJS question I've had.
For anyone curious of the answer, here it is.
stream$
.groupBy(stream => stream.lane)
.mergeMap(stream =>
stream.scan((acc, cur) => {
if (cur.action === 'start') {
acc = [];
}
acc.push(cur)
return acc;
}, [])
.filter(stream => stream[stream.length-1].action === 'sold')
)
.subscribe(
x => console.log(x),
(e) => console.error(e),
() => console.log('complete')
)
http://jsbin.com/tuxitev/edit?js,console
Related
I have a costly server ajax request which has one input (full: boolean). If full is false, the server can return either a partial or a full response (response.isFull == true); but if full is true, the server will return a full response. Normally the partial response is good enough, but there are certain conditions that will require a full response. I need to avoid requesting a full response explicitly as much as possible, so I thought I'd start with a BehaviorSubject which I can eventually feed with true and combine it with distinctUntilChanged if I ever need to get the full response. This will give me an observable with false initially and that can give me true if I feed that into it:
const fullSubject = new BehaviorSubject<boolean>(false);
Then I've got a function that takes a boolean parameter and returns an observable with the server request (retried, transformed, etc.). As said, the answer can be partial or full, but it can be full even if the input parameter was false at the server's discretion. For example:
interface IdentityData {
...
isFull: boolean;
}
private getSimpleIdentity(full: boolean): Observable<IdentityData> {
return Axios.get(`/api/identity${full?"?full=true":""}`)
.pipe( ... retry logic ...,
... transformation logic ...,
shareReplay(1) );
}
I need to know how can I combine these so that the following is true:
The server needs to be queried at most twice.
If the first answer is a full answer, no further queries must be performed to the server.
If the first answer is a partial answer, and true is fed into fullSubject, a full answer must be requested.
The expected output from all this is an observable that emits either one full response, or a partial response and, when asked, a full response.
Environment: Vue 2.6.11, RxJS 6.5.5, Axios 0.19.2, TypeScript 3.7.5.
Thanks in advance
Here would be my approach:
const fullSubject = new BehaviorSubject(false);
const src$ = fullSubject.pipe(
switchMap(isFull => Axios.get('...')),
take(2), // Server required at most twice
takeWhile(response => !response.isFull, true), // When `isFull`, it will complete & unsubscribe -> no more requests to the server
shareReplay(1),
);
src$.subscribe(() => { /* ... */ });
function getFullAnswer () {
fullSubject.next(true);
}
takeWhile takes a second argument, inclusive. When set to true, when the predicate function evaluates to false(e.g isFull is true) it will send that value as well. –
if I've got it correctly
private getSimpleIdentity(): Observable<IdentityData> {
return fullSubject.pipe(
switchMap(full => Axios.get(`/api/identity${full ? "?full=true" : ""}`)),
shareReplay(1),
);
}
Uses the retryWhen() operator
const source = of("").pipe(map(() => Math.floor(Math.random() * 10 + 1)));
const example = source
.pipe(
tap((val) => console.log("tap", val)),
map((val) => {
//error will be picked up by retryWhen
if (val !== 5) throw val;
return val;
}),
retryWhen((errors) =>
errors.pipe(
tap(() => console.log("--Wait 1 seconds then repeat")),
delay(1000)
)
)
)
.subscribe((val) => console.log("subscription", val));
/*
output:
tap 3
--Wait 1 seconds then repeat
tap 8
--Wait 1 seconds then repeat
tap 1
--Wait 1 seconds then repeat
tap 4
--Wait 1 seconds then repeat
tap 7
--Wait 1 seconds then repeat
tap 5
subscription 5
*/
I have the following, and it does work, it keeps increasing the delay and eventually timing out which is what I wanted.
But because I am using Concatmap i lose the original value from the interval.
let x = 1
let source2$ = interval(500)
.pipe(
concatMap(() => {
x++
let newtime = x * 500
console.log("newtime ", newtime)
return of(5).pipe(delay(newtime))
}),
timeout(3000),
map((data) => {
return 'Source 2: ' + data
})
)
so it prints Source 2: 5.. where as i want it to print the value of the interval.
I got working what i wanted using the concatmap but i think its the wrong operator as I lose the original value.
Can somebody help?
More info
TO summarize, all i would like to do is emit values using the interval and after each emit increase the delay time - eventually it hits the timeout of 3000 ms and errors out.
I've mentioned in comments that you can use concatMap for this that receives ever increasing index from interval:
concatMap(index => {
let newtime = index * 500
console.log("newtime ", newtime)
return of(index).pipe(delay(newtime))
}),
Notice, that I'm returning the value back to the stream by of(index).
I think I understand what were you concerned about returning another Observable. Since you want to emit items in sequence (emit one only after the previous one completes) then you have to use concatMap with another inner Observable. There isn't a special operator only for this functionality because this is "composable behavior" which means you can achieve this behavior by combining existing operators.
const source2$ = interval(500)
.pipe(
map(x => x * 500),
switchMap(x => timer(x)),
timeout(3000),
map(data => 'Source 2: ' + data)
)
UPDATE:
https://stackblitz.com/edit/rxjs-iywcm6?devtoolsheight=60
const source2$ = interval(500)
.pipe(
tap(x => console.log('Tick before delay', x)),
concatMap(x => timer((x + 1) * 500).pipe(mapTo(x))),
tap(x => console.log('Tick after delay', x)),
map(data => 'Source 2: ' + data),
timeout(3000)
).subscribe(
(data) => console.log(data),
e => console.error('Timeout', e))
Hope someone can help me with this problem.
I have 2 streams that I need to use the operator combineLatest on. After a while I need to add dynamically streams that also need to use combineLatest on.
Here is what I need to do:
stream a ---------a---------------b-------------c------------------------>
stream b ----1--------2-----3-------------------------4------------------>
stream c (not defined at start) -----z-----------------x------------>
stream d (not defined at start) ----------k------>
(combineLatest)
result ---------(a1)(a2)--(a3)--(b3)----(b3z)-(c3z)-(c4z)-(c4x)-(c4xk)->
UPDATE
To be more specific I want to turn this STREAM (link)
To this result:
A----B---B0-C0--D0--D1--E1--E1a--F1a--F2a---G2a---G3a--H3a-H3b--I3b
The idea it that everything is a stream. Even stream of streams :)
const onNew$ = new Rx.Subject();
const a$ = Rx.Observable.interval(1000).mapTo('a');
const b$ = Rx.Observable.interval(1000).mapTo('b');
const comb$ = Rx.Observable
.merge(
onNew$,
Rx.Observable.from([a$, b$]),
)
.scan((acc, v) => {
acc.push(v);
return acc;
}, [])
.switchMap(vs => Rx.Observable.combineLatest(vs))
comb$.take(4).subscribe(v => console.log(v));
setTimeout(
() => onNew$.next(Rx.Observable.interval(1000).mapTo('c')),
2000,
);
setTimeout(
() => onNew$.next(Rx.Observable.interval(1000).mapTo('d')),
4000,
);
<script src="https://unpkg.com/rxjs/bundles/Rx.min.js"></script>
Taking Oles' answer, simplifying a little and adding test data as given in question update
const Subject = Rx.ReplaySubject
const ReplaySubject = Rx.ReplaySubject
const newStream = new Subject()
// Set up output, no streams yet
const streamOfStreams = newStream
.scan( (acc, stream) => {
acc.push(stream);
return acc;
}, [])
.switchMap(vs => Observable.combineLatest(vs))
.map(arrayOfValues => arrayOfValues.join('')) // declutter
.subscribe(console.log)
// Add a stream
const s1 = new ReplaySubject()
newStream.next(s1)
// emit on streams
s1.next('A'); s1.next('B')
// Add a stream
const s2 = new ReplaySubject()
newStream.next(s2)
// emit on streams
s2.next('0'); s1.next('C')
s1.next('D'); s2.next('1'); s1.next('E');
// Add a stream
const s3 = new ReplaySubject()
newStream.next(s3)
// emit on streams
s3.next('a');
s1.next('F'); s2.next('2'); s1.next('G'); s2.next('3'); s1.next('H');
s3.next('b'); s1.next('I')
Working example: CodePen
Update
Christian has kindly supplied some test streams which are more 'real world' than the sequenced Subjects I've used above. Unfortunately, these highlight a bug in the solution as it stands.
For reference, the new test streams are
const streamA = Rx.Observable.timer(0,800).map(x => String.fromCharCode(x+ 65));
const streamB = Rx.Observable.timer(0,1300).map(x => x);
const streamC = Rx.Observable.timer(1100, 2000).map(x => String.fromCharCode(x+ 97));
setTimeout(() => newStream.next(streamA), 500);
setTimeout(() => newStream.next(streamB), 2000);
setTimeout(() => newStream.next(streamC), 3000);
Problem #1
The first problem stems from the core line in streamOfStreams,
.switchMap(vs => Observable.combineLatest(vs))
This essentially says, every time a new array of streams appears, map it to a combineLatest() of the new array and switch to the new observable. However, the test observables are cold, which means each re-subscription gets the full stream.
Ref: Introduction to Rx - Hot and Cold observables
Some observable sequences can appear to be hot when they are in fact
cold. A couple of examples that surprise many is Observable.Interval
and Observable.Timer
So we get
- expected A--B--B0...
- actual A--B--A0--B0...
The obvious solution is to turn the cold observables hot,
const asHot = (stream) => {
const hot = stream.multicast(() => new Rx.Subject())
hot.connect()
return hot
}
but this omits B0 from the sequence, A--B--C0..., so we want hot + 1 previous which can be had with a buffer size one
const asBuffered = (stream) => {
const bufferOne = new ReplaySubject(1)
stream.subscribe(value => bufferOne.next(value))
return bufferOne
}
Problem #2
The second problem comes from the fact that streamC delays it's first emit by 1100ms (good test Christian!).
This results is
- expected A--B--B0--C0--D0--D1--E1--E1a...
- actual A--B--B0--C0--D0--E1a...
which means we need to delay adding a stream until it's first emit
const addStreamOnFirstEmit = (stream) => {
const buffered = asBuffered(stream)
buffered.first().subscribe( _ => {
newStream.next(buffered)
})
}
Working example: CodePen
Notes on the CodePen
I've left in the various streamAdder functions for experimentation, and there are also _debug versions that emit the streams and the addStream events to show the sequence.
Also limited the source streams so that the console doesn't scroll too much.
Note on the expected output
The new solution diverges from the expected output given in the question after 'G3a'
expected A----B---B0-C0--D0--D1--E1--F1---F2---F2a---G2a---G3a--H3a--H3b--I3b
actual A----B---B0-C0--D0--D1--E1--E1a--F1a--F2a---G2a---G3a--G3b--H3b--I3b
which is due to the simultaneous emission of of 'H' and 'b'. Problem #3?
One more test
In order to see if the solution failed if streamC delayed first emission until after two emits of streamA & streamB, I changed the delay to 1800ms
const streamC = Rx.Observable.timer(1800, 2000).map(x => String.fromCharCode(x+ 97));
I believe the output for this test is correct.
Can be done if you can unsubscribe and re-subscribe for each new stream
// Start with two streams
const s1 = new ReplaySubject(1)
const s2 = new ReplaySubject(1)
let out = Observable.combineLatest(s1, s2)
let subscription = out.subscribe(console.log)
s2.next('1'); s1.next('a'); s2.next('2'); s2.next('3'); s1.next('b')
// Add a new stream
subscription.unsubscribe()
const s3 = new ReplaySubject(1)
out = Observable.combineLatest(s1, s2, s3)
subscription = out.subscribe(console.log)
s3.next('z'); s1.next('c'); s2.next('4'); s3.next('x')
// Add a new stream
subscription.unsubscribe()
const s4 = new ReplaySubject(1)
out = Observable.combineLatest(s1, s2, s3, s4)
subscription = out.subscribe(console.log)
s4.next('k')
Working example: CodePen
I am learning Rxjs and wanted to try out a few examples on my own
but I can't seem to get my head around to think reactively.
I am trying to calculate the time a user's mouse pointer spends inside and outside a div.
see fiddle - https://jsfiddle.net/ishansoni22/44af3n3k/
<div class = "space">
<div>
let $space = $(".space")
let in$ = Rx.Observable.fromEvent($space, "mouseenter")
.map((event) => "in")
let out$ = Rx.Observable.fromEvent($space, "mouseleave")
.map((event) => "out")
let inOut$ = Rx.Observable.merge(in$, out$)
let time$ = Rx.Observable.interval(1000)
.buffer(inOut$)
.map((list) => list.length)
time$.subscribe((value) => console.log(value));
I am able to calculate the time but how do I relate it to the respective in/ out streams? I want the output to look something like :
inside, in - 20, out - 30
outside, in - 20, out - 35
inside, in - 100, out - 35
Also, can someone point me to some examples I could do so that I can start thinking in the reactive paradigm?
There are some examples in the official documentation (http://reactivex.io/rxjs) but they are a little bit scarce indeed.
I think I would some your sample something like this:
let $space = $(".space")
let in$ = Rx.Observable.fromEvent($space, "mouseenter")
let out$ = Rx.Observable.fromEvent($space, "mouseleave")
let durations$ = in$
.map(_ => Date.now())
.switchMap(inTime => out$
.take(1)
.map(_ => Date.now())
.map(outTime => outTime - inTime)
)
durations$
.scan((sum, next) => sum + next, 0)
.subscribe(total => console.log(total))
This would start listening to in$, then upon a mouseenter-event it starts to listen to mouseleaves, takes 1 of those events and calculate the duration.
I have written multiple maps below each other for clarity, but of course you can compose that into a single function.
One of the things I found most challenging when starting out with Rx was using streams of streams, and becoming comfortable with flatMap and switchMap. The problem you describe is most easily solved using exactly this approach. With your streams defined as follows (I prefer const over let to make it clear no mutation is occuring):
const in$ = Rx.Observable.fromEvent($space, 'mouseenter');
const out$ = Rx.Observable.fromEvent($space, 'mouseleave');
you can describe entering and then leaving as follows:
const inThenOut$ = in$.switchMap(() => out$);
To understand exactly what this is doing I urge you to learn about flatMap, become comfortable with streams of streams, and then learn how switchMap works by only maintaining a subscription to the most recent inner stream. For this I found the official rxjs documentation the best source. The included marble diagrams often tell complex stories with just a few dots and lines.
From here it's a relatively small step to get the time spent inside. First, we map our original streams into timestamp values:
const timestamp = () => + new Date();
const in$ = Rx.Observable.fromEvent($space, 'mouseenter').map(() => timestamp());
const out$ = Rx.Observable.fromEvent($space, 'mouseleave').map(() => timestamp());
(note: there is a timestamp method in rxjs you could use instead of doing this manually, but I feel this better illustrates how you can map your stream elements into anything you please).
From there, we can adjust our switchMap usage to access both the in and out values, and return the difference between them:
const inThenOut$ = in$.switchMap(() => out$, (x, y) => y - x);
Here's the whole thing working:
https://jsbin.com/qoruyoluho/edit?js,console,output
You could use RXJS - Timestamp operator to attach timestamp to each item emitted by an Observable indicating when it was emitted.
const { fromEvent } = Rx;
const { map, switchMap, timestamp, take, tap } = RxOperators;
const in$ = fromEvent($space, 'mouseenter').pipe(
timestamp(),
tap(x => console.log(`In: ${x.timestamp}`))
)
const out$ = fromEvent($space, 'mouseleave').pipe(
timestamp(),
tap(x => console.log(`Out: ${x.timestamp}`))
)
const duration$ = in$.pipe(
switchMap(start => out$.pipe(
take(1),
map(finish => finish.timestamp - start.timestamp),
tap(value => console.log(`Duration ms: ${value}`))
)
)
)
/* output example
In: 1552295324302
Out: 1552295325158
Duration ms: 856
*/
Try it here: https://rxviz.com/v/rOW5g9x8
I am trying RxJS.
My use case is to parse a log file and group lines by topic ( i.e.: the beginning of the group is the filename and then after that I have some lines with user, date/time and so on)
I can analyse the lines using regExp. I can determine the beginning of the group.
I use ".scan" to group the lines together, when I've the beginning of new group of line, I create an observer on the lines I've accumulated ... fine.
The issue is the end of the file. I've started a new group, I am accumulating lines but I can not trigger the last sequence as I do not have the information that the end. I would have expect to have the information in the complete (but not)
Here is an example using number. Begin of group can multi of 3 or 5. (remark: I work in typescript)
import * as Rx from "rx";
let r = Rx.Observable
.range(0, 8)
.scan( function(acc: number[], value: number): number[]{
if (( value % 3 === 0) || ( value % 5 === 0)) {
acc.push(value);
let info = acc.join(".");
Rx.Observable
.fromArray(acc)
.subscribe( (value) => {
console.log(info, "=>", value);
});
acc = [];
} else {
acc.push(value);
}
return acc;
}, [])
.subscribe( function (x) {
// console.log(x);
});
This emit:
0 => 0
1.2.3 => 1
1.2.3 => 2
1.2.3 => 3
4.5 => 4
4.5 => 5
6 => 6
I am looking how to emit
0 => 0
1.2.3 => 1
1.2.3 => 2
1.2.3 => 3
4.5 => 4
4.5 => 5
6 => 6
7.8 => 7 last items are missing as I do not know how to detect end
7.8 => 8
Can you help me, grouping items?
Any good idea, even not using scan, is welcome.
Thank in advance
You can use the materialize operator. See the documentation here and the marbles here, and an example of use from SO.
In your case, I would try something like (untested but hopefully you can complete it yourself, note that I don't know a thing about typescript so there might be some syntax errors):
import * as Rx from "rx";
let r = Rx.Observable
.range(0, 8)
.materialize()
.scan( function(acc: number[], materializedNumber: Rx.Notification<number>): number[]{
let rangeValue: number = materializedNumber.value;
if (( rangeValue % 3 === 0) || ( rangeValue % 5 === 0)) {
acc.push(rangeValue);
generateNewObserverOnGroupOf(acc);
acc = [];
} else if ( materializedNumber.kind === "C") {
generateNewObserverOnGroupOf(acc);
acc = [];
} else {
acc.push(rangeValue);
}
return acc;
}, [])
// .dematerialize()
.subscribe( function (x) {
// console.log(x);
});
function generateNewObserverOnGroupOf(acc: number[]) {
let info = acc.join(".");
Rx.Observable
.fromArray(acc)
.subscribe( (value) => {
console.log(info, "=>", value);
});
The idea is that the materialize and dematerialize works with notifications, which encodes whether the message being passed by the stream is one of next, error, completed kinds (respectively 'N', 'E', 'C' values for the kind property). If you have a next notification, then the value passed is in the value field of the notification object. Note that you need to dematerialize to return to the normal behaviour of the stream so it can complete and free resources when finished.