How do I get multiple subscribers waiting the same promise to resolve if it is already inflight with latecomers given a new resolution?
doSomething = () => {
return new Promise((resolve) => {
setTimeout(() => resolve(Math.random(), 1000)
})
}
// how to define obs?
obs.subscribe(v => console.log(v)); // 0.39458743297857473
obs.subscribe(v => console.log(v)); // 0.39458743297857473
obs.subscribe(v => console.log(v)); // 0.39458743297857473
setTimeout(() => obs.subscribe(v => console.log(v)), 2000); // 0.9485769395265746
I'd like the observable to remain cold until the first subscriber, then go cold again after the result is streamed to all subsequent concurrent subscribers. I basically don't want any concurrent requests to the same underlying function.
You can use defer as the creation-operator and then share the stream:
doSomething = () => {
return new Promise((resolve) => {
setTimeout(() => resolve(Math.random(), 1000));
});
}
const obs = Rx.Observable
.defer(doSomething)
.share();
obs.subscribe(console.log); // resolve #1
obs.subscribe(console.log); // resolve #1
obs.subscribe(console.log); // resolve #1
setTimeout(() => obs.subscribe(console.log), 2000); // resolve #2
<script src="https://unpkg.com/rxjs/bundles/Rx.min.js"></script>
Related
I have the next example
import { Subject } from "rxjs";
const subject = new Subject();
subject.subscribe(() => new Promise(res => {
setTimeout(() => console.log('!! 1'), 500);
}))
subject.subscribe(() => new Promise(res => {
setTimeout(() => console.log('!! 2'), 1000);
}))
console.log('>>> START')
subject.next();
console.log('<<< FINISH')
Console looks like
>>> START
<<< FINISH
!! 1
!! 2
I want the following behavious
>>> START
!! 1
!! 2
<<< FINISH
Can I reach expected behaviour or I should to use another aproach?
Ok so, this should work. I just forced the behavior of the Observables. To be clear, observables are async javascript and need to share values in async mode so that you can 'emit' a new value through a subject and all the observables can see that value while doing some other tasks. This code do exactly what you asked for but it makes no sense to emit a value and await for that value in the same place and moment, to achieve that you should probably think about another way of coding this module.
Said that, i've tested this snippet and it works, hope this will help
import { Subject } from "rxjs";
const subject = new Subject();
async function nextValue(value) {
return new Promise((resolve, reject) => {
subject.subscribe(subValue => {
setTimeout(() => {
console.log(subValue);
resolve();
}, 500);
}, err => reject(err));
subject.next(value);
});
}
(async () => {
console.log('>>> START');
await nextValue('myValue');
console.log('<<< FINISH');
})();
I perform http requests to my db and have noticed that if I send all the requests at once, some of them will get a timeout errors. I'd like to add a delay between calls so the server doesn't get overloaded. I'm trying to find the RxJS solution to this problem and don't want to add a setTimeout.
Here is what I currently do:
let observables = [];
for(let int = 0; int < 10000; int++){
observables.push(new Observable((observer) => {
db.add(doc[int], (err, result)=>{
observer.next();
observer.complete();
})
}))
}
forkJoin(observables).subscribe(
data => {
},
error => {
console.log(error);
},
() => {
db.close();
}
);
You can indeed achieve this with Rxjs quite nicely. You'll need higher order observables, which means you'll emit an observable into an observable, and the higher order observable will flatten this out for you.
The nice thing about this approach is that you can easily run X requests in // without having to manage the pool of requests yourself.
Here's the working code:
import { Observable, Subject } from "rxjs";
import { mergeAll, take, tap } from "rxjs/operators";
// this is just a mock to demonstrate how it'd behave if the API was
// taking 2s to reply for a call
const mockDbAddHtppCall = (id, cb) =>
setTimeout(() => {
cb(null, `some result for call "${id}"`);
}, 2000);
// I have no idea what your response type looks like so I'm assigning
// any but of course you should have your own type instead of this
type YourRequestType = any;
const NUMBER_OF_ITEMS_TO_FETCH = 10;
const calls$$ = new Subject<Observable<YourRequestType>>();
calls$$
.pipe(
mergeAll(3),
take(NUMBER_OF_ITEMS_TO_FETCH),
tap({ complete: () => console.log(`All calls are done`) })
)
.subscribe(console.log);
for (let id = 0; id < NUMBER_OF_ITEMS_TO_FETCH; id++) {
calls$$.next(
new Observable(observer => {
console.log(`Starting a request for ID "${id}""`);
mockDbAddHtppCall(id, (err, result) => {
if (err) {
observer.error(err);
} else {
observer.next(result);
observer.complete();
}
});
})
);
}
And a live demo on Stackblitz: https://stackblitz.com/edit/rxjs-z1x5m9
Please open the console of your browser and note that the console log showing when a call is being triggered starts straight away for 3 of them, and then wait for 1 to finish before picking up another one.
Looks like you could use an initial timer to trigger the http calls. e.g.
timer(delayTime).pipe(combineLatest(()=>sendHttpRequest()));
This would only trigger the sendHttpRequest() method after the timer observable had completed.
So with your solution. You could do the following...
observables.push(
timer(delay + int).pipe(combineLatest(new Observable((observer) => {
db.add(doc[int], (err, result)=>{
observer.next();
observer.complete();
}))
}))
Where delay could probably start off at 0 and you could increase it using the int index of your loop by some margin.
Timer docs: https://www.learnrxjs.io/learn-rxjs/operators/creation/timer
Combine latest docs: https://www.learnrxjs.io/learn-rxjs/operators/combination/combinelatest
merge with concurrent value:
mergeAll and mergeMap both allow you to define the max number of subscribed observables. mergeAll(1)/mergeMap(LAMBDA, 1) is basically concatAll()/concatMap(LAMBDA).
merge is basically just the static mergeAll
Here's how you might use that:
let observables = [...Array(10000).keys()].map(intV =>
new Observable(observer => {
db.add(doc[intV], (err, result) => {
observer.next();
observer.complete();
});
})
);
const MAX_CONCURRENT_REQUESTS = 10;
merge(...observables, MAX_CONCURRENT_REQUESTS).subscribe({
next: data => {},
error: err => console.log(err),
complete: () => db.close()
});
Of note: This doesn't batch your calls, but it should solve the problem described and it may be a bit faster than batching as well.
mergeMap with concurrent value:
Perhaps a slightly more RxJS way using range and mergeMap
const MAX_CONCURRENT_REQUESTS = 10;
range(0, 10000).pipe(
mergeMap(intV =>
new Observable(observer => {
db.add(doc[intV], (err, result) => {
observer.next();
observer.complete();
});
}),
MAX_CONCURRENT_REQUESTS
)
).subscribe({
next: data => {},
error: err => console.log(err),
complete: () => db.close()
});
I want the done to print only after the first and second is printed.
const obs1 = new Observable<any>((observer) => {
setTimeout(() => {
console.log('first');
observer.next();
observer.complete();
}, 10000);
});
const obs2 = new Observable<any>((observer) => {
setTimeout(() => {
console.log('second');
observer.next();
observer.complete();
}, 1000);
});
from([obs1, obs2]).pipe(concatAll()).subscribe(() => {
console.log('done');
});
You don't complete any of the two source Observables so no operator can know what you consider as "done". This means you could use merge or combineLatest and only handle next notifications.
However, if you know they'll always emit just once you can complete each source and then use forkJoin or concat:
const obs1 = new Observable<any>((observer) => {
setTimeout(() => {
console.log('first');
observer.next();
observer.complete();
}, 10000);
});
...
concat(obs1, obs2).subscribe({
complete: () => {
console.log('done');
}
});
I am learning to use RXJS. In this scenario, I am chaining a few async requests using rxjs. At the last mergeMap, I'd like to have access to the first mergeMap's params. I have explored the option using Global or withLatest, but neither options seem to be the right fit here.
const arraySrc$ = from(gauges).pipe(
mergeMap(gauge => {
return readCSVFile(gauge.id);
}),
mergeMap((csvStr: any) => readStringToArray(csvStr.data)),
map((array: string[][]) => transposeArray(array)),
mergeMap((array: number[][]) => forkJoin(uploadToDB(array, gauge.id))),
catchError(error => of(`Bad Promise: ${error}`))
);
readCSVFile is an async request which returns an observable to read CSV from a remote server.
readStringToArray is another async request which returns an observable to convert string to Arrays
transposeArray just does the transpose
uploadToDB is async DB request, which needs gague.id from the first mergeMap.
How do I get that? It would be great to take some advice on why the way I am doing it is bad.
For now, I am just passing the ID layer by layer, but it doesn't feel to be correct.
const arraySrc$ = from(gauges).pipe(
mergeMap(gauge => readCSVFile(gauge.id)),
mergeMap(({ data, gaugeId }: any) => readStringToArray(data, gaugeId)),
map(({ data, gaugeId }) => transposeArray(data, gaugeId)),
mergeMap(({ data, gaugeId }) => uploadToDB(data, gaugeId)),
catchError(error => of(`Bad Promise: ${error}`))
);
Why don't you do simply this?
const arraySrc$ = from(gauges).pipe(
mergeMap(gauge => readCSVFile(gauge.id).pipe(
mergeMap((csvStr: any) => readStringToArray(csvStr.data)),
map((array: string[][]) => transposeArray(array)),
mergeMap((array: number[][]) => forkJoin(uploadToDB(array, gauge.id)))
)),
catchError(error => of(`Bad Promise: ${error}`))
);
You can also wrap the inner observable in a function:
uploadCSVFilesFromGaugeID(gaugeID): Observable<void> {
return readCSVFile(gaugeID).pipe(
mergeMap((csvStr: any) => readStringToArray(csvStr.data)),
map((array: string[][]) => transposeArray(array)),
mergeMap((array: number[][]) => forkJoin(uploadToDB(array, gaugeID))
);
}
In order to do this at the end:
const arraySrc$ = from(gauges).pipe(
mergeMap(gauge => uploadCSVFileFromGaugeID(gauge.id)),
catchError(error => of(`Bad Promise: ${error}`))
);
MergeMap requires all observable inputs; else, previous values may be returned.
It is a difficult job to concatenate and display the merging response. But here is a straightforward example I made so you can have a better idea. How do we easily perform sophisticated merging.
async playWithBbservable() {
const observable1 = new Observable((subscriber) => {
subscriber.next(this.test1());
});
const observable2 = new Observable((subscriber) => {
subscriber.next(this.test2());
});
const observable3 = new Observable((subscriber) => {
setTimeout(() => {
subscriber.next(this.test3());
subscriber.complete();
}, 1000);
});
console.log('just before subscribe');
let result = observable1.pipe(
mergeMap((val: any) => {
return observable2.pipe(
mergeMap((val2: any) => {
return observable3.pipe(
map((val3: any) => {
console.log(`${val} ${val2} ${val3}`);
})
);
})
);
})
);
result.subscribe({
next(x) {
console.log('got value ' + x);
},
error(err) {
console.error('something wrong occurred: ' + err);
},
complete() {
console.log('done');
},
});
console.log('just after subscribe');
}
test1() {
return 'ABC';
}
test2() {
return 'PQR';
}
test3() {
return 'ZYX';
}
I just realized that inner-observables (like those defined in a mergeMap or switchMap operator) do not "stop" even when the outer-observable has no subscription left.
For a better example, let's show some code:
const {
Subject,
of: obsOf,
concat: obsConcat,
defer,
} = require("rxjs");
const {
finalize,
mergeMap,
tap,
takeUntil,
} = require("rxjs/operators");
const subject = new Subject();
obsOf(null).pipe(
mergeMap(() =>
obsConcat(
defer(() => {
console.log("side-effect 1");
return obsOf(1);
}),
defer(() => {
console.log("side-effect 2");
return obsOf(2);
}),
defer(() => {
console.log("side-effect 3");
return obsOf(3);
})
)
),
finalize(() => {
console.log("finalized");
})
)
.pipe(
takeUntil(subject),
tap((i) => {
if (i === 2) {
subject.next();
}
})
).subscribe(
(i) => { console.log("next", i); },
(e) => { console.log("error", e); },
() => { console.log("complete"); },
);
// Ouput:
// > side-effect 1
// > next 1
// > side-effect 2
// > complete
// > finalized
// > side-effect 3
The fact that the side-effect 3 line is logged is weird since the outer observable already called finalize.
As all those side-effects are in a defer, they could perfectly be avoided after unsubscription. From my point-of-view, those side-effects provide no value at all.
Any idea why RxJS still execute those ?
This is unfortunately by design (as of RxJS 6) - concat will buffer the observables and will subscribe to each buffered one even after you unsubscribe (if the subscription is closed it will subscribe and immediately unsubscribe).
You have to prevent the observables from getting buffered...
obsOf(null).pipe(
mergeMap(() => obsOf(
defer(() => {
console.log("side-effect 1");
return obsOf(1);
}),
defer(() => {
console.log("side-effect 2");
return obsOf(2);
}),
defer(() => {
console.log("side-effect 3");
return obsOf(3);
})
)),
concatAll(),
finalize(() => {
console.log("finalized");
}),
takeUntil(subject),
tap((i) => {
if (i === 2) {
subject.next();
}
})
).subscribe(
(i) => { console.log("next", i); },
(e) => { console.log("error", e); },
() => { console.log("complete"); },
);
One could think the code above works, but only until you delay one of the observables. Replace obsOf(1) with timer(100).pipe(mapTo(1)); and behavior is exactly the same.
The only workaround is to make sure you are not buffering anything (mean don't use concat* operators) or limit observable production some other way (use separate Subject and control the production manually).