I am new to Rxjs and am trying to implement the following workflow in it:
User clicks on a menu item that triggers an HTTP request
Before the response has arrived, the user clicks on a second request
The subscription to the first request is ended and a subscription to the second request is started
// The code below sits inside the onClick event of my menu
var callAction = function(someParameters) {
return Rx.Observable.create(function(observer) {
var subscribed = true;
myHttpApi.someActionCall(someParameters).then(
(data: any) => {
if (subscribed) {
// Send data to the client
observer.next(data);
// Immediately complete the sequence
observer.complete();
}
}).catch((err: any) => {
if (subscribed) {
// Inform the client that an error occurred.
observer.error(ex);
}
}
);
return function () {
subscribed = false;
}
});
};
The observer is further defined below:
var observer = {
// onNext in RxJS 4
next: function (data) {
// Do what you need to do in the interface
},
// onError in RxJS 4
error: function (err) {
// Handle the error in the interface
},
// onComplete in RxJS 4
complete: function () {
//console.log("The asynchronous operation has completed.");
}
};
let subscription = callAction(somParameters).subscribe(observer);
How do I now go about implementing #3, whereby the subscription to the first request is ended and a subscription to the new request (in this example, the same block of code is executed for different menu options and therefore different requests based on the parameters) is started?
Breaking up the steps into discrete functions,
// Inner observable, calls the API
const callAction$ = function(someParameters) {
return Observable.fromPromise(
myHttpApi.someActionCall(someParameters)
)
}
// Outer observable, controls the click chain
const click$ = new Subject();
click$.switchMap(clickParams => {
return callAction$(clickParams)
})
.subscribe(
result => console.log('Result: ', result),
err => console.log('Error: ', err.message)
)
// Handler function, called from menu
const handleClick = function(clickParams) {
click$.next(clickParams)
}
Working example CodePen
Related
My ampq system seems loosing messages, so I'd like a way to see if messages are effectively queued before being consumed.
I have several MicroServices communicating by amqp messages on NodeJs, using CloudAmqp. One of this microservice MS[B] generates .pdf, the process it's pretty heavy and requires about a minute for each request. So I send the .pdf asyncronously, triggering a webhook once finished, and generate once per time using a PreFetch = 1
So one MS[A] collects all the requests from the user, answers back to them saying "ok, request received, listen on the webhook" and in parallel it asks to the MS[B] to generate pdfs. MS[B] has prefetch=1, so consumes just one request per time. Once finished, sends the response to the callback queue of MS[A], which triggers the user webhook saying "the pdf, it's ready".
The problem is that MS[B] misses all the messages while busy:
it consumes one request from MS[A]
starts generating the .pdf
while generating, it discards all the other messages that MS[A] sends, as if there would be not any queue
it finishes the .pdf, sending ACK to MS[A]
then it starts again accepting messages, taking the last one received after being idle, losing all the previous ones.
Why? How can I find the problem, what could I monitor?
Communications between other MSs works well, with messages correctly ordered in queues. Just this one, with prefetch=1, loses messages.
I am NOT using the NO-ACK rule. I don't know what try, what test and what monitor to find the problem.
How can I see (if) messages are correctly queued before being consumed, ora just lost?
Below, the implementation of the messaging system
Channel Creation
/*
Starting Point of a connection to the CLOUDAMQP_URL server, then exec the callback
*/
start(callback) {
var self = this;
// if the connection is closed or fails to be established at all, we will reconnect
amqp.connect(process.env.CLOUDAMQP_URL + "?heartbeat=60")
.then(
function (conn) {
// create queues and consume mechanism
self.amqpConn = conn;
setTimeout(() => {
startPublisher();
}, 200);
setTimeout(() => {
createCallbackQueue();
}, 1000);
setTimeout(() => {
callback();
}, 2000);
});
// create publisher channels
function startPublisher() {
self.amqpConn.createConfirmChannel()
.then(function (ch) {
self.pubChannel = ch;
logger.debug("🗣️ pubChannel ready");
while (true) {
var m = self.offlinePubQueue.shift();
if (!m) break;
self.publish(m[0], // exchange
m[1], // routingKey
m[2], // content,
undefined // correlationId
);
}
});
}
// create callback channel
function createCallbackQueue() {
self.amqpConn.createChannel()
.then(function (channel) {
channel.assertQueue(self.CALLBACK_QUEUE_NAME, {
durable: true,
exclusive: true, // callback are exclusive
})
.then(function (q) {
logger.debug(" đź‘‚ Waiting for RPC RESP in " + self.CALLBACK_QUEUE_NAME);
channel.consume(q.queue,
processCallback, {
noAck: false
}
);
});
// process messages of the callback
function processCallback(msg) {
var correlationId = msg.properties.correlationId;
}
//callback received
if (self.correlationIds_map[correlationId]) {
delete self.correlationIds_map[correlationId];
var content = JSON.parse(msg.content.toString());
self.eventEmitter.emit(correlationId, content);
}
}
});
}
return deferred.promise;
}
Consuming Messages
/*
#worker_queue - the name of the queue
*/
// Consume message from 'worker_queue', A worker that acks messages only if processed succesfully
startWorker(worker_queue, routes) {
var self = this;
logger.debug("startWorker " + self.CALLBACK_QUEUE_NAME);
var channel;
worker_queue = self.MICROSERVICE_NAME + worker_queue;
self.amqpConn.createChannel()
.then(
function (ch) {
channel = ch;
ch.prefetch(self.opt.prefetch); // = 1 for MS[B] generating pdf
channel.assertQueue(worker_queue, {
durable: true,
exclusive: true
})
.then(function (q) {
channel.consume(worker_queue, processMsg, {
noAck: false
});
});
});
// call the 'function from interface' passing params, and send the ACK
function processMsg(msg) {
work(msg)
.then(function (data) {
channel.ack(msg, false); // allUpTo = false
})
.catch(function (err) {
channel.ack(msg, false);
// channel.reject(msg, false); // requeue = false
// this.closeOnErr(e);
});
}
// execute the command, and queue back a response, checking if it's an error or not
function work(msg) {
var deferred = Q.defer();
var correlationId;
try {
correlationId = msg.properties.correlationId;
} catch (err) {}
work_function(msg.content, correlationId)
.then(function (resp) {
var content = {
data: resp
};
content = Buffer.from(JSON.stringify(content));
channel.sendToQueue(msg.properties.replyTo,
content, {
correlationId: correlationId,
content_type: 'application/json'
}
);
deferred.resolve(resp);
});
return deferred.promise;
}
}
Publish Messages
publish(exchange, routingKey, content, correlationId) {
var self = this;
var deferred = Q.defer();
self.correlationIds_map[correlationId] = true;
self.pubChannel.publish(exchange, routingKey, content,
{
replyTo: self.CALLBACK_QUEUE_NAME,
content_type : 'application/json',
correlationId: correlationId,
persistent : true
},
function(err, ok) {
if (err)
{
self.offlinePubQueue.push([exchange, routingKey, content]); // try again
self.pubChannel.connection.close();
deferred.resolve('requeued');
}
else
{
deferred.resolve(ok);
}
});
return deferred.promise;
}
We have an EnableRule (JS) for a ribbon button. Because it calls an api, it takes a bit and hence, should be async. The following implementation ilustrates the problem:
export async function ribbonEnabled():Promise<boolean> {
setInterval(() => {console.log(new Date().toLocaleTimeString())}, 1000);
await new Promise(r => setTimeout(r, 7000));
console.log('sleep is over');
return true;
}
Even tough the function is declared as async and awaits the api call (or in the example the timeout), the ribbon is not rendered until a value is returned.
Here you can see that several seconds have passed and no ribbon is rendered:
And as soon as the await procedure is over, the message gets printed and the ribbon is rendered:
Is that a known issue? Are there any workarounds? I have seen solution where the function is synchronous, returns a boolean variable and in a promis changes this variable and calls ui.refreshRibbon. But this is not adviced and runs the whole function again, so more logic is needed.
You can return promise object in your function. This is only support in Unified Interface.
function EnableRule()
{
const request = new XMLHttpRequest();
request.open('GET', '/bar/foo');
return new Promise(function (resolve, reject)
{
request.onload = function (e)
{
if (request.readyState === 4)
{
if (request.status === 200)
{
resolve(request.responseText === "true");
}
else
{
reject(request.statusText);
}
}
};
request.onerror = function (e)
{
reject(request.statusText);
};
request.send(null);
});
}
Notice If the promise does not resolve within 10 seconds, the rule will resolve with a false value.
You can get more detail here
I'm not sure what's going on here. I have set up an API route in NextJS that returns before the data has been loaded. Can anyone point out any error here please?
I have this function that calls the data from makeRequest():
export async function getVendors() {
const vendors = await makeRequest(`Vendor.json`);
console.log({ vendors });
return vendors;
}
Then the route: /api/vendors.js
export default async (req, res) => {
const response = await getVendors();
return res.json(response);
};
And this is the makeRequest function:
const makeRequest = async (url) => {
// Get Auth Header
const axiosConfig = await getHeader();
// Intercept Rate Limited API Errors & Retry
api.interceptors.response.use(
function (response) {
return response;
},
async function (error) {
await new Promise(function (res) {
setTimeout(function () {
res();
}, 2000);
});
const originalRequest = error.config;
if (error.response.status === 401 && !originalRequest._retry) {
token[n] = null;
originalRequest._retry = true;
const refreshedHeader = await getHeader();
api.defaults.headers = refreshedHeader;
originalRequest.headers = refreshedHeader;
return Promise.resolve(api(originalRequest));
}
return Promise.reject(error);
}
);
// Call paginated API and return number of requests needed.
const getQueryCount = await api.get(url, axiosConfig).catch((error) => {
throw error;
});
const totalItems = parseInt(getQueryCount.data['#attributes'].count);
const queriesNeeded = Math.ceil(totalItems / 100);
// Loop through paginated API and push data to dataToReturn
const dataToReturn = [];
for (let i = 0; i < queriesNeeded; i++) {
setTimeout(async () => {
try {
const res = await api.get(`${url}?offset=${i * 100}`, axiosConfig);
console.log(`adding items ${i * 100} through ${(i + 1) * 100}`);
const { data } = res;
const arrayName = Object.keys(data)[1];
const selectedData = await data[arrayName];
selectedData.map((item) => {
dataToReturn.push(item);
});
if (i + 1 === queriesNeeded) {
console.log(dataToReturn);
return dataToReturn;
}
} catch (error) {
console.error(error);
}
}, 3000 * i);
}
};
The issue that I'm having is that getVendors() is returned before makeRequest() has finished getting the data.
Looks like your issue stems from your use of setTimeout. You're trying to return the data from inside the setTimeout call, and this won't work for a few reasons. So in this answer, I'll go over why I think it's not working as well as a potential solution for you.
setTimeout and the event loop
Take a look at this code snippet, what do you think will happen?
console.log('start')
setTimeout(() => console.log('timeout'), 1000)
console.log('end')
When you use setTimeout, the inner code is pulled out of the current event loop to run later. That's why end is logged before the timeout.
So when you use setTimeout to return the data, the function has already ended before the code inside the timeout even starts.
If you're new to the event loop, here's a really great talk: https://youtu.be/cCOL7MC4Pl0
returning inside setTimeout
However, there's another fundamental problem here. And it's that data returned inside of the setTimeout is the return value of the setTimeout function, not your parent function. Try running this, what do you think will happen?
const foo = () => {
setTimeout(() => {
return 'foo timeout'
}, 1000)
}
const bar = () => {
setTimeout(() => {
return 'bar timeout'
}, 1000)
return 'bar'
}
console.log(foo())
console.log(bar())
This is a result of a) the event loop mentioned above, and b) inside of the setTimeout, you're creating a new function with a new scope.
The solution
If you really need the setTimeout at the end, use a Promise. With a Promise, you can use the resolve parameter to resolve the outer promise from within the setTimeout.
const foo = () => {
return new Promise((resolve) => {
setTimeout(() => resolve('foo'), 1000)
})
}
const wrapper = async () => {
const returnedValue = await foo()
console.log(returnedValue)
}
wrapper()
Quick note
Since you're calling the setTimeout inside of an async function, you will likely want to move the setTimeout into it's own function. Otherwise, you are returning a nested promise.
// don't do this
const foo = async () => {
return new Promise((resolve) => resolve(true))
}
// because then the result is a promise
const result = await foo()
const trueResult = await result()
I need to create a chart based on the GET request.
For this I have 3 functions which should be executed in the following order:
1 - EventListener to trigger the function that draws the charts.
2 - The function that requests the object to draw the chart.
3 - The function that draws the chart.
For some reason I can't understand, the function that draws the chart seems to be ready before the GET request.
I have tried giving Async/Await to both the function that draws the charts and the function that request the object. But Iam doing something wrong :
searchForm.addEventListener("submit" , function(event)
{
event.preventDefault(); // Evita a submissão normal do formulário
changeLogoImage(searchBar.value);
giveCompanieSymbol(searchBar.value);
turnDisplayBlock(mainContainer);
console.log('This should come first')
displayChart();
}
)
async function displayChart()
{
await objectForChart();
console.log('This should come last');
let ano = arrayOfMonths.slice(0,10)
var chart = new Chart(ctx, {
(...)
}
async function objectForChart()
{
let http = new XMLHttpRequest();
http.open('GET' , 'url');
http.send();
http.onreadystatechange = function()
{
if (http.readyState == XMLHttpRequest.DONE)
{
let responseObject = JSON.parse(http.response);
for(let month in responseObject['Monthly Time Series'] )
{
arrayOfMonths.push(month);
}
console.log('This should come 2nd');
}
}
}
The console gives me the logs in the following order :
-This should come first
-This should come last
-This should come second
you need to await the request in objectForChart. try using fetch (promise based) instead of XMLHttpRequest (callback based).
async function objectForChart() {
const data = await fetch(url);
return data.json();
}
I've got 3 functions.
Cron job lambda function
Event driven function which detects when a new record is added to the DynamoDB
A reusable function which is currently called by the 2 above functions
The Cron job function
export async function scheduledFunction() {
const detailsHistory = await sharedFunction(param1);
}
The event driven function
export async function eventFunction(event) {
event.Records.forEach(async record => {
if (record.eventName === 'INSERT') {
await sharedFunction(param1)
}
}
}
The function called by both of the event and scheduled function
const sharedFunction = async (param1) {
const apiUrl = 'xxxxxx';
const details = await axios.get(apiUrl, {
headers: {
'x-api-key': xxxx
}
});
}
The event function works when the DynamoDB has a new insert and then calls the 3rd party API which works as expected
The scheduled function fires every 4 hours and is works and gets to the sharedFunction, but when its gets to the API call await axios.get it just does nothing, I'm not getting any errors in the CloudWatch. I've placed console.logs() before and after the call and it logs the one before but nothing after.
You should always put async code inside try ... catch block. Also forEach won't work with promise you will need to use for loop. Try this:
export async function eventFunction(event) {
try {
for (let record of event.Records) {
if (record.eventName === 'INSERT') {
await sharedFunction(param1)
}
}
}
catch (err) {
console.log(err);
return err;
}
}
Shared function:
const sharedFunction = async (param1) => {
try {
const apiUrl = 'xxxxxx';
return await axios.get(apiUrl, {
headers: {
'x-api-key': xxxx
}
});
}
catch (err) {
return err;
}
}