I'm looping through several items and making an ajax request for each of them (using jQuery). I want them to execute independently, but populate into the DOM in the order they were called, not the order they are returned (for some reason some requests are taking longer than others). Any tips on the best practice for this type of thing?
Well the results can come back in any undefined order, they are asynchronous and subject to the vagaries of the internet and servers.
What you can do is deal with the problem in the same way TCP does over UDP. You use sequence identifiers.
Keep a sequence identifier going, and increment it every time you send out a request. As requests come back, check them off in order and only process them as they come in. Keep a list of what has returned with the data in order, and have a routine fire to check that list after each update to it. When the first expected is in, it should process the whole list down to the first gap.
Bare in mind that you could lose a request, so a suitable timeout before you ignore a given sequence identifier would be in order.
The answer to this ended up being a jQuery plugin called ajaxManager. This did exactly what I needed:
https://github.com/aFarkas/Ajaxmanager
You could send all the success result objects to a queue. Have an index that was sent with the original request, and continually check that queue for the next index.
But generally browsers only allow two simultaneous ajax requests, so it might be worth it to just send the next ajax request on success of the previous request.
Here's a start at the code:
var results = {}, lastProcessedIndex = 0;
var totalLength = $('a.myselector').each(function(el, index){
$.ajax({
url: $(this).attr('href'),
success: function(result){
results[index] = result; // add to results object
}
});
}).length;
var intervalId = setInterval(function(){
if(results[lastProcessedIndex]){
// use object
lastProcessedIndex++;
}
else if(totalLength == lastProcessedIndex){
clearInterval(intervalId);
}
}, 1000); // every 1 second
I'll be taking a stab in the dark with this one but it might help. Maybe you could create a global buffer array and then whenever the AJAX returns you can add the result to the buffer. You could then set up a timer that, when triggered, will check the contents of the buffer. If they are in order it will output it accordingly.
Related
I am using cypress to test our web application.
In certain pages there are different endpoint requests that are executed multiple times. [ e.g. GET /A GET /B GET /A].
What would be the best practise in cypress in order to wait for all requests to finish and guarantee that page has been fully loaded.
I don't want to use a ton cy.wait() commands to wait for all request to be processed. (there are a lot of different sets of requests in each page)
You can use the cy.route() feature from cypress. Using this you can intercept all your Get requests and wait till all of them are executed:
cy.server()
cy.route('GET', '**/users').as('getusers')
cy.visit('/')
cy.wait('#getusers')
I'm sure this is not recommended practice but here's what I came up with. It effectively waits until there's no response for a certain amount of time:
function debouncedWait({ debounceTimeout = 3000, waitTimeout = 4000 } = {}) {
cy.intercept('/api/*').as('ignoreMe');
let done = false;
const recursiveWait = () => {
if (!done) {
// set a timeout so if no response within debounceTimeout
// send a dummy request to satisfy the current wait
const x = setTimeout(() => {
done = true; // end recursion
fetch('/api/blah');
}, debounceTimeout);
// wait for a response
cy.wait('#ignoreMe', { timeout: waitTimeout }).then(() => {
clearTimeout(x); // cancel this wait's timeout
recursiveWait(); // wait for the next response
});
}
};
recursiveWait();
}
According to Cypress FAQ there is no definite way. But I will share some solutions I use:
Use the JQuery sintax supported by cypress
$('document').ready(function() {
//Code to run after it is ready
});
The problem is that after the initial load - some action on the page can initiate a second load.
Select an element like an image or select and wait for it to load. The problem with this method is that some other element might need more time.
Decide on a maindatory time you will wait for the api requests (I personaly use 4000 for my app) and place a cy.wait(mandatoryWaitTime) where you need your page to be loaded.
I faced the same issue with our large Angular application doing tens of requests as you navigate through it.
At first I tried what you are asking: to automatically wait for all requests to complete. I used https://github.com/bahmutov/cypress-network-idle as suggested by #Xiao Wang in this post. This worked and did the job, but I eventually realized I was over-optimizing my tests. Tests became slow. Test was waiting for all kinds of calls to finish, even those that weren't needed at that point in time to finish (like 3rd party analytics etc).
So I'd suggest not trying to wait for everything at a step, but instead finding the key API calls (you don't need to know the full path, even api/customers is enough) in your test step, use cy.intercept() and create an alias for it. Then use cy.wait() with your alias. The result is that you are waiting only when needed and only for the calls that really matter.
// At this point, there are lots of GET requests that need to finish in order to continue the test
// Intercept calls that contain a GET request with a request path containing /api/customer/
cy.intercept({ method: 'GET', url: '**/api/customer/**' }).as("customerData");
// Wait for all the GET requests with path containing /api/customer/ to complete
cy.wait("#customerData");
// Continue my test knowing all requested data is available..
cy.get(".continueMyTest").click()
Dependent on a selection, made on a website, plots are generated on a server where Flask runs.
The issue that I have is that generating e.g. 10 different plots can take up to 30s. What I like to achieve is to start updating the website as soon as the first plot is ready and then load automatically the others as soon as they are ready.
Currently, the following AJAX function is executed as soon as the user hits the "process" button on the website:
$.ajax({
type: "POST",
url: "/single",
data: { athleteName1: $('#athleteName1').val(), style: $('#style').val()},
success: function (results) {
$('#results').empty().append(results);
$('#results').show();
$('#submitbutton').prop('disabled', false);
},
error: function (error) {
console.log(error);
}
});
On the server site, plots are created and embedded in div-containers. They are subsequently concatenated an returned to the website at once as "diagStr":
#app.route('/single', methods=['POST', 'GET'])
def single():
loop 10 times:
diagStr += generate_plot()
return Markup(diagStr)
Doing it with "Streaming Contents" can only be part of the solution as AJAX waits until the the entire response is received.
Any idea how this is solved with today's technology?
There are multiple way you could achieve this, but some simple examples:
do your looping on the client side, and generate 10 separate Ajax requests, updating the web page when each response is received.
if you don't know in advance on the client side, how many loops you will have, then use a single request and have the server send the response as soon as the first loop is complete, along with a flag indicating whether there are more loops or not - the client can look at this flag and create a new Ajax request if there are more loops.
TLDR;
I managed to simplify my question after a good night's sleep. Here's the simpler question.
I want to upload N files to a server, which would process them together and return a single response (e.g. Total foobars in all files combined = XYZ).
What's the best way to send this single response back to the client?
Thanks.
&
Below is the old question, left behind as a lesson for me.
I'm using Dropzone.js to build D&D functionality into my app.
Please note: I know there are a couple of questions already that discuss multifile uploads. But they are different from my question. They talk about how to get a single callback call instead of multiple ones.
My issue is related to the situation where I drag and drop multiple files into the dropzone, but am seeing the single server response being duplicated multiple times. Here is my config:
Dropzone.options.inner = {
init: function() {
this.on("dragenter", function(e) {
$('#inner').addClass('drag-over');
//// TODO - find out WTF this isn't working (low priority)
}),
this.on("completemultiple", function(file, resp) {
//// TODO
})
},
url: "php/...upload...php",
timeout: 120000, // 2m
uploadMultiple: true,
autoProcessQueue: false,
clickable: false,
};
//// ... Some other stuff
//// ...
$(document).ready(function() {
$('#inner').click(function() {
Dropzone.forElement('.dropzone').processQueue();
});
In the beginning I intercepted the "complete" event, rather than "completemultiple". That resulted in its handler being invoked multiple separate times (once for each file), even though the server-side php was only being invoked once. Each invocation returned a duplicate copy of the same server-side message.
I didn't want that, so I changed it to "completemultiple", and now I can confirm that the handler only gets called once with an array of files, but the single server response is now buried within each file object returned - each has a duplicate copy of the exact same response.
It doesn't matter ultimately because it is the same message, after all. But the whole esthetics of the thing now seems off which indicates to me I'm doing something wrong - the response seems to indicate two independent uploads, but they were part of a single invocation of the server side php. Why make the client "believe" there were two separate upload requests when the server-side script only has one opportunity to respond (i.e. The php is not sending back different messages for each file - should it? And if so, what's the best way to do it?)
How can I make it so that if I have a scenario in which it's all-or-none, I get a single response back from the php script?
This is especially important to me because my server response will contain the status and some other data. The script does more than simply receiving the uploaded files (hence the longer timeout).
I thought maybe that's a sign that I should separate the uploading part from the processing part and trigger the processing once the upload is complete.
But that means that the server side upload script can't clean up after itself. It needs to persist data beyond its own life. Also it now needs to return a handle to this data back to the client, which would dispatch the server-side processor in a different ajax call passing it this handle - and the subsequent call needs to clean up the files left by the uploader after it is done processing them.
This seems the less elegant solution. Is this something I just need to get used to? Or is there a better way of accomplishing what I want?
Also, any other free tips and hints from the front-end gurus in my network will be gratefully accepted.
Thanks.
&
The following approach works. Until something better can be found.
Dropzone.options.inner = {
// . . .
init: function() {
this.on("completemultiple", function(file) {
var code = JSON.parse(file[0].xhr.response).code;
var data = { "code" : code };
$.post('php/......php', data, function(res) {
// TODO - surface the res back to the user
});
})
},
};
&
I want to call 1 million different URLs with AJAX.
What I did is (Javascript, jQuery used):
var numbers = [1, 2, 3...1000000]; // numbers.length = 1000000
$(function () {
$.each(numbers, function(key, val) {
$.ajax({
url: '/getter.php',
data: { id: val},
success: function () {
console.info(id);
}
});
});
});
I loop over 1 million integers, and passing them to my getter.php (which is doing something cool with that numbers).
The problem is after ~1,5k of requests I get Google Chrome dead.
I know I do this ineffective way, that's why am asking for help - how to actually do it right? How to GET request a php script 1 million times (not necessary with JavaScript!)?
You could use a persistent connection between your php script and the client requesting the data.
I think you are bumping into a limitation of the time to live on the single request you are calling. Also the HTTP requests functions as a Request - Response why, when you put it in your foreach statement, each single statement is being processed alone like:
1. iteration: GET /getter.php with value 1 .... wait... Oh theres a response
2. iteration GET /getter.php with value 2 .... wait Oh another response
This is a seemingly long and wrong proces as you might already have figured out.
Another approach would be to set up a persistent socket, which functions like the TCP procotol:
1. open the connection
2. send all the data
3. close the connection
Have you considered trying with a websocket?
Heres a few tutorials:
HTML5websocket
http://www.tutorialspoint.com/html5/html5_websocket.htm
PHP socket:
http://www.phpbuilder.com/articles/application-architecture/optimization/creating-real-time-applications-with-php-and-websockets.html
EDIT:
Also see this article on the difference between AJAX and websocket.
"AJAX is great if you aren’t in a hurry, but if you’re moving a high volume of data then the overhead of creating an HTTP connection every time is going to be a bottleneck. You need a persistent connection instead. In addition, AJAX always has to poll the server for data rather than receive it via push from the server. If you want speed and efficiency you need WebSockets."
http://blog.safe.com/2014/08/websockets-ajax-webhooks-comparison/
I am new to angular and want to use it to send data to my app's backend. In several occasions, I have to make several http post calls that should either all succeed or all fail. This is the scenario that's causing me a headache: given two http post calls, what if one call succeeds, but the other fails? This will lead to inconsistencies in the database. I want to know if there's a way to cancel the succeeding calls if at least one call has failed. Thanks!
Without knowing more about your specific situation I would urge you to use the promise error handling if you are not already doing so. There's only one situation that I know you can cancel a promise that has been sent is by using the timeout option in the $http(look at this SO post), but you can definitely prevent future requests. What happens when you make a $http call is that it returns a promise object(look at $q here). What this does is it returns two methods that you can chain on your $http request called success and failure so it looks like $http.success({...stuff...}).error({...more stuff..}). So if you do have error handling in each of these scenarios and you get a .error, dont make the next call.
You can cancel the next requests in the chain, but the previous ones have already been sent. You need to provide the necessary backend functionality to reverse them.
If every step is dependent on the other and causes changes in your database, it might be better to do the whole process in the backend, triggered by a single "POST" request. I think it is easier to model this process synchronously, and that is easier to do in the server than in the client.
However, if you must do the post requests in the client side, you could define each request step as a separate function, and chain them via then(successCallback, errorCallback) (Nice video example here: https://egghead.io/lessons/angularjs-chained-promises).
In your case, at each step you can check if the previous one failed an take action to reverse it by using the error callback of then:
var firstStep = function(initialData){
return $http.post('/some/url', data).then(function(dataFromServer){
// Do something with the data
return {
dataNeededByNextStep: processedData,
dataNeededToReverseThisStep: moreData
}
});
};
var secondStep = function(dataFromPreviousStep){
return $http.post('/some/other/url', data).then(function(dataFromServer){
// Do something with the data
return {
dataNeededByNextStep: processedData,
dataNeededToReverseThisStep: moreData
}
}, function(){
// On error
reversePreviousStep(dataFromPreviousStep.dataNeededToReverseThisStep);
});
};
var thirdFunction = function(){ ... };
...
firstFunction(initialData).then(secondFunction)
.then(thirdFunction)
...
If any of the steps in the chain fails, it's promise would fail, and next steps will not be executed.