Dependent on a selection, made on a website, plots are generated on a server where Flask runs.
The issue that I have is that generating e.g. 10 different plots can take up to 30s. What I like to achieve is to start updating the website as soon as the first plot is ready and then load automatically the others as soon as they are ready.
Currently, the following AJAX function is executed as soon as the user hits the "process" button on the website:
$.ajax({
type: "POST",
url: "/single",
data: { athleteName1: $('#athleteName1').val(), style: $('#style').val()},
success: function (results) {
$('#results').empty().append(results);
$('#results').show();
$('#submitbutton').prop('disabled', false);
},
error: function (error) {
console.log(error);
}
});
On the server site, plots are created and embedded in div-containers. They are subsequently concatenated an returned to the website at once as "diagStr":
#app.route('/single', methods=['POST', 'GET'])
def single():
loop 10 times:
diagStr += generate_plot()
return Markup(diagStr)
Doing it with "Streaming Contents" can only be part of the solution as AJAX waits until the the entire response is received.
Any idea how this is solved with today's technology?
There are multiple way you could achieve this, but some simple examples:
do your looping on the client side, and generate 10 separate Ajax requests, updating the web page when each response is received.
if you don't know in advance on the client side, how many loops you will have, then use a single request and have the server send the response as soon as the first loop is complete, along with a flag indicating whether there are more loops or not - the client can look at this flag and create a new Ajax request if there are more loops.
I'm trying to synchronize my POSTs to an endpoint in Angular. I did see some examples of doing a synchronized GET but had trouble understanding the examples well enough to apply them to POSTs.
The POSTs are pretty simple, at least from my perspective as the front-end developer. I send an object with an parent group ID and a sub group ID to a /parentgroups endpoint. On the backend, however, async calls cause the data to get overwritten.
Apologies for lack of an example, but I am pretty far from having one that's close to working how I need. My code is still async and a loop over calls to $http.post().
You actually cannot do real synchronous (as in blocking) http calls in Angular, it forces you do use async. If you can't do it with callbacks then you have a problem with your architecture that the entire team should focus on solving ASAP. If your current architecture requires the frontend to do blocking calls then your architecture is quite simply broken and needs to be fixed.
Anyway, while I recommend against it you could always register your request in a list, and then in each callback you pop the next request from the list and run it. That way you can just keep pushing requests into the list without knowing how many there will be. Something like this (untested, but the general principle should work):
var requestList = [];
requestList.push(function() {
$http.post('/someUrl', {})
.success(function(data, status, headers, config) {
// Remove the next request from list and call it
requestList.shift()();
});
});
requestList.push(function() {
$http.post('/someOtherUrl', {})
.success(function(data, status, headers, config) {
// Remove the next request from list and call it
requestList.shift()();
});
});
// Start the first request
requestList.shift()();
This is fairly clean, but still a bit of a hack. It would probably work fine but I would be taking a good long look at why the API forces you to do something like this.
I am new to angular and want to use it to send data to my app's backend. In several occasions, I have to make several http post calls that should either all succeed or all fail. This is the scenario that's causing me a headache: given two http post calls, what if one call succeeds, but the other fails? This will lead to inconsistencies in the database. I want to know if there's a way to cancel the succeeding calls if at least one call has failed. Thanks!
Without knowing more about your specific situation I would urge you to use the promise error handling if you are not already doing so. There's only one situation that I know you can cancel a promise that has been sent is by using the timeout option in the $http(look at this SO post), but you can definitely prevent future requests. What happens when you make a $http call is that it returns a promise object(look at $q here). What this does is it returns two methods that you can chain on your $http request called success and failure so it looks like $http.success({...stuff...}).error({...more stuff..}). So if you do have error handling in each of these scenarios and you get a .error, dont make the next call.
You can cancel the next requests in the chain, but the previous ones have already been sent. You need to provide the necessary backend functionality to reverse them.
If every step is dependent on the other and causes changes in your database, it might be better to do the whole process in the backend, triggered by a single "POST" request. I think it is easier to model this process synchronously, and that is easier to do in the server than in the client.
However, if you must do the post requests in the client side, you could define each request step as a separate function, and chain them via then(successCallback, errorCallback) (Nice video example here: https://egghead.io/lessons/angularjs-chained-promises).
In your case, at each step you can check if the previous one failed an take action to reverse it by using the error callback of then:
var firstStep = function(initialData){
return $http.post('/some/url', data).then(function(dataFromServer){
// Do something with the data
return {
dataNeededByNextStep: processedData,
dataNeededToReverseThisStep: moreData
}
});
};
var secondStep = function(dataFromPreviousStep){
return $http.post('/some/other/url', data).then(function(dataFromServer){
// Do something with the data
return {
dataNeededByNextStep: processedData,
dataNeededToReverseThisStep: moreData
}
}, function(){
// On error
reversePreviousStep(dataFromPreviousStep.dataNeededToReverseThisStep);
});
};
var thirdFunction = function(){ ... };
...
firstFunction(initialData).then(secondFunction)
.then(thirdFunction)
...
If any of the steps in the chain fails, it's promise would fail, and next steps will not be executed.
I have read some posts about this topic and the answers are comet, reverse ajax, http streaming, server push, etc.
How does incoming mail notification on Gmail works?
How is GMail Chat able to make AJAX requests without client interaction?
I would like to know if there are any code references that I can follow to write a very simple example. Many posts or websites just talk about the technology. It is hard to find a complete sample code. Also, it seems many methods can be used to implement the comet, e.g. Hidden IFrame, XMLHttpRequest. In my opinion, using XMLHttpRequest is a better choice. What do you think of the pros and cons of different methods? Which one does Gmail use?
I know it needs to do it both in server side and client side.
Is there any PHP and Javascript sample code?
The way Facebook does this is pretty interesting.
A common method of doing such notifications is to poll a script on the server (using AJAX) on a given interval (perhaps every few seconds), to check if something has happened. However, this can be pretty network intensive, and you often make pointless requests, because nothing has happened.
The way Facebook does it is using the comet approach, rather than polling on an interval, as soon as one poll completes, it issues another one. However, each request to the script on the server has an extremely long timeout, and the server only responds to the request once something has happened. You can see this happening if you bring up Firebug's Console tab while on Facebook, with requests to a script possibly taking minutes. It is quite ingenious really, since this method cuts down immediately on both the number of requests, and how often you have to send them. You effectively now have an event framework that allows the server to 'fire' events.
Behind this, in terms of the actual content returned from those polls, it's a JSON response, with what appears to be a list of events, and info about them. It's minified though, so is a bit hard to read.
In terms of the actual technology, AJAX is the way to go here, because you can control request timeouts, and many other things. I'd recommend (Stack overflow cliche here) using jQuery to do the AJAX, it'll take a lot of the cross-compability problems away. In terms of PHP, you could simply poll an event log database table in your PHP script, and only return to the client when something happens? There are, I expect, many ways of implementing this.
Implementing:
Server Side:
There appear to be a few implementations of comet libraries in PHP, but to be honest, it really is very simple, something perhaps like the following pseudocode:
while(!has_event_happened()) {
sleep(5);
}
echo json_encode(get_events());
The has_event_happened function would just check if anything had happened in an events table or something, and then the get_events function would return a list of the new rows in the table? Depends on the context of the problem really.
Don't forget to change your PHP max execution time, otherwise it will timeout early!
Client Side:
Take a look at the jQuery plugin for doing Comet interaction:
Project homepage: http://plugins.jquery.com/project/Comet
Google Code: https://code.google.com/archive/p/jquerycomet/ - Appears to have some sort of example usage in the subversion repository.
That said, the plugin seems to add a fair bit of complexity, it really is very simple on the client, perhaps (with jQuery) something like:
function doPoll() {
$.get("events.php", {}, function(result) {
$.each(result.events, function(event) { //iterate over the events
//do something with your event
});
doPoll();
//this effectively causes the poll to run again as
//soon as the response comes back
}, 'json');
}
$(document).ready(function() {
$.ajaxSetup({
timeout: 1000*60//set a global AJAX timeout of a minute
});
doPoll(); // do the first poll
});
The whole thing depends a lot on how your existing architecture is put together.
Update
As I continue to recieve upvotes on this, I think it is reasonable to remember that this answer is 4 years old. Web has grown in a really fast pace, so please be mindful about this answer.
I had the same issue recently and researched about the subject.
The solution given is called long polling, and to correctly use it you must be sure that your AJAX request has a "large" timeout and to always make this request after the current ends (timeout, error or success).
Long Polling - Client
Here, to keep code short, I will use jQuery:
function pollTask() {
$.ajax({
url: '/api/Polling',
async: true, // by default, it's async, but...
dataType: 'json', // or the dataType you are working with
timeout: 10000, // IMPORTANT! this is a 10 seconds timeout
cache: false
}).done(function (eventList) {
// Handle your data here
var data;
for (var eventName in eventList) {
data = eventList[eventName];
dispatcher.handle(eventName, data); // handle the `eventName` with `data`
}
}).always(pollTask);
}
It is important to remember that (from jQuery docs):
In jQuery 1.4.x and below, the XMLHttpRequest object will be in an
invalid state if the request times out; accessing any object members
may throw an exception. In Firefox 3.0+ only, script and JSONP
requests cannot be cancelled by a timeout; the script will run even if
it arrives after the timeout period.
Long Polling - Server
It is not in any specific language, but it would be something like this:
function handleRequest () {
while (!anythingHappened() || hasTimedOut()) { sleep(2); }
return events();
}
Here, hasTimedOut will make sure your code does not wait forever, and anythingHappened, will check if any event happend. The sleep is for releasing your thread to do other stuff while nothing happens. The events will return a dictionary of events (or any other data structure you may prefer) in JSON format (or any other you prefer).
It surely solves the problem, but, if you are concerned about scalability and perfomance as I was when researching, you might consider another solution I found.
Solution
Use sockets!
On client side, to avoid any compatibility issues, use socket.io. It tries to use socket directly, and have fallbacks to other solutions when sockets are not available.
On server side, create a server using NodeJS (example here). The client will subscribe to this channel (observer) created with the server. Whenever a notification has to be sent, it is published in this channel and the subscriptor (client) gets notified.
If you don't like this solution, try APE (Ajax Push Engine).
Hope I helped.
According to a slideshow about Facebook's Messaging system, Facebook uses the comet technology to "push" message to web browsers. Facebook's comet server is built on the open sourced Erlang web server mochiweb.
In the picture below, the phrase "channel clusters" means "comet servers".
Many other big web sites build their own comet server, because there are differences between every company's need. But build your own comet server on a open source comet server is a good approach.
You can try icomet, a C1000K C++ comet server built with libevent. icomet also provides a JavaScript library, it is easy to use as simple as:
var comet = new iComet({
sign_url: 'http://' + app_host + '/sign?obj=' + obj,
sub_url: 'http://' + icomet_host + '/sub',
callback: function(msg){
// on server push
alert(msg.content);
}
});
icomet supports a wide range of Browsers and OSes, including Safari(iOS, Mac), IEs(Windows), Firefox, Chrome, etc.
Facebook uses MQTT instead of HTTP. Push is better than polling.
Through HTTP we need to poll the server continuously but via MQTT server pushes the message to clients.
Comparision between MQTT and HTTP: http://www.youtube.com/watch?v=-KNPXPmx88E
Note: my answers best fits for mobile devices.
One important issue with long polling is error handling.
There are two types of errors:
The request might timeout in which case the client should reestablish the connection immediately. This is a normal event in long polling when no messages have arrived.
A network error or an execution error. This is an actual error which the client should gracefully accept and wait for the server to come back on-line.
The main issue is that if your error handler reestablishes the connection immediately also for a type 2 error, the clients would DOS the server.
Both answers with code sample miss this.
function longPoll() {
var shouldDelay = false;
$.ajax({
url: 'poll.php',
async: true, // by default, it's async, but...
dataType: 'json', // or the dataType you are working with
timeout: 10000, // IMPORTANT! this is a 10 seconds timeout
cache: false
}).done(function (data, textStatus, jqXHR) {
// do something with data...
}).fail(function (jqXHR, textStatus, errorThrown ) {
shouldDelay = textStatus !== "timeout";
}).always(function() {
// in case of network error. throttle otherwise we DOS ourselves. If it was a timeout, its normal operation. go again.
var delay = shouldDelay ? 10000: 0;
window.setTimeout(longPoll, delay);
});
}
longPoll(); //fire first handler
I'm looping through several items and making an ajax request for each of them (using jQuery). I want them to execute independently, but populate into the DOM in the order they were called, not the order they are returned (for some reason some requests are taking longer than others). Any tips on the best practice for this type of thing?
Well the results can come back in any undefined order, they are asynchronous and subject to the vagaries of the internet and servers.
What you can do is deal with the problem in the same way TCP does over UDP. You use sequence identifiers.
Keep a sequence identifier going, and increment it every time you send out a request. As requests come back, check them off in order and only process them as they come in. Keep a list of what has returned with the data in order, and have a routine fire to check that list after each update to it. When the first expected is in, it should process the whole list down to the first gap.
Bare in mind that you could lose a request, so a suitable timeout before you ignore a given sequence identifier would be in order.
The answer to this ended up being a jQuery plugin called ajaxManager. This did exactly what I needed:
https://github.com/aFarkas/Ajaxmanager
You could send all the success result objects to a queue. Have an index that was sent with the original request, and continually check that queue for the next index.
But generally browsers only allow two simultaneous ajax requests, so it might be worth it to just send the next ajax request on success of the previous request.
Here's a start at the code:
var results = {}, lastProcessedIndex = 0;
var totalLength = $('a.myselector').each(function(el, index){
$.ajax({
url: $(this).attr('href'),
success: function(result){
results[index] = result; // add to results object
}
});
}).length;
var intervalId = setInterval(function(){
if(results[lastProcessedIndex]){
// use object
lastProcessedIndex++;
}
else if(totalLength == lastProcessedIndex){
clearInterval(intervalId);
}
}, 1000); // every 1 second
I'll be taking a stab in the dark with this one but it might help. Maybe you could create a global buffer array and then whenever the AJAX returns you can add the result to the buffer. You could then set up a timer that, when triggered, will check the contents of the buffer. If they are in order it will output it accordingly.