I have a Perl script that generates a web page. It takes a non-trivial amount of time to run. I would like to be able to render a complete HTML table to the user so they know what results to expect, but fill in the details slowly as the Perl script generates them.
What approach should I be taking here?
My initial assumption was that I would be able to assign an ID to my various table data elements and then adjust their innerHTML properties as and when I got the results in. But it doesn't seem like I can perform such manipulations whilst the page is still loading.
There's no consistantly reliable way to modify a web page as it's loading.
You can create the effect by initially loading a compact loading page, and then loading the rest of the content via AJAX calls back to the server to get the individual components.
You can then load those components as your AJAX calls are completed.
EDIT
As the comments have pointed out...while this would achieve the results you want, it's a terrible idea.
Search Engine Indexing being the primary reason. You're also relying on Javascript to do a lot of heavy lifting...and it might not always be enabled.
One solution would be to progressively load the data via AJAX. You would need to do something like this:
Load the webpage
Using javascript, query the webserver for table values
Populate table with received values
Loop until table filled
Obviously this solution presents problems if the data is meant to be crawled. Since crawlers don't take into account dynamic data via javascript.
The other issue to consider is usability. Web Users are not used to this type of progressive loading, so informing them that the data is still being loaded would be very important. Also some type of accurate progress bar would provide good usability.
As suggested, AJAX is probably the way to go.
Create a basic HTML page with an empty div to hold your data, then using repeating AJAX calls, fill in the div.
This page describes how to do this:
link text
Related
I'm generating a form from JSON data that I load via $http.get(), hence use bunch of custom/3rd party directives (ui-select, Bootstrap UI, ...) to get the desired end result. Just to make things more interesting, forms are nested, and with ng-repeat, things still feel pretty sloppy, especially on mobile. Form is quite lengthy and I've split it in several sections, so putting ng-if and displaying one section at a time, as well as using bindonce does improve performance a bit, but not to the extent that I find suitable from UX POV.
The catch 22 is that underlying JSON data is unlikely to change, so ideally I'd like to go with the sloppy version in development, but in production I'd like to build/compile the form, and make it load faster.
I know that 3rd party libraries (namely ui-select) introduce bottleneck, but apart from using $templateCache with $compile in app.run() section, or rendering the form with templating engine such as ejs, what other tweaks should I take into consideration in order to improve performance?
U can also go for caching of data using IndexedDb or Local Storage to cache the JSON Data which can result to load form more faster.
The title pretty much sums up my question.
When is it more efficient to generate a static page, that a user can access, as apposed to using dynamically generated pages that query a database? As in what situations would one be better than the other.
To serve up a static page, your web server just needs to read the page off the disk and send it. Virtually no processing will be required. If the page is frequently accessed, it will probably be cached in memory, so even the disk access will not be needed.
Generating pages dynamically obviously has more overhead. There is a cost for every DB access you make, no matter how simple the query is. (On a project I worked on recently, I measured a minimum overhead of 0.7ms for each query, even for SELECT 1;) So if you can just generate a static page and save it to disk, page accesses will be faster. How much faster? It just depends on how much work is being done to generate the page dynamically. We don't know what you are doing, so we can't comment on that.
Now, if you generate a static page and save it to disk, that means you need to re-generate it every time the data which went into generating that page changes. If the data changes more often than the page is actually accessed, you could be doing more work rather than less! But in most cases, that's a very unlikely situation.
More likely, the biggest problem you will experience from generating static pages and saving them to disk is coding (and maintaining) the logic for re-generating the pages whenever necessary. You will need to keep track of exactly what data goes into each page, and in every place in the code where data can be changed, you will need to invoke re-generation of all the relevant pages. If you forget just one, then your users may be looking at stale data some of the time.
If you mix dynamic generation per-request and caching generated pages on disk, then your code will be harder to read and maintain, because of mixing the two styles.
And you can't really cache generated pages on disk in certain situations -- like responding to POST requests which come from a form submission. Or imagine that when your users invoke certain actions, you have to send a request to a 3rd party API, and the data which comes back from that API will be used in the page. What comes back from the API may be different each time, so in this case, you need to generate the page dynamically each time.
Static pages (or better resources) are filled with content, that does not change or at least not often, and does not allow further queries on it: About Page, Contact, ...
In this case it doesn't make any sense to query these pages. On the other side we have Data (e.g. in a Database) and want to query it/give the user the opportunity to query it. In this case you give the User a page with the possibility to specify the query and return a rendered page with the dynamically generated data.
In my opinion it depends on the result you want to present to the user. Either it is only an information or it is the possibility to query a Datasource. The first result is known before you do something, the second (query data) is known after you have the query parameters, which means you don't know the result beforehand (it could be empty or invalid).
It depends on your architecture, but when you consider that GET Requests should be idempotent it should be also easy to cache dynamic Pages with a Proxy, and invalidate the cache, when something new happens to the data which is displayed on the cached path. In this case one could save a lot of time, because the system behaves like the cached pages would be static, but instead coming from the filesystem, they come from your memory, which is really fast.
Cheers
Laidback
I have a few perl CGIs which almost query the whole table with more than 5000 rows as result and send that data to browsers. The size of html data generated is around 1MB.
Earlier I was using tables(which should be ideal approach).
Unfortunately most users use IE and it does not display data till it receives closing table tag. Can we do something about it.
To push output as soon as its generated, I used another approach where in I was using printf and <pre>. Which reduced the response size by 200kb and and it appears more faster in display.
Again IE (not any other browser) eats up CPU and hangs for couple of seconds... :-( ..
Can we do something about it too.
FYI I am using IE8.
Perhaps it would be wise from a UX perspective to use some sort of pagination method. Having a single page with thousands upon thousands of rows sounds entirely unfriendly for the end user. Something like a simple means of pagination ("Skip to page =dropdown=") would certainly solve your problem, as well as decrease load times and increase usability.
There are also several solutions which are pre-built and would likely integrate rather easily. One which comes to mind almost immediately is Sencha's Paging Grid:
http://dev.sencha.com/deploy/dev/examples/grid/paging.html
http://www.sencha.com/products/js/
It's pretty nifty and you'll likely get some kudos for using a hip new technology. There are other options, too:
Ingrid: http://www.reconstrukt.com/ingrid/src/example3.html
YUI Data Grid: http://developer.yahoo.com/yui/datatable/#data
MyTableGrid: http://pabloaravena.info/mytablegrid/
Hope this helps!
More people use ie?
http://www.w3schools.com/browsers/browsers_stats.asp
anyways why do you still use tables?
we all know they are easy to maintain, easy to get confused and in your case slow...
output it as div which will show all the data and does not need to wait for a table tag to finish. as div and span have the least properties.
Is it better-practice to AJAX every form element separately (eg. send request onChange, etc) or collect all the data, then submit with 1 click save?
Essentially, auto-save or user-initiated-save?
I would generally say that a user-initiated save is the way to go for most web-applications. If for nothing else, this is how users are used to interacting with web apps; familiarity and ease of use is extremely important in web applications. Not to mention it can cut down on unnecessary traffic.
This is not to say that auto-saving does not have it's place, but often it can be cause unnecessary traffic. For example, if I am auto-saving a contact form, fill out my name, then email, then back to name to change it, that is already 3 requests that have been sent with no benefit - this is extra work for no added advantage.
Once again, I think it does have a lot to do with your application or where you are planning on using it. Inline edits are something that often uses auto-saving and there I think it is useful, whereas a contact form/signup form would not be a good idea.
I'd say that depends on the nature of your application and whether "auto-save" is a behaviour desired by your users.
"User initiated save" is what a user would expect from their experience with web forms nowadays - I would not deviate from that unless there's a good reason.
Depends on following factors:
What kind of data are you trying to save. E.g. is it okay to be able to save the data partly or you need to save it all at once?
How much data do you want to save? If you have many fields, you might want to send data in chunks (In case of wizards) or save everything at once
Its also a good idea to have data saved (in background) for large forms in a temp way if the user may take a long time to fill in the data (e.g. emails saved as drafts)
It also depends on your web app and the way you have designed your forms. In some forms you may allow certain fields to be modified and saved inplace, so that you can fetch additional data for example
In most cases it would be good to have an explicit "Save" action for your data forms
For context: this is an HTML app, with little or no browser side JavaScript. I can't easily change that so need to do this on the server.
CouchDB is built to not have side effects. This is fair enough. But there seems to be no method that i can conceive of with shows, views, lists to change what is shown to a user with subsequent requests, or based on user objects, without writing data.
And can a get request for document result in the creation of a new record? Im guessing not as that would be a side effect.
But if you can, you could just create a log and then have a view that picks an advert firm a set of documents describing adverts which is affected by the change in the log when a previous ad was shown.
I'm not actually going to show adverts on my site, I'm going to have tips, and article summaries and minor features that vary from page load to page load.
Any suggestions appreciated.
I've wrapped my head around how to work with the grain for the rest of the functionality I need, but this bit seems contrary to the way couchdb works.
I think you're going to need a list function that receives a set of documents from the view and then chooses only one to return, either at random or some other method. However, because you're inside a list function you gain access to the user's request details, including cookies (which you can also set, btw.) That sounds more like what you want.
In addition, you could specify different Views for the list function to use at query-time. This means you could, say, have only random articles show up on the homepage, but any type of content show up on all others.
Note: You can't get access to the request in a map/reduce function and you'll run into problems if you do something like Math.random() inside a map function.
So a list function is the way to go.
http://guide.couchdb.org/draft/transforming.html
Look into the various methods of selecting a random document from a view. That should enable you to choose a random document (presumably representing an ad, tip, etc.) to display.