,This question is likely subjective, but a lot of "grid" Javascript plugins have come out to help paginate and sort tables. They usually work in 2 ways, the first and simplest is that it takes an existing HTML <table> and converts it into a sortable and searchable information. The second is that it passes info to the server and has the server select info from the database to be displayed.
My question is this: At what point (size wise) is it more efficient to use server-side processing vs displaying all the data and have the "grid plugin" convert it to a sortable/searchable table client-side?
Using datatables as an example, I have to execute at least 3 queries to get total rows in the table, total filtered results for pagination, and the filtered results to be displayed for the specific selected page. Then every time I sort, I am querying again. Every time I move to another page, or search in the table, more queries.
If I was to pull the data once when the client visits the page, I would be executing a single query, and then formatting and pushing the results to the client all at once. This increases the page size, and possibly delays loading of the page once it gets too big. The upside is there will only one query, and all the sorting, searching, and pagination is handled by the plugin, so no waiting for a response and no more queries.
If I was to have just a few rows, I imagine just pushing the formatted table data to the client at the page load would be the fastest. But with thousands of rows, switching to server-side would be the most efficient way.
Where is the tipping point? Is there a tipping point, or is server-side or client-side the way to go 100% of the time?
The answer on your question can be only subjective. So I explain how I personally understand the problem and give me recommendation.
In my opinion the data with 2-3 row and 3-4 column can be displayed in HTML table without usage any plugin. The data you display for the user the more important will be that the user will be able to grasp the information which will be displayed. So I think that the information for example have to be good formatted and marked with colors and icons for example. This with help to grasp information from probably 10 rows of data, but not much more. If you just display table with 100 rows or more then you overtax the user. The user will have to analyse the data to get any helpful information from the table. Scrolling of the data makes this not easier.
So I think that one should give the user comfortable or at least convenient interface to sort and to filter the data from the table. The exact interface is mostly the matter of taste. For example the grid can have an additional filter bar
For filtering and even for sorting of the data it's important to have not pure strings, but to be able to distinguish the data types like integer (10 should be after 9 and not between 1 and 2), numbers (correct interpret '.' and ',' inside of numbers), dates (3/20/2012 should be grater as 4/15/2010) and so on. If you just convert HTML table to some grid you will have problems with correct filtering or sorting. Even if you use pure local JavaScript data to display in grid it would be important to have datasource which has some kind of type information and then to create the grid based in the data. In the case you can gives date as JavaScript Date or as ISO 8601 string "2012-03-20" and in the grid display the data corresponds the specified formatter as 3/20/2012 or 20-Mar-2012.
Whether you implement filtering, sorting and paging on the server side or on the client side is not really important for the user who open the page. It's important only that all works quickly enough. The exact choose of the grid plugin, the filtering (with filter toolbar or external controls) and styling of the grid depend on your taste and the project requirements.
Related
I am an occaisional cocoa programmer who needs to write an app to display a table of data. I need a push in the right direction so I can avoid blind alleys. Stock data would provide a reasonable example of what I need to do. The raw data is collected by another program and written to a file, let's say stock trading volume, open, high, low, and close price. I need to parse the input file data and display it, but also add data (columns) that are derived from the other entries (rows), like average volume and moving average close price. I need to be able to scroll through a large file, and be able to find things, like a particular date or close price greater than some value.
An NSTableView – dataSource gives my code execution control for each data item, which would allow me to calculate derived items on the fly. It also requires the most code.
NSArrayController wraps up a lot of code, but I don't know if my code would get execution control to parse the input data. I'm also fuzzy on how to dictate what range of rows gets displayed (by scrolling or by searching).
Core data seems to tighten things up even more, but seems geared toward data that is created, saved, and read by the application, rather than something that is saved by another application in a format that differs from what needs to be displayed.
Thanks for any guidance you may provide.
Scott
Binding MVC grid to stored procedure with large amount of data:
I want to bind MVC grid to an object result that returned from SP
Normally the grid request only data needed to be shown to the user and that will be very good when binding to a table with large amount of data that lets the grid fast and its performance would be good.
i have 2 ways to bind MVC grid with SP:
Bind to SP without using .ToList() it gives me an error "The result of a query can not be enumerated more than once."
Bind to sp using .ToList() will resolve that error but it will loads all records from database first and the performance will be bad and the grid loading, paging, sorting and filtering would be very slow.
Please tell me a solution to bind MVC grid with SP that returns large amount of data with good performance.
Thank you
I have used jqgrid in the past, jqgrid implements paging so not all the content of the grid is actually displayed at once, you pick the number of rows per page and jqgrid automatically will hook the navigation to your controller. There are a lot of examples of this on the web. If you click the next page it will retrieve the appropriate data for that page and so forth.
The jqgrid page has a lot of examples in loading data that ilustrate this: http://www.trirand.com/blog/jqgrid/jqgrid.html, here is another page that talks how that is implemented at the server side using mvc http://www.codersource.net/AspNet/ASPNetAdvanced/jqGridPaginginaspnetmvc.aspx, I'm sure if you look around you'll find a lot of information on how to go about this approach.
Finally I usually avoid showing the user a lot of data anyways, mainly because it is hard for a human been to make any sense of more than 100 rows of data having a way to search further in it. So I rather try and attempt to shrink the data size giving them a way to filter it down further, but this is not always possible.
Hope this helps.
I am using GWT 2.4. There are times when I have to show huge amount of records for example: 50,000 records on my screen in a gridtable or flextable. But it takes very long to load that screen say around 30 mins or so; or, ultimately the screen hangs or at times IE displays an error saying that this might take too long and your application will stop working, so do you wish to continue.
Is there any solution to improve gwt performance?
Don't bring all data at once, you should bring it in pages, as the comments suggested here.
However, paging not be trivial , as it might be that during paging your db is filled with more entries, and if you're using some sorting algorithm for the results,
the new entries might ruin your sorting (for example, when trying to fetch page #2, some entries that should have been on the first page are inserted.
You may decided that you create some sort of "cursor" for paging purposes and it will reflect the state of your database at the point you created it, so you will ignore entires that are entered during traversal between pages.
Another option you may consider, as part of paging is providing only a small version for each record - i.e - only the most important details, and let the user double click if he wants to see the whole details for the record - this can also provide you some performance improvement within each page.
I'm setting up a jqGrid (in a Google Chrome Extension) which will handle local JSON data.
My concern is performance due to my unique use case. I have thousands of records getting dynamically generated on the client side over a few minutes and I can't wait for the data to be generated so currently I add this data to the grid row by row using 'addRowData'.
But the problem is, when I'm adding data to the grid I have to check if that data already exists and if it does I need to update the existing record. I'm just having trouble understanding the best way to accomplish this, is the only way I can search the grid by calling 'getCol' and then searching the array. My concern with calling getCol is I presume this searches the DOM? But I could be wrong, I have scroll: 1 set and I'm starting to think this might mean its pulling data directly from an array?
Or maybe I should be implementing this a totally different way? It would of been so much easier if I could of just inserted all of this data into an array and then loaded the grid but due to the time taken to generate the data the user needs to see it ASAP.
Large datasets, millions of records, need special programming to maintain speed in DBGrids.
I want to know if there are any ready-made components for Delphi (DBGrids) that do this automatically?
EDIT For Example: Some databases have features such as fetch 1st X records (eg 100 records). When I reach the bottom with scrolling, I want to auto fetch the next 100. Conversely when I reach the beginning, I want to fetch the previous 100. I know I can program this, but it sure is possible to propagate that feature to a DBGrid control where the DBGrid does the buffering. It will save quite a bit of programming - you simply have to set the "buffer size" so to speak.
You might want to take a look at the wonderful (free, open source, dual licensed as MPL 1.1 and GPL thus usable in closed source apps) Virtual TreeView and its user-supplied descendants (scroll down the page to find those.)
Edit to reflect the question's edit: Virtual TreeView not only allows you to handle millions of nodes without keeping them in memory, but that is in fact the preferred way of using it. You supply the data through event callbacks when it's needed, and you can tell the tree to cache that data (or not.)
Oh, and of course it also has a grid / report mode where it can function as a table (just set the GridExtensions property to True.)
I would have a look at Developer Express QuantumGrid Suite. (#birger: you just were a tick faster ;-) ) So I'm not just duplicating the answer, some elaboration:
The DevExpress Grid uses a data controller that has several modes to controll the data bound to the grid. One of these is exactly what you are looking for:
Grid Mode
When using Grid Mode, only a fixed
number of dataset records is loaded
into memory. Because only a limited
set of records are retrieved from the
dataset, automatic sorting, filtering
and summary calculations are disabled
in Grid Mode (must be controlled
manually instead). By default, this
mode is disabled and the
ExpressDataController loads all
records in a dataset.
It does have some drawbacks, which seem pretty obvious: you cannot make a summary, sort, or filter if you do not have all records at hand.
NextGrid is light, fast and nice looking grid for Delphi
http://www.bergsoft.net/component/next-grid/features.htm
HANDLING LARGE AMOUNT OF CELLS WITHOUT LOOSING SPEED
NextGrid can handle very large amount
of cells without losing speed. Speed
of adding, modifying and deleting data
doesn't depend of the amount of cells.
In NextGrid demo you can see how fast
NextGrid work with 100,000 rows and 10
columns = 1,000,000 cells
I think the DevExpress Quantumgrid supports this very good.
sorry, I just saw your comment to Neftalí
if you would to bring 100 record per time, and then fetch the next 100, this work related to database access components, look at devart components, they are offer direct access components to most used database, and they have the feature you are asking about and more:
http://www.devart.com/products-vcl.html