I am using DataTable plugin in Laravel. I have a record of 3000 entries in some
But when i load that page it loads all 3000 records in the browser then create pagination, this slow down the page loading.
How to fix this or correct way
Use server-side processing.
Get help from some Laravel Packages. Such as Yajra's: https://yajrabox.com/docs/laravel-datatables/
Generally you can solve pagination either on the front end, the back end (server or database side), or a combination of both.
Server side processing, without a package, would mean setting up TOP/FETCH or make rows in data being returned from your server.
You could also load a small amount (say 20) and then when the user scrolls to the bottom of the list, load another 20 or so. I mention the inclusion of front end processing as well because I’m not sure what your use cases are, but I imagine it’s pretty rare any given user actually needs to see 3000 rows at a time.
Given that Data Tables seems to have built-in functionality for paginating data, I think that #tersakyan is essentially correct — what you want is some form of back-end filtering or paginating of rows of data to limit what’s being sent to the front end.
I don’t know if that package works for you or not or what your setup looks like, but pagination can also be achieved directly from a DataBase returning data via the SQL (using TOP/FETCH for example) or could be implemented in a Controller or Service by tracking pages of data and “loading a page at a time” both from the server and then into the table. All you would need is a unique key to associate each "set of pages" for a specific request.
But for performance, you want to avoid both large data requests and operations on large sets of data. So the more you limit how much data is being grabbed or processed at any stage of your application using it, the more performant your application will be in principle.
Related
I noticed that in all the examples provided by the website, all data is retrieved from the database at page load. However, in my case this would result in having to retrieve a significant amount of data, from which only part of it is being edited. I am no expert, but this would seem inefficient in terms of performance. I've also seen PhpGrid.org and they seem to use AJAX calls for each page.
Would you agree, or could someone explain why the current method is in fact more efficient?
Is there a way to only retrieve data for the page that is being viewed (i.e. via AJAX calls) ?
Can someone convince me? :-)
Please refer to DataTables manual page about data for more information. Below is an excerpt:
DataTables has two different modes of processing data (ordering,
searching, etc. of data):
Client-side processing - the full data set is loaded up-front and data processing is done in the browser.
Server-side processing - an Ajax request is made for every table redraw, with only the data required for each display returned. The
data processing is performed on the server.
Each has its own advantages and disadvantages, but the key indicator
for which mode you should select is based on the number of rows in
your table.
You're referring to client-side processing mode which loads all data at once. However, in server-side processing mode only current page data is loaded which improves performance.
We have a TDBGrid that connected to TClientDataSet via TDataSetProvider in Delphi 7 with Oracle database.
It goes fine to show content of small tables, but the program hangs when you try to open a table with many rows (for ex 2 million rows) because TClientDataSet tries to load the whole table in memory.
I tried to set "FetchOnDemand" to True for our TClientDataSet and "poFetchDetailsOnDemand" to True in Options for TDataSetProvider, but it does not help to solve the problem. Any ides?
Update:
My solution is:
TClientDataSet.FetchOnDemand = T
TDataSetProvider.Options.poFetchDetailsOnDemand = T
TClientDataSet.PacketRecords = 500
I succeeded to solve the problem by setting the "PacketRecords" property for TCustomClientDataSet. This property indicates the number or type of records in a single data packet. PacketRecords is automatically set to -1, meaning that a single packet should contain all records in the dataset, but I changed it to 500 rows.
When working with RDBMS, and especially with large datasets, trying to access a whole table is exactly what you shouldn't do. That's a typical newbie mistake, or a borrowing from old file based small database engines.
When working with RDBMS, you should load the rows you're interested in only, display/modify/update/insert, and send back changes to the database. That means a SELECT with a proper WHERE clause and also an ORDER BY - remember row ordering is never assured when you issue a SELECT without an OREDER BY, a database engine is free to retrieve rows in the order it sees fit for a given query.
If you have to perform bulk changes, you need to do them in SQL and have them processed on the server, not load a whole table client side, modify it, and send changes row by row to the database.
Loading large datasets client side may fali for several reasons, lack of memory (especially 32 bit applications), memory fragmentation, etc. etc., you will flood the network probably with data you don't need, force the database to perform a full scan, maybe flloding the database cache as well, and so on.
Thereby client datasets are not designed to handle millions of billions of rows. They are designed to cache the rows you need client side, and then apply changes to the remote data. You need to change your application logic.
I'm working on a project where I have to fetch data from a database then present results in tables with at least 200 rows until 3000 rows.
I tried first Pagedlist and noticed that only the fixed number of records per page is displayed in the client side and each time when the user change the page, there is a new request to the server.
For tablesorter, I noticed that all the results are displayed and paging is only visual (only the presentation of table) but everything is in the client side
Is my understanding correct?
What I want to know is which approach is better (in time execution)?
Actually I'm working on localhost and there is only me as user so I can't notice the difference even tablesorter take more time to load in the first time but after it's very quick meanwhile the pagedlist method is faster in loading page but each time it request the server to change page and load the according data
When finished the application will be in the server and many users will have access to the application to add, search delete...
Which of these two approaches is a better choice?
Thanks
I've got 2 views with CI: Header and Content.
In my controller I load my header, execute a query and then load this into the controller. This is bringing back all my content.
However, I have a search bar in my Header, what I'd like it to do is when the user is typing, I would like it to filter the content on the page as they type.
Is there anyway to do this without constantly executing database queries?
Yes, you make one database query and cache all the results, and then use autocomplete to query your cache store. Depending on the searchable content this may or may not be feasible. If you have a giant dataset with several million records, it would make more sense to rely on database cache like Memcached - if you're searching geo locations or something similar you might want to rely on Ajax API calls to a Solr instance.
I am working on an MVC3 and Razor website. The user has to select their way through a few choices before finally working on the data.
For example:
Client List -> Version List (Filtered by client) -> Etc (Filtered by version)
Once a user selects a client, they select a version for the client. So I'm passing the client id on the querystring. For each mode of the controller of version I'm passing around the client id. On views that I want to show the client name, I'm querying the database for the client and stuffing it into the ViewBag. This seems very inefficient. I feel like I could use a cookie to hold the client id & name.
Now that I've got my version controller done, I'm facing the same pattern again with each subsequent controller, but now I need to persist both client and version...
What is a preferred approach for persisting information like this across requests?
This seems very inefficient
That's what database are made and optimized for => query data based on fields and if you put indexes on those fields it will be screamingly fast. Of course Session, Cookies, Cache are some common techniques that you could employ to limit the number of queries to the database but you will have to assume the possible staleness of data that you are getting this way (if some other thread/process modified the data in the database you no longer get correct results).
So before doing any premature optimizations here's what I would recommend you: hammer your database until you discover that this is actually a bottleneck for your application. Databases might become bottleneck in some very high traffic applications where you should resort to one of the afforementioned techniques (or in some poorly written applications of course but let's exclude this possibility for the moment).
You should use TempData, which allows you to pass data between the current and next HTTP requests. Be sure to keep in mind that it uses the session.
Greg Shackles has a great article all about TempData here
see this similar question MVC3 multi step form - How to persist model object