I have a rad grid which fetches the data from database and displays required number of records even though there are number of records
eg: if there are 10,000 records which comes under search criteria. But we are retriving only the records equals to page size. ie., if paze size is 10 then 10 records will be retrived even though there are 10,000.
This is done to overcome performance issues while loading millions of records.
Now the problem is if user clicks on grid column to sort only 10 records are getting sorted. But i need all the records in a sorted order then required records get displayed in the page.
Two possibilities pop up in my mind:
If you use LINQ or Entity Framework, the telerik grid will perform the paging on the database server and only the items on the current page will be loaded. See demo here.
Use manual custom paging or binding to ObjectDataSource as in this sample.
Related
I have a Kendo MVC grid which contains five columns. Some of the rows in two of the columns could contain quite a bit of data, perhaps 500 to 1000 characters. The data is displayed fine but if I attempt to delete the row containing the large amount of data, the callback to my delete function errors out with a 500 server error. My delete function is never getting executed. However, if I attempt to delete a row with a much smaller amount of data, I have no issues.
What can I do to get this to work correctly?
Populating an Apex 5.1 select list of employees with about 25,000 names is proving to be a performance problem when loading the page. Is there a method of limiting the initial list to a set number (such as 200), and dynamically populating with chunks of additional names as the user scrolls through the list? Are there other options I should consider that would not slow down the page load?
I am currently using a dynamic LOV, and have tried adjusting this LOV to include Oracle row limiting code; however, there is no way of fetching past the initial set of rows. The source of the data is a view on a materialized view.
I appreciate any ideas
I'd use a pop-up LOV with a search function, not showing any records until the user enters a search value (more than 3 characters). I know it's tedious to use a pop-up LOV but it seems the only way to prevent waiting for a slow list to display.
I'd try with cascading lists of values. I don't know what those 25.000 names represent, but - suppose it is a large company. Then you'd
1st LoV: continent
2nd Lov: country
which refers to the previous LoV as where country.continent = :P1_CONTINENT
3rd LoV: city
which refers the previous LoV as where city.coutry = :P1_COUNTRY
4rd Lov is actually your current query:
which refers to the previous Lov as where person.city = :P1_CITY
Now your list of values wouldn't contain 25.000 rows, but - hopefully - a lot less.
In kendo-grid there is option to sort the column data by clicking on column header, but sort is applied on only the current page data.
Suppose I fetched 100 records from database, and choose to keep pagination of 10 records at a time on GUI,so that at a time we can see 10 records on the webpage.
Now, my requirement is to apply sort not on only 10 records, rather all fetched record.
Any suggestion is welcome.
Server sorting is the only way to go. Please check this:
http://docs.telerik.com/kendo-ui/api/javascript/data/datasource#configuration-serverSorting
I have 100,000 records right now (will grow in future). I have JSON api call (remote URL, however, within same server) to get those records. If I use KendoUI with paging turned on (say 50 per page), will KendoUI datasource going to fetch all those records and bring them into client and apply paging? Or is it something I need to pass to the server (page size) to be able to only get only needed data for display? If I need to pass, do I have to write custom data source query methods?
Same question goes for using filter input boxes in toolbar within KendoUI.
There are two (efficient) ways of loading that amount of data:
Setting serverPaging to true in the DataSourcedefinition.
Using serverPaging plyst (as #bobosov534 and #gitsitgo suggests) virtual scrolling.
In the both you receive in the server tow parameters: top indicating the number of records to retrieve (what you have defined as pageSize) and skip for the number of records to ignore (no skip means the first top records).
The difference is that int the first you see a pagination bar in the bottom of the grid and in the second you see additional records as you scroll down.
In DataSource.serverPaging you find detailed information on the fields sent to the server for managing pagination.
I read a lot of documents about AppFabric caching but most of them cover simple scenarios.
For example adding city list data or shopping card data to the cache.
But I need adding product catalog data to the cache.
I have 4 tables:
Product (1 million rows), ProductProperty (25 million rows), Property (100 rows), PropertyOption (300 rows)
I display paged search results querying with some filters for Product and ProductProperty tables.
I am creating criteria set over searched result set. For example (4 Items New Product, 34 Items Phone, 26 Items Book etc.)
I query for grouping over Product table with columns of IsNew, CategoryId, PriceType etc.
and also another query for grouping over ProductProperty table with PropertyId and PropertyOptionId columns to get which property have how many items
Therefore to display search results I make one query for search result and 2 for creating criteria list (with counts)
Search result query took 0,7 second and 2 grouping queryies took 1,5 second in total.
When I run load test I reach 7 request per second and %10 dropped by IIS becasue db could not give response.
This is why I want to cache Product and property records.
If I follow items below (in AppFabric);
Create named cache
Create region for product catalog data (a table which have 1 million rows and property table which have 25 million rows)
Tagging item for querying data and grouping.
Can I query with some tags and get 1st or 2nd page of results ?
Can I query with some tags and get counts of some grouping results. (displaying filter options with count)
And do I have to need 3 servers ? Can I provide a solution with only one appfabric server (And of course I know risk.)
Do you know any article or any document explains those scenarios ?
Thanks.
Note:
Some additional test:
I added about 30.000 items to the cache and its size is 900 MB.
When I run getObjectsInRegion method, it tooks about 2 minutes. "IList> dataList = this.DataCache.GetObjectsInRegion(region).ToList();"
The problem is converting to IList. If I use IEnumerable it works very quicly. But How can I get paging or grouping result without converting it to my type ?
Another test:
I tried getting grouping count with 30.000 product item and getting result for grouping took 4 seconds. For example GetObjectByTag("IsNew").Count() and other nearly 50 query like that.
There is, unfortunately, no paging API for AppFabric in V1. Any of the bulk APIs, like GetObjectsByTag, are going to perform the query on the server and stream back all the matching cache entries to the client. From there you can obviously use any LINQ operators you want on the IEnumerable (e.g. Skip/Take/Count), but be aware that you're always pulling the full result set back from the server.
I'm personally hoping that AppFabric V2 will provide support via IQueryable instead of IEnumerable which will give the ability to remote the full request to the server so it could page results there before returning to the client much like LINQ2SQL or ADO.NET EF.
For now, one potential solution, depending on the capabilities of your application, is you can actually calculate some kind of paging as you inject the items into the cache. You can build ordered lists of entity keys representing each page and store those as single entries in the cache which you can pull out in one request and then individually (in parallel) or bulk fetch the items in the list from the cache and join them together with an in-memory LINQ query. If you wanted to trade off CPU for Memory, just cache the actual list of full entities rather than IDs and having to do the join for the entities.
You would obviously have to come up with some kind of keying mechanism to quickly pull these lists of objects from the cache based on the incoming search criteria. Some kind of keying like this might work:
private static string BuildPageListCacheKey(string entityTypeName, int pageSize, int pageNumber, string sortByPropertyName, string sortDirection)
{
return string.Format("PageList<{0}>[pageSize={1};pageNumber={2};sortedBy={3};sortDirection={4}]", entityTypeName, pageSize, pageNumber, sortByPropertyName, sortDirection);
}
You may want to consider doing this kind of thing with a separate process or worker thread that's keeping the cache up to date rather than doing it on demand and forcing the users wait if the cache entry isn't populated yet.
Whether or not this approach ultimately works for you depends on several factors of your application and data. If it doesn't exactly fit your scenarios maybe it will at least help shift your mind into a different way of thinking about solving the problem.