I'm trying to use syslistview32 as a container to diplay a dataset from db. About 10K records(120+ columns), the db fetch operation very fast,less then 2 seconds.but the marco ListView_InsertItem/ListView_SetItem，very slow. the total loading procedure cost almost 15 seconds(or even more).
what makes the loading processs so slow ? what's the major difference between syslistview32 and sql plus dataview (tsgrid as the spy shows) ? if i insisted in using the syslistview32 ， what should i do to improving the process?
Related
I have an interactive report in one of my APEX application. The SQL query used in the IR runs pretty fine when executed in SQL Developer.
But, at times in the application it gets stuck and requires more time than usual to load the IR. (Usually it takes less than 5 secs to load but at times more than 50 secs).
What might be the possible reasons for it to load slow ?
The query is well tuned and IR has default settings with no modification. I have also checked the stats on the tables and it is fresh.
The SQL query used in IR fetches 10k records.
If you go into Component View and then click Interactive Report under Regions, there is a setting near the bottom under the Performance heading called Maximum Rows To Process. Also limiting the number of rows to display sped things up for me.
Sorry but i can't write comments. Is there any database view in your query?
I have similar situation where query from database view with 6 mil. records take around 3 min to complete in Oracle Apex IR and 10-15 seconds in SQL Developer. So after some research i try to put sql from view directly into IR and result was almost same as this in SQL Developer.
Also You can remove pagination from IR or change it from "x to y from z" to be only "x to y".
I hope this can help you.
Query response time in SQL Developer versus any other Web browser cannot be compared directly. Some of the reasons for its slugishness could be related to server setup, server load, current user traffic, page load processes, page and region rendering, number of regions,components and plugins, navigation menu query, report query, number or columns and rows being displayed, row content length, apex items especially LOV with SQL queries, etc.
From your question, it looks like performance issue is not consistent and so, I think issue may be related to server setup or traffic. Try to check if you see any difference in load time after bouncing the server, if that's an option. Try to isolate the problem and if the issue is specific to interactive report, build a classic report and compare times.
Another thing that has helped me in past is to compare and verify compute times using APEX Debugger, here is the screenshot.
Also look at network and timeline tabs in Chrome debugger,
Implement indexes on your tables
Verify with your DBA if you have database locks
Verify the amount of logs in Database
Switch to classic reports.
Regards
I am using kendoUi Grid with 20k records, When i am changing page size 20 to 200 grid taking 40 to 50 sec to work. some times it taking min. paging is client side only.
For large datasets, it's better to use server paging, mainly because:
Faster loads
Avoid wasting memory
Avoid unnecessary database and network loads, database just gives you the number of records of the current page
You should consider enabling server paging at datasource level, and then read pagination values on backend side before performing the query to the database.
http://docs.telerik.com/kendo-ui/api/javascript/data/datasource#configuration-serverPaging
good question, i am also faced this type of issue.
Please use MVVM logic instance of MVC for bind the grid,
for more please find this below link.
http://demos.telerik.com/kendo-ui/grid/mvvm
We have a TDBGrid that connected to TClientDataSet via TDataSetProvider in Delphi 7 with Oracle database.
It goes fine to show content of small tables, but the program hangs when you try to open a table with many rows (for ex 2 million rows) because TClientDataSet tries to load the whole table in memory.
I tried to set "FetchOnDemand" to True for our TClientDataSet and "poFetchDetailsOnDemand" to True in Options for TDataSetProvider, but it does not help to solve the problem. Any ides?
Update:
My solution is:
TClientDataSet.FetchOnDemand = T
TDataSetProvider.Options.poFetchDetailsOnDemand = T
TClientDataSet.PacketRecords = 500
I succeeded to solve the problem by setting the "PacketRecords" property for TCustomClientDataSet. This property indicates the number or type of records in a single data packet. PacketRecords is automatically set to -1, meaning that a single packet should contain all records in the dataset, but I changed it to 500 rows.
When working with RDBMS, and especially with large datasets, trying to access a whole table is exactly what you shouldn't do. That's a typical newbie mistake, or a borrowing from old file based small database engines.
When working with RDBMS, you should load the rows you're interested in only, display/modify/update/insert, and send back changes to the database. That means a SELECT with a proper WHERE clause and also an ORDER BY - remember row ordering is never assured when you issue a SELECT without an OREDER BY, a database engine is free to retrieve rows in the order it sees fit for a given query.
If you have to perform bulk changes, you need to do them in SQL and have them processed on the server, not load a whole table client side, modify it, and send changes row by row to the database.
Loading large datasets client side may fali for several reasons, lack of memory (especially 32 bit applications), memory fragmentation, etc. etc., you will flood the network probably with data you don't need, force the database to perform a full scan, maybe flloding the database cache as well, and so on.
Thereby client datasets are not designed to handle millions of billions of rows. They are designed to cache the rows you need client side, and then apply changes to the remote data. You need to change your application logic.
we are working on a .net desktop application. On the GUI, there are number of tabs and panels. graph tab, imges tab, result grid tab etc. The task is to fetch about 50,000 records and be able to take its different views. e.g can have graph amoung two columns, can go throug the results in the grid and can view images of the records.
Application is developed but its performence is too bad. We are trying to target it atleast for 50,000 records, but its response gets weired on about 5000 records.
Facts:
1- Queryies are complex which include number on joins. On the avg 10 to 12 tables joins. And sometimes subquery table as join. It takes about 8 to 10 seconds to return results.
What can be done to achive performance at this level.
- Index are used properly.
Can using SSIS (Sql Server Integration Services) help in the context?
2- Graphs support very less amount of data and start getting exshast on about 4000 records. What can be done to improve graph's performance?
Pagging can't be used when graphs are involve.
Please post your schema and a sample query so that it can be improved upon.
As for the graph performance, here's some general performance tips (assuming WinForms):
Does the graph object have a .BeginUpdate() or .BeginDataUpdate() and a corresponding .EndUpdate() / .EndDataUpdate() method? If it has those then you should be using that. The same applies to the GridControl as well.
Are you adding the items to the graph/grid (or their datasource) one by one, or are you calling .AddRange() or setting the data source / data bindings. If you're adding the adding the items one by one then it will often remake the list over and over, this was a common problem with the .NET 1.1 ListView control, because the items underneath were stored in an array and each .Add(..) call recreated the array so it very quickly went O(n^2) for adding items.
What graphing and grid controls are you using?
I am trying to develop my first web project using the entity framework, while I love the way that you can use linq instead of writing sql, I do have some severe performance issuses. I have a lot of unhandled data in a table which I would like to do a few transformations on and then insert into another table. I run through all objects and then inserts them into my new table. I need to do some small comparisons (which is why I need to insert the data into another table) but for performance tests I have removed them. The following code (which approximately 12-15 properties to set) took 21 seconds, which is quite a long time. Is it usually this slow, and what might I do wrong?
DataLayer.MotorExtractionEntities mee = new DataLayer.MotorExtractionEntities();
List<DataLayer.CarsBulk> carsBulkAll = ((from c in mee.CarsBulk select c).Take(100)).ToList();
foreach (DataLayer.CarsBulk carBulk in carsBulkAll)
{
DataLayer.Car car = new DataLayer.Car();
car.URL = carBulk.URL;
car.color = carBulk.SellerCity.ToString();
car.year = //... more properties is set this way
mee.AddToCar(car);
}
mee.SaveChanges();
You cannot create batch updates using Entity Framework.
Imagine you need to update rows in a table with a SQL statement like this:
UPDATE table SET col1 = #a where col2 = #b
Using SQL this is just one roundtrip to the server. Using Entity Framework, you have (at least) one roundtrip to the server loading all the data, then you modify the rows on the client, then it will send it back row by row.
This will slow things down especially if your network connection is limited, and if you have more than just a couple of rows.
So for this kind of updates a stored procedure is still a lot more efficient.
I have been experimenting with the entity framework quite a lot and I haven't seen any real performance issues.
Which row of your code is causing the big delay, have you tried debugging it and just measuring which method takes the most time?
Also, the complexity of your database structure could slow down the entity framework a bit, but not to the speed you are saying. Are there some 'infinite loops' in your DB structure? Without the DB structure it is really hard to say what's wrong.
can you try the same in straight SQL?
The problem might be related to your database and not the Entity Framework. For example, if you have massive indexes and lots of check constraints, inserting can become slow.
I've also seen problems at insert with databases which had never been backed-up. The transaction log could not be reclaimed and was growing insanely, causing a single insert to take a few seconds.
Trying this in SQL directly would tell you if the problem is indeed with EF.
I think I solved the problem. I have been running the app locally, and the database is in another country (neighbor, but never the less). I tried to load the application to the server and run it from there, and it then only took 2 seconds to run instead of 20. I tried to transfer 1000 records which took 26 seconds, which is quite an update, though I don't know if this is the "regular" speed for saving the 1000 records to the database?