I have a SQL query that fetches roughly 200 columns from multiple tables and normally runs in a matter of minutes.
A Java program kicked off by cron calls the SQL every 4 hours, but occasionally hangs forever(=not fetching any data. Neither updates nor inserts are involved).
Here are some outputs from V$SESSION.
STATUS: ACTIVE
ROW_WAIT_OBJ#: 22392 ←not changing
ROW_WAIT_FILE#: 6 ←not changing
ROW_WAIT_BLOCK#: 8896642 ←not changing
ROW_WAIT_ROW#: 0 ←not changing
LAST_CALL_ET: 5632 ←keeps incresing
★No other heavy SQL queries are running at the same time
What could be the cause of this and what should I look into to solve it?
You can use TKPROF or SQL Profiler. This reports can help you. We can not replay your question now.
If you attach your tuning reports, we can help you. Because many things can cause performance problems. A comprehensive study is needed to understand this.
Follow this link;
https://docs.oracle.com/cd/E11882_01/server.112/e41573/perf_overview.htm
Related
While sending periodically the same graph query to Hasura server, I have observed significantly different execution times
In one of the these cases, the query was executed under a single seconds, where as in another case the same query took more than 150 seconds. The execution times were captures from the Hasura "http-log" statements.
An additional observation from the corresponding "query-log" statements is that, the SQLs are generated in both cases, within similar times.
Any reason for the generated SQL being executed after a significant and considerable delay compared to the other.
Any specific reason for this inconsistent behaviour and any specific configurations that can be made to overcome this issue.
I don't know if that counts as an answer, it's certainly not "a general case answer" as it reflects only our experience.
We encountered similar problem: inconsistent latencies for the same queries.
Where we looked and what we found.
1. hasura
Hasura itself is a very thin and predictable layer above postgresql (and now other DBs too).
I'm not a haskel expert but I got impression that SQL generation comes from here: https://github.com/hasura/graphql-engine/blob/b2461c5899a881183ad2d269ebe8a2c6f55e46af/server/src-lib/Hasura/GraphQL/Execute/LiveQuery.hs
(I could be wrong and I will be grateful if somebody will correct me)
So:
hasura always generate the same SQL for the same query
this process is predictable
it has a low cost
Conclusion: hasura itself could not be source of different latencies. We need to look on DB level
2. What we encountered on DB level
We build a simple test: running the same query on DB
And we discovered that the same query is running as 100ms-100ms-2 seconds - 150 ms - 3 seconds - 90 ms.
We search for locks - and did not found them.
We looked on buffering - and discovered that almost all DB is cached in memory.
Finally our suspicion was that it's Azure Database (we used cloud postgresql from MS) is misbehaving.
We contacted support (and we had other questions to them) and finally we discovered that we simply hit IOPS limit.
This hypothesis was supported by simple fact: if we run VACUUM/REINDEX/ REFRESH MATERIALIZED VIEW/heavy procedures then DB became much less responsive for an amount of time.
We considered upgrading Azure Database but we had other problems and we wanted to upgrade postgresql version so finally we decided to migrate to Amazon RDS.
(That's not bashing Azure or promoting Amazon, personally I think that running on-premise would be the best)
After that all strange execution times disappeared.
Think yourself how that reflects your case.
In general I recommend to look on DB level only.
I have an interactive report in one of my APEX application. The SQL query used in the IR runs pretty fine when executed in SQL Developer.
But, at times in the application it gets stuck and requires more time than usual to load the IR. (Usually it takes less than 5 secs to load but at times more than 50 secs).
What might be the possible reasons for it to load slow ?
The query is well tuned and IR has default settings with no modification. I have also checked the stats on the tables and it is fresh.
The SQL query used in IR fetches 10k records.
If you go into Component View and then click Interactive Report under Regions, there is a setting near the bottom under the Performance heading called Maximum Rows To Process. Also limiting the number of rows to display sped things up for me.
Sorry but i can't write comments. Is there any database view in your query?
I have similar situation where query from database view with 6 mil. records take around 3 min to complete in Oracle Apex IR and 10-15 seconds in SQL Developer. So after some research i try to put sql from view directly into IR and result was almost same as this in SQL Developer.
Also You can remove pagination from IR or change it from "x to y from z" to be only "x to y".
I hope this can help you.
Query response time in SQL Developer versus any other Web browser cannot be compared directly. Some of the reasons for its slugishness could be related to server setup, server load, current user traffic, page load processes, page and region rendering, number of regions,components and plugins, navigation menu query, report query, number or columns and rows being displayed, row content length, apex items especially LOV with SQL queries, etc.
From your question, it looks like performance issue is not consistent and so, I think issue may be related to server setup or traffic. Try to check if you see any difference in load time after bouncing the server, if that's an option. Try to isolate the problem and if the issue is specific to interactive report, build a classic report and compare times.
Another thing that has helped me in past is to compare and verify compute times using APEX Debugger, here is the screenshot.
Also look at network and timeline tabs in Chrome debugger,
Implement indexes on your tables
Verify with your DBA if you have database locks
Verify the amount of logs in Database
Switch to classic reports.
Regards
I wrote an application that queries oracle v$sqlarea and dumps data to my own database for further analysis. I noticed something very strange - sometimes data in the v$sqlarea shows less executions than before. I'm pretty sure that the oracle cache was not cleaned (the first load time of query is still the same, and since I query oracle each minute I dont believe that in this one minute the query was executed 100k+ times).
Can anybody explain how this is possible?
Ok, so I asked this on Oracle forum as well, and I believe that the correct answer is this one
https://community.oracle.com/message/12980175#12980175
I have an Oracle bind query that is extremely slow (about 2 minutes) when it executes in my C# program but runs very quickly in SQL Developer. It has two parameters that hit the tables index:
select t.Field1, t.Field2
from theTable t
where t.key1=:key1
and t.key2=:key2
Also, if I remove the bind variables and create dynamic sql, it runs just like it does in SQL Developer.
Any suggestion?
BTW, I'm using ODP.
If you are replacing the bind variables with static varibles in sql developer, then you're not really running the same test. Make sure you use the bind varibles, and if it's also slow you're just getting bit by a bad cached execution plan. Updating the stats on that table should resolve it.
However if you are actually using bind variables in sql developers then keep reading. The TLDR version is that parameters that ODP.net run under sometimes cause a slightly more pessimistic approach. Start with updating the stats, but have your dba capture the execution plan under both scenarios and compare to confirm.
I'm reposting my answer from here: https://stackoverflow.com/a/14712992/852208
I considered flagging yours as a duplicate but your title is a little more concise since it identifies the query does run fast in sql developer. I'll welcome advice on handling in another manner.
Adding the following to your config will send odp.net tracing info to a log file:
This will probably only be helpful if you can find a large gap in time. Chances are rows are actually coming in, just at a slower pace.
Try adding "enlist=false" to your connection string. I don't consider this a solution since it effecitively disables distributed transactions but it should help you isolate the issue. You can get a little bit more information from an oracle forumns post:
From an ODP perspective, all we can really point out is that the
behavior occurs when OCI_ATR_EXTERNAL_NAME and OCI_ATR_INTERNAL_NAME
are set on the underlying OCI connection (which is what happens when
distrib tx support is enabled).
I'd guess what you're not seeing is that the execution plan is actually different (meaning the actual performance hit is actually occuring on the server) between the odp.net call and the sql developer call. Have your dba trace the connection and obtain execution plans from both the odp.net call and the call straight from SQL Developer (or with the enlist=false parameter).
If you confirm different execution plans or if you want to take a preemptive shot in the dark, update the statistics on the related tables. In my case this corrected the issue, indicating that execution plan generation doesn't really follow different rules for the different types of connections but that the cost analysis is just slighly more pesimistic when a distributed transaction might be involved. Query hints to force an execution plan are also an option but only as a last resort.
Finally, it could be a network issue. If your odp.net install is using a fresh oracle home (which I would expect unless you did some post-install configuring) then the tnsnames.ora could be different. Host names in tnsnams might not be fully qualified, creating more delays resolving the server. I'd only expect the first attempt (and not subsequent attempts) to be slow in this case so I don't think it's the issue but I thought it should be mentioned.
Are the parameters bound to the correct data type in C#? Are the columns key1 and key2 numbers, but the parameters :key1 and :key2 are strings? If so, the query may return the correct results but will require implicit conversion. That implicit conversion is like using a function to_char(key1), which prevents an index from being used.
Please also check what is the number of rows returned by the query. If the number is big then possibly C# is fetching all rows and the other tool first pocket only. Fetching all rows may require many more disk reads in that case, which is slower. To check this try to run in SQL Developer:
SELECT COUNT(*) FROM (
select t.Field1, t.Field2
from theTable t
where t.key1=:key1
and t.key2=:key2
)
The above query should fetch the maximum number of database blocks.
Nice tool in such cases is tkprof utility which shows SQL execution plan which may be different in cases above (however it should not be).
It is also possible that you have accidentally connected to different databases. In such cases it is nice to compare results of queries.
Since you are raising "Bind is slow" I assume you have checked the SQL without binds and it was fast. In 99% using binds makes things better. Please check if query with constants will run fast. If yes than problem may be implicit conversion of key1 or key2 column (ex. t.key1 is a number and :key1 is a string).
I have a stored procedure that returns about 50000 records in 10sec using at most 2 cores in SSMS. The SSRS report using the stored procedure was taking 20min and would max out the processor on an 8 core server for the entire time. The report was relatively simple (i.e. no graphs, calculations). The report did not appear to be the issue as I wrote the 50K rows to a temp table and the report could display the data in a few seconds. I tried many different ideas for testing altering the stored procedure each time, but keeping the original code in a separate window to revert back to. After one Alter of the stored procedure, going back to the original code, the report and server utilization started running fast, comparable to the performance of the stored procedure alone. Everything is fine for now, but I am would like to get to the bottom of what caused this in case it happens again. Any ideas?
I'd start with a SQL Profiler trace of both the stored procedure when you execute it normally, and then the same SP when it's called by SSRS. Make sure you include the execution plans involved, so you can see if it's making some bad decisions (though that seems unlikely - the SQL Server should execute an optimal - or at least consistent - plan regardless of the query's source).
We used to have cases where Business Objects would execute stored procs dozens of times for no aparent reason and it lead to occasionally horrible performance, though I've never seen that same behavior with SSRS. It may be somewhere to start, though. You'll also see the execution begin/end times - that will make it clear if it's the database layer that's hanging up, or if the SQL Server hands back the data in 10 seconds and then it's the SSRS service that's choking somewhere.
The primary solution to speeding SSRS reports is to cache the reports. If one does this (either my preloading the cache at 7:30 am for instance) or caches the reports on-hit, one will find massive gains in load speed.
You may also find that monthly restarts of SSRS application domain to resolve your issue.
Please note that I do this daily and professionally and am not simply waxing poetic on SSRS
Caching in SSRS
http://msdn.microsoft.com/en-us/library/ms155927.aspx
Pre-loading the Cache
http://msdn.microsoft.com/en-us/library/ms155876.aspx
If you do not like initial reports taking long and your data is static i.e. a daily general ledger or the like, meaning the data is relatively static over the day, you may increase the cache life-span.
Finally, you may also opt for business managers to instead receive these reports via email subscriptions, which will send them a point in time Excel report which they may find easier and more systematic.
You can also use parameters in SSRS to allow for easy parsing by the user and faster queries. In the query builder type IN(#SSN) under the Filter column that you wish to parameterize, you will then find it created in the parameter folder just above data sources in the upper left of your BIDS GUI.
[If you do not see the data source section in SSRS, hit CTRL+ALT+D.
See a nearly identical question here: Performance Issuses with SSRS