In the spirit of Dimitre Novatchev's answer at XSLT Performance, I want to create a profile that shows where the time consumed by my XSL transform has gone. Using the Saxon -TP:profile.html option, we created an “Analysis of Stylesheet Execution Time” HTML document.
At the top of that document, we see:
Total time: 1102316.688 milliseconds
This figure (1,102 seconds) corresponds with my measured program execution time.
However, the sum of the “total time (net)” column values is less than 2% of this total. I assume, per http://www.saxonica.com/html/documentation/using-xsl/performanceanalysis.html, that the “total time (net)” column values are reported in milliseconds.
I would normally work a profile from the top down, but in this case, I don’t want to invest effort into optimizing a template that is reported to have contributed less than 0.5% of my total response time.
How can I find out where my time has really gone? Specifically, how can I learn where the unreported 98% of my program's time has been consumed?
Related
I'm getting WORKLOAD REPOSITORY COMPARE PERIOD REPORT that says
Load Profile
1st per sec
DB time: 1.3
I'm confused, Db time should be in time units, doesn't it?
BELOW are context and history of what's I've researched about AWRs and how I came to the answer I posted eventually.
I have ARW report that says
Elapsed Time (min) DB time (min)
60 80
That I read e.g. here https://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:229943800346471782 it's explained how DB time can exceed elapsed time. And Db time is time, it's measured in time units (min = minute?), so far so good.
Then Load Profile says:
1st per sec
DB time: 1.3
If DB time is 80 minutes in 60 minutes, than per sec by math should be 80/60/60, where that division by 60 to get per second go?
EDIT: my guess now as the question have been posted that this metric is in seconds, although units are not mentioned in AWR and I could not find about it in web by awr db time in sec search. Please provide link where it's confimed for sure (if it is so).
EDIT 2: WORKLOAD REPOSITORY report says, DB Time(s): per sec in Load profile section, where as WORKLOAD REPOSITORY COMPARE PERIOD REPORT just says Db time per sec. So now with like 99% assurance I can guess compare report uses same units, it's still not 100% sure fact. I actually get the reports via automated system, so cannot be sure they not mangled along the way...
P.S. by the way, I tried to do pretty formating of output, wanting to insert tabs, but could not find how, e,g. here Tab space in Markdown it says for similar it's not possible in markdown. Please add in comment if it can be done on stackoverflow.
My guess is that due to lots of info to fit on one line of compare AWR developers of the report decided to skip (s): which is present on the same place in ordinary (not compare) AWR.
I've looked at WORKLOAD REPOSITORY report, it says: DB Time(s): per sec 1.4 in Load profile section, where as WORKLOAD REPOSITORY COMPARE PERIOD REPORT just says Db time per sec and states 2nd 1.4. So now with like 99% assurance I can guess compare report uses same units - seconds for per seconds metric. Not 100% sure, but what are things we are 100% sure anyway?
Scenario:
Suppose we have infinite cache memory size. Caching is just limited by timeout, value of this timeout is half an hour. Cache is initially empty.
Problem:
We have 50,000 distinct request. Our system is querying, randomly, at the rate of 15 request/second i.e. 27,000 request in half an hour . What kind of curve or average value of cache hit rate could we expect for first 5 hours?
Note: This scenario is fixed. I need an approach to find out hit rate. If you think tag is wrong, please suggest appropriate tag.
I think you're right and this is a math question (certainly not a programming
problem).
One approach is to consider the extremes -- what is the hit rate for the
first query when the the system starts running? For the second query?
After one second? After 10? After a minute? And what is the likelyhood
that any random query will be found in the cache once the system has been
running a long time?
These are few specific values, and together they give you a curve.
I don't think great numeric precision is necessary; the long-term average
and the shape of the curve is more interesting.
In the Client Statistics Window in Sql Server Management Studio, I get the total
execution time.
However, this time is often muss less then the time the query actually took.
So what is the additional time spend for?
For example, here I got ~5,6 seconds of total execution time, but my query took 13 seconds to finish
The total execution time is the time until the result is available for display. But then, depending on the result set size and the way you display the data, the time until everything has been rendered is usually much higher.
I was trying to analyze the AWR report generated for a particular process with a duration of one hour. I am trying to find out which query is taking much time
to while running the process.
When I have gone through the report, I can see SQL ordered by Gets,SQL ordered by CPU Time,SQL ordered by Executions,SQL ordered by Parse Calls,
SQL ordered by Sharable Memory,SQL ordered by Elapsed Time etc.
I can see the SQL Text from the table SQL ordered by Elapsed Time.
My question: Is this the right way to identify the expensive query ? Please advise in this regard.
Elapsed Time (s) SQL Text
19,477.05 select abc.....
7,644.04 select def...
SQL Ordered by Elapsed Time , includes SQL statements that took significant execution time during processing.We have to look at Executions,Elapsed time per Exec (s) etc. along with Elapsed time to analyze.
For example,a query has low Executions and high Elapsed time per Exec (s) and this query could be a candidate for troubleshooting or optimizations.
The best reference I found so far: http://www.dbas-oracle.com/2013/05/10-steps-to-analyze-awr-report-in-oracle.html
AWR is used to see database health. So, I think this is not the good tools to trace a process.
You should use other tools like sql_trace (with tkprof) or dbms_profiler. It will concenrate on your own process.
If you are using sql_trace, you need to connect to the server (or ask to the dba tem) to analyse the trace.
In SQL Ordered by Elapsed time you always need to check the Query which is having low Execution numbers and higher Elapsed time . This would always be the problematic Query . Since Elapsed time is the defined task for a respective Query in this case if it is higher with less number of Executions it means that for some reason the Query is performing not up to expectations .
There is some parameter need to check so we are find issue in progress.
Buffer get is less expensive than physical read because database has to work harder (and more) to get the data. Basically time it would have taken if available in buffer cache + time actually taken to find out from physical block.
If you suspect that excessive parsing is hurting your database’s performance:
check “time model statistics” section (hard parse elapsed time, parse time elapsed etc.)
see if there are any signs of library cache contention in the top-5 events
see if CPU is an issue.
Establishing a new database connection is also expensive (and even more expensive in case of audit or triggers).
“Logon storms” are known to create very serious performance problems.
If you suspect that high number of logons is degrading your performance, check “connection management elapsed time” in “Time model statistics”.
Soft Parsing being low indicates bind variable and versioning issues. With 99.25 % for the soft parse meaning that about 0.75 % (100 – soft parse) is happening for hard parsing. Low hard parse is good for us.
If Latch Hit % is <99%, you may have a latch problem. Tune latches to reduce cache contention.
Library hit % is great when it is near 100%. If this was under 95% we would investigate the size of the shared pool.
In this ration is low then we may need to:
• Increase the SHARED_POOL_SIZE init parameter.
• CURSOR_SHARING may need to be set to FORCE.
• SHARED_POOL_RESERVED_SIZE may be too small.
• Inefficient sharing of SQL, PLSQL or JAVA code.
• Insufficient use of bind variables
Are there any widgets for predicting when a download (or any other process) will finish based on percent done history?
The trivial version would just do a 2 point fit based on the start time, current time and percent done but better option are possible.
A GUI widgest would be nice but a class that just returns the value would be just fine.
For the theoretical algorithm that I would attempt, if I would write such a widget, would be something like:
Record the amount of data transferred within a one second period (a literal KiB/s)
Remember the last 5 or 10 such periods (to get an an recent average KiB/s)
Subtract the total size from the transferred size (to get a "bytes remaining")
???
Widget!
That oughta do it...
(the missing step being: kibibytes remaining divided by average KiB/s)
Most progress bar widgets have an update() method which takes a percentage done.
They then calculate the time remainging based on the time since start and the latest percentage. If you have a process with a long setup time you might want to add more functionality so that it doesn't include this time, perhaps by having the prediction clock reset when sent a 0% update.
It would sort of have the same effect as some of the other proposals but by collecting percent done data as a function of time statistical methods could be used to generate a R^2 best fit line and project it to 100% done. To make it take into account the current operation a heavier weight could be placed on the newer data (or the older data could be thinned) and to make it ignore short term fluctuations, a heavy weighting can be put on the first data point.