I have a table and a query (within a PL/SQL packge) accessing that table. Statistics are collected weekly normally.
A large update has been run on the table, resulting in significantly different data distribution on a particular indexed column. The query plan used by Oracle (which I can see from v$sqlarea) is sub-optimal. If I take an explain plan on the same* query from SQL*Plus, a good plan is returned.
I have since collected statistics on the table. Oracle is still using the query plan that it originally came up with. v$sqlarea.last_load_time suggests this was a plan generated prior to the statistics generation. I thought regenerating statistics would have invalidated plans in the SQL cache.
Is there any way to remove just this statement from the SQL cache?
(* Not character-for-character, matches-in-the-SQL-cache same, but the same statement).
If you are using 10.2.0.4 or later, you should be able to use the DBMS_SHARED_POOL package to purge a single cursor from the shared pool.
I found out (when researching something else) that what I should have done was to use
no_invalidate => FALSE
When collecting the statistics by calling gather_table_stats. This would have caused all SQL plans referencing the table to immediately be invalidated.
The Oracle docs say:
Does not invalidate the dependent cursors if set to TRUE. The procedure
invalidates the dependent cursors immediately if set to FALSE. Use
DBMS_STATS.AUTO_INVALIDATE. to have Oracle decide when to invalidate dependent
cursors. This is the default.
The default of AUTO_INVALIDATE seems to cause invalidation of SQL statements within the next 5 hours. This is to stop massive number of hard-parses if you are collecting statistics on lots of objects.
Related
I have some SQL that runs quickly on its own, but is 60 times slower when the same SQL is inside a stored proc. The SQL uses temp tables in many of its queries. Why is it slower in a sp?
tl;dr
After populating the (temp) table then consider a) running create index and/or b) running update statistics on the table; both of these provide the following benefits:
forces a recompilation of any follow-on queries that reference the (temp) table
provides the compiler with stats about the (temp) table [and index]
Hard to say with 100% certainty without a minimal, reproducible example (to include ASE version and query plans) but some ideas to consider:
I'm guessing the SQL that runs quickly on its own is based on a
multi-batch process (eg, temp table created in one batch, temp table
populated in one batch, query(s) run in yet another batch) in which
case the query(s) is compiled with the benefit of having access to
some stats for the non-empty temp table
with the proc the steps (create temp table, populate temp table, run
query(s)) is all part of a single batch and therefore, as pointed
out in eric's answer, the
query(s) will (initially) be compiled based on some assumptions
about the temp table size
another possibiilty: 1st time proc is executed it's compiled based
on user1's (temp table) data set; user2 then runs the proc and ends
up using the cached version of user1's execution plan, but if
user2's (temp table) data set is considerably different (such that
user2's data would generate a different query plan) then applying
user1's query plan to user2's data could lead to a poor/slow
execution
Regardless of why the proc's query(s) is run with a less-than-optimal query plan the general 'solution' is to make sure the query(s) is compiled with some up-to-date info about the temp table.
How/when to (re)compile the proc's query(s) depends on ASE version and any number of associated scenarios:
newer ASE versions have config parameters deferred name resolution, procedure deferred compilation and optimize temp table resolution which can dictate when individual queries, within a proc, are compiled; but even this may not be enough if idea #2 (above) is in play
the proc could be created with recompile (to force a recompilation on each run) but this may not be enough if (again) idea #2 (above) is in play
creating an index (after the temp table has been populated) and/or running update statistics (again, preferably after temp table has been populated) should force a recompilation of any follow-on queries
(re)compilation of large, complex queries may extend the overall run time for the query (as opposed to re-using a pre-existing query plan); net result is that excessive recompilations can lead to overall degradation in performance
the use of abstract query plans (as well as other optimizer hints), while not capable of eliminating recompiations, can help reduce the time to (re)compile queries [this starts getting into advanced P&T work - likely a bit more than what is needed for this particular Q&A]
While the documentation's recommendation to create the temp table(s) prior to calling the proc may have some benefit, there are a few pitfalls here, too:
having to manage/remember the relationship between parent process (batch or proc where temp table is created) and the child proc (where the problematic query is run)
for a child proc called by multiple/different parent processes the same temp table DDL must be used by all parent processes otherwise the child proc could end up with excessive recompilations due to schema changes in the temp table; the issue is that if the proc notices a different temp table structure (eg, different datatypes, columns named in different order, different indexes) then it will force a recompilation
Sybase says, "When you create a table in the same stored procedure or batch where it is used, the query optimizer cannot determine how large the table is because the table was not created when the query was optimized. This applies to both temporary tables and regular user tables."
Sybase recommends creating the temp tables outside of the stored proc. Also some solutions if you create indices in certain ways. See Sybase docs for more specifics
Does you stored procedure has parameters?
If yes, SQL Server uses a process called parameter sniffing when it executes stored procedures that have parameters. When the procedure is compiled or recompiled, the value passed into the parameter is evaluated and used to create an execution plan. That value is then stored with the execution plan in the plan cache. On subsequent executions, that same value – and same plan – is used.
What happens when the values in a table you’re querying aren’t evenly distributed? What if one value would return 10 rows and another value would return 10,000 rows, or 10 million rows?
What will happen is that the first time the procedure is run and the plan is compiled, whatever value is passed in is stored with the plan. Every time it’s executed, until it’s recompiled, the same value and plan will be used – regardless of whether it is the fastest or best plan for that value.
You can force SQL Server to recompile the stored procedure each time it is run. The benefit here is that the best query plan will be created each time it is run. However, recompiling is a CPU-intensive operation. This may not be an ideal solution for stored procedures that are run frequently, or on a server that is constrained by CPU resources already.
ALTER PROCEDURE SP_Test
#ProductID INT
WITH RECOMPILE
AS
SELECT OrderID, OrderQty
FROM SalesOrderDetail
WHERE ProductID = #ProductID
I need clarification on how DBMS_STATS.GATHER_TABLE_STATS works.
I have a scenario,were i am creating table and the same table is used in the package creation also.In other words the package compilation is depended on table creation.
Is it mandatory to include DBMS_STATS.GATHER_TABLE_STATS after index creation command in table creation script.
In which situation DBMS_STATS.GATHER_TABLE_STATS works,whether it's for package compilation or package execution.Please confirm.
DBMS_STATS provides information to the Oracle optimizer, enabling Oracle to build efficient execution plans for SQL statements. Optimizer statistics are never necessary for compiling objects.
Gathering statistics is such a complicated subject it can be difficult to even ask the right questions at first. I'd recommend starting by reading the manual, such as Managing Optimizer Statistics: Basic Topics.
Typically your statistics gathering strategy should look something like this:
Gather statistics manually whenever an object's data significantly changes.
Use the default autotask to automatically gather statistics for objects that have been slowly changing or were accidentally forgotten.
To answer some specific questions, you probably do not want to gather statistics right after creating a table. Only gather statistics after changing the data. Gathering statistics on an empty table can be dangerous; Oracle will think the table has 0 rows until the next time statistics are gathered. It would be better to have no statistics at all, then Oracle can use dynamic sampling to estimate the statistics.
(But on the other hand, if the table is created as a SELECT statement and contains a lot of data you may want to gather statistics. Then again, if you're using 12c it might gather statistics automatically when creating a table.)
You also generally do not need to gather statistics after creating an index. Oracle automatically gathers index statistics when an index is created or rebuilt.
I am incrementally pulling records from a Web Service. I need to set the status of any records in our local table that were not returned by the Web Service to 'DELETED';
I had intended to store the complete list of record IDs in a PL/SQL table and then perform a single UPDATE statement based on that.
I now estimate the memory usage of the record set to be ~615MB. How does the memory footprint of a PL/SQL table compare to using a global temporary table instead? Would it use a different part of Oracle's memory, PGA vs SGA for instance?
Performance isn't a major concern because this nightly job already runs in Production in an acceptable amount of time. I don't believe adding the 'DELETED' status piece will increase the run duration to affect users.
A global temporary table would not use memory in that way. It stores the values in a temporary segment to which only your session has access, and which is dropped when no longer needed.
Have you considered what happens if your session disconnects? In either method you lose the values you have accumulated. You might like to just use a regular table.
Recently, a very strange scenario has been reported from one of of our sites.
Based on our fields, we found that there should be some kind of delete that must have happenend for that scenario
In our application code, there is no delete for that table itself. So we checked in gv$sqlarea(since we use RAC) table whether there are any delete sql for this table. We found nothing.
Then we tried to do the same kind of delete through our PL/SQL Developer. We are able to track all delete through gv$sqlarea or gv$session. But when we use below query, lock, edit and commit in plsql developer, there is no trace
select t.*, t.rowid
from <table>
Something which we are able to find is sys.mon_mods$ has the count of deletes. But it is not stored for a long time, so that we can trace by timestamp
Can anyone help me out to track this down
Oracle Version: 11.1.0.7.0
Type : RAC (5 instances)
gv$sqlarea just shows the SQL statements that are in the shared pool. If the statement is only executed once, depending on how large the shared pool and how many distinct SQL statements are executed, a statement might not be in the shared pool very long. I certainly wouldn't expect that a one-time statement would still be in the shared pool of a reasonably active system after a couple hours.
Assuming that you didn't enable auditing and that you don't have triggers that record deletes, is the system in ARCHIVELOG mode? Do you have the archived logs from the point in time where the row was deleted? If so, you could potentially use LogMiner to look through the archived logs to find the statement in question.
If I rebuild an unusable index using alter index x rebuild, will the execution plan for any SQL that used that index previously be re-evaluated?
I know that the statistics are re-calculated as part of the rebuild in the DB version I'm using - Oracle 10.2.0.4.0.
I don't want to use the dbms_stats package to force the issue on this, seeing as I've already got fresh stats!
since the index is currently unusable, no SQL query uses the index. When you collect statistics (either through a rebuild or the dbms_stats package), all statements against the base table will be reparsed (hard-parse) next time they are submitted. Plans may change due to the stats update.
Rebuilding an unusable index will therefore make it visible and usable by all statements, even those that have been previously parsed.