In ODI we used the DBMS_STATS.GATHER_SCHEMA_STATS to recompute the stats only when a table changed by a certain percentage with the option (options => 'GATHER AUTO'). (http://docs.oracle.com/cd/B19306_01/appdev.102/b14258/d_stats.htm#i1036456)
Now I want to move the calculation of statistics to the table level (in the IKL) but DBMS_STATS.GATHER_TABLE_STATS does not seem to have a setting to only recompute the stats if they need an update (determined by Oracle).
(http://docs.oracle.com/cd/B19306_01/appdev.102/b14258/d_stats.htm#i1036461)
Always recomputing the statistics of all our tables is too costly.
Does anyone know a way to check if a table needs its statistics updated or a hidden option in DBMS_STATS.GATHER_TABLE_STATS.
DBMS_STATS.GATHER_SCHEMA_STATS has an option to LIST STALE objects; you could run that first and decide if your table is in the list of returned objects.
Check the column STALE_STATS from USER_TAB_STATISTICS / ALL_TAB_STATISTICS / DBA_TAB_STATISTICS
Related
Does an extended metadata table exist in oracle 19 holding column statistics from all tables? I know there is the table ALL_TAB_COL_STATISTICS which stores histogram, min, max, num distinct values etc. but I need additional statistics such as mean, median, or percentiles? If there is such a statistics table how and when gets the table updated? I do not want to calculate the statistics myself but want to extract them as metadata.
Thanks for your help.
I believe you are looking for something that Oracle never intended to. Oracle Statistics are information useful for the CBO ( Cost Based Optimizer ) in order to determine the best execution plan for any SQL statement. It stores information regarding the density and cardinality of the data stored there, but you want analytical data of it, which is not the purpose of these statistics.
You always might rely in Oracle Functions to retrieve those:
select avg(column) as mean_rating , MEDIAN(column) OVER () AS median_value from
yourtable
Regarding percentiles, I would check PERCENTIL_CONT and PERCENTILE_DISC. But like I said, you won't find those in any metadata dictionary table.
I needed to truncate and reload a table.
I learned that truncate needs stats gathering on the table as its successor process so the database gets the actual statistics, otherwise previous stats are not cleared by the truncate statement.
After doing these two operations (truncate and stats gathering on the empty table), ran the insert... but don't see new statistics in all_tab_statistics table for my table. Sample_size is still 0.
Why is that? Shouldn't have Oracle done the automatic stats gathering after the insert?
Do I need to rerun the stats or is it just fine considering the performance around this table (please note it's going to truncate and reload each time)?
Consider the following approach. It has the advantage of the table always being present.
Create an empty new table like the old one.
Load the data into the new table. This is the slowest step.
Do whatever cleanup you might need, such as refreshing the statistics.
RENAME tables to swap the new table in place. This step is fast enough so you won't notice.
I know it's a long time since I posted my question above. But recently, we again faced the similar situation and this time below steps worked towards a much better performance on a table with 800 million rows.
Take a backup of the original table.
Truncate the original table.
Gather stats on the truncated table, so that statistics show 0 in the DB. Us CASCADE=>TRUE in the command to also include indexes in the process.
Drop the indexes on the truncated table and Insert the required data from the backup table.
Recreate the indexes and gather stats again (ofcourse, with CASCADE=>TRUE; however recreation of the indexes should ideally have calculated the appropriate stats).
Drop the backup table if not needed.
Consider this Oracle docs about indexes, this about speed of insert and this question on StackOverflow lead me to conclusion that:
Indexes helps us locate information faster
Primary and Unique Keys are indexed automatically
Inserting with indexes can cause worse performance
However every time indexes are discussed there are only SELECT operations shown as examples.
My question is: are indexes used in INSERT and UPDATE operations? When and how?
My suggestions are:
UPDATE can use index in WHERE clause (if the column in the clause has index)
INSERT can use index when uses SELECT (but in this case, index is from another table)
or probably when checking integrity constraints
but I don't have such deep knowledge of using indexes.
For UPDATE statements, index can be used by the optimiser if it deems the index can speed it up. The index would be used to locate the rows to be updated. The index is also a table in a manner of speaking, so if the indexed column is getting updated, it obviously needs to UPDATE the index as well. On the other hand if you're running an update without a WHERE clause the optimiser may choose not to use an index as it has to access the whole table, a full table scan may be more efficient (but may still have to update the index). The optimiser makes those decisions at runtime based on several parameters such as if there are valid stats against the tables and indexes in question, how much data is affected, what type of hardware, etc.
For INSERT statements though the INSERT itself does not need the index, the index will also need to be 'inserted into', so will need to be accessed by oracle. Another case where INSERT can cause the index to be used is an INSERT like this:
INSERT INTO mytable (mycolmn)
SELECT mycolumn + 10 FROM mytable;
Insert statement has no direct benefit for index. But more index on a table cause slower insert operation. Think about a table that has no index on it and if you want to add a row on it, it will find table block that has enough free space and store that row. But if that table has indexes on it database must make sure that these new rows also found via indexes, So to add new rows on a table that has indexes, also need to entry in indexes too. That multiplies the insert operation. So more index you have, more time you need to insert new rows.
For update it depends on whether you update indexed column or not. If you are not updating indexed column then performance should not be affected. Index can also speed up a update statements if the where conditions can make use of indexes.
I am trying to update statistics to some of our tables whose names I receive as input to my procedure. But, I couldn't compile the procedure with the below code.
update index statistics #tableName
Aren't dynamic table names allowed? Or, would the below statement work?
select #statsCmd = 'update index statistics '+#tableName
exec(#statsCmd)
Also, what are the notable differences between "update statistics" and "update index statistics"?
It does appear that update statistics does not allow dynamic table names, but the second statement should work without issue.
Regarding update statistics & update index statistics:
Update statistics can be run against tables without indexes, and other non-index objects, as well as against indexes. If run against an index, it actually executes an update index statistics behind the scenes. Update index statistics only updates statistics for the indices on the specified table.
Also, have you looked into using the Job Scheduler, and the datachange function to automate your update statistics?
I use DBMS_STATS.GATHER_SCHEMA_STATS procedure to collect statistics in my code.
BEGIN
DBMS_STATS.gather_schema_stats
(ownname => '<SCHEMA NAME',
estimate_percent => DBMS_STATS.auto_sample_size,
options => 'GATHER STALE',
DEGREE => NULL,
CASCADE => DBMS_STATS.auto_cascade,
granularity => 'auto'
);
end;
My question: After I executed this code, i checked the last analyzed date of tables using TOAD tool and it is showing some other past date instead of the current date when i ran the DBMS_STATS.GATHER_SCHEMA_STATS procedure.
Does this mean that the DBMS_STATS will not analyze the table?
When you specify the option GATHER STALE, you are telling Oracle that you only want to gather statistics on objects that have undergone "significant" change since the last time statistics were gathered on that object. If Oracle determines that the table has not changed much since statistics were last gathered, it will not gather statistics again.
Oracle determines that tables have changed significantly by monitoring DML on those tables (Oracle by default monitors tables in 11g-- you had to enable monitoring in earlier versions). That data is written to DBA_TAB_MODIFICATIONS periodically (roughly every few hours). When DBMS_STATS runs with the GATHER STALE option, Oracle compares the approximate number of changes in DBA_TAB_MODIFICATIONS against the prior statistics for the table to determine whether enough rows have changed to make it worthwhile to gather statistics again.