So I've been wracking my brain on this and admittedly am not very good with Oracle. We have a table that holds about 60 million records with values stored in it for buildings. Have added appropriate indexes where I thought was fit, but still poor performance. Here is the query as that should help:
SELECT count(*)
FROM viewBuildings
INNER JOIN tblValues
ON viewBuildings.bldg_id = tblValues.bldg_id
WHERE bldg_deleted = 0
AND (bldg_summary = 1
OR (bldg_root = 0 AND bldg_def = 0)
OR bldg_parent = 1)
AND field_id IN (207)
AND UPPER(dbms_lob.substr(v_value, 2000, 1)) = UPPER('2320')
So the above is just one example of a query that can be constructed. It's looking in tblValues on the v_value CLOB field for a match of '2320'. It uppercases as it can search both numeric and text values. tblValues has 60 million records. It is indexed by the building id and also the field id.
I may need to supply more info, but as far as stats go, the number that jumped out to me was "consistent gets". Consistent gets = 74069. Is that a large number?
Any advice would be great, primarily in dealing with a CLOB field on a large database table. Can't use context type indexing as I need exact matches and the data being looked up can be numeric or string.
EDIT (more information):
tblBuildings is part of viewBuildings (a view), has 80,000 records
tblValues has the values of each building, has 68,000,000 records
tblValues has about 550 fields per building (field_id)
Desired result: Query to return results in < 5 seconds. Is this unreasonable? Sometimes it will indefinitely run, other times maybe 80 seconds.
Explain Plan results
Plan hash value: 1480138519
-----------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------------------------------------|
| 0 | SELECT STATEMENT | | 1 | 192 | 32 (0)| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 192 | | |
| 2 | NESTED LOOPS | | 1 | 192 | 15 (0)| 00:00:01 |
| 3 | NESTED LOOPS | | 1 | 183 | 12 (0)| 00:00:01 |
|* 4 | FILTER | | | | | |
| 5 | NESTED LOOPS OUTER | | 1 | 64 | 10 (0)| 00:00:01 |
|* 6 | TABLE ACCESS BY INDEX ROWID | TBLBUILDINGS | 1 | 60 | 9 (0)| 00:00:01 |
|* 7 | INDEX RANGE SCAN | SAA_4 | 17 | | 3 (0)| 00:00:01 |
| 8 | NESTED LOOPS | | 1 | 21 | 3 (0)| 00:00:01 |
| 9 | TABLE ACCESS BY INDEX ROWID| TBLBUILDINGSTATUSES | 1 | 15 | 2 (0)| 00:00:01 |
|* 10 | INDEX RANGE SCAN | IDX_BUILDINGSTATUS_EXCLUDEQUERY | 1 | | 1 (0)| 00:00:01 |
|* 11 | INDEX RANGE SCAN | IDX_BUILDING_STATUS_ASID_DELETED | 1 | 6 | 1 (0)| 00:00:01 |
| 12 | TABLE ACCESS BY INDEX ROWID | TBLBUILDINGSTATUSES | 1 | 4 | 1 (0)| 00:00:01 |
|* 13 | INDEX UNIQUE SCAN | PK_TBLBUILDINGSTATUS | 1 | | 0 (0)| 00:00:01 |
|* 14 | TABLE ACCESS BY INDEX ROWID | TBLVALUES | 1 | 119 | 2 (0)| 00:00:01 |
|* 15 | INDEX UNIQUE SCAN | PK_SAA_6 | 1 | | 1 (0)| 00:00:01 |
| 16 | INLIST ITERATOR | | | | | |
|* 17 | INDEX RANGE SCAN | SAA_7 | 1 | 9 | 3 (0)| 00:00:01 |
-----------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
4 - filter("TBLBUILDINGSTATUSES"."BUILDING_STATUS_HIDE_REPORTS" IS NULL OR
"TBLBUILDINGSTATUSES"."BUILDING_STATUS_HIDE_REPORTS"=0)
6 - filter("TBLBUILDINGS"."BLDG_SUMMARY"=1 OR "TBLBUILDINGS"."BLDG_SUB_BUILDING_PARENT"=1 OR
"TBLBUILDINGS"."BLDG_BUILDING_DEF"=0 AND "TBLBUILDINGS"."BLDG_ROOT"=0)
7 - access("TBLBUILDINGS"."BLDG_DELETED"=0)
filter( NOT EXISTS (SELECT 0 FROM "TBLBUILDINGSTATUSES" "TBLBUILDINGSTATUSES","TBLBUILDINGS" "TBLBUILDINGS" WHERE
"TBLBUILDINGS"."BLDG_ID"=:B1 AND "TBLBUILDINGSTATUSES"."BUILDING_STATUS_ID"="TBLBUILDINGS"."BUILDING_STATUS_ID" AND
"TBLBUILDINGSTATUSES"."BUILDING_STATUS_EXCLUDE_QUERY"=1))
10 - access("TBLBUILDINGSTATUSES"."BUILDING_STATUS_EXCLUDE_QUERY"=1)
11 - access("TBLBUILDINGS"."BLDG_ID"=:B1 AND "TBLBUILDINGSTATUSES"."BUILDING_STATUS_ID"="TBLBUILDINGS"."BUILDING_STATUS_ID")
filter("TBLBUILDINGSTATUSES"."BUILDING_STATUS_ID"="TBLBUILDINGS"."BUILDING_STATUS_ID")
13 - access("TBLBUILDINGSTATUSES"."BUILDING_STATUS_ID"(+)="TBLBUILDINGS"."BUILDING_STATUS_ID")
14 - filter(UPPER("DBMS_LOB"."SUBSTR"("TBLVALUES"."V_VALUE",2000,1))=U'2320')
15 - access("TBLVALUES"."FE_ID"=207 AND "TBLBUILDINGS"."BLDG_ID"="TBLVALUES"."BLDG_ID")
17 - access("TBLINSPECTORBUILDINGMAP"."IN_ID"=1 AND ("TBLINSPECTORBUILDINGMAP"."IAM_BUILDING_ACCESS_LEVEL"=0 OR
"TBLINSPECTORBUILDINGMAP"."IAM_BUILDING_ACCESS_LEVEL"=1) AND "TBLBUILDINGS"."BLDG_ID"="TBLINSPECTORBUILDINGMAP"."BLDG_ID")
44 rows selected
Plan hash value: 2137789089
---------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
---------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 8168 | 16336 | 29 (0)| 00:00:01 |
| 1 | COLLECTION ITERATOR PICKLER FETCH| DISPLAY | 8168 | 16336 | 29 (0)| 00:00:01 |
---------------------------------------------------------------------------------------------
Okay, I gathered statistics as you suggested and then here is the plan_table_output. Looks like IDX_CURVAL_FE_ID is the problem here? That is an index on the values table for the field id.
SQL_ID d4aq8nsr1p6uw, child number 0
-------------------------------------
SELECT /*+ gather_plan_statistics */ count(*) FROM
viewAssetsForUser1 INNER JOIN tblCurrentValues ON
viewAssetsForUser1.as_id = tblCurrentValues.as_id WHERE as_deleted =
:"SYS_B_0" AND (as_summary = :"SYS_B_1" OR (as_root =
:"SYS_B_2" AND as_asset_def = :"SYS_B_3") OR
as_sub_asset_parent = :"SYS_B_4") AND fe_id IN (:"SYS_B_5")
AND UPPER(dbms_lob.substr(cv_value, :"SYS_B_6", :"SYS_B_7")) =
UPPER(:"SYS_B_8")
Plan hash value: 4033422776
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads | OMem | 1Mem | Used-Mem |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1 |00:08:43.19 | 56589 | 56084 | | | |
| 1 | SORT AGGREGATE | | 1 | 1 | 1 |00:08:43.19 | 56589 | 56084 | | | |
|* 2 | FILTER | | 1 | | 0 |00:08:43.19 | 56589 | 56084 | | | |
| 3 | NESTED LOOPS | | 1 | | 0 |00:08:43.19 | 56589 | 56084 | | | |
| 4 | NESTED LOOPS | | 1 | 115 | 0 |00:08:43.19 | 56589 | 56084 | | | |
|* 5 | FILTER | | 1 | | 0 |00:08:43.19 | 56589 | 56084 | | | |
|* 6 | HASH JOIN RIGHT OUTER | | 1 | 82 | 0 |00:08:43.19 | 56589 | 56084 | 1348K| 1348K| 742K (0)|
| 7 | TABLE ACCESS FULL | TBLASSETSTATUSES | 1 | 4 | 4 |00:00:00.01 | 3 | 0 | | | |
| 8 | NESTED LOOPS | | 1 | | 0 |00:08:43.19 | 56586 | 56084 | | | |
| 9 | NESTED LOOPS | | 1 | 163 | 0 |00:08:43.19 | 56586 | 56084 | | | |
|* 10 | TABLE ACCESS BY INDEX ROWID | TBLCURRENTVALUES | 1 | 163 | 0 |00:08:43.19 | 56586 | 56084 | | | |
|* 11 | INDEX RANGE SCAN | IDX_CURVAL_FE_ID | 1 | 16283 | 61357 |00:00:05.98 | 132 | 132 | | | |
|* 12 | INDEX RANGE SCAN | SAA_1 | 0 | 1 | 0 |00:00:00.01 | 0 | 0 | | | |
|* 13 | TABLE ACCESS BY INDEX ROWID | TBLASSETS | 0 | 1 | 0 |00:00:00.01 | 0 | 0 | | | |
|* 14 | INDEX UNIQUE SCAN | PK_TBLINSPECTORBRIDGEMAP2 | 0 | 1 | 0 |00:00:00.01 | 0 | 0 | | | |
|* 15 | TABLE ACCESS BY GLOBAL INDEX ROWID| TBLINSPECTORASSETMAP | 0 | 1 | 0 |00:00:00.01 | 0 | 0 | | | |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - filter(:SYS_B_0=0)
5 - filter(("TBLASSETSTATUSES"."ASSET_STATUS_HIDE_REPORTS" IS NULL OR "TBLASSETSTATUSES"."ASSET_STATUS_HIDE_REPORTS"=0))
6 - access("TBLASSETSTATUSES"."ASSET_STATUS_ID"="TBLASSETS"."ASSET_STATUS_ID")
10 - filter(UPPER("DBMS_LOB"."SUBSTR"("TBLCURRENTVALUES"."CV_VALUE",:SYS_B_6,:SYS_B_7))=SYS_OP_C2C(UPPER(:SYS_B_8)))
11 - access("TBLCURRENTVALUES"."FE_ID"=:SYS_B_5)
12 - access("TBLASSETS"."AS_DELETED"=:SYS_B_0 AND "TBLASSETS"."AS_ID"="TBLCURRENTVALUES"."AS_ID")
13 - filter((("TBLASSETS"."AS_ROOT"=:SYS_B_2 AND "TBLASSETS"."AS_ASSET_DEF"=:SYS_B_3) OR "TBLASSETS"."AS_SUMMARY"=:SYS_B_1 OR
"TBLASSETS"."AS_SUB_ASSET_PARENT"=:SYS_B_4))
14 - access("TBLASSETS"."AS_ID"="TBLINSPECTORASSETMAP"."AS_ID" AND "TBLINSPECTORASSETMAP"."IN_ID"=1)
15 - filter(("TBLINSPECTORASSETMAP"."IAM_ASSET_ACCESS_LEVEL"=0 OR "TBLINSPECTORASSETMAP"."IAM_ASSET_ACCESS_LEVEL"=1))
Bad Index Cost If the stats are fresh, and the optimizer has a relatively good cardinality estimate, why would it pick a bad plan? Perhaps there is a parameter making indexes look artificially cheap. Take a look at: select * from v$parameter where name in ('optimizer_index_cost_adj', 'optimizer_index_caching'); Are they significantly different than their default values, 100 and 0?
Also, take a look at select * from sys.aux_stats$; Maybe your system statistics make full table scans look too expensive. Some versions of Oracle have bugs with workload statistics where the numbers are wrong by several orders of magnitude.
Or perhaps your table is just incredibly huge, and 16K index reads is the best access path. Look at DBA_SEGMENTS.BYTES to find the size of your table and LOB segment.
Even if the table is medium-sized, and the plan changed to a full table scan, that probably won't get the run time to within 5 seconds. But combined with your idea to partition, it might be enough.
LOB STORAGE Given your example, I assume most of the CLOBs are relatively small? Perhaps you have an unusual LOB setting wasting a lot of space, such as DISABLE STORAGE IN ROW. You may want to check your table DDL, or post all of it here. Or if you can replace the CLOB with a VARCHAR2, that would be even better.
FBI A function-based index on the CLOB may significantly speed things up. But it may be a very large index: create index TBLCURRENTVALUES_FBI on TBLCURRENTVALUES(UPPER(dbms_lob.substr(v_value, 2000, 1)));
CURSOR_SHARING The queries are changing a bit, which makes tuning difficult. Looks like this latest version has CURSOR_SHARING=FORCE, which is unusual. For an expensive query, using literals can be a good thing - the extra time spent building query plans is probably worth it. If the system parameter can't change, look into the hint /*+ cursor_sharing_exact */.
You can do any number of optimization but ultimately its the huge amount of data which causes the problem. When you execute the query and track it on performance graph on OEM , you will that major amount of time will be spent on IO. That is taking data in and out of memory.
So whats the solution: It will be to partition the table. Whenever data is huge , you should look to partition the table so that you deal with only relevant data.
In order to partition the table you need some point to segregate the data and looking at your data it can be building id.
You can read more about it at this url : http://docs.oracle.com/cd/E11882_01/server.112/e25523/partition.htm#g471747
Partitioning comes up with many other features, like local indexes which help to optimize queries even more.
Partitioning will not be a solution if you are deal with whole of large table data all the time but then that puts a question mark over database schema.
SO yes query optimization will help but as data is large you should evaluate table partitioning as well.
Related
I have an application that runs automated process against the database on a very regular basis. Unfortunately, the statement was crafted inside an ORM and can't be rewritten.
The statement is as follows:
SELECT t0.id FROM SCHEMA.CMTS t0, SCHEMA.CMNTSRCH t3, SCHEMA.CONT t1, SCHEMA.YU t2 WHERE (
(
t0.YUtypeyn = 0
OR
t0.YUtypeyn = 1
)
AND
(
(
t1.contcd IN ('FD')
OR
t1.contcd IS NULL
)
AND
(
t2.TRNSMDCD = 'RD'
AND
t2.spid = 1
AND
t0.compyn = NULL
AND
(
NOT
(
upper(t3.curryuloc) LIKE upper('ABC%') ESCAPE '\'
)
AND
NOT
(
upper(t3.curryuloc) LIKE upper('DEF%') ESCAPE '\'
)
AND
NOT
(
upper(t3.curryuloc) LIKE upper('GHI%') ESCAPE '\'
)
AND
NOT
(
upper(t3.curryuloc) LIKE upper('%JKL%') ESCAPE '\'
)
AND
NOT
(
upper(t3.curryuloc) LIKE upper('%MNO%') ESCAPE '\'
)
AND
NOT
(
upper(t3.curryuloc) LIKE upper('%PQR%') ESCAPE '\'
)
AND
NOT
(
upper(t3.curryuloc) LIKE upper('%STU%') ESCAPE '\'
)
AND
NOT
(
upper(t3.curryuloc) LIKE upper('%VWX%') ESCAPE '\'
)
AND
NOT
(
upper(t3.curryuloc) LIKE upper('%YZ%') ESCAPE '\'
)
)
AND
t0.cancelledyn = NULL
)
)
)
AND
t0.CMNTSRCHid = t3.id
AND
t0.contcd = t1.contcd
AND
t0.YUid = t2.id(+)
Performance issues have been noticed while this statement is running. When monitoring the v$session_longops, I noticed a full table scan against CMNTSRCH.
2640 seconds elapsed so far for a full table scan - which raised concerns.
Explain plan is as follows:
------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 59 | 0 (0)| |
|* 1 | FILTER | | | | | |
| 2 | NESTED LOOPS | | 1 | 59 | 26677 (1)| 00:00:02 |
| 3 | NESTED LOOPS | | 1 | 59 | 26677 (1)| 00:00:02 |
| 4 | MERGE JOIN CARTESIAN | | 1 | 27 | 26674 (1)| 00:00:02 |
| 5 | NESTED LOOPS | | 1 | 13 | 24369 (1)| 00:00:01 |
|* 6 | INDEX UNIQUE SCAN | CONT_PK | 1 | 5 | 0 (0)| 00:00:01 |
|* 7 | TABLE ACCESS FULL | CMNTSRCH | 1 | 8 | 24369 (1)| 00:00:01 |
| 8 | BUFFER SORT | | 36824 | 503K| 2305 (1)| 00:00:01 |
|* 9 | INDEX FAST FULL SCAN | IDX_YU20220408 | 36824 | 503K| 2305 (1)| 00:00:01 |
|* 10 | INDEX RANGE SCAN | CMNT_013 | 1 | | 2 (0)| 00:00:01 |
|* 11 | TABLE ACCESS BY INDEX ROWID| CMNT | 1 | 32 | 3 (0)| 00:00:01 |
------------------------------------------------------------------------------------------------------
Reviewing the statement, I decided to create a function based index on t3.id and t3.curryuloc and reviewed the explain plan - it appeared to improve CPU cost and now performs an Index Fast Full scan as opposed to a Table Scan.
New explain plan (after Function Based index creation):
----------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 59 | 0 (0)| |
|* 1 | FILTER | | | | | |
| 2 | NESTED LOOPS | | 1 | 59 | 3975 (1)| 00:00:01 |
| 3 | NESTED LOOPS | | 1 | 59 | 3975 (1)| 00:00:01 |
| 4 | MERGE JOIN CARTESIAN | | 1 | 27 | 3972 (1)| 00:00:01 |
| 5 | NESTED LOOPS | | 1 | 13 | 1667 (2)| 00:00:01 |
|* 6 | INDEX UNIQUE SCAN | CONT_PK | 1 | 5 | 0 (0)| 00:00:01 |
|* 7 | INDEX FAST FULL SCAN | IDX_CMNTSRCH_20220407 | 1 | 8 | 1667 (2)| 00:00:01 |
| 8 | BUFFER SORT | | 36824 | 503K| 2305 (1)| 00:00:01 |
|* 9 | INDEX FAST FULL SCAN | IDX_YU20220408 | 36824 | 503K| 2305 (1)| 00:00:01 |
|* 10 | INDEX RANGE SCAN | CMNT_013 | 1 | | 2 (0)| 00:00:01 |
|* 11 | TABLE ACCESS BY INDEX ROWID| CMNT | 1 | 32 | 3 (0)| 00:00:01 |
----------------------------------------------------------------------------------------------------------------
However, this operation is also taking an incredibly long time
Monitoring the v$session_longops, I can see when a new session runs the statement. SOFAR starts at about 4000 blocks, and gets slower and slower, almost grinding to a halt by the time it gets to 5500 blocks. Forcing the use of the Function based index doesn't appear to have made any difference to the time it takes for the operation to complete.
Oddly though, if I run the statement manually with the same arguments the app binds to the parameters, it take almost no time to execute (there are no rows returned).
What should be my next steps to troubleshoot this issue?
You have a whole mess of WHERE clauses like this:
AND (NOT(upper(t3.curryuloc) LIKE upper('ABC%') ESCAPE '\')
See the upper() function on the column CMNTSRCH.curryuloc? It can't random-access an ordinary index.
But Oracle has function indexes. You may be able to get better performance if you create an index like this:
CREATE INDEX whatever_1 ON CMNTSRCH (UPPER(curryuloc), id);
or, reversing the column order, like this:
CREATE INDEX whatever_2 ON CMNTSRCH (id, UPPER(curryuloc));
It's worth a try.
But, it has to be said, that cascade of AND...NOT...LIKE clauses isn't written for efficiency.
My table has millions of records. In this query below, can I make Oracle 12c examine the first X rows only instead of doing a full table scan?
The value of X, I imagine should be Offset + Fetch Next , so in this case 15
SELECT * FROM table OFFSET 5 ROWS FETCH NEXT 10 ROWS ONLY;
Thanks in advance
Edit 1
These are the tables involved and this is the actual query
Orders - This table has 113k records in my test DB ( and over 8 million in prod db like my original question mentioned)
--------------------------
| Id | SKUField1|SKUField2|
--------------------------
| 1 | Value1 | Value2 |
| 2 | Value2 | Value2 |
| 3 | Value1 | Value3 |
--------------------------
Products - This table has 2 million records in my test DB ( prod db is similar)
---------------
| PId| SKU_NUM|
---------------
| 1 | Value1 |
| 2 | Value2 |
| 3 | Value3 |
---------------
Note that values of Orders.SKUField1 and Orders.SKUField2 come from the Products.SKU_NUM values
Actual Query:
SELECT /*+ gather_plan_statistics */ Id, PId, SKUField1, SKUField2, SKU_NUM
FROM Orders
LEFT JOIN (
-- this inner query reduces size of Products from 2 million rows down to 1462 rows
select * from Products where SKU_NUM in (
select SKUField1 from Orders
)
) p1 ON SKUField1 = p1.SKU_NUM
LEFT JOIN (
-- this inner query reduces size of table B from 2 million rows down to 459 rows
select * from Products where SKU_NUM in (
select SKUField2 from Orders
)
) p4 ON SKUField2 = p4.SKU_NUM
OFFSET 5 ROWS FETCH NEXT 10 ROWS ONLY
Execution Plan:
--------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Time | A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem |
--------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 10 |00:00:00.06 | 8013 | | | |
|* 1 | VIEW | | 1 | 00:00:01 | 10 |00:00:00.06 | 8013 | | | |
|* 2 | WINDOW NOSORT STOPKEY | | 1 | 00:00:01 | 15 |00:00:00.06 | 8013 | 27M| 1904K| |
|* 3 | HASH JOIN RIGHT OUTER | | 1 | 00:00:01 | 15 |00:00:00.06 | 8013 | 1162K| 1162K| 1344K (0)|
| 4 | VIEW | | 1 | 00:00:01 | 1462 |00:00:00.04 | 6795 | | | |
| 5 | NESTED LOOPS | | 1 | 00:00:01 | 1462 |00:00:00.04 | 6795 | | | |
| 6 | NESTED LOOPS | | 1 | 00:00:01 | 1462 |00:00:00.04 | 5333 | | | |
| 7 | SORT UNIQUE | | 1 | 00:00:01 | 1469 |00:00:00.04 | 3010 | 80896 | 80896 |71680 (0)|
| 8 | TABLE ACCESS FULL | Orders | 1 | 00:00:01 | 113K|00:00:00.02 | 3010 | | | |
|* 9 | INDEX UNIQUE SCAN | UIX_Product_SKU_NUM | 1469 | 00:00:01 | 1462 |00:00:00.01 | 2323 | | | |
| 10 | TABLE ACCESS BY INDEX ROWID | Products | 1462 | 00:00:01 | 1462 |00:00:00.01 | 1462 | | | |
|* 11 | HASH JOIN RIGHT OUTER | | 1 | 00:00:01 | 15 |00:00:00.02 | 1218 | 1142K| 1142K| 1335K (0)|
| 12 | VIEW | | 1 | 00:00:01 | 459 |00:00:00.02 | 1213 | | | |
| 13 | NESTED LOOPS | | 1 | 00:00:01 | 459 |00:00:00.02 | 1213 | | | |
| 14 | NESTED LOOPS | | 1 | 00:00:01 | 459 |00:00:00.02 | 754 | | | |
| 15 | SORT UNIQUE | | 1 | 00:00:01 | 462 |00:00:00.02 | 377 | 24576 | 24576 |22528 (0)|
| 16 | INDEX FAST FULL SCAN | Orders_SKUField2_IDX6 | 1 | 00:00:01 | 113K|00:00:00.01 | 377 | | | |
|* 17 | INDEX UNIQUE SCAN | UIX_Product_SKU_NUM | 462 | 00:00:01 | 459 |00:00:00.01 | 377 | | | |
| 18 | TABLE ACCESS BY INDEX ROWID| Products | 459 | 00:00:01 | 459 |00:00:00.01 | 459 | | | |
| 19 | TABLE ACCESS FULL | Orders | 1 | 00:00:01 | 15 |00:00:00.01 | 5 | | | |
--------------------------------------------------------------------------------------------------------------------------------------------------
Hence, based on the "A-Rows" column values for row Ids 8 and 16 in the execution plan, it seems like there are full table scans on the Orders table (though row id 16 atleast seems to be using an index). So my question is is it true that there is a full table scan on the orders table even though I am using Offset/Fetch Next
Although your FETCH clause may use a full table scan, Oracle will still only fetch the first X rows from the table.
In the following example, the "TABLE ACCESS FULL" operation does start to read the entire table, but it gets cutoff part of the way through by the "WINDOW NOSORT STOPKEY" operation. Not all full table scans actually scan the full table. You would see similar behavior if your code ended with WHERE ROWNUM <= 50.
CREATE TABLE some_table AS SELECT * FROM all_objects;
EXPLAIN PLAN FOR SELECT * FROM some_table OFFSET 5 ROWS FETCH NEXT 10 ROWS ONLY;
SELECT * FROM TABLE(dbms_xplan.display);
Plan hash value: 2559837639
-------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 15 | 7410 | 2 (0)| 00:00:01 |
|* 1 | VIEW | | 15 | 7410 | 2 (0)| 00:00:01 |
|* 2 | WINDOW NOSORT STOPKEY| | 15 | 2010 | 2 (0)| 00:00:01 |
| 3 | TABLE ACCESS FULL | SOME_TABLE | 15 | 2010 | 2 (0)| 00:00:01 |
-------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("from$_subquery$_002"."rowlimit_$$_rownumber"<=15 AND
"from$_subquery$_002"."rowlimit_$$_rownumber">5)
2 - filter(ROW_NUMBER() OVER ( ORDER BY NULL )<=15)
The performance implications get more complicated if you also want to order the results. If that is the case, you may want to post the full query and execution plan.
(EDIT: 2022-09-25)
Yes, there is a full table scan on the ORDERS table happening on line 8 of the execution plan. As you mentioned, you can look at the "A-rows" column to tell what's really happening.
But the third full table scan of ORDERS, on line 19, is not a "full" full table scan. The operation "WINDOW NOSORT STOPKEY" stops that full table scan as soon as the 15 necessary rows are read. So the FETCH syntax is helping at least a little.
Applying a FETCH to a query does not mean that every single table will be limited. Although, in your query, it does seem like there ought to be a way to reduce the full table scans. Perhaps an index on SKUField1 would help?
Since Oracle as I know don't provide something like limit or top you can created by yourself like the following:
what is happening here, the inner query gets all the first 10 records and the outer query get those, you can still use any clauses like where or order or any others
SELECT * FROM (
SELECT * FROM Customers WHERE CustomerID <= 10 ORDER BY CustomerID
)
The full article will be found about this topic here at Oracle-Fetch
I am using Online Oracle so you can try it from your end, please let me know if you still have a problem.
I seem to have an odd issue regarding an Oracle view that has functions defined for columns and when those functions are evaluated.
Let's say I have the following view and function definition:
CREATE OR REPLACE VIEW test_view_one AS
SELECT column_one,
a_package.function_that_returns_a_value(column_one) function_column
FROM a_table;
CREATE OR REPLACE PACKAGE BODY a_package AS
FUNCTION function_that_returns_a_value(p_key VARCHAR2) RETURN VARCHAR2 IS
CURSOR a_cur IS
SELECT value
FROM table_b
WHERE key = p_key;
p_temp VARCHAR2(30);
BEGIN
-- Code here to write into a temp table. The function call is autonomous.
OPEN a_cur;
FETCH a_cur INTO p_temp;
CLOSE a_cur;
RETURN p_temp;
END function_that_returns_a_value;
END a_package;
In general, I would expect that if function_column is included in a query then for every row brought back by that query, the function would be run. This seems to be true in some circumstances but not for others.
For example, let's say I have the following:
SELECT pageouter,*
FROM(WITH page_query AS (SELECT *
FROM test_view_one
ORDER BY column_one)
SELECT page_query.*, ROWNUM as innerrownum
FROM page_query
WHERE rownum <= 25) pageouter WHERE pageouter.innerrownum >= 1
In this scenario, that inner query (the one querying test_view_one) brings back around 90,000 records.
If I define the function as inserting into a temporary table then I can tell that the function ran 25 times, once for each row brought back. Exactly what I would expect.
However, if I add a significant where clause on to that inner query, e.g.
SELECT pageouter,*
FROM(WITH page_query AS (SELECT *
FROM test_view_one
WHERE EXISTS (SELECT 'x' FROM some_table WHERE ...)
AND NOT EXISTS (SELECT 'x' FROM some_other_table WHERE ...)
AND EXISTS (SELECT 'x' FROM another_table WHERE ...)
ORDER BY column_one)
SELECT page_query.*, ROWNUM as innerrownum
FROM page_query
WHERE rownum <= 25) pageouter WHERE pageouter.innerrownum >= 1
Then the number of rows being brought back by the inner query is 60,000 and if I then query the temporary table, I can tell the function has run 60,000 times. Unsurprisingly, this pretty much destroys performance of the query.
The queries above are run as part of a paging implementation which is why we only ever bring back 25 rows and is why we only ever need the functions to be run for those 25 rows.
I should add, if I change the WHERE clause (i.e. I remove some of the conditions) then the query goes back to behaving it self, only running the functions for the 25 rows that are actually brought back.
Does anyone have any idea as to when functions in views are evaluated? Or anyway in determining what causes it or a way of identifying when the functions are evaluated (I've checked the explain plan and there's nothing in there which seems to give it away). If I knew that then I could hopefully find a solution to the problem but there seems to be little documentation other than "They'll run for each row brought back" which is clearly not the case in some scenarios.
I fully appreciate it's difficult to work out what's going on without a working schema but if you need anymore info then please feel free to ask.
Many Thanks
Additional Info as Requested.
Below is the actual explain plan that I get out of the production environment. The table names don't match the above query (in fact there's considerably more tables involved but they're all joined by NOT EXISTS statements within the WHERE clause.)
The DEMISE table, is the equivalent of the A_TABLE in the above query.
It's worth noting that stats were gathered just before I ran the explain plan to make it as accurate as possible.
My understanding of this is that the VIEW row is where the functions would be evaluated, which occurs AFTER the rows have been filtered down. My understanding is obviously flawed!
So this is the bad plan, the one that calls the function 60,000 times...
Execution Plan
----------------------------------------------------------
-------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|
-------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 5 | 10230 | 984 (1)|
| 1 | FAST DUAL | | 1 | | 2 (0)|
| 2 | FAST DUAL | | 1 | | 2 (0)|
|* 3 | VIEW | | 5 | 10230 | 984 (1)|
|* 4 | COUNT STOPKEY | | | | |
| 5 | VIEW | | 5 | 10165 | 984 (1)|
|* 6 | SORT ORDER BY STOPKEY | | 5 | 340 | 984 (1)|
| 7 | COUNT | | | | |
|* 8 | FILTER | | | | |
|* 9 | HASH JOIN RIGHT OUTER | | 5666 | 376K| 767 (1)|
|* 10 | INDEX RANGE SCAN | USERDATAI1 | 1 | 12 | 2 (0)|
|* 11 | HASH JOIN RIGHT ANTI | | 5666 | 309K| 765 (1)|
|* 12 | INDEX FAST FULL SCAN | TNNTMVINI1 | 1 | 17 | 35 (0)|
|* 13 | HASH JOIN RIGHT ANTI | | 6204 | 236K| 729 (1)|
|* 14 | INDEX RANGE SCAN | CODESGENI3 | 1 | 10 | 2 (0)|
|* 15 | INDEX FULL SCAN | DEMISEI4 | 6514 | 184K| 727 (1)|
| 16 | NESTED LOOPS | | 1 | 25 | 3 (0)|
| 17 | NESTED LOOPS | | 1 | 25 | 3 (0)|
|* 18 | INDEX RANGE SCAN | PROPERTY_GC | 1 | 15 | 2 (0)|
|* 19 | INDEX UNIQUE SCAN | CODESGENI1 | 1 | | 0 (0)|
|* 20 | TABLE ACCESS BY INDEX ROWID| CODESGEN | 1 | 10 | 1 (0)|
| 21 | TABLE ACCESS FULL | QCDUAL | 1 | | 3 (0)|
|* 22 | INDEX RANGE SCAN | DMSELEASI4 | 1 | 21 | 2 (0)|
|* 23 | INDEX RANGE SCAN | TNNTMVINI1 | 1 | 17 | 1 (0)|
| 24 | TABLE ACCESS FULL | QCDUAL | 1 | | 3 (0)|
-------------------------------------------------------------------------------------------
This is the good plan. This calls the function 25 times but has some of the not exists statements removed from the where clause.
Execution Plan
----------------------------------------------------------
----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|
----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 25 | 54200 | 144 (0)|
| 1 | FAST DUAL | | 1 | | 2 (0)|
| 2 | FAST DUAL | | 1 | | 2 (0)|
|* 3 | VIEW | | 25 | 54200 | 144 (0)|
|* 4 | COUNT STOPKEY | | | | |
| 5 | VIEW | | 26 | 56030 | 144 (0)|
| 6 | COUNT | | | | |
|* 7 | FILTER | | | | |
| 8 | NESTED LOOPS ANTI | | 30 | 3210 | 144 (0)|
| 9 | NESTED LOOPS OUTER | | 30 | 2580 | 114 (0)|
| 10 | NESTED LOOPS ANTI | | 30 | 2220 | 84 (0)|
| 11 | NESTED LOOPS ANTI | | 32 | 1824 | 52 (0)|
| 12 | TABLE ACCESS BY INDEX ROWID| DEMISE | 130K| 5979K| 18 (0)|
| 13 | INDEX FULL SCAN | DEMISEI4 | 34 | | 3 (0)|
|* 14 | INDEX RANGE SCAN | CODESGENI3 | 1 | 10 | 1 (0)|
|* 15 | INDEX RANGE SCAN | TNNTMVINI1 | 1 | 17 | 1 (0)|
|* 16 | INDEX RANGE SCAN | USERDATAI1 | 1 | 12 | 1 (0)|
|* 17 | INDEX RANGE SCAN | DMSELEASI4 | 1 | 21 | 1 (0)|
| 18 | TABLE ACCESS FULL | QCDUAL | 1 | | 3 (0)|
----------------------------------------------------------------------------------------
I fully appreciate the second plan is doing less but that doesn't explain why the functions aren't being evaluated... at least not that I can work out.
The Pagination with ROWNUM may be performed
in two ways:
A) full scan the row source with optimized sorting (limited to the top N rows) or
B) index access of the row source with no sort at all
Here simplified example of case A
SELECT *
FROM
(SELECT a.*,
ROWNUM rnum
FROM
( SELECT * FROM test_view_one ORDER BY id
) a
WHERE ROWNUM <= 25
)
WHERE rnum >= 1
The corresponding execution plan looks as follows (Note that I presend also part
of column projection - I will soon explain why):
-----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 25 | 975 | | 1034 (1)| 00:00:01 |
|* 1 | VIEW | | 25 | 975 | | 1034 (1)| 00:00:01 |
|* 2 | COUNT STOPKEY | | | | | | |
| 3 | VIEW | | 90000 | 2285K| | 1034 (1)| 00:00:01 |
|* 4 | SORT ORDER BY STOPKEY| | 90000 | 439K| 1072K| 1034 (1)| 00:00:01 |
| 5 | TABLE ACCESS FULL | TEST | 90000 | 439K| | 756 (1)| 00:00:01 |
-----------------------------------------------------------------------------------------
Column Projection Information (identified by operation id):
-----------------------------------------------------------
...
3 - "A"."ID"[NUMBER,22], "A"."FUNCTION_COLUMN"[NUMBER,22]
4 - (#keys=1) "ID"[NUMBER,22], "MY_PACKAGE"."MY_FUNCTION"("ID")[22]
5 - "ID"[NUMBER,22]
Within the execution the table is accessed with FULL SCAN, i.e. all records are red.
The optimization is in the SORT operation: SORT ORDER BY STOPKEY means that not all
rows are sorted, but only the top 25 are kept and sortet.
Here the execution plan for case B
--------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 25 | 975 | 2 (0)| 00:00:01 |
|* 1 | VIEW | | 25 | 975 | 2 (0)| 00:00:01 |
|* 2 | COUNT STOPKEY | | | | | |
| 3 | VIEW | | 26 | 676 | 2 (0)| 00:00:01 |
|* 4 | INDEX RANGE SCAN| TEST_IDX | 26 | 130 | 2 (0)| 00:00:01 |
--------------------------------------------------------------------------------
Here are accessed only the required 25 rows and therefore the function can't be called more that the N times.
Now the important consideration, in case A, the function can, but need not be called for each row. How do we see it?
The answer is in the column projection in the explain plan.
4 - (#keys=1) "ID"[NUMBER,22], "MY_PACKAGE"."MY_FUNCTION"("ID")[22]
The relevant line 4 show, that the function is called in the SORT operation and therefor for each line. (Sort gets all the rows).
Conclusion
My test on 11.2 shows that in case A (FULL SCAN with SORT ORDER BY STOPKEY) the view function is called
once per each row.
I guess the only workaround is to select only the ID, limit the result and than join back the original view to get the function value.
Final notes
I tested this in 12.1 as well and see below the shift in the column projection.
The function is calculated first in the VIEW (line 3), i.e. both cases works fine.
Column Projection Information (identified by operation id):
-----------------------------------------------------------
...
3 - "A"."ID"[NUMBER,22], "A"."FUNCTION_COLUMN"[NUMBER,22]
4 - (#keys=1) "ID"[NUMBER,22]
5 - "ID"[NUMBER,22]
And of course in 12c the new feature of OFFSET - FETCH NEXT could be used.
Good Luck!
we are facing issue with one of query which joins between few tables.
even though there are few hundreds of records in the table the plan is going to Merge join thinking only one record in the table, please find the below plan.
when the Merge sort plan is used the query fails with temp space issue.
oracle choose Merge plan only when job loaded to newly created partition. but the rest of old partitions it is choosing Hash Join where we get results in few seconds.
for information. All the joined table has same volume.
could you please explain why this is happening?
Merg join( query hung)
-----------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
-----------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 712 | 36 (3)| 00:00:01 | | |
|* 1 | HASH JOIN | | 1 | 712 | 36 (3)| 00:00:01 | | |
| 2 | MERGE JOIN CARTESIAN | | 1 | 679 | 28 (0)| 00:00:01 | | |
| 3 | MERGE JOIN CARTESIAN | | 1 | 615 | 21 (0)| 00:00:01 | | |
| 4 | MERGE JOIN CARTESIAN | | 1 | 388 | 14 (0)| 00:00:01 | | |
| 5 | PARTITION RANGE SINGLE | | 1 | 105 | 7 (0)| 00:00:01 | 4 | 4 |
|* 6 | TABLE ACCESS FULL | PCA_DCM_CLNT_BSPKE_REFS_M_LND | 1 | 105 | 7 (0)| 00:00:01 | 4 | 4 |
| 7 | BUFFER SORT | | 1 | 283 | 7 (0)| 00:00:01 | | |
| 8 | PARTITION RANGE SINGLE| | 1 | 283 | 7 (0)| 00:00:01 | 4 | 4 |
|* 9 | TABLE ACCESS FULL | PCA_DCM_INDBTDNS_BLK_M_LND | 1 | 283 | 7 (0)| 00:00:01 | 4 | 4 |
| 10 | BUFFER SORT | | 1 | 227 | 14 (0)| 00:00:01 | | |
| 11 | PARTITION RANGE SINGLE | | 1 | 227 | 7 (0)| 00:00:01 | 4 | 4 |
|* 12 | TABLE ACCESS FULL | PCA_DCM_DELPHI_BLK_M_LND | 1 | 227 | 7 (0)| 00:00:01 | 4 | 4 |
| 13 | BUFFER SORT | | 1 | 64 | 21 (0)| 00:00:01 | | |
| 14 | PARTITION RANGE SINGLE | | 1 | 64 | 7 (0)| 00:00:01 | 4 | 4 |
|* 15 | TABLE ACCESS FULL | PCA_DCM_APACS_BLK_M_LND | 1 | 64 | 7 (0)| 00:00:01 | 4 | 4 |
| 16 | PARTITION RANGE SINGLE | | 1 | 33 | 7 (0)| 00:00:01 | 4 | 4 |
|* 17 | TABLE ACCESS FULL | PCA_DCM_SCORE_BLK_M_LND | 1 | 33 | 7 (0)| 00:00:01 | 4 | 4 |
-----------------------------------------------------------------------------------------------------------------------------
Hash join(Few seconds we get the results)
----------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
----------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 491 | 341K| 74 (3)| 00:00:01 | | |
|* 1 | HASH JOIN | | 491 | 341K| 74 (3)| 00:00:01 | | |
| 2 | PARTITION RANGE SINGLE | | 493 | 109K| 14 (0)| 00:00:01 | 3 | 3 |
|* 3 | TABLE ACCESS FULL | PCA_DCM_DELPHI_BLK_M_LND | 493 | 109K| 14 (0)| 00:00:01 | 3 | 3 |
|* 4 | HASH JOIN | | 491 | 232K| 60 (4)| 00:00:01 | | |
|* 5 | HASH JOIN | | 492 | 99384 | 45 (3)| 00:00:01 | | |
|* 6 | HASH JOIN | | 492 | 47724 | 31 (4)| 00:00:01 | | |
| 7 | PARTITION RANGE SINGLE| | 493 | 16269 | 14 (0)| 00:00:01 | 3 | 3 |
|* 8 | TABLE ACCESS FULL | PCA_DCM_SCORE_BLK_M_LND | 493 | 16269 | 14 (0)| 00:00:01 | 3 | 3 |
| 9 | PARTITION RANGE SINGLE| | 493 | 31552 | 16 (0)| 00:00:01 | 3 | 3 |
|* 10 | TABLE ACCESS FULL | PCA_DCM_APACS_BLK_M_LND | 493 | 31552 | 16 (0)| 00:00:01 | 3 | 3 |
| 11 | PARTITION RANGE SINGLE | | 493 | 51765 | 14 (0)| 00:00:01 | 3 | 3 |
|* 12 | TABLE ACCESS FULL | PCA_DCM_CLNT_BSPKE_REFS_M_LND | 493 | 51765 | 14 (0)| 00:00:01 | 3 | 3 |
| 13 | PARTITION RANGE SINGLE | | 493 | 136K| 14 (0)| 00:00:01 | 3 | 3 |
|* 14 | TABLE ACCESS FULL | PCA_DCM_INDBTDNS_BLK_M_LND | 493 | 136K| 14 (0)| 00:00:01 | 3 | 3 |
----------------------------------------------------------------------------------------------------------------------------
Please find the query
SELECT
substr(BLK.ACC_NUM,1,14) AS ACCOUNT_NUMBER,
CASE WHEN SUBSTR(BLK.ACC_NUM,20,1) = '1' THEN 'F1'
WHEN SUBSTR(BLK.ACC_NUM,20,1) = ' ' THEN 'F1'
WHEN SUBSTR(BLK.ACC_NUM,20,1) = '0' THEN 'F1'
WHEN SUBSTR(BLK.ACC_NUM,20,1) = '2' THEN 'F2'
END FLTR,
DELPHI.ND_SPA_CII_SPA
FROM
BUR_LND.PCA_DCM_SCORE_BLK_M_LND BLK
INNER JOIN BUR_LND.PCA_DCM_CLNT_BSPKE_REFS_M_LND REFFS
ON BLK.ACC_NUM= REFFS.ACC_NUM
INNER JOIN BUR_LND.PCA_DCM_INDBTDNS_BLK_M_LND IND
ON BLK.ACC_NUM= IND.ACC_NUM
INNER JOIN BUR_LND.PCA_DCM_DELPHI_BLK_M_LND DELPHI
ON BLK.ACC_NUM= DELPHI.ACC_NUM
INNER JOIN BUR_LND.PCA_DCM_APACS_BLK_M_LND APACS
ON BLK.ACC_NUM= APACS.ACC_NUM
WHERE
BLK.EFF_DT=TO_DATE('2013-10-30','YYYY-MM-DD')
AND REFFS.EFF_DT=TO_DATE('2013-10-30','YYYY-MM-DD')
AND IND.EFF_DT=TO_DATE('2013-10-30','YYYY-MM-DD')
AND DELPHI.EFF_DT=TO_DATE('2013-10-30','YYYY-MM-DD')
AND APACS.EFF_DT=TO_DATE('2013-10-30','YYYY-MM-DD')
Thanks in advance for help.
Thanks
arkesh
The plans are bad because the new partition is missing statistics. Statistics should be updated after partition changes, ideally using incremental statistics. If that's not possible then a hint like /*+ dynamic_sampling(4) */ can help.
Statistics can be accurate, inaccurate, or missing. Missing statistics are generally not a huge problem because of dynamic sampling. With the default dynamic sampling level, 2, Oracle will automatically gather statistics if a statement includes tables without statistics.
Unfortunately for this case, Oracle only considers missing statistics per table, not per partition. (That would be a good feature request, but that won't help you right now.) With the literals in the SQL statement Oracle appears to know exactly which partition to look in. Since there are no statistics for that partition it assumes there are no rows, leading to the bad plans.
Example
Create a sample partitioned table with 1000 rows but no gathered statistics.
create table partition_test
(
a number,
b number
)
partition by range (a)
(
partition p1 values less than (2)
);
insert into partition_test select 1, level from dual connect by level <= 1000;
When there are no statistics Oracle will use dynamic sampling automatically and get a good row count. You can't see it in this simple plan, but normally this would lead to better access methods and better join operations.
explain plan for select * from partition_test where a = 1 and b <= 1000;
select * from table(dbms_xplan.display);
Plan hash value: 4097357352
---------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
---------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1000 | 26000 | 16 (0)| 00:00:01 | | |
| 1 | PARTITION RANGE SINGLE| | 1000 | 26000 | 16 (0)| 00:00:01 | 1 | 1 |
|* 2 | TABLE ACCESS FULL | PARTITION_TEST | 1000 | 26000 | 16 (0)| 00:00:01 | 1 | 1 |
---------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - filter("A"=1 AND "B"<=1000)
Note
-----
- dynamic statistics used: dynamic sampling (level=2)
Gather stats, then create a new partition with data. Although that partition has 1000 rows and no stats, Oracle does not know that and just assumes it's empty.
begin
dbms_stats.gather_table_stats(user, 'partition_test');
end;
/
alter table partition_test add partition p2 values less than (3);
insert into partition_test select 2, level from dual connect by level <= 1000;
explain plan for select * from partition_test where a = 2 and b <= 1000;
select * from table(dbms_xplan.display);
Plan hash value: 4097357352
---------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
---------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 7 | 9 (0)| 00:00:01 | | |
| 1 | PARTITION RANGE SINGLE| | 1 | 7 | 9 (0)| 00:00:01 | 2 | 2 |
|* 2 | TABLE ACCESS FULL | PARTITION_TEST | 1 | 7 | 9 (0)| 00:00:01 | 2 | 2 |
---------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - filter("A"=2 AND "B"<=1000)
Explicitly requesting dynamic sampling will fix the cardinality estimates, which would likely solve your execution plan problems.
explain plan for select /*+ dynamic_sampling(4) */ * from partition_test where a = 2 and b <= 1000;
select * from table(dbms_xplan.display);
Plan hash value: 4097357352
---------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
---------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1000 | 7000 | 9 (0)| 00:00:01 | | |
| 1 | PARTITION RANGE SINGLE| | 1000 | 7000 | 9 (0)| 00:00:01 | 2 | 2 |
|* 2 | TABLE ACCESS FULL | PARTITION_TEST | 1000 | 7000 | 9 (0)| 00:00:01 | 2 | 2 |
---------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - filter("A"=2 AND "B"<=1000)
Note
-----
- dynamic statistics used: dynamic sampling (level=4)
So, I'm in the process of writing a process that will populate a table using about 10 different selects. As part of the process I've picked up a few things, however I'm facing some inconsistent behaviour at times when performing an INSERT/SELECT. When I run the select query alone, I get results back within 10 seconds (after I flush buffer cache and shared_pool), but at soon as I tag on the insert portion, takes over 10 min...
Here's what I'm doing after I clear the table... Added the "Append" hint on insert which appeared to have helped quite a bit earlier this week, but now it's back to taking longer than usual, but only when performing the insert!!
--// Disable GL_JLOG_DETAILS INDEXES
Execute immediate 'alter index IDX_GLDTL1 unusable';
Execute immediate 'alter index IDX_GLDTL2 unusable';
--// SECTION 1
INSERT /*+ APPEND PARALLEL(6) */ INTO GL_JLOG_DETAILS ( MAT_SECTION
,JLOG_KEY
,SRC_CD
,SRC_KEY
,CASE_KEY
,CASE_MBR_KEY
,UNALLOC_ACCT_CD
,FDRT_KEY
,FDRT_TR_CD
,TR_CD
,TR_REF_NO
,STAT_CD
,FD_DESC_ID
,FD_NO
,FD_TYP_CD
,BKT_NO
,RVSL_CD
,CO_CD
,BEN_RPT_TYP_CD
,DB_CR_CD
,ACCT_NO
,SUB_ACCT_NO
,JRNL_AMT
,TDTL_AMT
,RVSL_ERROR
,RVSL_SAME_DAY
,TDTL_REF_KEY)
---// SECTION 1 //--------
Select /*+ USE_NL(TMAP,TMOV,ACCT,SBNT,BNTP,FDRT) */ '1' MAT_SECTION,
JLOG.JLOG_KEY,
JLOG.SRC_CD,
TDTL.TDTL_KEY AS SRC_KEY,
TDTL.CASE_KEY,
TDTL.CASE_MBR_KEY,
TDTL.UNALLOC_ACCT_CD,
TDTL.FDRT_KEY,
FDRT.TR_CD FDRT_TR_CD,
TDTL.TR_CD,
TDTL.TR_REF_NO,
TDTL.STAT_CD,
TDTL.FD_DESC_ID,
TDTL.FD_NO,
FDDC.FD_TYP_CD,
TDTL.BKT_NO,
JLOG.RVSL_CD,
BNTP.CO_CD,
SBNT.BEN_RPT_TYP_CD,
JLOG.DB_CR_CD,
ACCT.ACCT_NO,
ACCT.SUB_ACCT_NO,
JLOG.JRNL_AMT,
ABS (TDTL.AMT) AS TDTL_AMT,
CASE
WHEN TDTL.PROC_DT < TDTL.RVSL_CYC_DT AND TDTL.ORIG_INBS_KEY IS NULL
THEN
1
ELSE
0
END RVSL_ERROR,
CASE
WHEN TDTL.PROC_DT = TDTL.RVSL_CYC_DT AND NOT TDTL.TR_CD IN ('3002','3004','1501','1502','1503','1504','1505')
THEN 1
ELSE 0
END RVSL_SAME_DAY,
TDTL.REF_KEY TDTL_REF_KEY
from GL_JOURNAL_LOGS JLOG,
Transact_Details TDTL,
FUND_DESC FDDC,
FD_RATES FDRT,
BEN_TYPES BNTP,
SYS_BEN_TYPES SBNT,
LEDGER_ACCOUNTS ACCT,
TRANS_MAP_OVRD TMOV,
TRANSACTION_MAP TMAP
WHERE JLOG.JRNL_CD = '0'
AND JLOG.SRC_CD = '2'
AND JLOG.MKEY_FD_NUM <> 0
AND NVL(JLOG.TMOV_KEY, -1) > 0
AND NVL(JLOG.ORIG_SCAT_KEY, 1) = 1
AND JLOG.Scat_key = TDTL.SCAT_KEY
AND JLOG.TR_CD = TDTL.TR_CD
AND JLOG.CASE_KEY = TDTL.CASE_KEY
AND JLOG.TR_REF_NO = TDTL.TR_REF_NO
AND JLOG.ACCT_KEY = ACCT.ACCT_KEY
AND JLOG.TMOV_KEY = TMOV.TMOV_KEY
AND NVL(TDTL.ORIG_SCAT_KEY, 1) = 1
AND TDTL.STAT_CD <> '4'
AND TDTL.FD_DESC_ID = FDDC.FD_DESC_ID
AND TDTL.FDRT_KEY = FDRT.FDRT_KEY
AND BNTP.BNTP_KEY = FDRT.BNTP_KEY
AND BNTP.SBNT_KEY (+) = SBNT.SBNT_KEY
AND TMOV.TMAP_KEY = TMAP.TMAP_KEY
AND TMOV.CO_CD = BNTP.CO_CD
AND DECODE(FDDC.MKEY_FD_NUM, NULL, TMAP.MKEY_FD_NUM, FDDC.MKEY_FD_NUM) = TMAP.MKEY_FD_NUM;
Any tips / advice would be greatly appreciated!
Explain Plan
Plan hash value: 4157721082
-----------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------------------------
| 0 | INSERT STATEMENT | | 1 | 212 | 1596 (1)| 00:00:20 |
| 1 | LOAD AS SELECT | GL_JLOG_DETAILS | | | | |
| 2 | NESTED LOOPS | | | | | |
| 3 | NESTED LOOPS | | 1 | 212 | 1596 (1)| 00:00:20 |
| 4 | NESTED LOOPS | | 1 | 195 | 1595 (1)| 00:00:20 |
| 5 | NESTED LOOPS | | 1 | 190 | 1594 (1)| 00:00:20 |
| 6 | NESTED LOOPS | | 12 | 2172 | 1582 (1)| 00:00:19 |
|* 7 | HASH JOIN | | 12 | 2004 | 1570 (1)| 00:00:19 |
| 8 | TABLE ACCESS FULL | FUND_DESC | 168 | 1176 | 4 (0)| 00:00:01 |
| 9 | NESTED LOOPS | | | | | |
| 10 | NESTED LOOPS | | 257 | 41120 | 1566 (1)| 00:00:19 |
| 11 | NESTED LOOPS | | 257 | 22359 | 537 (0)| 00:00:07 |
| 12 | NESTED LOOPS | | 257 | 20817 | 280 (0)| 00:00:04 |
|* 13 | TABLE ACCESS BY INDEX ROWID| GL_JOURNAL_LOGS | 257 | 18504 | 23 (0)| 00:00:01 |
|* 14 | INDEX RANGE SCAN | IDX_ORIGSCATKEY | 690 | | 4 (0)| 00:00:01 |
| 15 | TABLE ACCESS BY INDEX ROWID| TRANS_MAP_OVRD | 1 | 9 | 1 (0)| 00:00:01 |
|* 16 | INDEX UNIQUE SCAN | TMOV_PK | 1 | | 0 (0)| 00:00:01 |
| 17 | TABLE ACCESS BY INDEX ROWID | TRANSACTION_MAP | 1 | 6 | 1 (0)| 00:00:01 |
|* 18 | INDEX UNIQUE SCAN | TMAP_PK | 1 | | 0 (0)| 00:00:01 |
|* 19 | INDEX RANGE SCAN | IX_AML8890 | 3 | | 3 (0)| 00:00:01 |
|* 20 | TABLE ACCESS BY INDEX ROWID | TRANSACT_DETAILS | 1 | 73 | 4 (0)| 00:00:01 |
| 21 | TABLE ACCESS BY INDEX ROWID | FD_RATES | 1 | 14 | 1 (0)| 00:00:01 |
|* 22 | INDEX UNIQUE SCAN | FDRT_PK | 1 | | 0 (0)| 00:00:01 |
|* 23 | TABLE ACCESS BY INDEX ROWID | BEN_TYPES | 1 | 9 | 1 (0)| 00:00:01 |
|* 24 | INDEX UNIQUE SCAN | BNTP_PK | 1 | | 0 (0)| 00:00:01 |
| 25 | TABLE ACCESS BY INDEX ROWID | SYS_BEN_TYPES | 1 | 5 | 1 (0)| 00:00:01 |
|* 26 | INDEX UNIQUE SCAN | SBNT_PK | 1 | | 0 (0)| 00:00:01 |
|* 27 | INDEX UNIQUE SCAN | ACCT_PK | 1 | | 0 (0)| 00:00:01 |
| 28 | TABLE ACCESS BY INDEX ROWID | LEDGER_ACCOUNTS | 1 | 17 | 1 (0)| 00:00:01 |
-----------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
7 - access("TDTL"."FD_DESC_ID"="FDDC"."FD_DESC_ID")
filter("TMAP"."MKEY_FD_NUM"=DECODE(TO_CHAR("FDDC"."MKEY_FD_NUM"),NULL,"TMAP"."MKEY_FD_NUM","
FDDC"."MKEY_FD_NUM"))
13 - filter("JLOG"."TMOV_KEY" IS NOT NULL AND "JLOG"."SRC_CD"='2' AND "JLOG"."MKEY_FD_NUM"<>0
AND NVL("TMOV_KEY",(-1))>0 AND "JLOG"."JRNL_CD"='0')
14 - access(NVL("ORIG_SCAT_KEY",1)=1)
16 - access("JLOG"."TMOV_KEY"="TMOV"."TMOV_KEY")
18 - access("TMOV"."TMAP_KEY"="TMAP"."TMAP_KEY")
19 - access("JLOG"."SCAT_KEY"="TDTL"."SCAT_KEY" AND "JLOG"."CASE_KEY"="TDTL"."CASE_KEY")
filter("TDTL"."FD_DESC_ID" IS NOT NULL AND "TDTL"."STAT_CD"<>'4' AND
"JLOG"."CASE_KEY"="TDTL"."CASE_KEY")
20 - filter("TDTL"."FDRT_KEY" IS NOT NULL AND NVL("TDTL"."ORIG_SCAT_KEY",1)=1 AND
"JLOG"."TR_CD"="TDTL"."TR_CD" AND "JLOG"."TR_REF_NO"="TDTL"."TR_REF_NO")
22 - access("TDTL"."FDRT_KEY"="FDRT"."FDRT_KEY")
23 - filter("TMOV"."CO_CD"="BNTP"."CO_CD")
24 - access("BNTP"."BNTP_KEY"="FDRT"."BNTP_KEY")
26 - access("BNTP"."SBNT_KEY"="SBNT"."SBNT_KEY")
27 - access("JLOG"."ACCT_KEY"="ACCT"."ACCT_KEY")
A couple of problems with "Direct path" load a.k.a. /*+ APPEND */.
Firstly, it isn't necessarily faster in general. It does a direct path load to disk - bypassing the buffer cache. In many cases - especially with smaller sets of data - where the direct path load to disk would be slower than a conventional path load into the cache.
Second, DPL works above the high watermark of the table which means it does not re-use the space. So unless you TRUNCATE the whole table every time, it probably isn't gonna be necessarily faster.
Third, only one seesion/user can DPL on to a table at a time. That may cause concurrency issues, here all the modifications will become serialized (like a serial circuit, rather than a parallel circuit). No insert/update/delete or merge into this table will be allowed until the transaction that direct paths, commits.
Lastly, the indexes of your table in question matters. It is pretty difficult to say without looking at the data as of what kind of index will be appropriate and on which columns. Try a combination of Local and global bitmap indexes, but be careful while chosing the columns. Making an index 'unusable' is not a good idea in general, there is a purpose why it was created and altering it almost conflicts to its immediate interests.
To start with, remove all hints, try indexing. and iff it is a reloadable table then truncate it prior to every load.