I have several instances: dbA, dbB, dbC.., then i want to query data from local and link to other instance over dblink, and expected is l_ids will be send to remote site cause l_ids is ligh and table#dbA is numerous, just as followering:
DECLARE
out_row_data SYS_REFCURSOR;
l_ids UDT_TBL_NUMBER; -- TABLE OF NUMBER(10)
l_sql VARCHAR2(18000);
BEGIN
l_sql := 'SELECT * FROM tableX#dbA
WHERE column IN (SELECT * FROM TABLE(:l_ids));'
OPEN out_row_data FOR l_sql
USING l_ids;
END;/
But execution plan shows data of tableX fetched first, then join with table(l_ids).
Another try, using driving_site hint to specify it, but didn't work either:
DECLARE
out_row_data SYS_REFCURSOR;
l_ids UDT_TBL_NUMBER; -- TABLE OF NUMBER(10)
l_sql VARCHAR2(18000);
BEGIN
l_sql := 'SELECT /*+ MONITOR driving_site(X) */ * FROM TABLE(:l_ids) GIDS
LEFT JOIN tableX#dbA X ON GIDS.column_value = X.column;'
OPEN out_row_data FOR l_sql
USING l_ids;
END;/
I have no idea now, above statement all process successfully, but execution plan aren't as expected.
Can someone help me or need more info? :(
Update
I think if driving_site is work, the tableX REMOTE operation shouldn't exist.
Streamline dynamic sqltext:
l_sql := '
WITH matched_Y AS (
SELECT *
FROM tableY#dbB
), matched_X AS (
SELECT /*+ MONITOR no_merge(X) */
column1
,sum(column2)
FROM
(
SELECT /*+ MONITOR driving_site(RX) DRSITE */ * FROM
TABLE(:ids) GIDS
LEFT JOIN tableX#dbA RX
ON GIDS.column_value = RX.column1
) X
LEFT JOIN TABLE(:currency_table) ct
ON column3 >= ct.column3
GROUP BY column1
), matched_X2 AS (
SELECT *
FROM (matched_X)
GROUP BY column1
)
SELECT *
FROM matched_Y g
LEFT JOIN matched_X2 w ON g.column1 = w.column1
ORDER BY g.column4 DESC';
Execution plan
**---------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | Inst |IN-OUT|
---------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | | 3141 (100)| | | |
| 1 | SORT ORDER BY | | 11790 | 9936K| 10M| 3141 (1)| 00:00:01 | | |
|* 2 | HASH JOIN RIGHT OUTER | | 11790 | 9936K| | 1001 (1)| 00:00:01 | | |
| 3 | VIEW | | 1 | 130 | | 60 (2)| 00:00:01 | | |
| 4 | HASH GROUP BY | | 1 | 96 | | 60 (2)| 00:00:01 | | |
| 5 | NESTED LOOPS OUTER | | 20 | 1920 | | 59 (0)| 00:00:01 | | |
| 6 | VIEW | | 1 | 94 | | 30 (0)| 00:00:01 | | |
|* 7 | FILTER | | | | | | | | |
|* 8 | HASH JOIN | | 1 | 109 | | 30 (0)| 00:00:01 | | |
| 9 | REMOTE | tableX| 1 | 107 | | 1 (0)| 00:00:01 | DBLIN~ | R->S |
| 10 | COLLECTION ITERATOR PICKLER FETCH| | 100 | 200 | | 29 (0)| 00:00:01 | | |
|* 11 | COLLECTION ITERATOR PICKLER FETCH | | 20 | 40 | | 29 (0)| 00:00:01 | | |
|* 12 | VIEW | | 11790 | 8439K| | 941 (1)| 00:00:01 | | |
| 13 | REMOTE | | | | | | | DBLIN~ | R->S |
---------------------------------------------------------------------------------------------------------------------------**
In a true distributed query, the optimization is done on the sending site. Because your local site may not have access to the CBO statistics on the remote site.
The driving_site hint forces query execution to be done at a different site than the initiating instance. This is done when the remote table is much larger than the local table and you want the work (join, sorting) done remotely to save the back-and-forth network traffic.
Your plan is behaving correctly, from the perspective of the driving site, and without proper knowledge of the filter operations done by the optimizer ( this part of the plan is not in your post ), I would suggest to try and get rid of the nested loops outer produced by the left join, to see whether a hash join can behave better.
WITH matched_Y AS (
SELECT /*+ materialize */ *
FROM tableY#dbB
), matched_X AS (
SELECT /*+ MONITOR no_merge(X) use_hash(ct,x) */
column1
,sum(column2)
FROM
(
SELECT /*+ MONITOR driving_site(RX) DRSITE */ * FROM
TABLE(:ids) GIDS
LEFT JOIN tableX#dbA RX
ON GIDS.column_value = RX.column1
) X
LEFT JOIN TABLE(:currency_table) ct
ON column3 >= ct.column3
GROUP BY column1
), matched_X2 AS (
SELECT *
FROM (matched_X)
GROUP BY column1
)
SELECT *
FROM matched_Y g
LEFT JOIN matched_X2 w ON g.column1 = w.column1
ORDER BY g.column4 DESC
Related
My table has millions of records. In this query below, can I make Oracle 12c examine the first X rows only instead of doing a full table scan?
The value of X, I imagine should be Offset + Fetch Next , so in this case 15
SELECT * FROM table OFFSET 5 ROWS FETCH NEXT 10 ROWS ONLY;
Thanks in advance
Edit 1
These are the tables involved and this is the actual query
Orders - This table has 113k records in my test DB ( and over 8 million in prod db like my original question mentioned)
--------------------------
| Id | SKUField1|SKUField2|
--------------------------
| 1 | Value1 | Value2 |
| 2 | Value2 | Value2 |
| 3 | Value1 | Value3 |
--------------------------
Products - This table has 2 million records in my test DB ( prod db is similar)
---------------
| PId| SKU_NUM|
---------------
| 1 | Value1 |
| 2 | Value2 |
| 3 | Value3 |
---------------
Note that values of Orders.SKUField1 and Orders.SKUField2 come from the Products.SKU_NUM values
Actual Query:
SELECT /*+ gather_plan_statistics */ Id, PId, SKUField1, SKUField2, SKU_NUM
FROM Orders
LEFT JOIN (
-- this inner query reduces size of Products from 2 million rows down to 1462 rows
select * from Products where SKU_NUM in (
select SKUField1 from Orders
)
) p1 ON SKUField1 = p1.SKU_NUM
LEFT JOIN (
-- this inner query reduces size of table B from 2 million rows down to 459 rows
select * from Products where SKU_NUM in (
select SKUField2 from Orders
)
) p4 ON SKUField2 = p4.SKU_NUM
OFFSET 5 ROWS FETCH NEXT 10 ROWS ONLY
Execution Plan:
--------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Time | A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem |
--------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 10 |00:00:00.06 | 8013 | | | |
|* 1 | VIEW | | 1 | 00:00:01 | 10 |00:00:00.06 | 8013 | | | |
|* 2 | WINDOW NOSORT STOPKEY | | 1 | 00:00:01 | 15 |00:00:00.06 | 8013 | 27M| 1904K| |
|* 3 | HASH JOIN RIGHT OUTER | | 1 | 00:00:01 | 15 |00:00:00.06 | 8013 | 1162K| 1162K| 1344K (0)|
| 4 | VIEW | | 1 | 00:00:01 | 1462 |00:00:00.04 | 6795 | | | |
| 5 | NESTED LOOPS | | 1 | 00:00:01 | 1462 |00:00:00.04 | 6795 | | | |
| 6 | NESTED LOOPS | | 1 | 00:00:01 | 1462 |00:00:00.04 | 5333 | | | |
| 7 | SORT UNIQUE | | 1 | 00:00:01 | 1469 |00:00:00.04 | 3010 | 80896 | 80896 |71680 (0)|
| 8 | TABLE ACCESS FULL | Orders | 1 | 00:00:01 | 113K|00:00:00.02 | 3010 | | | |
|* 9 | INDEX UNIQUE SCAN | UIX_Product_SKU_NUM | 1469 | 00:00:01 | 1462 |00:00:00.01 | 2323 | | | |
| 10 | TABLE ACCESS BY INDEX ROWID | Products | 1462 | 00:00:01 | 1462 |00:00:00.01 | 1462 | | | |
|* 11 | HASH JOIN RIGHT OUTER | | 1 | 00:00:01 | 15 |00:00:00.02 | 1218 | 1142K| 1142K| 1335K (0)|
| 12 | VIEW | | 1 | 00:00:01 | 459 |00:00:00.02 | 1213 | | | |
| 13 | NESTED LOOPS | | 1 | 00:00:01 | 459 |00:00:00.02 | 1213 | | | |
| 14 | NESTED LOOPS | | 1 | 00:00:01 | 459 |00:00:00.02 | 754 | | | |
| 15 | SORT UNIQUE | | 1 | 00:00:01 | 462 |00:00:00.02 | 377 | 24576 | 24576 |22528 (0)|
| 16 | INDEX FAST FULL SCAN | Orders_SKUField2_IDX6 | 1 | 00:00:01 | 113K|00:00:00.01 | 377 | | | |
|* 17 | INDEX UNIQUE SCAN | UIX_Product_SKU_NUM | 462 | 00:00:01 | 459 |00:00:00.01 | 377 | | | |
| 18 | TABLE ACCESS BY INDEX ROWID| Products | 459 | 00:00:01 | 459 |00:00:00.01 | 459 | | | |
| 19 | TABLE ACCESS FULL | Orders | 1 | 00:00:01 | 15 |00:00:00.01 | 5 | | | |
--------------------------------------------------------------------------------------------------------------------------------------------------
Hence, based on the "A-Rows" column values for row Ids 8 and 16 in the execution plan, it seems like there are full table scans on the Orders table (though row id 16 atleast seems to be using an index). So my question is is it true that there is a full table scan on the orders table even though I am using Offset/Fetch Next
Although your FETCH clause may use a full table scan, Oracle will still only fetch the first X rows from the table.
In the following example, the "TABLE ACCESS FULL" operation does start to read the entire table, but it gets cutoff part of the way through by the "WINDOW NOSORT STOPKEY" operation. Not all full table scans actually scan the full table. You would see similar behavior if your code ended with WHERE ROWNUM <= 50.
CREATE TABLE some_table AS SELECT * FROM all_objects;
EXPLAIN PLAN FOR SELECT * FROM some_table OFFSET 5 ROWS FETCH NEXT 10 ROWS ONLY;
SELECT * FROM TABLE(dbms_xplan.display);
Plan hash value: 2559837639
-------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 15 | 7410 | 2 (0)| 00:00:01 |
|* 1 | VIEW | | 15 | 7410 | 2 (0)| 00:00:01 |
|* 2 | WINDOW NOSORT STOPKEY| | 15 | 2010 | 2 (0)| 00:00:01 |
| 3 | TABLE ACCESS FULL | SOME_TABLE | 15 | 2010 | 2 (0)| 00:00:01 |
-------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("from$_subquery$_002"."rowlimit_$$_rownumber"<=15 AND
"from$_subquery$_002"."rowlimit_$$_rownumber">5)
2 - filter(ROW_NUMBER() OVER ( ORDER BY NULL )<=15)
The performance implications get more complicated if you also want to order the results. If that is the case, you may want to post the full query and execution plan.
(EDIT: 2022-09-25)
Yes, there is a full table scan on the ORDERS table happening on line 8 of the execution plan. As you mentioned, you can look at the "A-rows" column to tell what's really happening.
But the third full table scan of ORDERS, on line 19, is not a "full" full table scan. The operation "WINDOW NOSORT STOPKEY" stops that full table scan as soon as the 15 necessary rows are read. So the FETCH syntax is helping at least a little.
Applying a FETCH to a query does not mean that every single table will be limited. Although, in your query, it does seem like there ought to be a way to reduce the full table scans. Perhaps an index on SKUField1 would help?
Since Oracle as I know don't provide something like limit or top you can created by yourself like the following:
what is happening here, the inner query gets all the first 10 records and the outer query get those, you can still use any clauses like where or order or any others
SELECT * FROM (
SELECT * FROM Customers WHERE CustomerID <= 10 ORDER BY CustomerID
)
The full article will be found about this topic here at Oracle-Fetch
I am using Online Oracle so you can try it from your end, please let me know if you still have a problem.
I'm facing unsolvable and impossible performace drop while using UNION ALL with two sub-queries in one cursor (at least I think that's the problem). PL/SQL Developer just freezes when opening cursor results in test window.
If I turn off no matter which sub-query - everything works fine.
If I take the whole query out of cursor to regular SQL Query windows - everything is okay without any need to turn off some parts.
Procedure structure is down below, looking forward any help:
procedure p_proc(p_param varchar2,
outcur out sys_refcursor) is
begin
open outcur for
select *
from (select -- visible cols
si.item_full_name
, si.final_price
, si.full_price
, si.receipt_num
, si.receipt_date
, si.vendor_code
, case when det.br_summary is null and mr.motiv_rate_value is not null then mr.motiv_rate_value
when det.br_summary is not null then det.br_summary
end personal_bonus_amount
, case when det.br_summary is null and mr.motiv_rate_value is not null then 1
when det.br_summary is not null then det.cross_sale_kt
end personal_bonus_koeff
-- service cols
, case when det.br_summary is null and mr.motiv_rate_value is not null then 'approximate'
when det.br_summary is not null then 'definite'
end personal_bonus_type
, coalesce(det.sale_stream, mr.sale_stream, 'Not defined') item_group_name
, si.operation_type
, si.src
-- pagination
, row_number() over (order by si.receipt_date desc) rn
from (-- curr day
select b.cost final_price
, case when b.discount = 0 then null else b.price
end full_price
, b.doc_number receipt_num
, b.receipt_date receipt_date
, i.item_code vendor_code
, i.full_name item_full_name
, b.subsite code_op
, b.operator_id
, to_char(b.businessday, 'yyyymm') sale_period
, b.oper_type operation_type
, 'bill' src
from scheme.bills b
join scheme.items i on i.item_code = b.item
where b.businessday = trunc(p_date_to)
and b.subsite = p_office_id
and b.operator_id = p_emp_id
union all
-- prev days
select l.txn_amount final_price
, case when l.disc = 0 then null else l.price
end full_price
, t.receipt_num receipt_num
, t.ts receipt_date
, i.item_code vendor_code
, i.full_name item_full_name
, s.office_code code_op
, e.emp_code operator_id
, to_char(l.dt,'yyyymm') sale_period
, l.txn_type operation_type
, 'txn' src
from scheme.txn t
join scheme.txn_lines l on t.rtl_txn_id = l.rtl_txn_id
join scheme.items i on l.item_id = i.item_id
join scheme.offices s on t.subsite_id = s.subsite_id
join scheme.employees e on t.employee_id = e.employee_id
where t.ts between trunc(p_date_from) and trunc(p_date_to)
and t.subsite_id = v_op_id
and t.employee_id = v_emp_id
) si
/* fact */
left join scheme.sales_details det on si.sale_period = det.period
and si.code_op = det.op_code
and ltrim(si.operator_id,'0') = ltrim(det.tab_num,'0')
and si.receipt_num = det.rcpt_num
and si.vendor_code = det.item_article
/* prognosis */
left join scheme.rates mr on si.sale_period = mr.motiv_rate_period
and si.code_op = mr.code_op
and si.vendor_code = mr.code_1c
where 1 = 1
and si.final_price between nvl(p_price_from, si.final_price) and nvl(p_price_to, si.final_price)
/* if no filters */
and (item_group_cnt = 0 or coalesce(det.sale_stream, mr.sale_stream, 'Not defined') in (select * from table(p_item_group)))
and si.receipt_num = nvl(p_receipt_num, si.receipt_num)
)
where rn between p_page_num * p_page_size + 1 and (p_page_num + 1) * p_page_size;
end;
UPD Explain plan for the whole query used in a cursor:
----------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost | Time |
----------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 10 | 32810 | 62 | 00:00:01 |
| * 1 | VIEW | | 10 | 32810 | 62 | 00:00:01 |
| * 2 | WINDOW SORT PUSHED RANK | | 2 | 2956 | 62 | 00:00:01 |
| 3 | NESTED LOOPS OUTER | | 2 | 2956 | 61 | 00:00:01 |
| 4 | NESTED LOOPS OUTER | | 2 | 2826 | 53 | 00:00:01 |
| 5 | VIEW | | 2 | 2728 | 46 | 00:00:01 |
| 6 | UNION-ALL | | | | | |
| 7 | NESTED LOOPS | | 1 | 138 | 32 | 00:00:01 |
| 8 | NESTED LOOPS | | 1 | 138 | 32 | 00:00:01 |
| 9 | PARTITION RANGE SINGLE | | 1 | 66 | 29 | 00:00:01 |
| * 10 | TABLE ACCESS BY LOCAL INDEX ROWID BATCHED | F003_BILL | 1 | 66 | 29 | 00:00:01 |
| * 11 | INDEX RANGE SCAN | IX_SUBSITE_DOCNUM_BUSINDAY_SEQ | 1 | | 5 | 00:00:01 |
| * 12 | INDEX RANGE SCAN | IX_D001_CODE_1C_ITEM_ID | 1 | | 2 | 00:00:01 |
| 13 | TABLE ACCESS BY INDEX ROWID | D001_ITEM | 1 | 72 | 3 | 00:00:01 |
| 14 | NESTED LOOPS | | 1 | 183 | 14 | 00:00:01 |
| 15 | NESTED LOOPS | | 1 | 183 | 14 | 00:00:01 |
| 16 | NESTED LOOPS | | 1 | 104 | 12 | 00:00:01 |
| 17 | NESTED LOOPS | | 1 | 70 | 7 | 00:00:01 |
| 18 | NESTED LOOPS | | 1 | 30 | 4 | 00:00:01 |
| 19 | TABLE ACCESS BY INDEX ROWID | D005_EMPLOYEE | 1 | 18 | 3 | 00:00:01 |
| * 20 | INDEX UNIQUE SCAN | PK_D005 | 1 | | 2 | 00:00:01 |
| 21 | TABLE ACCESS BY INDEX ROWID | D018_SUBSITE | 1 | 12 | 1 | 00:00:01 |
| * 22 | INDEX UNIQUE SCAN | PK_D018 | 1 | | 0 | 00:00:01 |
| 23 | PARTITION RANGE ITERATOR | | 1 | 40 | 3 | 00:00:01 |
| 24 | PARTITION HASH SINGLE | | 1 | 40 | 3 | 00:00:01 |
| * 25 | TABLE ACCESS FULL | F007_RTL_TXN | 1 | 40 | 3 | 00:00:01 |
| * 26 | TABLE ACCESS BY GLOBAL INDEX ROWID BATCHED | F008_RTL_TXN_LI | 1 | 34 | 5 | 00:00:01 |
| * 27 | INDEX RANGE SCAN | IX_F008_RTL_TXN_ID | 7 | | 3 | 00:00:01 |
| * 28 | INDEX UNIQUE SCAN | PK_D001 | 1 | | 1 | 00:00:01 |
| 29 | TABLE ACCESS BY INDEX ROWID | D001_ITEM | 1 | 79 | 2 | 00:00:01 |
| * 30 | TABLE ACCESS BY INDEX ROWID BATCHED | T_OP_MOTIVATION_RATE_MYRTK | 1 | 49 | 7 | 00:00:01 |
| * 31 | INDEX RANGE SCAN | IDX02_CODE_OP_1C | 3 | | 3 | 00:00:01 |
| * 32 | TABLE ACCESS BY INDEX ROWID BATCHED | DET_SALES_PPT_DWH | 1 | 65 | 4 | 00:00:01 |
| * 33 | INDEX RANGE SCAN | IDX_03_RCPT_NUM | 3 | | 2 | 00:00:01 |
----------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
------------------------------------------
* 1 - filter("RN">=1 AND "RN"<=10)
* 2 - filter(ROW_NUMBER() OVER ( ORDER BY INTERNAL_FUNCTION("SI"."RECEIPT_DATE") DESC )<=10)
* 10 - filter("F003"."OPERATOR_ID"='000189513' AND "F003"."COST">=TO_NUMBER(TO_CHAR("F003"."COST")) AND "F003"."COST"<=TO_NUMBER(TO_CHAR("F003"."COST")))
* 11 - access("F003"."SUBSITE"='S165' AND "F003"."BUSINESSDAY"=TO_DATE(' 2021-11-23 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
* 11 - filter("F003"."BUSINESSDAY"=TO_DATE(' 2021-11-23 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND "F003"."DOC_NUMBER" IS NOT NULL)
* 12 - access("I"."D001_CODE_1C"="F003"."ITEM")
* 12 - filter("I"."D001_CODE_1C" IS NOT NULL)
* 20 - access("E"."EMPLOYEE_ID"=3561503543)
* 22 - access("S"."SUBSITE_ID"=29260)
* 25 - filter("T"."EMPLOYEE_ID"=3561503543 AND "T"."SUBSITE_ID"=29260 AND "T"."F007_TS"<=TO_DATE(' 2021-11-23 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND "T"."F007_RCPT_NUM_1C" IS NOT NULL)
* 26 - filter("L"."F008_AMOUNT">=TO_NUMBER(TO_CHAR("L"."F008_AMOUNT")) AND "L"."F008_AMOUNT"<=TO_NUMBER(TO_CHAR("L"."F008_AMOUNT")))
* 27 - access("T"."RTL_TXN_ID"="L"."RTL_TXN_ID")
* 28 - access("L"."ITEM_ID"="I"."ITEM_ID")
* 30 - filter("SI"."SALE_PERIOD"="MR"."MOTIV_RATE_PERIOD"(+))
* 31 - access("SI"."CODE_OP"="MR"."CODE_OP"(+) AND "SI"."VENDOR_CODE"="MR"."CODE_1C"(+))
* 32 - filter("SI"."CODE_OP"="DET"."OP_CODE"(+) AND "SI"."VENDOR_CODE"="DET"."ITEM_ARTICLE"(+) AND "DET"."ITEM_ARTICLE"(+) IS NOT NULL AND "DET"."PERIOD"(+)=TO_NUMBER("SI"."SALE_PERIOD") AND
LTRIM("SI"."OPERATOR_ID",'0')=LTRIM("DET"."TAB_NUM_RTK"(+),'0'))
* 33 - access("SI"."RECEIPT_NUM"="DET"."RCPT_NUM"(+))
* 33 - filter("DET"."RCPT_NUM"(+) IS NOT NULL)
Actual solution
Managed to get procedure execution plan from DBA. The problem was that optimizer chose another index for joining scheme.sales_details table when executing query inside the procedure. Added INDEX HINT with the same index which was used in regular query and everything works just fine.
Deprecated ideas down below
As far as I understood the problem is in Oracle optimizer which "thought" that doing UNION ALL first is better than pushing predicate into the sub-query. Separating this union into two single queries make him push pred without any hesitations.
Probably this can be fixed by playing with hints, that's wip for now.
Temporary workaround is to regroup the query, going from this structure
select *
from (select row_number() rn
, u.*
from (select *
from first_query
union all
select *
from second_query) u
-- some joins
join first_table ft
join second_table st
-- predicate block
where 1=1
and a = b
)
where rn between c and d;
to this
select *
from (select row_number() rn
, u.*
from (select *
from first_query) u
-- some joins
join first_table ft
join second_table st
-- predicate block
where 1=1
and a = b
union all
select row_number() rn
, u.*
from (select *
from second_query) u
-- some joins
join first_table ft
join second_table st
-- predicate block
where 1=1
and a = b
)
where rn between c and d;
That's not the perfect solution cause it doubles the JOIN section but at least it works.
I have created a materialized view on SH2 but, when I run my explain plan statement I can't see the materialized view being used in the plan table output. I'm not sure if it's a more complex materialized view with additional key columns to join to other dimensions, so I'm a bit confused why the materialized view isn't being utilized as I'm refering to the SH2 prefix in my select query.
CREATE MATERIALIZED VIEW fweek_pscat_sales_mv
PCTFREE 5
BUILD IMMEDIATE
REFRESH COMPLETE
ENABLE QUERY REWRITE
AS
SELECT t.week_ending_day
, p.prod_subcategory
, sum(s.amount_sold) AS Money
, s.channel_id
, s.promo_id
FROM sales s
, times t
, products p
WHERE s.time_id = t.time_id
AND s.prod_id = p.prod_id
GROUP BY t.week_ending_day
, p.prod_subcategory
, s.channel_id
, s.promo_id;
CREATE BITMAP INDEX FW_PSC_S_MV_SUBCAT_BIX
ON fweek_pscat_sales_mv(prod_subcategory);
CREATE BITMAP INDEX FW_PSC_S_MV_CHAN_BIX
ON fweek_pscat_sales_mv(channel_id);
CREATE BITMAP INDEX FW_PSC_S_MV_PROMO_BIX
ON fweek_pscat_sales_mv(promo_id);
CREATE BITMAP INDEX FW_PSC_S_MV_WD_BIX
ON fweek_pscat_sales_mv(week_ending_day);
spool &data_dir.EXP_query_on_SH2_2.txt
alter session set query_rewrite_integrity = TRUSTED;
alter session set query_rewrite_enabled = TRUE;
set timing on
EXPLAIN PLAN FOR
SELECT t.week_ending_day
, p.prod_subcategory
, sum(s.amount_sold) AS Money
, s.channel_id
, s.promo_id
FROM SH2.sales s
, SH2.times t
, SH2.products p
WHERE s.time_id = t.time_id
AND s.prod_id = p.prod_id
GROUP BY t.week_ending_day
, p.prod_subcategory
, s.channel_id
, s.promo_id;
REM Now Let us Display the Output of the Explain Plan
SET pagesize 9999
set linesize 250
set markup html preformat on
select * from table(dbms_xplan.display());
set linesize 80
spool off
----------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time | Pstart| Pstop |
----------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1016K| 60M| | 17365 (1)| 00:00:01 | | |
| 1 | HASH GROUP BY | | 1016K| 60M| 70M| 17365 (1)| 00:00:01 | | |
|* 2 | HASH JOIN | | 1016K| 60M| | 2178 (1)| 00:00:01 | | |
| 3 | VIEW | index$_join$_003 | 10000 | 224K| | 74 (0)| 00:00:01 | | |
|* 4 | HASH JOIN | | | | | | | | |
| 5 | INDEX FAST FULL SCAN | PRODUCTS_PK | 10000 | 224K| | 41 (0)| 00:00:01 | | |
| 6 | INDEX FAST FULL SCAN | PRODUCTS_PROD_SUBCAT_IX | 10000 | 224K| | 51 (0)| 00:00:01 | | |
|* 7 | HASH JOIN | | 1016K| 37M| | 2101 (1)| 00:00:01 | | |
| 8 | PART JOIN FILTER CREATE | :BF0000 | 1016K| 37M| | 2101 (1)| 00:00:01 | | |
| 9 | TABLE ACCESS FULL | TIMES | 1461 | 23376 | | 13 (0)| 00:00:01 | | |
| 10 | PARTITION RANGE JOIN-FILTER| | 1016K| 22M| | 2086 (1)| 00:00:01 |:BF0000|:BF0000|
| 11 | TABLE ACCESS FULL | SALES | 1016K| 22M| | 2086 (1)| 00:00:01 |:BF0000|:BF0000|
----------------------------------------------------------------------------------------------------------------------------------
Why would you expect reference to the MV from the baseline query. For that to happen Oracle would have to compare the query to all MVs to find a match. Even further it would require every query be compared to every MV. MVs are typically created to avoid running the baseline query by accessing the MV directly (that is why MVs are the stored results of a query). If you want the MV just select from it directly.
SELECT week_ending_day
, prod_subcategory
, Money
, channel_id
, promo_id
from fweek_pscat_sales_mv;
I would like to know if creating an index in the field CREATION_DATE will make any improvement in this query
select TO_CHAR(CREATION_DATE, 'HH24') || ':mm' HOUR, count ('1') MESSAGE_HOUR
from t_hotel
where TO_CHAR(CREATION_DATE, 'dd/mm/yyyy') = TO_CHAR(SYSDATE, 'dd/mm/yyyy')
group by TO_CHAR(CREATION_DATE, 'HH24')
order by TO_CHAR(CREATION_DATE, 'HH24') ;
Yes, you can take advantage of a function-based index on CREATION_DATE column.
Take the following demo as an example where I have a table with a column of date type.
create table t_hotel (creation_date date);
insert into t_hotel select sysdate+0.5 from dual connect by level <=100;
100 rows affected
insert into t_hotel select sysdate+0.2 from dual connect by level <=100;
100 rows affected
explain plan for select TO_CHAR(CREATION_DATE, 'HH24') || ':mm' HOUR, count ('1') MESSAGE_HOUR
from t_hotel
where trunc(creation_date)= trunc(sysdate)
group by TO_CHAR(CREATION_DATE, 'HH24')
order by TO_CHAR(CREATION_DATE, 'HH24') ;
select * from table(dbms_xplan.display());
| PLAN_TABLE_OUTPUT |
| :----------------------------------------------------------------------------- |
| Plan hash value: 353888308 |
| |
| ------------------------------------------------------------------------------ |
| | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | |
| ------------------------------------------------------------------------------ |
| | 0 | SELECT STATEMENT | | 200 | 1800 | 4 (25)| 00:00:01 | |
| | 1 | SORT GROUP BY | | 200 | 1800 | 4 (25)| 00:00:01 | |
| |* 2 | TABLE ACCESS FULL| T_HOTEL | 200 | 1800 | 3 (0)| 00:00:01 | |
| ------------------------------------------------------------------------------ |
| |
| Predicate Information (identified by operation id): |
| --------------------------------------------------- |
| |
| 2 - filter(TRUNC(INTERNAL_FUNCTION("CREATION_DATE"))=TRUNC(SYSDATE#!) |
| ) |
| |
| Note |
| ----- |
| - dynamic sampling used for this statement (level=2) |
create index idx_cd on t_hotel(trunc(creation_date));
explain plan for select TO_CHAR(CREATION_DATE, 'HH24') || ':mm' HOUR, count ('1') MESSAGE_HOUR
from t_hotel
where trunc(creation_date)= trunc(sysdate)
group by TO_CHAR(CREATION_DATE, 'HH24')
order by TO_CHAR(CREATION_DATE, 'HH24') ;
select * from table(dbms_xplan.display());
| PLAN_TABLE_OUTPUT |
| :--------------------------------------------------------------------------------------- |
| Plan hash value: 2841908389 |
| |
| ---------------------------------------------------------------------------------------- |
| | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | |
| ---------------------------------------------------------------------------------------- |
| | 0 | SELECT STATEMENT | | 2 | 36 | 3 (34)| 00:00:01 | |
| | 1 | SORT GROUP BY | | 2 | 36 | 3 (34)| 00:00:01 | |
| | 2 | TABLE ACCESS BY INDEX ROWID| T_HOTEL | 2 | 36 | 2 (0)| 00:00:01 | |
| |* 3 | INDEX RANGE SCAN | IDX_CD | 1 | | 1 (0)| 00:00:01 | |
| ---------------------------------------------------------------------------------------- |
| |
| Predicate Information (identified by operation id): |
| --------------------------------------------------- |
| |
| 3 - access(TRUNC(INTERNAL_FUNCTION("CREATION_DATE"))=TRUNC(SYSDATE#!)) |
| |
| Note |
| ----- |
| - dynamic sampling used for this statement (level=2) |
dbfiddle here
1) where clause should be written differently.
where creation_date >= trunc(sysdate) and creation_date < trunc(sysdate+1)
2) You can create two index.
- normal create index idx_creation_date on t_hotel(creation_date); improve where clause.
- fucntion base index create index idx_creation_date_hh24 on t_hotel(TO_CHAR(CREATION_DATE, 'HH24') ); probably it will improve group by and orderby but i'm not sure.
How to optimize an update query:
UPDATE frst_usage vfm1 SET
(vfm1.account_dn,
vfm1.usage_date,
vfm1.country,
vfm1.feature_name,
vfm1.hu_type,
vfm1.make,
vfm1.region,
vfm1.service_hits,
vfm1.maint_last_ts,
vfm1.accountdn_hashcode) = (
SELECT
(SELECT vst.account_dn FROM services_track vst WHERE vst.accountdn_hashcode = vrd1.account_dn_hashcode AND rownum = 1),
min(usage_date),
country,
feature_name,
hu_type,
make,
region,
service_hits,
SYSDATE,
account_dn_hashcode
FROM raw_data vrd1
WHERE vrd1.vin_hashcode = vfm1.vin_hashcode
AND vrd1.usage_date IS NOT NULL AND rownum = 1
GROUP BY account_dn, country, feature_name, hu_type, make, region, service_hits, vfm1.maint_last_ts, account_dn_hashcode
);
the tables have indexes on all the columns available in the where conditions.
Still the execution is taking more than 4 hours. Below is the explain plan
From the execution plan i could see that the select is good but the update is consuming more time resources, Is there a way i could optimize this.
I think correlated subqueries may be an issue:
WHERE vrd1.vin_hashcode = vfm1.vin_hashcode
You should try merge clause, it could have dramatic impact on performance
http://docs.oracle.com/cd/E11882_01/server.112/e41084/statements_9016.htm
Below is example similar to yours. 10k sample rows, all columns indexed and statistics gathered:
Update (16s)
SQL> update x1 set (v1, v2, v3, v4) =
2 (
3 select v1, v2, v3, min(v4)
4 from x2
5 where x1.nr = x2.nr
6 group by v1,v2,v3
7 );
9999 rows updated.
Elapsed: 00:00:16.56
Execution Plan
----------------------------------------------------------
Plan hash value: 3497322513
----------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------
| 0 | UPDATE STATEMENT | | 9999 | 859K| 1679K (5)| 05:35:59 |
| 1 | UPDATE | X1 | | | | |
| 2 | TABLE ACCESS FULL | X1 | 9999 | 859K| 40 (0)| 00:00:01 |
| 3 | SORT GROUP BY | | 1 | 88 | 41 (3)| 00:00:01 |
|* 4 | TABLE ACCESS FULL| X2 | 1 | 88 | 40 (0)| 00:00:01 |
----------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
4 - filter("X2"."NR"=:B1)
Merge(1,5s)
SQL> merge into x1 using (
2 select nr, v1, v2, v3, min(v4) v4
3 from x2
4 group by nr, v1,v2,v3
5 ) xx2
6 on (x1.nr = xx2.nr)
7 when matched then update set
8 x1.v1 = xx2.v1, x1.v2 = xx2.v2, x1.v3 = xx2.v3, x1.v4 = xx2.v4;
9999 rows merged.
Elapsed: 00:00:01.25
Execution Plan
----------------------------------------------------------
Plan hash value: 1113810112
---------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
---------------------------------------------------------------------------------------
| 0 | MERGE STATEMENT | | 9999 | 58M| | 285 (1)| 00:00:04 |
| 1 | MERGE | X1 | | | | | |
| 2 | VIEW | | | | | | |
|* 3 | HASH JOIN | | 9999 | 58M| | 285 (1)| 00:00:04 |
| 4 | TABLE ACCESS FULL | X1 | 9999 | 859K| | 40 (0)| 00:00:01 |
| 5 | VIEW | | 9999 | 57M| | 244 (1)| 00:00:03 |
| 6 | SORT GROUP BY | | 9999 | 859K| 1040K| 244 (1)| 00:00:03 |
| 7 | TABLE ACCESS FULL| X2 | 9999 | 859K| | 40 (0)| 00:00:01 |
---------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("X1"."NR"="XX2"."NR")
Indexes on the the target table reduce the performance.Disable the index before updating the table and rebuild the index once the update completed.
Able to solve this using below query, and now the explain plan looks good.
merge INTO FRST_USAGE vfu USING
(SELECT tmp1.*
FROM
(SELECT ROW_NUMBER() OVER (PARTITION BY tmp.vin_hash ORDER BY tmp.usage_date) AS rn,
tmp.*
FROM
(SELECT vrd.VIN_HASHCODE AS vin_hash,
vrd.ACCOUNT_DN_HASHCODE AS actdn_hash,
vst.ACCOUNT_DN AS actdn,
vrd.FEATURE_NAME AS feature,
vrd.MAKE AS make,
vrd.COUNTRY AS country,
vrd.HU_TYPE AS hu,
vrd.REGION AS region,
vrd.SERVICE_HITS AS hits,
MIN(vrd.USAGE_DATE) AS usage_date,
sysdate AS maintlastTs
FROM RAW_DATA vrd,
SERVICES_TRACK vst
WHERE vrd.ACCOUNT_DN_HASHCODE=vst.ACCOUNTDN_HASHCODE
GROUP BY vrd.VIN_HASHCODE,
vrd.ACCOUNT_DN_HASHCODE,
vst.ACCOUNT_DN,
vrd.FEATURE_NAME,
vrd.MAKE,
vrd.COUNTRY,
vrd.HU_TYPE,
vrd.REGION,
vrd.SERVICE_HITS,
sysdate
ORDER BY vrd.VIN_HASHCODE,
MIN(vrd.USAGE_DATE)
) tmp
)tmp1
WHERE tmp1.rn =1
) tmp2 ON (vfu.VIN_HASHCODE = tmp2.vin_hash)
WHEN matched THEN
UPDATE
SET vfu.ACCOUNTDN_HASHCODE=tmp2.actdn_hash,
vfu.account_dn =tmp2.actdn,
vfu.FEATURE_NAME =tmp2.feature,
vfu.MAKE =tmp2.make,
vfu.COUNTRY =tmp2.country,
vfu.HU_TYPE =tmp2.hu,
vfu.REGION =tmp2.region,
vfu.SERVICE_HITS =tmp2.hits,
vfu.usage_date =tmp2.usage_date,
vfu.MAINT_LAST_TS =tmp2.maintlastTs;
below is the Explan plan:
Suggestions are allowed if there is any more optimizations i can do on this.