Optimize Oracle Query - oracle

I asked this question before but from comments I realized that the question was not enough information in the question to get help so here I am providing the whole query + explain plan:
Query:
WITH gtt_1 AS
(SELECT
r.user_id, r.role_id, r.participant_code, MAX(status_id)
FROM
user_role r,
cmp_role c
WHERE
r.role_id = c.role_id
AND r.participant_code IS NOT NULL
AND c.group_id = 3
GROUP BY
r.user_id, r.role_id, r.participant_code
HAVING MAX(status_id) IN (SELECT b.status_id FROM USER_ROLE b
WHERE (b.ACTIVE = 1 OR ( b.ACTIVE IN ( 0,3 )
AND SYSDATE BETWEEN b.effective_from_date AND b.effective_to_date
))
)
)
SELECT c.role_id,
c.subgroup,
c.subgroup_description,
COUNT(a.USER_ID) user_count
FROM
(SELECT b.user_id, b.role_id FROM gtt_1 b, user e
WHERE e.user_id = RTRIM(b.user_id)
) a
RIGHT OUTER JOIN CMP_ROLE c ON a.role_id = c.role_id
WHERE c.group_id = 3
GROUP BY c.role_id,c.subgroup,c.subgroup_description
ORDER BY c.subgroup;
Tables used:
SQL> select count(*) from user_role;
COUNT(*)
----------
803513
SQL> select count(*) from cmp_role;
COUNT(*)
----------
27
SQL> select count(*) from user;
COUNT(*)
----------
73893
Explain plan:
SQL> select * from table(dbms_xplan.display);
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------------------------------------------------
Plan hash value: 269840545
--------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Inst |IN-OUT|
--------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 256 | 18944 | 1683 (4)| 00:00:21 | | |
| 1 | SORT GROUP BY | | 256 | 18944 | 1683 (4)| 00:00:21 | | |
|* 2 | HASH JOIN OUTER | | 2776 | 200K| 1682 (4)| 00:00:21 | | |
| 3 | TABLE ACCESS BY INDEX ROWID | ROLE | 8 | 504 | 2 (0)| 00:00:01 | | |
|* 4 | INDEX RANGE SCAN | N_ROLE_IDX2 | 8 | | 1 (0)| 00:00:01 | | |
| 5 | VIEW | | 9370 | 100K| 1683 (5)| 00:00:21 | | |
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------------------------------------------------
| 6 | NESTED LOOPS | | 9370 | 164K| 1683 (5)| 00:00:21 | | |
| 7 | VIEW | | 9370 | 100K| 1677 (4)| 00:00:21 | | |
|* 8 | FILTER | | | | | | | |
| 9 | HASH GROUP BY | | 9370 | 283K| 1677 (4)| 00:00:21 | | |
|* 10 | HASH JOIN | | 232K| 7051K| 1654 (3)| 00:00:20 | | |
| 11 | TABLE ACCESS BY INDEX ROWID| ROLE | 8 | 56 | 2 (0)| 00:00:01 | | |
|* 12 | INDEX RANGE SCAN | N_ROLE_IDX2 | 8 | | 1 (0)| 00:00:01 | | |
|* 13 | TABLE ACCESS FULL | USER_ROLE | 786K| 17M| 1643 (2)| 00:00:20 | | |
|* 14 | TABLE ACCESS BY INDEX ROWID | USER_ROLE | 1 | 30 | 3 (0)| 00:00:01 | | |
|* 15 | INDEX UNIQUE SCAN | U_USER_ROLE_IDX1 | 1 | | 2 (0)| 00:00:01 | | |
| 16 | REMOTE | MV_P113PT_USER_HUB | 1 | 7 | 0 (0)| 00:00:01 | HWVAU~ | R->S |
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("A"."ROLE_ID"(+)="ROLE"."ROLE_ID")
4 - access("ROLE"."GRP_ID"=3)
8 - filter( EXISTS (SELECT 0 FROM "P181"."USER_ROLE" "USER_ROLE" WHERE "USER_ROLE"."STTS_ID"=:B1 AND
(TO_NUMBER("USER_ROLE"."ACTV_CD")=1 OR (TO_NUMBER("USER_ROLE"."ACTV_CD")=0 OR TO_NUMBER("USER_ROLE"."ACTV_CD")=3)
AND "USER_ROLE"."EFCTV_TO_DTTM">=SYSDATE#! AND "USER_ROLE"."EFCTV_FROM_DTTM"<=SYSDATE#!)))
10 - access("USER_ROLE"."ROLE_ID"="ROLE"."ROLE_ID")
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------------------------------------------------
12 - access("ROLE"."GRP_ID"=3)
13 - filter("USER_ROLE"."PRTPT_CD" IS NOT NULL)
14 - filter(TO_NUMBER("USER_ROLE"."ACTV_CD")=1 OR (TO_NUMBER("USER_ROLE"."ACTV_CD")=0 OR
TO_NUMBER("USER_ROLE"."ACTV_CD")=3) AND "USER_ROLE"."EFCTV_TO_DTTM">=SYSDATE#! AND
"USER_ROLE"."EFCTV_FROM_DTTM"<=SYSDATE#!)
15 - access("USER_ROLE"."STTS_ID"=:B1)
I do not have enough priveleges for tkprof

Does following query produce the same results?
SELECT c.role_id
, c.subgroup
, c.subgroup_description
, COUNT(a.USER_ID) user_count
FROM cmp_role c
LEFT OUTER JOIN (
SELECT r.user_id
, r.role_id
FROM (
SELECT r.user_id
, r.role_id
, r.participant_code
, MAX(r.status_id)
FROM user_role r
INNER JOIN cmp_role c ON c.role_id = r.role_id
WHERE r.participant_code IS NOT NULL
AND c.group_id = 3
GROUP BY
r.user_id
, r.role_id
, r.participant_code
HAVING MAX(r.status_id) IN (
SELECT status_id
FROM user_role
WHERE (Active = 1
OR (Active IN ( 0,3 )
AND SYSDATE BETWEEN effective_from_date AND effective_to_date)
)
)
) r
INNER JOIN user e ON e.user_id = RTRIM(r.user_id)
) a ON a.role_id = c.role_id
WHERE c.group_id = 3
GROUP BY
c.role_id
, c.subgroup
, c.subgroup_description
ORDER BY
c.subgroup
It is easier to read (for me). If it produces the same results, you can already start with dropping r.participant_codeand MAX(r.status_id)from the inner select.

Related

Performance drop returning cursor with union all

I'm facing unsolvable and impossible performace drop while using UNION ALL with two sub-queries in one cursor (at least I think that's the problem). PL/SQL Developer just freezes when opening cursor results in test window.
If I turn off no matter which sub-query - everything works fine.
If I take the whole query out of cursor to regular SQL Query windows - everything is okay without any need to turn off some parts.
Procedure structure is down below, looking forward any help:
procedure p_proc(p_param varchar2,
outcur out sys_refcursor) is
begin
open outcur for
select *
from (select -- visible cols
si.item_full_name
, si.final_price
, si.full_price
, si.receipt_num
, si.receipt_date
, si.vendor_code
, case when det.br_summary is null and mr.motiv_rate_value is not null then mr.motiv_rate_value
when det.br_summary is not null then det.br_summary
end personal_bonus_amount
, case when det.br_summary is null and mr.motiv_rate_value is not null then 1
when det.br_summary is not null then det.cross_sale_kt
end personal_bonus_koeff
-- service cols
, case when det.br_summary is null and mr.motiv_rate_value is not null then 'approximate'
when det.br_summary is not null then 'definite'
end personal_bonus_type
, coalesce(det.sale_stream, mr.sale_stream, 'Not defined') item_group_name
, si.operation_type
, si.src
-- pagination
, row_number() over (order by si.receipt_date desc) rn
from (-- curr day
select b.cost final_price
, case when b.discount = 0 then null else b.price
end full_price
, b.doc_number receipt_num
, b.receipt_date receipt_date
, i.item_code vendor_code
, i.full_name item_full_name
, b.subsite code_op
, b.operator_id
, to_char(b.businessday, 'yyyymm') sale_period
, b.oper_type operation_type
, 'bill' src
from scheme.bills b
join scheme.items i on i.item_code = b.item
where b.businessday = trunc(p_date_to)
and b.subsite = p_office_id
and b.operator_id = p_emp_id
union all
-- prev days
select l.txn_amount final_price
, case when l.disc = 0 then null else l.price
end full_price
, t.receipt_num receipt_num
, t.ts receipt_date
, i.item_code vendor_code
, i.full_name item_full_name
, s.office_code code_op
, e.emp_code operator_id
, to_char(l.dt,'yyyymm') sale_period
, l.txn_type operation_type
, 'txn' src
from scheme.txn t
join scheme.txn_lines l on t.rtl_txn_id = l.rtl_txn_id
join scheme.items i on l.item_id = i.item_id
join scheme.offices s on t.subsite_id = s.subsite_id
join scheme.employees e on t.employee_id = e.employee_id
where t.ts between trunc(p_date_from) and trunc(p_date_to)
and t.subsite_id = v_op_id
and t.employee_id = v_emp_id
) si
/* fact */
left join scheme.sales_details det on si.sale_period = det.period
and si.code_op = det.op_code
and ltrim(si.operator_id,'0') = ltrim(det.tab_num,'0')
and si.receipt_num = det.rcpt_num
and si.vendor_code = det.item_article
/* prognosis */
left join scheme.rates mr on si.sale_period = mr.motiv_rate_period
and si.code_op = mr.code_op
and si.vendor_code = mr.code_1c
where 1 = 1
and si.final_price between nvl(p_price_from, si.final_price) and nvl(p_price_to, si.final_price)
/* if no filters */
and (item_group_cnt = 0 or coalesce(det.sale_stream, mr.sale_stream, 'Not defined') in (select * from table(p_item_group)))
and si.receipt_num = nvl(p_receipt_num, si.receipt_num)
)
where rn between p_page_num * p_page_size + 1 and (p_page_num + 1) * p_page_size;
end;
UPD Explain plan for the whole query used in a cursor:
----------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost | Time |
----------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 10 | 32810 | 62 | 00:00:01 |
| * 1 | VIEW | | 10 | 32810 | 62 | 00:00:01 |
| * 2 | WINDOW SORT PUSHED RANK | | 2 | 2956 | 62 | 00:00:01 |
| 3 | NESTED LOOPS OUTER | | 2 | 2956 | 61 | 00:00:01 |
| 4 | NESTED LOOPS OUTER | | 2 | 2826 | 53 | 00:00:01 |
| 5 | VIEW | | 2 | 2728 | 46 | 00:00:01 |
| 6 | UNION-ALL | | | | | |
| 7 | NESTED LOOPS | | 1 | 138 | 32 | 00:00:01 |
| 8 | NESTED LOOPS | | 1 | 138 | 32 | 00:00:01 |
| 9 | PARTITION RANGE SINGLE | | 1 | 66 | 29 | 00:00:01 |
| * 10 | TABLE ACCESS BY LOCAL INDEX ROWID BATCHED | F003_BILL | 1 | 66 | 29 | 00:00:01 |
| * 11 | INDEX RANGE SCAN | IX_SUBSITE_DOCNUM_BUSINDAY_SEQ | 1 | | 5 | 00:00:01 |
| * 12 | INDEX RANGE SCAN | IX_D001_CODE_1C_ITEM_ID | 1 | | 2 | 00:00:01 |
| 13 | TABLE ACCESS BY INDEX ROWID | D001_ITEM | 1 | 72 | 3 | 00:00:01 |
| 14 | NESTED LOOPS | | 1 | 183 | 14 | 00:00:01 |
| 15 | NESTED LOOPS | | 1 | 183 | 14 | 00:00:01 |
| 16 | NESTED LOOPS | | 1 | 104 | 12 | 00:00:01 |
| 17 | NESTED LOOPS | | 1 | 70 | 7 | 00:00:01 |
| 18 | NESTED LOOPS | | 1 | 30 | 4 | 00:00:01 |
| 19 | TABLE ACCESS BY INDEX ROWID | D005_EMPLOYEE | 1 | 18 | 3 | 00:00:01 |
| * 20 | INDEX UNIQUE SCAN | PK_D005 | 1 | | 2 | 00:00:01 |
| 21 | TABLE ACCESS BY INDEX ROWID | D018_SUBSITE | 1 | 12 | 1 | 00:00:01 |
| * 22 | INDEX UNIQUE SCAN | PK_D018 | 1 | | 0 | 00:00:01 |
| 23 | PARTITION RANGE ITERATOR | | 1 | 40 | 3 | 00:00:01 |
| 24 | PARTITION HASH SINGLE | | 1 | 40 | 3 | 00:00:01 |
| * 25 | TABLE ACCESS FULL | F007_RTL_TXN | 1 | 40 | 3 | 00:00:01 |
| * 26 | TABLE ACCESS BY GLOBAL INDEX ROWID BATCHED | F008_RTL_TXN_LI | 1 | 34 | 5 | 00:00:01 |
| * 27 | INDEX RANGE SCAN | IX_F008_RTL_TXN_ID | 7 | | 3 | 00:00:01 |
| * 28 | INDEX UNIQUE SCAN | PK_D001 | 1 | | 1 | 00:00:01 |
| 29 | TABLE ACCESS BY INDEX ROWID | D001_ITEM | 1 | 79 | 2 | 00:00:01 |
| * 30 | TABLE ACCESS BY INDEX ROWID BATCHED | T_OP_MOTIVATION_RATE_MYRTK | 1 | 49 | 7 | 00:00:01 |
| * 31 | INDEX RANGE SCAN | IDX02_CODE_OP_1C | 3 | | 3 | 00:00:01 |
| * 32 | TABLE ACCESS BY INDEX ROWID BATCHED | DET_SALES_PPT_DWH | 1 | 65 | 4 | 00:00:01 |
| * 33 | INDEX RANGE SCAN | IDX_03_RCPT_NUM | 3 | | 2 | 00:00:01 |
----------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
------------------------------------------
* 1 - filter("RN">=1 AND "RN"<=10)
* 2 - filter(ROW_NUMBER() OVER ( ORDER BY INTERNAL_FUNCTION("SI"."RECEIPT_DATE") DESC )<=10)
* 10 - filter("F003"."OPERATOR_ID"='000189513' AND "F003"."COST">=TO_NUMBER(TO_CHAR("F003"."COST")) AND "F003"."COST"<=TO_NUMBER(TO_CHAR("F003"."COST")))
* 11 - access("F003"."SUBSITE"='S165' AND "F003"."BUSINESSDAY"=TO_DATE(' 2021-11-23 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
* 11 - filter("F003"."BUSINESSDAY"=TO_DATE(' 2021-11-23 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND "F003"."DOC_NUMBER" IS NOT NULL)
* 12 - access("I"."D001_CODE_1C"="F003"."ITEM")
* 12 - filter("I"."D001_CODE_1C" IS NOT NULL)
* 20 - access("E"."EMPLOYEE_ID"=3561503543)
* 22 - access("S"."SUBSITE_ID"=29260)
* 25 - filter("T"."EMPLOYEE_ID"=3561503543 AND "T"."SUBSITE_ID"=29260 AND "T"."F007_TS"<=TO_DATE(' 2021-11-23 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND "T"."F007_RCPT_NUM_1C" IS NOT NULL)
* 26 - filter("L"."F008_AMOUNT">=TO_NUMBER(TO_CHAR("L"."F008_AMOUNT")) AND "L"."F008_AMOUNT"<=TO_NUMBER(TO_CHAR("L"."F008_AMOUNT")))
* 27 - access("T"."RTL_TXN_ID"="L"."RTL_TXN_ID")
* 28 - access("L"."ITEM_ID"="I"."ITEM_ID")
* 30 - filter("SI"."SALE_PERIOD"="MR"."MOTIV_RATE_PERIOD"(+))
* 31 - access("SI"."CODE_OP"="MR"."CODE_OP"(+) AND "SI"."VENDOR_CODE"="MR"."CODE_1C"(+))
* 32 - filter("SI"."CODE_OP"="DET"."OP_CODE"(+) AND "SI"."VENDOR_CODE"="DET"."ITEM_ARTICLE"(+) AND "DET"."ITEM_ARTICLE"(+) IS NOT NULL AND "DET"."PERIOD"(+)=TO_NUMBER("SI"."SALE_PERIOD") AND
LTRIM("SI"."OPERATOR_ID",'0')=LTRIM("DET"."TAB_NUM_RTK"(+),'0'))
* 33 - access("SI"."RECEIPT_NUM"="DET"."RCPT_NUM"(+))
* 33 - filter("DET"."RCPT_NUM"(+) IS NOT NULL)
Actual solution
Managed to get procedure execution plan from DBA. The problem was that optimizer chose another index for joining scheme.sales_details table when executing query inside the procedure. Added INDEX HINT with the same index which was used in regular query and everything works just fine.
Deprecated ideas down below
As far as I understood the problem is in Oracle optimizer which "thought" that doing UNION ALL first is better than pushing predicate into the sub-query. Separating this union into two single queries make him push pred without any hesitations.
Probably this can be fixed by playing with hints, that's wip for now.
Temporary workaround is to regroup the query, going from this structure
select *
from (select row_number() rn
, u.*
from (select *
from first_query
union all
select *
from second_query) u
-- some joins
join first_table ft
join second_table st
-- predicate block
where 1=1
and a = b
)
where rn between c and d;
to this
select *
from (select row_number() rn
, u.*
from (select *
from first_query) u
-- some joins
join first_table ft
join second_table st
-- predicate block
where 1=1
and a = b
union all
select row_number() rn
, u.*
from (select *
from second_query) u
-- some joins
join first_table ft
join second_table st
-- predicate block
where 1=1
and a = b
)
where rn between c and d;
That's not the perfect solution cause it doubles the JOIN section but at least it works.

Union all and function based index

I have big tables with identical function based indexes. When one table is queried then query plan uses the index. When I join second table with union all then I have full scans.
I have found one working solution (WORKING JOIN WITH TABLE), but it has some limitations:
I cannot use ROWNUM not CONNECT BY in the x query merged to union
I cannot use pipeline function in a query.
How to overcome the mentioned limitations?
I tried hints like CARDINALITY to tell that the number of record is small, and some rewrite (REWRITE, PUSH_PRED) without success.
I cannot use partitioning since the problem is on Oracle SE.
The problem shown here is a simplification of my actual problem which is:
I have huge data set divided into identical tables
Each table contains data from a different month
I have a view which union all the underlying tables
It is kind of partitioning style practiced on oracle before v8.0
In real I may have adhoc queries and joins with many different tables. Thus I cannot simply push the join into a union which will be the simplest solution.
Here is the DDL script.
-- FULL SCAN FOR IN SUBQUERY
SELECT * FROM (
SELECT * FROM TEST_EXPV1
UNION ALL SELECT * FROM TEST_EXPV2
)
WHERE DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value) IN (SELECT Id FROM test_10r)
-----------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 200K| 5078K| 916 (3)| 00:00:11 |
|* 1 | HASH JOIN | | 200K| 5078K| 916 (3)| 00:00:11 |
| 2 | VIEW | VW_NSO_1 | 10 | 130 | 4 (25)| 00:00:01 |
| 3 | HASH UNIQUE | | 10 | 30 | 4 (25)| 00:00:01 |
| 4 | TABLE ACCESS FULL| TEST_10R | 10 | 30 | 3 (0)| 00:00:01 |
| 5 | VIEW | | 2000K| 24M| 902 (2)| 00:00:11 |
| 6 | UNION-ALL | | | | | |
| 7 | TABLE ACCESS FULL| TEST_EXPV1 | 1000K| 3906K| 451 (2)| 00:00:06 |
| 8 | TABLE ACCESS FULL| TEST_EXPV2 | 1000K| 3906K| 451 (2)| 00:00:06 |
-----------------------------------------------------------------------------------
-- CORRECT RANGE INDEX SCAN for bound value
SELECT * FROM (
SELECT * FROM TEST_EXPV1
UNION ALL SELECT * FROM TEST_EXPV2
)
WHERE DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value) = :b1001
----------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 20000 | 253K| 979 (10)| 00:00:12 |
| 1 | VIEW | | 20000 | 253K| 979 (10)| 00:00:12 |
| 2 | UNION-ALL | | | | | |
| 3 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV1 | 2 | 8 | 3 (0)| 00:00:01 |
|* 4 | INDEX RANGE SCAN | I_TEST_EXPV1 | 2 | | 1 (0)| 00:00:01 |
| 5 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV2 | 2 | 8 | 3 (0)| 00:00:01 |
|* 6 | INDEX RANGE SCAN | I_TEST_EXPV2 | 2 | | 1 (0)| 00:00:01 |
----------------------------------------------------------------------------------------------
-- WORKING JOIN WITH TABLE
WITH x AS (SELECT Id FROM test_10r WHERE Id BETWEEN :a AND :b)
SELECT /*+ FIRST_ROWS */ * FROM (
SELECT * FROM TEST_EXPV1
UNION ALL SELECT * FROM TEST_EXPV2
) U, x
WHERE DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value) = x.id
-------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 16 | 996 (11)| 00:00:12 |
|* 1 | FILTER | | | | | |
|* 2 | HASH JOIN | | 500 | 8000 | 996 (11)| 00:00:12 |
|* 3 | TABLE ACCESS FULL | TEST_10R | 10 | 30 | 3 (0)| 00:00:01 |
| 4 | VIEW | | 5000 | 65000 | 993 (11)| 00:00:12 |
| 5 | UNION-ALL | | | | | |
|* 6 | FILTER | | | | | |
| 7 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV1 | 2500 | 10000 | 4192 (1)| 00:00:51 |
|* 8 | INDEX RANGE SCAN | I_TEST_EXPV1 | 4500 | | 11 (0)| 00:00:01 |
|* 9 | FILTER | | | | | |
| 10 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV2 | 2500 | 10000 | 4192 (1)| 00:00:51 |
|* 11 | INDEX RANGE SCAN | I_TEST_EXPV2 | 4500 | | 11 (0)| 00:00:01 |
-------------------------------------------------------------------------------------------------
You're dong the UNION ALL, which is doing a full scan of both tables to get combined result set; and only then filtering on the values from the third table.
A more usual pattern for subquery factoring is to refer to the CTE in each branch of the union:
WITH x AS (SELECT Id FROM test_10r)
SELECT TEST_EXPV1.* FROM x JOIN TEST_EXPV1
ON DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value) = x.Id
UNION ALL SELECT TEST_EXPV2.* FROM x JOIN TEST_EXPV2
ON DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value) = x.Id;
----------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 32 | 224 | 66 (0)| 00:00:01 |
| 1 | UNION-ALL | | | | | |
| 2 | NESTED LOOPS | | 16 | 112 | 33 (0)| 00:00:01 |
| 3 | TABLE ACCESS FULL | TEST_10R | 10 | 30 | 3 (0)| 00:00:01 |
| 4 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV1 | 2 | 8 | 3 (0)| 00:00:01 |
|* 5 | INDEX RANGE SCAN | I_TEST_EXPV1 | 2 | | 1 (0)| 00:00:01 |
| 6 | NESTED LOOPS | | 16 | 112 | 33 (0)| 00:00:01 |
| 7 | TABLE ACCESS FULL | TEST_10R | 10 | 30 | 3 (0)| 00:00:01 |
| 8 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV2 | 2 | 8 | 3 (0)| 00:00:01 |
|* 9 | INDEX RANGE SCAN | I_TEST_EXPV2 | 2 | | 1 (0)| 00:00:01 |
----------------------------------------------------------------------------------------------
Pushing predicates into a UNION ALL can be temperamental. Try this:
SELECT /*+ PUSH_PRED(v) */ *
FROM (SELECT * FROM test_expv1
UNION ALL
SELECT * FROM test_expv2) v INNER JOIN test_10r ON
(DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value)) = test_10r.id;
Here are my results for the above query from an 11.2.0.4 instance, using the OP's DDL:
SQL_ID df6dvkgjwjsq1, child number 1
-------------------------------------
SELECT /*+ PUSH_PRED(v) */ * FROM (SELECT * FROM test_expv1
UNION ALL SELECT * FROM test_expv2) v INNER JOIN test_10r ON
(DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS
NUMBER(38)), Value)) = test_10r.id
Plan hash value: 191389749
---------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers |
---------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 16 |00:00:00.01 | 69 |
| 1 | NESTED LOOPS | | 1 | 2000K| 16 |00:00:00.01 | 69 |
| 2 | TABLE ACCESS FULL | TEST_10R | 1 | 10 | 10 |00:00:00.01 | 22 |
| 3 | VIEW | | 10 | 32 | 16 |00:00:00.01 | 47 |
| 4 | UNION ALL PUSHED PREDICATE | | 10 | | 16 |00:00:00.01 | 47 |
| 5 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV1 | 10 | 158 | 8 |00:00:00.01 | 24 |
|* 6 | INDEX RANGE SCAN | I_TEST_EXPV1 | 10 | 2 | 8 |00:00:00.01 | 16 |
| 7 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV2 | 10 | 158 | 8 |00:00:00.01 | 23 |
|* 8 | INDEX RANGE SCAN | I_TEST_EXPV2 | 10 | 2 | 8 |00:00:00.01 | 15 |
---------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
6 - access("TEST_EXPV1"."SYS_NC00002$"="TEST_10R"."ID")
8 - access("TEST_EXPV2"."SYS_NC00002$"="TEST_10R"."ID")

oracle 12c query is slow

below code works well in 11g BUT only in 12c works so slow (over 10 sec).
SELECT * FROM DMProgValue_00001
WHERE 1=1
AND ProgressOID IN (
SELECT P.OID FROM (
SELECT OID FROM (
SELECT A.OID, ROWNUM as seqNum FROM (
SELECT OID FROM DMProgress_00001 WHERE 1=1
AND Project = 'Q539'
ORDER BY actCode
) A
WHERE ROWNUM <= 40
) WHERE seqNum > 20
) P
);
Plan hash value: 763232015
-----------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 189 | 171 (1)| 00:00:01 |
|* 1 | FILTER | | | | | |
| 2 | TABLE ACCESS FULL | DMPROGVALUE_00001 | 1 | 189 | 68 (0)| 00:00:01 |
|* 3 | VIEW | | 20 | 800 | 103 (1)| 00:00:01 |
|* 4 | COUNT STOPKEY | | | | | |
| 5 | VIEW | | 2916 | 78732 | 103 (1)| 00:00:01 |
|* 6 | SORT ORDER BY STOPKEY| | 2916 | 130K| 103 (1)| 00:00:01 |
|* 7 | TABLE ACCESS FULL | DMPROGRESS_00001 | 2916 | 130K| 102 (0)| 00:00:01 |
-----------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter( EXISTS (SELECT 0 FROM (SELECT "A"."OID" "OID",ROWNUM "SEQNUM" FROM
(SELECT "OID" "OID" FROM "DMPROGRESS_00001" "DMPROGRESS_00001" WHERE
"PHASE"='Construction' AND "PROJECT"='Q539' ORDER BY "ACTCODE") "A" WHERE ROWNUM<=40)
"from$_subquery$_003" WHERE "SEQNUM">20 AND "OID"=:B1))
3 - filter("SEQNUM">20 AND "OID"=:B1)
4 - filter(ROWNUM<=40)
6 - filter(ROWNUM<=40)
7 - filter("PHASE"='Construction' AND "PROJECT"='Q539')
DMProgress_0001 stats
NUM_ROW : 10385
BLOCKS : 370
AVG_ROW_LEN : 176
SMAPLE_SIZE : 8263
DMProgvalue_0001 stats
NUM_ROW : 15703
BLOCKS : 244
AVG_ROW_LEN : 49
SMAPLE_SIZE : 5033
It's only 10k rows and Indexs are well made ( I can tell because of 11g experience). I know some detour way to make fast ( below code - 0.001 sec) BUT I want to know real problem and fix it.
I cannot understand it only has one sub query and 10k rows for each table. Not just compared to 11g, There is no way that this query takes over 10 sec.
SELECT * FROM DMProgValue_00001 V,
(SELECT OID FROM (
SELECT A.OID, ROWNUM as seqNum FROM (
SELECT OID FROM DMProgress_00001 WHERE 1=1
AND Project = 'Q539'
ORDER BY actCode
) A
WHERE ROWNUM <= 40
) WHERE seqNum > 20
) P
WHERE 1=1
AND V.ProgressOID = P.OID;
added 11g similar query plan
Plan hash value: 3049049852
-----------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 9 | 684 | 49 (5)| 00:00:01 |
|* 1 | HASH JOIN RIGHT SEMI | | 9 | 684 | 49 (5)| 00:00:01 |
| 2 | VIEW | VW_NSO_1 | 3 | 81 | 35 (3)| 00:00:01 |
|* 3 | VIEW | | 3 | 75 | 35 (3)| 00:00:01 |
|* 4 | COUNT STOPKEY | | | | | |
| 5 | VIEW | | 3 | 36 | 35 (3)| 00:00:01 |
|* 6 | SORT ORDER BY STOPKEY| | 3 | 144 | 35 (3)| 00:00:01 |
|* 7 | TABLE ACCESS FULL | DMPROGRESS_00037 | 3 | 144 | 34 (0)| 00:00:01 |
| 8 | TABLE ACCESS FULL | DMPROGVALUE_00037 | 5106 | 244K| 13 (0)| 00:00:01 |
------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("PROGRESSOID"="OID")
3 - filter("SEQNUM">20)
4 - filter(ROWNUM<=40)
6 - filter(ROWNUM<=40)
7 - filter("DISPLINE"='Q340' AND "PHASE"='Procurement' AND "PROJECT"='Moho')
oracle 11g automatically change the query as Hash join BUT 12c does not. I think this is point. They are same structure.

why do they use (+) in where clause for checking not null in a column eg emp_name(+) IS NOT NULL

Why do we use a (+) operator in the where clause for instance emp_name(+) IS NOT NULL, emp_name IS NOT NULL AND emp_name(+) IS NOT NULL is the same
Because removing the (+) from the column you're checking is not null turns the join from an outer join into what is effectively an inner join. Leaving the (+) in tells oracle to get all rows from the "main" table, and then match any rows from the outer joined table where that column is not null.
See the below for an example of why the "extra" (+) is needed:
with t1 as (select 1 id, 'a' val from dual union all
select 2 id, 'b' val from dual union all
select 3 id, 'c' val from dual),
t2 as (select 1 id, null val from dual union all
select 3 id, 'd' val from dual)
select *
from t1,
t2
where t1.id = t2.id (+)
and t2.val (+) is not null
order by t1.id;
ID VAL ID_1 VAL_1
---------- --- ---------- -----
1 a
2 b
3 c 3 d
with t1 as (select 1 id, 'a' val from dual union all
select 2 id, 'b' val from dual union all
select 3 id, 'c' val from dual),
t2 as (select 1 id, null val from dual union all
select 3 id, 'd' val from dual)
select *
from t1,
t2
where t1.id = t2.id (+)
and t2.val is not null
order by t1.id;
ID VAL ID_1 VAL_1
---------- --- ---------- -----
3 c 3 d
You can see the difference easier if you convert the query to the ANSI join syntax:
with t1 as (select 1 id, 'a' val from dual union all
select 2 id, 'b' val from dual union all
select 3 id, 'c' val from dual),
t2 as (select 1 id, null val from dual union all
select 3 id, 'd' val from dual)
select *
from t1
left outer join t2 on (t1.id = t2.id and t2.val is not null)
order by t1.id;
ID VAL ID_1 VAL_1
---------- --- ---------- -----
1 a
2 b
3 c 3 d
with t1 as (select 1 id, 'a' val from dual union all
select 2 id, 'b' val from dual union all
select 3 id, 'c' val from dual),
t2 as (select 1 id, null val from dual union all
select 3 id, 'd' val from dual)
select *
from t1
left outer join t2 on (t1.id = t2.id)
where t2.val is not null
order by t1.id;
ID VAL ID_1 VAL_1
---------- --- ---------- -----
3 c 3 d
In other words, it's the difference between the "col is not null" predicate being a part of the outer join condition, or a filter in the where clause.
You'll note as well that having the "t2.val is not null" in the where clause has the effect of turning the outer join into an inner join, despite the fact that you've requested an outer join:
with t1 as (select 1 id, 'a' val from dual union all
select 2 id, 'b' val from dual union all
select 3 id, 'c' val from dual),
t2 as (select 1 id, null val from dual union all
select 3 id, 'd' val from dual)
select *
from t1
left outer join t2 on (t1.id = t2.id)
--where t2.val is not null
order by t1.id;
-----------------------------------------------------------------------------------------------------
| Id | Operation | Name | E-Rows |E-Bytes| Cost (%CPU)| E-Time | OMem | 1Mem | O/1/M |
-----------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | 12 (100)| | | | |
| 1 | SORT ORDER BY | | 3 | 33 | 12 (17)| 00:00:01 | 2048 | 2048 | 1/0/0|
|* 2 | HASH JOIN OUTER| | 3 | 33 | 11 (10)| 00:00:01 | 1156K| 1156K| 1/0/0|
| 3 | VIEW | | 3 | 18 | 6 (0)| 00:00:01 | | | |
| 4 | UNION-ALL | | | | | | | | |
| 5 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | | | |
| 6 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | | | |
| 7 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | | | |
| 8 | VIEW | | 2 | 10 | 4 (0)| 00:00:01 | | | |
| 9 | UNION-ALL | | | | | | | | |
| 10 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | | | |
| 11 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | | | |
-----------------------------------------------------------------------------------------------------
with t1 as (select 1 id, 'a' val from dual union all
select 2 id, 'b' val from dual union all
select 3 id, 'c' val from dual),
t2 as (select 1 id, null val from dual union all
select 3 id, 'd' val from dual)
select *
from t1
left outer join t2 on (t1.id = t2.id)
where t2.val is not null
order by t1.id;
-----------------------------------------------------------------------------------------------------
| Id | Operation | Name | E-Rows |E-Bytes| Cost (%CPU)| E-Time | OMem | 1Mem | O/1/M |
-----------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | 10 (100)| | | | |
| 1 | SORT ORDER BY | | 1 | 11 | 10 (20)| 00:00:01 | 2048 | 2048 | 3/0/0|
|* 2 | HASH JOIN | | 1 | 11 | 9 (12)| 00:00:01 | 1156K| 1156K| 3/0/0|
| 3 | VIEW | | 2 | 10 | 2 (0)| 00:00:01 | | | |
| 4 | UNION-ALL | | | | | | | | |
|* 5 | FILTER | | | | | | | | |
| 6 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | | | |
| 7 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | | | |
| 8 | VIEW | | 3 | 18 | 6 (0)| 00:00:01 | | | |
| 9 | UNION-ALL | | | | | | | | |
| 10 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | | | |
| 11 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | | | |
| 12 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | | | |
-----------------------------------------------------------------------------------------------------
Note the change from HASH JOIN OUTER to HASH JOIN in the row with id = 2 in the 2nd explain plan.

Problem with predicte pushing in a Max and Min statistic about each user in a group

I have two tables in an ORACLE 10g DB:
The table USERS with columns:
UserId | GroupId | Other details
where there's 1 record for each User, total 400, and there c.a. 1-10 users in each Group.
and the table USAGE with columns:
UserId | Date | Amount
where there's 10000 rows for each user. (so 400*10000 rows)
I have the Users table indexed on UserId, GroupId and (UserId,GroupId) and the table Usage is indexed on (UserId,Date)
I want to see the Max and Total amount of usage for each user in a given group.
Something like this:
select User.UserId, maxAmount, Total from
Users
JOIN
( select UserId,max(Amount) as maxAmount from USAGE group by UserId ) A
ON User.UserId = A.UserId
JOIN
( select UserId,min(Amount) as minAmount from USAGE group by UserId ) B
ON User.UserId = B.UserId
WHERE User.GroupId = 'some_group_id'
But it is very slow (approx 20 sec), but when I do them separately (just Max and just Min, but not both at once), like this:
select User.UserId, maxAmount, minAmount from
Users
JOIN
( select UserId,max(Amount) as maxAmount from USAGE group by UserId ) A
ON User.UserId = A.UserId
WHERE User.GroupId = 'some_group_id'
it runs in a flash!
It just makes no sense that I can join them separately very fast, but when I join them all three it's very slow.
Here are the plans, where I created the inline views in the preceeding statements as actual views:
create view usage_min as select user_id, min(amount) from usage group by user_id;
create view usage_max as select user_id, max(amount) from usage group by user_id;
explain plan for select * from usage_min join users using(user_id) where group_id='1212882339';
select * from table(dbms_xplan.display);
Plan hash value: 3874246446
-----------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 139 | 1162 (1)| 00:00:14 |
| 1 | HASH GROUP BY | | 1 | 139 | 1162 (1)| 00:00:14 |
| 2 | TABLE ACCESS BY INDEX ROWID | USAGE | 7977 | 116K| 1157 (1)| 00:00:14 |
| 3 | NESTED LOOPS | | 11085 | 1504K| 1160 (1)| 00:00:14 |
| 4 | MAT_VIEW ACCESS BY INDEX ROWID| USERS | 1 | 124 | 3 (0)| 00:00:01 |
|* 5 | INDEX RANGE SCAN | USERS_KT | 1 | | 1 (0)| 00:00:01 |
|* 6 | INDEX RANGE SCAN | SYS_C0099818 | 7977 | | 79 (0)| 00:00:01 |
-----------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
5 - access("USERS"."group_id"='1212882339')
6 - access("user_id"="USERS"."user_id")
explain plan for select * from users join usage_max using(user_id) where group_id='1212882339';
select * from table(dbms_xplan.display);
Plan hash value: 2384977958
-----------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 145 | 519 (2)| 00:00:07 |
| 1 | HASH GROUP BY | | 1 | 145 | 519 (2)| 00:00:07 |
| 2 | TABLE ACCESS BY INDEX ROWID | USAGE2 | 18251 | 374K| 512 (1)| 00:00:07 |
| 3 | NESTED LOOPS | | 25362 | 3591K| 515 (1)| 00:00:07 |
| 4 | MAT_VIEW ACCESS BY INDEX ROWID| USERS | 1 | 124 | 3 (0)| 00:00:01 |
|* 5 | INDEX RANGE SCAN | USERS_KT | 1 | | 1 (0)| 00:00:01 |
|* 6 | INDEX RANGE SCAN | T2_user_id | 18251 | | 25 (4)| 00:00:01 |
-----------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
5 - access("USERS"."group_id"='1212882339')
6 - access("USERS"."user_id"="user_id")
explain plan for select * from users join usage_max using(user_id) join usage_min using(user_id) where group_id='1212882339';
select * from table(dbms_xplan.display);
Plan hash value: 3190011991
-----------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 100 | 16600 | 6782 (19)| 00:01:22 |
|* 1 | HASH JOIN | | 100 | 16600 | 6782 (19)| 00:01:22 |
| 2 | MERGE JOIN | | 100 | 14500 | 3176 (19)| 00:00:39 |
| 3 | SORT JOIN | | 2920K| 58M| 3172 (19)| 00:00:39 |
| 4 | VIEW | USAGE_MAX | 2920K| 58M| 3172 (19)| 00:00:39 |
| 5 | HASH GROUP BY | | 2920K| 58M| 3172 (19)| 00:00:39 |
| 6 | TABLE ACCESS FULL | USAGE2 | 2920K| 58M| 2686 (5)| 00:00:33 |
|* 7 | SORT JOIN | | 1 | 124 | 4 (25)| 00:00:01 |
| 8 | MAT_VIEW ACCESS BY INDEX ROWID| USERS | 1 | 124 | 3 (0)| 00:00:01 |
|* 9 | INDEX RANGE SCAN | USERS_KT | 1 | | 1 (0)| 00:00:01 |
| 10 | VIEW | USAGE_MIN | 398 | 8358 | 3605 (19)| 00:00:44 |
| 11 | HASH GROUP BY | | 398 | 5970 | 3605 (19)| 00:00:44 |
| 12 | TABLE ACCESS FULL | USAGE | 3174K| 45M| 3073 (4)| 00:00:37 |
-----------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("USAGE_MAX"."user_id"="USAGE_MIN"."user_id")
7 - access("USERS"."user_id"="USAGE_MAX"."user_id")
filter("USERS"."user_id"="USAGE_MAX"."user_id")
9 - access("USERS"."group_id"='1212882339')
What are the query plans?
Are you measuring the time to fetch the first row? Or are you measuring the time to fetch the last row?
At a minimum, I'd want to hit the USAGE table just once using something like
select User.UserId, maxAmount, Total from
Users
JOIN
( select UserId,
max(Amount) as maxAmount,
sum(Amount) as Total
from USAGE group by UserId ) A
ON User.UserId = A.UserId
WHERE User.GroupId = 'some_group_id'
or even more simply
SELECT user.userId,
max(usage.Amount) maxAmount,
sum(usage.Amount) total
FROM user
join usage on (user.userId = usage.userId)
WHERE user.GroupId = 'some_group_id'
GROUP BY user.userId

Resources