I'm facing unsolvable and impossible performace drop while using UNION ALL with two sub-queries in one cursor (at least I think that's the problem). PL/SQL Developer just freezes when opening cursor results in test window.
If I turn off no matter which sub-query - everything works fine.
If I take the whole query out of cursor to regular SQL Query windows - everything is okay without any need to turn off some parts.
Procedure structure is down below, looking forward any help:
procedure p_proc(p_param varchar2,
outcur out sys_refcursor) is
begin
open outcur for
select *
from (select -- visible cols
si.item_full_name
, si.final_price
, si.full_price
, si.receipt_num
, si.receipt_date
, si.vendor_code
, case when det.br_summary is null and mr.motiv_rate_value is not null then mr.motiv_rate_value
when det.br_summary is not null then det.br_summary
end personal_bonus_amount
, case when det.br_summary is null and mr.motiv_rate_value is not null then 1
when det.br_summary is not null then det.cross_sale_kt
end personal_bonus_koeff
-- service cols
, case when det.br_summary is null and mr.motiv_rate_value is not null then 'approximate'
when det.br_summary is not null then 'definite'
end personal_bonus_type
, coalesce(det.sale_stream, mr.sale_stream, 'Not defined') item_group_name
, si.operation_type
, si.src
-- pagination
, row_number() over (order by si.receipt_date desc) rn
from (-- curr day
select b.cost final_price
, case when b.discount = 0 then null else b.price
end full_price
, b.doc_number receipt_num
, b.receipt_date receipt_date
, i.item_code vendor_code
, i.full_name item_full_name
, b.subsite code_op
, b.operator_id
, to_char(b.businessday, 'yyyymm') sale_period
, b.oper_type operation_type
, 'bill' src
from scheme.bills b
join scheme.items i on i.item_code = b.item
where b.businessday = trunc(p_date_to)
and b.subsite = p_office_id
and b.operator_id = p_emp_id
union all
-- prev days
select l.txn_amount final_price
, case when l.disc = 0 then null else l.price
end full_price
, t.receipt_num receipt_num
, t.ts receipt_date
, i.item_code vendor_code
, i.full_name item_full_name
, s.office_code code_op
, e.emp_code operator_id
, to_char(l.dt,'yyyymm') sale_period
, l.txn_type operation_type
, 'txn' src
from scheme.txn t
join scheme.txn_lines l on t.rtl_txn_id = l.rtl_txn_id
join scheme.items i on l.item_id = i.item_id
join scheme.offices s on t.subsite_id = s.subsite_id
join scheme.employees e on t.employee_id = e.employee_id
where t.ts between trunc(p_date_from) and trunc(p_date_to)
and t.subsite_id = v_op_id
and t.employee_id = v_emp_id
) si
/* fact */
left join scheme.sales_details det on si.sale_period = det.period
and si.code_op = det.op_code
and ltrim(si.operator_id,'0') = ltrim(det.tab_num,'0')
and si.receipt_num = det.rcpt_num
and si.vendor_code = det.item_article
/* prognosis */
left join scheme.rates mr on si.sale_period = mr.motiv_rate_period
and si.code_op = mr.code_op
and si.vendor_code = mr.code_1c
where 1 = 1
and si.final_price between nvl(p_price_from, si.final_price) and nvl(p_price_to, si.final_price)
/* if no filters */
and (item_group_cnt = 0 or coalesce(det.sale_stream, mr.sale_stream, 'Not defined') in (select * from table(p_item_group)))
and si.receipt_num = nvl(p_receipt_num, si.receipt_num)
)
where rn between p_page_num * p_page_size + 1 and (p_page_num + 1) * p_page_size;
end;
UPD Explain plan for the whole query used in a cursor:
----------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost | Time |
----------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 10 | 32810 | 62 | 00:00:01 |
| * 1 | VIEW | | 10 | 32810 | 62 | 00:00:01 |
| * 2 | WINDOW SORT PUSHED RANK | | 2 | 2956 | 62 | 00:00:01 |
| 3 | NESTED LOOPS OUTER | | 2 | 2956 | 61 | 00:00:01 |
| 4 | NESTED LOOPS OUTER | | 2 | 2826 | 53 | 00:00:01 |
| 5 | VIEW | | 2 | 2728 | 46 | 00:00:01 |
| 6 | UNION-ALL | | | | | |
| 7 | NESTED LOOPS | | 1 | 138 | 32 | 00:00:01 |
| 8 | NESTED LOOPS | | 1 | 138 | 32 | 00:00:01 |
| 9 | PARTITION RANGE SINGLE | | 1 | 66 | 29 | 00:00:01 |
| * 10 | TABLE ACCESS BY LOCAL INDEX ROWID BATCHED | F003_BILL | 1 | 66 | 29 | 00:00:01 |
| * 11 | INDEX RANGE SCAN | IX_SUBSITE_DOCNUM_BUSINDAY_SEQ | 1 | | 5 | 00:00:01 |
| * 12 | INDEX RANGE SCAN | IX_D001_CODE_1C_ITEM_ID | 1 | | 2 | 00:00:01 |
| 13 | TABLE ACCESS BY INDEX ROWID | D001_ITEM | 1 | 72 | 3 | 00:00:01 |
| 14 | NESTED LOOPS | | 1 | 183 | 14 | 00:00:01 |
| 15 | NESTED LOOPS | | 1 | 183 | 14 | 00:00:01 |
| 16 | NESTED LOOPS | | 1 | 104 | 12 | 00:00:01 |
| 17 | NESTED LOOPS | | 1 | 70 | 7 | 00:00:01 |
| 18 | NESTED LOOPS | | 1 | 30 | 4 | 00:00:01 |
| 19 | TABLE ACCESS BY INDEX ROWID | D005_EMPLOYEE | 1 | 18 | 3 | 00:00:01 |
| * 20 | INDEX UNIQUE SCAN | PK_D005 | 1 | | 2 | 00:00:01 |
| 21 | TABLE ACCESS BY INDEX ROWID | D018_SUBSITE | 1 | 12 | 1 | 00:00:01 |
| * 22 | INDEX UNIQUE SCAN | PK_D018 | 1 | | 0 | 00:00:01 |
| 23 | PARTITION RANGE ITERATOR | | 1 | 40 | 3 | 00:00:01 |
| 24 | PARTITION HASH SINGLE | | 1 | 40 | 3 | 00:00:01 |
| * 25 | TABLE ACCESS FULL | F007_RTL_TXN | 1 | 40 | 3 | 00:00:01 |
| * 26 | TABLE ACCESS BY GLOBAL INDEX ROWID BATCHED | F008_RTL_TXN_LI | 1 | 34 | 5 | 00:00:01 |
| * 27 | INDEX RANGE SCAN | IX_F008_RTL_TXN_ID | 7 | | 3 | 00:00:01 |
| * 28 | INDEX UNIQUE SCAN | PK_D001 | 1 | | 1 | 00:00:01 |
| 29 | TABLE ACCESS BY INDEX ROWID | D001_ITEM | 1 | 79 | 2 | 00:00:01 |
| * 30 | TABLE ACCESS BY INDEX ROWID BATCHED | T_OP_MOTIVATION_RATE_MYRTK | 1 | 49 | 7 | 00:00:01 |
| * 31 | INDEX RANGE SCAN | IDX02_CODE_OP_1C | 3 | | 3 | 00:00:01 |
| * 32 | TABLE ACCESS BY INDEX ROWID BATCHED | DET_SALES_PPT_DWH | 1 | 65 | 4 | 00:00:01 |
| * 33 | INDEX RANGE SCAN | IDX_03_RCPT_NUM | 3 | | 2 | 00:00:01 |
----------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
------------------------------------------
* 1 - filter("RN">=1 AND "RN"<=10)
* 2 - filter(ROW_NUMBER() OVER ( ORDER BY INTERNAL_FUNCTION("SI"."RECEIPT_DATE") DESC )<=10)
* 10 - filter("F003"."OPERATOR_ID"='000189513' AND "F003"."COST">=TO_NUMBER(TO_CHAR("F003"."COST")) AND "F003"."COST"<=TO_NUMBER(TO_CHAR("F003"."COST")))
* 11 - access("F003"."SUBSITE"='S165' AND "F003"."BUSINESSDAY"=TO_DATE(' 2021-11-23 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
* 11 - filter("F003"."BUSINESSDAY"=TO_DATE(' 2021-11-23 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND "F003"."DOC_NUMBER" IS NOT NULL)
* 12 - access("I"."D001_CODE_1C"="F003"."ITEM")
* 12 - filter("I"."D001_CODE_1C" IS NOT NULL)
* 20 - access("E"."EMPLOYEE_ID"=3561503543)
* 22 - access("S"."SUBSITE_ID"=29260)
* 25 - filter("T"."EMPLOYEE_ID"=3561503543 AND "T"."SUBSITE_ID"=29260 AND "T"."F007_TS"<=TO_DATE(' 2021-11-23 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND "T"."F007_RCPT_NUM_1C" IS NOT NULL)
* 26 - filter("L"."F008_AMOUNT">=TO_NUMBER(TO_CHAR("L"."F008_AMOUNT")) AND "L"."F008_AMOUNT"<=TO_NUMBER(TO_CHAR("L"."F008_AMOUNT")))
* 27 - access("T"."RTL_TXN_ID"="L"."RTL_TXN_ID")
* 28 - access("L"."ITEM_ID"="I"."ITEM_ID")
* 30 - filter("SI"."SALE_PERIOD"="MR"."MOTIV_RATE_PERIOD"(+))
* 31 - access("SI"."CODE_OP"="MR"."CODE_OP"(+) AND "SI"."VENDOR_CODE"="MR"."CODE_1C"(+))
* 32 - filter("SI"."CODE_OP"="DET"."OP_CODE"(+) AND "SI"."VENDOR_CODE"="DET"."ITEM_ARTICLE"(+) AND "DET"."ITEM_ARTICLE"(+) IS NOT NULL AND "DET"."PERIOD"(+)=TO_NUMBER("SI"."SALE_PERIOD") AND
LTRIM("SI"."OPERATOR_ID",'0')=LTRIM("DET"."TAB_NUM_RTK"(+),'0'))
* 33 - access("SI"."RECEIPT_NUM"="DET"."RCPT_NUM"(+))
* 33 - filter("DET"."RCPT_NUM"(+) IS NOT NULL)
Actual solution
Managed to get procedure execution plan from DBA. The problem was that optimizer chose another index for joining scheme.sales_details table when executing query inside the procedure. Added INDEX HINT with the same index which was used in regular query and everything works just fine.
Deprecated ideas down below
As far as I understood the problem is in Oracle optimizer which "thought" that doing UNION ALL first is better than pushing predicate into the sub-query. Separating this union into two single queries make him push pred without any hesitations.
Probably this can be fixed by playing with hints, that's wip for now.
Temporary workaround is to regroup the query, going from this structure
select *
from (select row_number() rn
, u.*
from (select *
from first_query
union all
select *
from second_query) u
-- some joins
join first_table ft
join second_table st
-- predicate block
where 1=1
and a = b
)
where rn between c and d;
to this
select *
from (select row_number() rn
, u.*
from (select *
from first_query) u
-- some joins
join first_table ft
join second_table st
-- predicate block
where 1=1
and a = b
union all
select row_number() rn
, u.*
from (select *
from second_query) u
-- some joins
join first_table ft
join second_table st
-- predicate block
where 1=1
and a = b
)
where rn between c and d;
That's not the perfect solution cause it doubles the JOIN section but at least it works.
I have a complex Oracle UNION query working fine. The query execution take time since it has to traverse through too many records. I need to analyze this query and optimize as much as possible.
SELECT DISTINCT APP.APPLICATION_ID, APE.ENTITY_ID ENTITY_ID, APE.PARENT_ENTITY_ID, RP.ROLE_ID, RP.ACTION_ID, COALESCE(APE_MLD.ENTITY_NAME, APE.ENTITY_NAME)
ENTITY_NAME, RP.CURRENT_RECORD_STATUS_ID, COALESCE(ACT_MLD.ACTION_NAME, ACT.ACTION_NAME) ACTION_NAME, APE.ENTITY_PATH, APE.ADMIN_TYPE_ID, APE.IS_ADMIN_PAGE,
'RO.CR_NO', APE.ENTITY_CODE
FROM AUTH_APPLICATION_ENTITIES APE
INNER JOIN AUTH_APPLICATIONS APP ON APE.APPLICATION_ID = APP.APPLICATION_ID
LEFT JOIN AUTH_APPLICATION_ENTITIES_MLD APE_MLD ON APE.ENTITY_ID = APE_MLD.ENTITY_ID AND APE_MLD.LANGUAGE_ID = 1
LEFT JOIN AUTH_APPLICATION_ENTITIES PARENT_APE ON APE.PARENT_ENTITY_ID = PARENT_APE.ENTITY_ID
LEFT JOIN AUTH_APPLICATION_ENTITIES_MLD PARENT_APE_MLD ON PARENT_APE.ENTITY_ID = PARENT_APE_MLD.ENTITY_ID AND PARENT_APE_MLD.LANGUAGE_ID = 1
INNER JOIN AUTH_ROLES_PRIVILEGES RP ON APE.ENTITY_ID = RP.ENTITY_ID AND RP.CURRENT_RECORD_STATUS_ID NOT IN (2, 3, 6, 8, 9, 10)
INNER JOIN AUTH_ACTIONS ACT ON RP.ACTION_ID = ACT.ACTION_ID
LEFT JOIN AUTH_ACTIONS_MLD ACT_MLD ON ACT.ACTION_ID = ACT_MLD.ACTION_ID AND ACT_MLD.LANGUAGE_ID = 1
INNER JOIN AUTH_USERS_ROLES RO ON RP.ROLE_ID = RO.ROLE_ID AND RO.CURRENT_RECORD_STATUS_ID NOT IN (2, 3, 6, 8, 9, 10)
WHERE APE.CURRENT_RECORD_STATUS_ID ! = 2
AND RP.CURRENT_RECORD_STATUS_ID ! = 2
AND (APE.APPLICATION_ID = 715 OR 715 = -99)
AND RO.USER_ID = 1255162
AND (RO.ROLE_ID = -99 OR -99 = -99)
AND (APE.IS_ADMIN_PAGE = 0 OR USER_HAS_ADMIN_PRIVILEGES(APE.APPLICATION_ID, APE.ADMIN_TYPE_ID, 1255162) = 1)
UNION
SELECT DISTINCT APP.APPLICATION_ID, APE.ENTITY_ID ENTITY_ID, APE.PARENT_ENTITY_ID, RP.ROLE_ID, RP.ACTION_ID, COALESCE(APE_MLD.ENTITY_NAME, APE.ENTITY_NAME)
ENTITY_NAME, RP.CURRENT_RECORD_STATUS_ID, COALESCE(ACT_MLD.ACTION_NAME, ACT.ACTION_NAME) ACTION_NAME, APE.ENTITY_PATH, APE.ADMIN_TYPE_ID, APE.IS_ADMIN_PAGE,
'CR_NO' CR_NO, APE.ENTITY_CODE
FROM AUTH_APPLICATION_ENTITIES APE
INNER JOIN AUTH_APPLICATIONS APP ON APE.APPLICATION_ID = APP.APPLICATION_ID
LEFT JOIN AUTH_APPLICATION_ENTITIES_MLD APE_MLD ON APE.ENTITY_ID = APE_MLD.ENTITY_ID AND APE_MLD.LANGUAGE_ID = 1
LEFT JOIN AUTH_APPLICATION_ENTITIES PARENT_APE ON APE.PARENT_ENTITY_ID = PARENT_APE.ENTITY_ID
LEFT JOIN AUTH_APPLICATION_ENTITIES_MLD PARENT_APE_MLD ON PARENT_APE.ENTITY_ID = PARENT_APE_MLD.ENTITY_ID AND PARENT_APE_MLD.LANGUAGE_ID = 1
INNER JOIN AUTH_ROLES_PRIVILEGES RP ON APE.ENTITY_ID = RP.ENTITY_ID AND RP.CURRENT_RECORD_STATUS_ID NOT IN (2, 3, 6, 8, 9, 10)
INNER JOIN AUTH_ACTIONS ACT ON RP.ACTION_ID = ACT.ACTION_ID
LEFT JOIN AUTH_ACTIONS_MLD ACT_MLD ON ACT.ACTION_ID = ACT_MLD.ACTION_ID AND ACT_MLD.LANGUAGE_ID = 1
INNER JOIN AUTH_ROLE_TYPE_ROLES RTO ON RP.ROLE_ID = RTO.ROLE_ID AND RTO.CURRENT_RECORD_STATUS_ID NOT IN (2, 3, 6, 8, 9, 10)
WHERE APE.CURRENT_RECORD_STATUS_ID ! = 2
AND RP.CURRENT_RECORD_STATUS_ID ! = 2
AND (APE.APPLICATION_ID = 715 OR 715 = -99)
AND RTO.ROLE_TYPE_ID IN (SELECT ROLE_TYPE_ID FROM AUTH_USER_TYPES WHERE USER_ID = 1255162)
AND (RTO.ROLE_ID = -99 OR -99 = -99)
AND (APE.IS_ADMIN_PAGE = 0 OR USER_HAS_ADMIN_PRIVILEGES(APE.APPLICATION_ID, APE.ADMIN_TYPE_ID, 1255162) = 1);
I have generated the explain plan and it provide this result:
----------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 40 | 11232 | 1485 (63)| 00:00:18 |
| 1 | SORT UNIQUE | | 40 | 11232 | 1485 (63)| 00:00:18 |
| 2 | UNION-ALL | | | | | |
|* 3 | HASH JOIN OUTER | | 26 | 7228 | 631 (13)| 00:00:08 |
| 4 | NESTED LOOPS OUTER | | 22 | 5016 | 625 (13)| 00:00:08 |
| 5 | NESTED LOOPS OUTER | | 22 | 4862 | 625 (13)| 00:00:08 |
| 6 | NESTED LOOPS | | 22 | 4774 | 625 (13)| 00:00:08 |
|* 7 | HASH JOIN OUTER | | 22 | 4686 | 625 (13)| 00:00:08 |
|* 8 | HASH JOIN | | 22 | 4246 | 622 (13)| 00:00:08 |
|* 9 | HASH JOIN | | 22 | 3872 | 620 (13)| 00:00:08 |
|* 10 | TABLE ACCESS BY INDEX ROWID| AUTH_APPLICATION_ENTITIES | 40 | 5920 | 7 (0)| 00:00:01 |
|* 11 | INDEX RANGE SCAN | AUTH_AE_IDX_APP | 49 | | 1 (0)| 00:00:01 |
| 12 | NESTED LOOPS | | 597 | 16716 | 612 (13)| 00:00:08 |
|* 13 | TABLE ACCESS FULL | AUTH_USERS_ROLES | 5 | 65 | 607 (13)| 00:00:08 |
|* 14 | INDEX RANGE SCAN | AUTH_ROLES_PRIVILEGES_UK1 | 131 | 1965 | 1 (0)| 00:00:01 |
| 15 | TABLE ACCESS FULL | AUTH_ACTIONS | 11 | 187 | 2 (0)| 00:00:01 |
|* 16 | TABLE ACCESS FULL | AUTH_ACTIONS_MLD | 9 | 180 | 2 (0)| 00:00:01 |
|* 17 | INDEX UNIQUE SCAN | PK_SYS_MODULES | 1 | 4 | 0 (0)| 00:00:01 |
|* 18 | INDEX UNIQUE SCAN | PK_SYS_ENTITIES | 1 | 4 | 0 (0)| 00:00:01 |
|* 19 | INDEX UNIQUE SCAN | AUTH_APPLICATION_ENTITIES_UK1 | 1 | 7 | 0 (0)| 00:00:01 |
|* 20 | TABLE ACCESS FULL | AUTH_APPLICATION_ENTITIES_MLD | 1262 | 63100 | 5 (0)| 00:00:01 |
|* 21 | HASH JOIN OUTER | | 14 | 4004 | 852 (17)| 00:00:11 |
| 22 | NESTED LOOPS OUTER | | 12 | 2832 | 847 (17)| 00:00:11 |
| 23 | NESTED LOOPS OUTER | | 12 | 2748 | 847 (17)| 00:00:11 |
| 24 | NESTED LOOPS | | 12 | 2700 | 847 (17)| 00:00:11 |
|* 25 | HASH JOIN OUTER | | 12 | 2652 | 847 (17)| 00:00:11 |
|* 26 | HASH JOIN | | 12 | 2412 | 844 (17)| 00:00:11 |
|* 27 | HASH JOIN | | 12 | 2208 | 842 (17)| 00:00:11 |
|* 28 | TABLE ACCESS BY INDEX ROWID| AUTH_APPLICATION_ENTITIES | 40 | 5920 | 7 (0)| 00:00:01 |
|* 29 | INDEX RANGE SCAN | AUTH_AE_IDX_APP | 49 | | 1 (0)| 00:00:01 |
| 30 | NESTED LOOPS | | 308 | 11088 | 834 (17)| 00:00:11 |
|* 31 | HASH JOIN | | 3 | 63 | 831 (17)| 00:00:10 |
|* 32 | TABLE ACCESS FULL | AUTH_USER_TYPES | 4 | 40 | 829 (17)| 00:00:10 |
|* 33 | TABLE ACCESS FULL | AUTH_ROLE_TYPE_ROLES | 9 | 99 | 2 (0)| 00:00:01 |
|* 34 | INDEX RANGE SCAN | AUTH_ROLES_PRIVILEGES_UK1 | 89 | 1335 | 1 (0)| 00:00:01 |
| 35 | TABLE ACCESS FULL | AUTH_ACTIONS | 11 | 187 | 2 (0)| 00:00:01 |
|* 36 | TABLE ACCESS FULL | AUTH_ACTIONS_MLD | 9 | 180 | 2 (0)| 00:00:01 |
|* 37 | INDEX UNIQUE SCAN | PK_SYS_MODULES | 1 | 4 | 0 (0)| 00:00:01 |
|* 38 | INDEX UNIQUE SCAN | PK_SYS_ENTITIES | 1 | 4 | 0 (0)| 00:00:01 |
|* 39 | INDEX UNIQUE SCAN | AUTH_APPLICATION_ENTITIES_UK1 | 1 | 7 | 0 (0)| 00:00:01 |
|* 40 | TABLE ACCESS FULL | AUTH_APPLICATION_ENTITIES_MLD | 1262 | 63100 | 5 (0)| 00:00:01 |
----------------------------------------------------------------------------------------------------------------------
I need to optimize the above query by reading the explain plan result. I just need to know how I can find JOIN orders and slow queries etc. Can anyoe please help me on this?
With that is difficult to help, we don't know where your sql lost time.
run the query with the hint /*+ GATHER_PLAN_STATISTICS */ or alter your session with
alter session set statistics_level='ALL', and then put the result of the plan here with:
SELECT *
FROM table(DBMS_XPLAN.DISPLAY_CURSOR(FORMAT=>'ALLSTATS LAST'));
Or if you want is easy to read, use DBMS_SQL_MONITOR to check query
I have an oracle table DM_TEMP_SUMMING_DVC_BY_FW with below columns and sample data. Data below shows
-dmc_id ='408' there are 2109 devices which are having firmware_version='RT1' and it is the first firmware_version because image_prerequisite is null for this firmware_image.
-dmc_id ='408' there are 40 devices which are having firmware_version='RT2' moved from "RT1" and so on.
The requirement is to create the chain of firmware_versions and the count of devices moving through the chain. For example:
Firmware_version movement is RT1-->RT2-->RT3-->RT4
so the count of devices moved from RT1 to RT4 is (RT1+RT2+RT3+RT4)
count of devices moved from RT2 to RT4 is (RT2+RT3+RT4)
count of devices moved from RT3 to RT4 is (RT3+RT4)
count of devices at last firmware_version is RT4
Below is the input data in table DM_TEMP_SUMMING_DVC_BY_FW
+-----------+-------+-----------------+-----------------+------------------+-------------------+
device_count| dmc_id| firmware_version| cg_id |image_prerequisite| count_within_dmcid|
+-----------+-------+-----------------+-----------------+------------------+-------------------+
| 40 | 408 |RT2 |0 |RT1 | 24028 |
| 24 | 408 |RT3 |0 |RT2 | 24028 |
| 18 | 408 |RT4 |0 |RT3 | 24028 |
| 2109 | 408 |RT1 |0 |null | 24028 |
| 1 | 1422 |RT1 |0 |null | 7 |
| 1 | 1422 |RT2 |0 |RT1 | 7 |
| 1 | 408 |RT1 |HFOTA-0000000041 |null | 1 |
| 1 | 408 |RT1 |HFOTA-0000000334 |null | 2 |
| 1 | 408 |RT1 |HFOTA-0000000359 |null | 1 |
| 1 | 408 |RT1 |HFOTA-0000000441 |null | 1 |
| 1 | 408 |RT1 |HFOTA-0000001885 |null | 2 |
| 4 | 408 |SVP01 |0 |null | 24028 |
| 11 | 408 |Sanity01 |0 |null | 24028 |
| 1 | 408 |Sanity1 |0 |null | 24028 |
| 6 | 408 |TB_HT01 |0 |null | 24028 |
| 1 | 408 |TEST_1 |0 |null | 24028 |
| 5 | 408 |TK_ST001 |0 |null | 24028 |
+-----------+-------+-----------------+-----------------+------------------+-------------------+
Query that I have executed:
SELECT
chain,
firmware_version as fw,
device_count as cnt,
dmc_id,
cg_id,
count_within_dmcid,
sum( device_count ) over ( partition by dmc_id, cg_id order by lev desc ) as cumm
FROM (
select t.* , level lev,
sys_connect_by_path( firmware_version, '/' ) as chain
from DM_TEMP_SUMMING_DVC_BY_FW t
START WITH image_prerequisite IS null
CONNECT BY nocycle PRIOR firmware_version=image_prerequisite
AND PRIOR dmc_id = dmc_id
AND PRIOR cg_id = cg_id
) x
order by dmc_id, cg_id, lev desc
;
Results Query is returning:
+-----------------+---------+-----+------+----------------+------------------+----+
chain |fw |cnt |dmc_id| cg_id |count_within_dmcid|cumm|
+-----------------+---------+-----+------+----------------+------------------+----+
|/RT1/RT2/RT3/RT4 | RT4 | 18| 408 | 0 | 24028 | 18|
|/RT1/RT2/RT3 | RT3 | 24| 408 | 0 | 24028 | 42|
|/RT1/RT2 | RT2 | 40| 408 | 0 | 24028 | 82|
|/Sanity1 |Sanity1 | 1 | 408 | 0 | 24028 |2219|
|/TB_HT01 |TB_HT01 | 6 | 408 | 0 | 24028 |2219|
|/Sanity01 |Sanity01 | 11| 408 | 0 | 24028 |2219|
|/SVP01 |SVP01 | 4 | 408 | 0 | 24028 |2219|
|/RT1 | RT1 |2109 | 408 | 0 | 24028 |2219|
|/TEST_1 |TEST_1 | 1 | 408 | 0 | 24028 |2219|
|/TK_ST001 |TK_ST001 | 5 | 408 | 0 | 24028 |2219|
|/RT1 |RT1 | 1 | 408 |HFOTA-0000000041| 1 | 1 |
|/RT1 |RT1 |1 | 408 |HFOTA-0000000334| 2 | 1 |
|/RT1 |RT1 |1 | 408 |HFOTA-0000000359| 1 | 1 |
|/RT1 |RT1 |1 | 408 |HFOTA-0000000441| 1 | 1 |
|/RT1 |RT1 |1 | 408 |HFOTA-0000001885| 2 | 1 |
|/RT1/RT2 |RT2 |1 |1422 | 0 | 7 | 1 |
|/RT1 |RT1 |1 |1422 | 0 | 7 | 2 |
+-----------------+---------+-----+------+----------------+------------------+----+
Expected Results:
+-----------------+---------+-----+------+----------------+------------------+----+
chain |fw |cnt |dmc_id| cg_id |count_within_dmcid|cumm|
+-----------------+---------+-----+------+----------------+------------------+----+
|/RT1/RT2/RT3/RT4 | RT4 | 18| 408 | 0 | 24028 | 18|
|/RT1/RT2/RT3 | RT3 | 24| 408 | 0 | 24028 | 42|
|/RT1/RT2 | RT2 | 40| 408 | 0 | 24028 | 82|
|/Sanity1 |Sanity1 | 1 | 408 | 0 | 24028 | 1 |
|/TB_HT01 |TB_HT01 | 6 | 408 | 0 | 24028 | 6 |
|/Sanity01 |Sanity01 | 11| 408 | 0 | 24028 | 11 |
|/SVP01 |SVP01 | 4 | 408 | 0 | 24028 | 4 |
|/RT1 | RT1 |2109 | 408 | 0 | 24028 |2191|
|/TEST_1 |TEST_1 | 1 | 408 | 0 | 24028 | 1 |
|/TK_ST001 |TK_ST001 | 5 | 408 | 0 | 24028 | 5 |
|/RT1 |RT1 | 1 | 408 |HFOTA-0000000041| 1 | 1 |
|/RT1 |RT1 |1 | 408 |HFOTA-0000000334| 2 | 1 |
|/RT1 |RT1 |1 | 408 |HFOTA-0000000359| 1 | 1 |
|/RT1 |RT1 |1 | 408 |HFOTA-0000000441| 1 | 1 |
|/RT1 |RT1 |1 | 408 |HFOTA-0000001885| 2 | 1 |
|/RT1/RT2 |RT2 |1 |1422 | 0 | 7 | 1 |
|/RT1 |RT1 |1 |1422 | 0 | 7 | 2 |
+-----------------+---------+-----+------+----------------+------------------+----+
Please suggest any solution for this problem.
Thanks in advance!
Another scenario with the split chains -
Input data in DM_TEMP_SUMMING_DVC_BY_FW table.
+-----------+-------+-----------------+-----------------+------------------+-------------------+
device_count| dmc_id| firmware_version| cg_id |image_prerequisite| count_within_dmcid|
+-----------+-------+-----------------+-----------------+------------------+-------------------+
| 5 | 3345 |SU.B |0000000000000000 |SU.A | 93 |
| 6 | 3345 |SU.C |0000000000000000 |SU.B | 93 |
| 8 | 3345 |SU.D |0000000000000000 |SU.C | 93 |
| 8 | 3345 |SU.E |0000000000000000 |SU.C | 93 |
| 20 | 3345 |SU.F |0000000000000000 |SU.D | 93 |
| 20 | 3345 |SU.F |0000000000000000 |SU.E | 93 |
| 10 | 3345 |SU.G |0000000000000000 |SU.F | 93 |
| 11 | 3345 |SU.H |0000000000000000 |SU.F | 93 |
| 20 | 3345 |SU.I |0000000000000000 |SU.G | 93 |
| 20 | 3345 |SU.I |0000000000000000 |SU.H | 93 |
| 5 | 3345 |SU.A |0000000000000000 |null | 93 |
| 40 | 408 |RT2 |0000000000000000 |RT1 | 24028 |
| 24 | 408 |RT3 |0000000000000000 |RT2 | 24028 |
| 18 | 408 |RT4 |0000000000000000 |RT3 | 24028 |
| 2109 | 408 |RT1 |0000000000000000 |null | 24028 |
| 1 | 142 |RT1 |0000000000000000 |null | 7 |
| 1 | 142 |RT2 |0000000000000000 |RT1 | 7 |
| 1 | 408 |RT1 |HFOTA-0000000041 |null | 1 |
| 1 | 408 |RT1 |HFOTA-0000000334 |null | 2 |
| 1 | 408 |RT1 |HFOTA-0000000359 |null | 1 |
| 1 | 408 |RT1 |HFOTA-0000000441 |null | 1 |
| 1 | 408 |RT1 |HFOTA-0000001885 |null | 2 |
| 4 | 408 |SVP01 |0000000000000000 |null | 24028 |
| 11 | 408 |Sanity01 |0000000000000000 |null | 24028 |
| 1 | 408 |Sanity1 |0000000000000000 |null | 24028 |
| 6 | 408 |TB_HT01 |0000000000000000 |null | 24028 |
| 1 | 408 |TEST_1 |0000000000000000 |null | 24028 |
| 5 | 408 |TK_ST001 |0000000000000000 |null | 24028 |
+-----------+-------+-----------------+-----------------+------------------+-------------------+
This query is breaking in the path split scenario data.
SU.A (5) -- Level 1
|
SU.B (5) -- Level 2
|
SU.C (6) -- Level 3
/ \
(8) SU.D SU.E (8) -- Level 4
\ /
SU.F (20) -- Level 5
/ \
(10) SU.G SU.H (11) -- Level 6
\ /
SU.I (20) -- Level 7
Query used:
SELECT chain,
root_fw,
firmware_version,
device_count,
dmc_id,
charging_group_id,
count_within_dmcid,
SUM(device_count) over(PARTITION BY dmc_id, cg_id, root_fw ORDER BY lev DESC),
lev
FROM (SELECT t.*,
LEVEL lev,
sys_connect_by_path(firmware_version, '/') AS chain,
connect_by_root(firmware_version) root_fw
FROM dm_temp_summing_dvc_by_fw t
START WITH image_prerequisite IS NULL
CONNECT BY nocycle PRIOR firmware_version = image_prerequisite
AND PRIOR dmc_id = dmc_id
AND PRIOR cg_id = cg_id) x
ORDER BY dmc_id,
cg_id,
lev DESC;
Ouput query is returning:
+-----------------------------------+--------+--------+-------------+------+----------------+------------------+----+---+
chain |root_fw |fw |device_count |dmc_id| cg_id |count_within_dmcid|cumm|lev|
+-----------------------------------+--------+--------+-------------+------+----------------+------------------+----+---+
|/SU.A/SU.B/SU.C/SU.D/SU.F/SU.G/SU.I|SU.A |SU.I | 20 |3345 |0000000000000000|93 |80 |7 |
|/SU.A/SU.B/SU.C/SU.E/SU.F/SU.H/SU.I|SU.A |SU.I | 20 |3345 |0000000000000000|93 |80 |7 |
|/SU.A/SU.B/SU.C/SU.E/SU.F/SU.G/SU.I|SU.A |SU.I | 20 |3345 |0000000000000000|93 |80 |7 |
|/SU.A/SU.B/SU.C/SU.D/SU.F/SU.H/SU.I|SU.A |SU.I | 20 |3345 |0000000000000000|93 |80 |7 |
|/SU.A/SU.B/SU.C/SU.E/SU.F/SU.G |SU.A |SU.G | 10 |3345 |0000000000000000|93 |122 |6 |
|/SU.A/SU.B/SU.C/SU.D/SU.F/SU.G |SU.A |SU.G | 10 |3345 |0000000000000000|93 |122 |6 |
|/SU.A/SU.B/SU.C/SU.D/SU.F/SU.H |SU.A |SU.H | 11 |3345 |0000000000000000|93 |122 |6 |
|/SU.A/SU.B/SU.C/SU.E/SU.F/SU.H |SU.A |SU.H | 11 |3345 |0000000000000000|93 |122 |6 |
|/SU.A/SU.B/SU.C/SU.E/SU.F |SU.A |SU.F | 20 |3345 |0000000000000000|93 |162 |5 |
|/SU.A/SU.B/SU.C/SU.D/SU.F |SU.A |SU.F | 20 |3345 |0000000000000000|93 |162 |5 |
|/SU.A/SU.B/SU.C/SU.E |SU.A |SU.E | 8 |3345 |0000000000000000|93 |178 |4 |
|/SU.A/SU.B/SU.C/SU.D |SU.A |SU.D | 8 |3345 |0000000000000000|93 |178 |4 |
|/SU.A/SU.B/SU.C |SU.A |SU.C | 6 |3345 |0000000000000000|93 |184 |3 |
|/SU.A/SU.B |SU.A |SU.B | 5 |3345 |0000000000000000|93 |189 |2 |
|/SU.A |SU.A |SU.A | 5 |3345 |0000000000000000|93 |194 |1 |
|/RT1/RT2/RT3/RT4 |RT1 |RT4 | 18 |408 |0000000000000000|24028 |18 |4 |
|/RT1/RT2/RT3 |RT1 |RT3 | 24 |408 |0000000000000000|24028 |42 |3 |
|/RT1/RT2 |RT1 |RT2 | 40 |408 |0000000000000000|24028 |82 |2 |
|/Sanity1 |Sanity1 |Sanity1 | 1 |408 |0000000000000000|24028 |1 |1 |
|/TB_HT01 |TB_HT01 |TB_HT01 | 6 |408 |0000000000000000|24028 |6 |1 |
|/Sanity01 |Sanity01|Sanity01| 11 |408 |0000000000000000|24028 |11 |1 |
|/SVP01 |SVP01 |SVP01 | 4 |408 |0000000000000000|24028 |4 |1 |
|/RT1 |RT1 |RT1 | 2109 |408 |0000000000000000|24028 |2191|1 |
|/TEST_1 |TEST_1 |TEST_1 | 1 |408 |0000000000000000|24028 |1 |1 |
|/TK_ST001 |TK_ST001|TK_ST001| 5 |408 |0000000000000000|24028 |5 |1 |
|/RT1 |RT1 |RT1 | 1 |408 |HFOTA-0000000041|1 |1 |1 |
|/RT1 |RT1 |RT1 | 1 |408 |HFOTA-0000000334|2 |1 |1 |
|/RT1 |RT1 |RT1 | 1 |408 |HFOTA-0000000359|1 |1 |1 |
|/RT1 |RT1 |RT1 | 1 |408 |HFOTA-0000000441|1 |1 |1 |
|/RT1 |RT1 |RT1 | 1 |408 |HFOTA-0000001885|2 |1 |1 |
|/RT1/RT2 |RT1 |RT2 | 1 |1422 |0000000000000000|7 |1 |2 |
|/RT1 |RT1 |RT1 | 1 |1422 |0000000000000000|7 |2 |1 |
+-----------------------------------+--------+--------+-------------+------+----------------+------------------+----+---+
Chain and cumulative counts are coming correct for regural chains A - B- C- D, but if we have split chains then it is doing summary of all the chain generated.
SU.A - 93 A+B+C+D+E+F+G+H+I
SU.B - 88 B+C+D+E+F+G+H+I
SU.C - 83 C+D+E+F+G+H+I
SU.D - 0 (Cummulative count will be zero if we have two or more FW at same level)
SU.E - 0 (Cummulative count will be zero if we have two or more FW at same level)
SU.F - 61 F+G+H+I
SU.G - 0 Cummulative count will be zero if we have two or more FW at same level)
SU.H - 0 Cummulative count will be zero if we have two or more FW at same level)
SU.I - 20 I
Expected output in this split scenario:
+---------------------------------------------+--------+--------+-------------+------+----------------+------------------+----+---+
chain |root_fw |fw |device_count |dmc_id| cg_id |count_within_dmcid|cumm|lev|
+---------------------------------------------+--------+--------+-------------+------+----------------+------------------+----+---+
|/SU.A/SU.B/SU.C/SU.D/SU.E/SU.F/SU.G/SU.H/SU.I|SU.A |SU.I | 20 |3345 |0000000000000000|93 |20 |7 |
|/SU.A/SU.B/SU.C/SU.D/SU.E/SU.F/SU.G/SU.H/ |SU.A |SU.H | 11 |3345 |0000000000000000|93 |0 |6 |
|/SU.A/SU.B/SU.C/SU.D/SU.E/SU.F/SU.G |SU.A |SU.G | 10 |3345 |0000000000000000|93 |0 |6 |
|/SU.A/SU.B/SU.C/SU.D/SU.E/SU.F |SU.A |SU.F | 20 |3345 |0000000000000000|93 |61 |5 |
|/SU.A/SU.B/SU.C/SU.D/SU.E |SU.A |SU.E | 8 |3345 |0000000000000000|93 |0 |4 |
|/SU.A/SU.B/SU.C/SU.D |SU.A |SU.D | 8 |3345 |0000000000000000|93 |0 |4 |
|/SU.A/SU.B/SU.C |SU.A |SU.C | 6 |3345 |0000000000000000|93 |83 |3 |
|/SU.A/SU.B |SU.A |SU.B | 5 |3345 |0000000000000000|93 |88 |2 |
|/SU.A |SU.A |SU.A | 5 |3345 |0000000000000000|93 |93 |1 |
|/RT1/RT2/RT3/RT4 |RT1 |RT4 | 18 |408 |0000000000000000|24028 |18 |4 |
|/RT1/RT2/RT3 |RT1 |RT3 | 24 |408 |0000000000000000|24028 |42 |3 |
|/RT1/RT2 |RT1 |RT2 | 40 |408 |0000000000000000|24028 |82 |2 |
|/Sanity1 |Sanity1 |Sanity1 | 1 |408 |0000000000000000|24028 |1 |1 |
|/TB_HT01 |TB_HT01 |TB_HT01 | 6 |408 |0000000000000000|24028 |6 |1 |
|/Sanity01 |Sanity01|Sanity01| 11 |408 |0000000000000000|24028 |11 |1 |
|/SVP01 |SVP01 |SVP01 | 4 |408 |0000000000000000|24028 |4 |1 |
|/RT1 |RT1 |RT1 | 2109 |408 |0000000000000000|24028 |2191|1 |
|/TEST_1 |TEST_1 |TEST_1 | 1 |408 |0000000000000000|24028 |1 |1 |
|/TK_ST001 |TK_ST001|TK_ST001| 5 |408 |0000000000000000|24028 |5 |1 |
|/RT1 |RT1 |RT1 | 1 |408 |HFOTA-0000000041|1 |1 |1 |
|/RT1 |RT1 |RT1 | 1 |408 |HFOTA-0000000334|2 |1 |1 |
|/RT1 |RT1 |RT1 | 1 |408 |HFOTA-0000000359|1 |1 |1 |
|/RT1 |RT1 |RT1 | 1 |408 |HFOTA-0000000441|1 |1 |1 |
|/RT1 |RT1 |RT1 | 1 |408 |HFOTA-0000001885|2 |1 |1 |
|/RT1/RT2 |RT1 |RT2 | 1 |1422 |0000000000000000|7 |1 |2 |
|/RT1 |RT1 |RT1 | 1 |1422 |0000000000000000|7 |2 |1 |
+---------------------------------------------+--------+--------+-------------+------+----------------+------------------+----+---+
It looks like what you're missing is partitioning on the root firmware_version, which you can get by using connect_by_root(), like so:
WITH dm_temp_summing_dvc_by_fw AS (SELECT 40 device_count, 408 dmc_id, 'RT2' firmware_version, '0000' cg_id, 'RT1' image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 24 device_count, 408 dmc_id, 'RT3' firmware_version, '0000' cg_id, 'RT2' image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 18 device_count, 408 dmc_id, 'RT4' firmware_version, '0000' cg_id, 'RT3' image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 2109 device_count, 408 dmc_id, 'RT1' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 1422 dmc_id, 'RT1' firmware_version, '0000' cg_id, NULL image_prerequisite, 7 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 1422 dmc_id, 'RT2' firmware_version, '0000' cg_id, 'RT1' image_prerequisite, 7 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'RT1' firmware_version, 'HFOTA1' cg_id, NULL image_prerequisite, 1 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'RT1' firmware_version, 'HFOTA2' cg_id, NULL image_prerequisite, 2 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'RT1' firmware_version, 'HFOTA3' cg_id, NULL image_prerequisite, 1 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'RT1' firmware_version, 'HFOTA4' cg_id, NULL image_prerequisite, 1 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'RT1' firmware_version, 'HFOTA5' cg_id, NULL image_prerequisite, 2 count_within_dmcid FROM dual UNION ALL
SELECT 4 device_count, 408 dmc_id, 'SVP01' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 11 device_count, 408 dmc_id, 'Sanity01' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'Sanity1' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 6 device_count, 408 dmc_id, 'TB_HT01' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'TEST_1' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 5 device_count, 408 dmc_id, 'TK_ST001' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual)
SELECT chain,
root_fw,
firmware_version AS fw,
device_count AS cnt,
dmc_id,
cg_id,
lev,
count_within_dmcid,
SUM(device_count) over(PARTITION BY dmc_id, cg_id, root_fw ORDER BY lev DESC) AS cumm
FROM (SELECT t.*,
LEVEL lev,
sys_connect_by_path(firmware_version, '/') AS chain,
connect_by_root(firmware_version) root_fw
FROM dm_temp_summing_dvc_by_fw t
START WITH image_prerequisite IS NULL
CONNECT BY nocycle PRIOR firmware_version = image_prerequisite
AND PRIOR dmc_id = dmc_id
AND PRIOR cg_id = cg_id) x
ORDER BY dmc_id,
cg_id,
lev DESC;
Which gives the results:
CHAIN ROOT_FW FW CNT DMC_ID CG_ID LEV COUNT_WITHIN_DMCID CUMM
----------------- -------- -------- ---------- ---------- ------ ---------- ------------------ ----------
/RT1/RT2/RT3/RT4 RT1 RT4 18 408 0000 4 24028 18
/RT1/RT2/RT3 RT1 RT3 24 408 0000 3 24028 42
/RT1/RT2 RT1 RT2 40 408 0000 2 24028 82
/Sanity1 Sanity1 Sanity1 1 408 0000 1 24028 1
/TB_HT01 TB_HT01 TB_HT01 6 408 0000 1 24028 6
/Sanity01 Sanity01 Sanity01 11 408 0000 1 24028 11
/SVP01 SVP01 SVP01 4 408 0000 1 24028 4
/RT1 RT1 RT1 2109 408 0000 1 24028 2191
/TEST_1 TEST_1 TEST_1 1 408 0000 1 24028 1
/TK_ST001 TK_ST001 TK_ST001 5 408 0000 1 24028 5
/RT1 RT1 RT1 1 408 HFOTA1 1 1 1
/RT1 RT1 RT1 1 408 HFOTA2 1 2 1
/RT1 RT1 RT1 1 408 HFOTA3 1 1 1
/RT1 RT1 RT1 1 408 HFOTA4 1 1 1
/RT1 RT1 RT1 1 408 HFOTA5 1 2 1
/RT1/RT2 RT1 RT2 1 1422 0000 2 7 1
/RT1 RT1 RT1 1 1422 0000 1 7 2
ETA:
With the additional requirement to handle cases where firmewares are at the same level as each other, I *think* this does what you're after:
WITH dm_temp_summing_dvc_by_fw AS (SELECT 40 device_count, 408 dmc_id, 'RT2' firmware_version, '0000' cg_id, 'RT1' image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 24 device_count, 408 dmc_id, 'RT3' firmware_version, '0000' cg_id, 'RT2' image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 18 device_count, 408 dmc_id, 'RT4' firmware_version, '0000' cg_id, 'RT3' image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 2109 device_count, 408 dmc_id, 'RT1' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 1422 dmc_id, 'RT1' firmware_version, '0000' cg_id, NULL image_prerequisite, 7 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 1422 dmc_id, 'RT2' firmware_version, '0000' cg_id, 'RT1' image_prerequisite, 7 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'RT1' firmware_version, 'HFOTA1' cg_id, NULL image_prerequisite, 1 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'RT1' firmware_version, 'HFOTA2' cg_id, NULL image_prerequisite, 2 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'RT1' firmware_version, 'HFOTA3' cg_id, NULL image_prerequisite, 1 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'RT1' firmware_version, 'HFOTA4' cg_id, NULL image_prerequisite, 1 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'RT1' firmware_version, 'HFOTA5' cg_id, NULL image_prerequisite, 2 count_within_dmcid FROM dual UNION ALL
SELECT 4 device_count, 408 dmc_id, 'SVP01' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 11 device_count, 408 dmc_id, 'Sanity01' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'Sanity1' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 6 device_count, 408 dmc_id, 'TB_HT01' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 1 device_count, 408 dmc_id, 'TEST_1' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 5 device_count, 408 dmc_id, 'TK_ST001' firmware_version, '0000' cg_id, NULL image_prerequisite, 24028 count_within_dmcid FROM dual UNION ALL
SELECT 5 device_count, 3345 dmc_id, 'SU.B' firmware_version, '0000' cg_id, 'SU.A' image_prerequisite, 93 count_within_dmcid FROM dual UNION ALL
SELECT 6 device_count, 3345 dmc_id, 'SU.C' firmware_version, '0000' cg_id, 'SU.B' image_prerequisite, 93 count_within_dmcid FROM dual UNION ALL
SELECT 8 device_count, 3345 dmc_id, 'SU.D' firmware_version, '0000' cg_id, 'SU.C' image_prerequisite, 93 count_within_dmcid FROM dual UNION ALL
SELECT 8 device_count, 3345 dmc_id, 'SU.E' firmware_version, '0000' cg_id, 'SU.C' image_prerequisite, 93 count_within_dmcid FROM dual UNION ALL
SELECT 20 device_count, 3345 dmc_id, 'SU.F' firmware_version, '0000' cg_id, 'SU.D' image_prerequisite, 93 count_within_dmcid FROM dual UNION ALL
SELECT 20 device_count, 3345 dmc_id, 'SU.F' firmware_version, '0000' cg_id, 'SU.E' image_prerequisite, 93 count_within_dmcid FROM dual UNION ALL
SELECT 10 device_count, 3345 dmc_id, 'SU.G' firmware_version, '0000' cg_id, 'SU.F' image_prerequisite, 93 count_within_dmcid FROM dual UNION ALL
SELECT 11 device_count, 3345 dmc_id, 'SU.H' firmware_version, '0000' cg_id, 'SU.F' image_prerequisite, 93 count_within_dmcid FROM dual UNION ALL
SELECT 20 device_count, 3345 dmc_id, 'SU.I' firmware_version, '0000' cg_id, 'SU.G' image_prerequisite, 93 count_within_dmcid FROM dual UNION ALL
SELECT 20 device_count, 3345 dmc_id, 'SU.I' firmware_version, '0000' cg_id, 'SU.H' image_prerequisite, 93 count_within_dmcid FROM dual UNION ALL
SELECT 5 device_count, 3345 dmc_id, 'SU.A' firmware_version, '0000' cg_id, NULL image_prerequisite, 93 count_within_dmcid FROM dual),
t1 AS (SELECT device_count,
dmc_id,
CASE WHEN COUNT(image_prerequisite) OVER (PARTITION BY dmc_id, cg_id, image_prerequisite) > 1
THEN listagg(firmware_version, '~') WITHIN GROUP (ORDER BY firmware_version) OVER (PARTITION BY dmc_id, cg_id, image_prerequisite)
ELSE firmware_version
END firmware_version,
cg_id,
image_prerequisite,
count_within_dmcid
FROM dm_temp_summing_dvc_by_fw),
t2 AS (SELECT sum(device_count) device_count,
dmc_id,
firmware_version,
cg_id,
image_prerequisite,
count_within_dmcid
FROM t1
GROUP BY dmc_id,
firmware_version,
cg_id,
image_prerequisite,
count_within_dmcid),
t3 AS (SELECT t.*,
LEVEL lev,
sys_connect_by_path(firmware_version, '/') AS chain,
connect_by_root(firmware_version) root_fw,
row_number() OVER (PARTITION BY dmc_id, cg_id, firmware_version, connect_by_root(firmware_version) ORDER BY LEVEL DESC, sys_connect_by_path(firmware_version, '/')) rn
FROM t2 t
START WITH image_prerequisite IS NULL
CONNECT BY nocycle PRIOR regexp_substr(firmware_version, '[^~]*') = image_prerequisite
AND PRIOR dmc_id = dmc_id
AND PRIOR cg_id = cg_id)
SELECT chain,
root_fw,
firmware_version AS fw,
device_count AS cnt,
dmc_id,
cg_id,
lev,
count_within_dmcid,
rn,
SUM(device_count) over(PARTITION BY dmc_id, cg_id, root_fw ORDER BY lev DESC) AS cumm
FROM t3
ORDER BY dmc_id,
cg_id,
lev DESC;
which gives us:
CHAIN ROOT_FW FW CNT DMC_ID CG_ID LEV COUNT_WITHIN_DMCID RN CUMM
--------------------------------------------- ---------- --------------- ---------- ---------- ------ ---------- ------------------ ---------- ----------
/RT1/RT2/RT3/RT4 RT1 RT4 18 408 0000 4 24028 1 18
/RT1/RT2/RT3 RT1 RT3 24 408 0000 3 24028 1 42
/RT1/RT2 RT1 RT2 40 408 0000 2 24028 1 82
/SVP01 SVP01 SVP01 4 408 0000 1 24028 1 4
/RT1 RT1 RT1 2109 408 0000 1 24028 1 2191
/Sanity01 Sanity01 Sanity01 11 408 0000 1 24028 1 11
/Sanity1 Sanity1 Sanity1 1 408 0000 1 24028 1 1
/TB_HT01 TB_HT01 TB_HT01 6 408 0000 1 24028 1 6
/TEST_1 TEST_1 TEST_1 1 408 0000 1 24028 1 1
/TK_ST001 TK_ST001 TK_ST001 5 408 0000 1 24028 1 5
/RT1 RT1 RT1 1 408 HFOTA1 1 1 1 1
/RT1 RT1 RT1 1 408 HFOTA2 1 2 1 1
/RT1 RT1 RT1 1 408 HFOTA3 1 1 1 1
/RT1 RT1 RT1 1 408 HFOTA4 1 1 1 1
/RT1 RT1 RT1 1 408 HFOTA5 1 2 1 1
/RT1/RT2 RT1 RT2 1 1422 0000 2 7 1 1
/RT1 RT1 RT1 1 1422 0000 1 7 1 2
/SU.A/SU.B/SU.C/SU.D~SU.E/SU.F/SU.G~SU.H/SU.I SU.A SU.I 20 3345 0000 7 93 1 20
/SU.A/SU.B/SU.C/SU.D~SU.E/SU.F/SU.G~SU.H SU.A SU.G~SU.H 21 3345 0000 6 93 1 41
/SU.A/SU.B/SU.C/SU.D~SU.E/SU.F SU.A SU.F 20 3345 0000 5 93 1 61
/SU.A/SU.B/SU.C/SU.D~SU.E SU.A SU.D~SU.E 16 3345 0000 4 93 1 77
/SU.A/SU.B/SU.C SU.A SU.C 6 3345 0000 3 93 1 83
/SU.A/SU.B SU.A SU.B 5 3345 0000 2 93 1 88
/SU.A SU.A SU.A 5 3345 0000 1 93 1 93
What this latest query does is first find out if there are firmwares that are at the same level as each other, and if so, we listagg them together (here, I've used ~ as a separator for clarity in the results - there's nothing to stop you from using the same separator as you use in the sys_connect_by_path).
Once we have that, we can do the connect by, but we need to look at just the first entry in the newly calculated firmware column (otherwise we won't find a match in the image_prerequisite column). Then it's just calculating the results in the same way the previous query did.
I have big tables with identical function based indexes. When one table is queried then query plan uses the index. When I join second table with union all then I have full scans.
I have found one working solution (WORKING JOIN WITH TABLE), but it has some limitations:
I cannot use ROWNUM not CONNECT BY in the x query merged to union
I cannot use pipeline function in a query.
How to overcome the mentioned limitations?
I tried hints like CARDINALITY to tell that the number of record is small, and some rewrite (REWRITE, PUSH_PRED) without success.
I cannot use partitioning since the problem is on Oracle SE.
The problem shown here is a simplification of my actual problem which is:
I have huge data set divided into identical tables
Each table contains data from a different month
I have a view which union all the underlying tables
It is kind of partitioning style practiced on oracle before v8.0
In real I may have adhoc queries and joins with many different tables. Thus I cannot simply push the join into a union which will be the simplest solution.
Here is the DDL script.
-- FULL SCAN FOR IN SUBQUERY
SELECT * FROM (
SELECT * FROM TEST_EXPV1
UNION ALL SELECT * FROM TEST_EXPV2
)
WHERE DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value) IN (SELECT Id FROM test_10r)
-----------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 200K| 5078K| 916 (3)| 00:00:11 |
|* 1 | HASH JOIN | | 200K| 5078K| 916 (3)| 00:00:11 |
| 2 | VIEW | VW_NSO_1 | 10 | 130 | 4 (25)| 00:00:01 |
| 3 | HASH UNIQUE | | 10 | 30 | 4 (25)| 00:00:01 |
| 4 | TABLE ACCESS FULL| TEST_10R | 10 | 30 | 3 (0)| 00:00:01 |
| 5 | VIEW | | 2000K| 24M| 902 (2)| 00:00:11 |
| 6 | UNION-ALL | | | | | |
| 7 | TABLE ACCESS FULL| TEST_EXPV1 | 1000K| 3906K| 451 (2)| 00:00:06 |
| 8 | TABLE ACCESS FULL| TEST_EXPV2 | 1000K| 3906K| 451 (2)| 00:00:06 |
-----------------------------------------------------------------------------------
-- CORRECT RANGE INDEX SCAN for bound value
SELECT * FROM (
SELECT * FROM TEST_EXPV1
UNION ALL SELECT * FROM TEST_EXPV2
)
WHERE DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value) = :b1001
----------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 20000 | 253K| 979 (10)| 00:00:12 |
| 1 | VIEW | | 20000 | 253K| 979 (10)| 00:00:12 |
| 2 | UNION-ALL | | | | | |
| 3 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV1 | 2 | 8 | 3 (0)| 00:00:01 |
|* 4 | INDEX RANGE SCAN | I_TEST_EXPV1 | 2 | | 1 (0)| 00:00:01 |
| 5 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV2 | 2 | 8 | 3 (0)| 00:00:01 |
|* 6 | INDEX RANGE SCAN | I_TEST_EXPV2 | 2 | | 1 (0)| 00:00:01 |
----------------------------------------------------------------------------------------------
-- WORKING JOIN WITH TABLE
WITH x AS (SELECT Id FROM test_10r WHERE Id BETWEEN :a AND :b)
SELECT /*+ FIRST_ROWS */ * FROM (
SELECT * FROM TEST_EXPV1
UNION ALL SELECT * FROM TEST_EXPV2
) U, x
WHERE DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value) = x.id
-------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 16 | 996 (11)| 00:00:12 |
|* 1 | FILTER | | | | | |
|* 2 | HASH JOIN | | 500 | 8000 | 996 (11)| 00:00:12 |
|* 3 | TABLE ACCESS FULL | TEST_10R | 10 | 30 | 3 (0)| 00:00:01 |
| 4 | VIEW | | 5000 | 65000 | 993 (11)| 00:00:12 |
| 5 | UNION-ALL | | | | | |
|* 6 | FILTER | | | | | |
| 7 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV1 | 2500 | 10000 | 4192 (1)| 00:00:51 |
|* 8 | INDEX RANGE SCAN | I_TEST_EXPV1 | 4500 | | 11 (0)| 00:00:01 |
|* 9 | FILTER | | | | | |
| 10 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV2 | 2500 | 10000 | 4192 (1)| 00:00:51 |
|* 11 | INDEX RANGE SCAN | I_TEST_EXPV2 | 4500 | | 11 (0)| 00:00:01 |
-------------------------------------------------------------------------------------------------
You're dong the UNION ALL, which is doing a full scan of both tables to get combined result set; and only then filtering on the values from the third table.
A more usual pattern for subquery factoring is to refer to the CTE in each branch of the union:
WITH x AS (SELECT Id FROM test_10r)
SELECT TEST_EXPV1.* FROM x JOIN TEST_EXPV1
ON DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value) = x.Id
UNION ALL SELECT TEST_EXPV2.* FROM x JOIN TEST_EXPV2
ON DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value) = x.Id;
----------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 32 | 224 | 66 (0)| 00:00:01 |
| 1 | UNION-ALL | | | | | |
| 2 | NESTED LOOPS | | 16 | 112 | 33 (0)| 00:00:01 |
| 3 | TABLE ACCESS FULL | TEST_10R | 10 | 30 | 3 (0)| 00:00:01 |
| 4 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV1 | 2 | 8 | 3 (0)| 00:00:01 |
|* 5 | INDEX RANGE SCAN | I_TEST_EXPV1 | 2 | | 1 (0)| 00:00:01 |
| 6 | NESTED LOOPS | | 16 | 112 | 33 (0)| 00:00:01 |
| 7 | TABLE ACCESS FULL | TEST_10R | 10 | 30 | 3 (0)| 00:00:01 |
| 8 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV2 | 2 | 8 | 3 (0)| 00:00:01 |
|* 9 | INDEX RANGE SCAN | I_TEST_EXPV2 | 2 | | 1 (0)| 00:00:01 |
----------------------------------------------------------------------------------------------
Pushing predicates into a UNION ALL can be temperamental. Try this:
SELECT /*+ PUSH_PRED(v) */ *
FROM (SELECT * FROM test_expv1
UNION ALL
SELECT * FROM test_expv2) v INNER JOIN test_10r ON
(DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS NUMBER(38)), Value)) = test_10r.id;
Here are my results for the above query from an 11.2.0.4 instance, using the OP's DDL:
SQL_ID df6dvkgjwjsq1, child number 1
-------------------------------------
SELECT /*+ PUSH_PRED(v) */ * FROM (SELECT * FROM test_expv1
UNION ALL SELECT * FROM test_expv2) v INNER JOIN test_10r ON
(DECODE(Value, -1, CAST(NULL AS NUMBER(38)), 0, CAST(NULL AS
NUMBER(38)), Value)) = test_10r.id
Plan hash value: 191389749
---------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers |
---------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 16 |00:00:00.01 | 69 |
| 1 | NESTED LOOPS | | 1 | 2000K| 16 |00:00:00.01 | 69 |
| 2 | TABLE ACCESS FULL | TEST_10R | 1 | 10 | 10 |00:00:00.01 | 22 |
| 3 | VIEW | | 10 | 32 | 16 |00:00:00.01 | 47 |
| 4 | UNION ALL PUSHED PREDICATE | | 10 | | 16 |00:00:00.01 | 47 |
| 5 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV1 | 10 | 158 | 8 |00:00:00.01 | 24 |
|* 6 | INDEX RANGE SCAN | I_TEST_EXPV1 | 10 | 2 | 8 |00:00:00.01 | 16 |
| 7 | TABLE ACCESS BY INDEX ROWID| TEST_EXPV2 | 10 | 158 | 8 |00:00:00.01 | 23 |
|* 8 | INDEX RANGE SCAN | I_TEST_EXPV2 | 10 | 2 | 8 |00:00:00.01 | 15 |
---------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
6 - access("TEST_EXPV1"."SYS_NC00002$"="TEST_10R"."ID")
8 - access("TEST_EXPV2"."SYS_NC00002$"="TEST_10R"."ID")