Oracle subquery performance - oracle

so I have this huge table SS(someID, someDate, ...). I need to join a subset of this table to the other table. The subset is determined by: select * from SS where someID in (select someID from SS where someDate is between date1 and date2).
When running this on Oracle XA data server in parallel, the execution takes a long time and TEMP space, even though Oracle can cell offloading efficiency of 99% on the SS table, but the subset query still bring back a large amount of data to the database server in joining with other table.
Is there anyway to make this more efficient? such as Oracle doesn't have to send back as much data and utilize more of the cell offloading efficiency?
Below is the query plan
PLAN_TABLE_OUTPUT
Plan hash value: 3198983388
---------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | TQ |IN-OUT| PQ Distrib |
---------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1044K| 589M| 46101 (1)| 00:01:33 | | | |
| 1 | PX COORDINATOR | | | | | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10003 | 1044K| 589M| 46101 (1)| 00:01:33 | Q1,03 | P->S | QC (RAND) |
|* 3 | HASH JOIN BUFFERED | | 1044K| 589M| 46101 (1)| 00:01:33 | Q1,03 | PCWP | |
| 4 | PX RECEIVE | | | | | | Q1,03 | PCWP | |
| 5 | PX SEND HASH | :TQ10001 | | | | | Q1,01 | P->P | HASH |
| 6 | NESTED LOOPS | | | | | | Q1,01 | PCWP | |
| 7 | NESTED LOOPS | | 523K| 135M| 38264 (1)| 00:01:17 | Q1,01 | PCWP | |
| 8 | SORT UNIQUE | | 29402 | 401K| 13751 (1)| 00:00:28 | Q1,01 | PCWP | |
| 9 | PX RECEIVE | | 29402 | 401K| 13751 (1)| 00:00:28 | Q1,01 | PCWP | |
| 10 | PX SEND HASH | :TQ10000 | 29402 | 401K| 13751 (1)| 00:00:28 | Q1,00 | P->P | HASH |
| 11 | PX BLOCK ITERATOR | | 29402 | 401K| 13751 (1)| 00:00:28 | Q1,00 | PCWC | |
|* 12 | INDEX STORAGE FAST FULL SCAN| SUPERSET_IDX1 | 29402 | 401K| 13751 (1)| 00:00:28 | Q1,00 | PCWP | |
|* 13 | INDEX RANGE SCAN | XU_SUPERSET_01 | 18 | | 1 (0)| 00:00:01 | Q1,01 | PCWP | |
| 14 | TABLE ACCESS BY INDEX ROWID | SUPERSET | 18 | 4644 | 2 (0)| 00:00:01 | Q1,01 | PCWP | |
| 15 | PX RECEIVE | | 2886K| 880M| 7834 (2)| 00:00:16 | Q1,03 | PCWP | |
| 16 | PX SEND HASH | :TQ10002 | 2886K| 880M| 7834 (2)| 00:00:16 | Q1,02 | P->P | HASH |
| 17 | PX BLOCK ITERATOR | | 2886K| 880M| 7834 (2)| 00:00:16 | Q1,02 | PCWC | |
| 18 | TABLE ACCESS STORAGE FULL | POL_DTL | 2886K| 880M| 7834 (2)| 00:00:16 | Q1,02 | PCWP | |
---------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access(SS.POL_ID=PD.POL_ID)
12 - storage(IMPT_DT<=TO_DATE(' 2014-11-20 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND IMPT_DT>=TO_DATE(' 2014-10-28
00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
filter(IMPT_DT<=TO_DATE(' 2014-11-20 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND IMPT_DT>=TO_DATE(' 2014-10-28
00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
13 - access(SS.POL_ID=POL_ID)
Note
-----
- Degree of Parallelism is 4 because of session

There may not be much you can do to improve this query. The execution plan looks pretty good:
Good objects The indexes seem to fit the query well, although it's hard to tell without the full definitions.
Good cardinality The estimated rows and actual rows are close. This strongly implies the optimizer is doing a good job and is picking a near-optimal plan. If it can estimate the number of rows correctly it will make wise decisions about the access paths, join methods, join order, etc. Even the time estimate is close, which is rare. It looks like there are good table and system statistics.
Cell offloading The storage predicates and the active report Cell offloading imply that cell offloading is working as expected, at least once.
Parallelism Large objects are already being processed in parallel. I don't see any obvious parallel problems.
Here are some ideas for improvement but don't expect drastic improvements:
Full table scan Force a full table scan instead of an index range scan with a hint like --+ no_index(superset XU_SUPERSET_01). With multiblock reads (use for full scans) and cell offloading (used for a direct path read for a full scan not used for an index range scan that uses the buffer cache), a full table scan reading all the data may be more efficient than an index range scan reading less data.
Covering index If the full table scan doesn't work, create a skinny version of the table with an index that includes all returned and queried columns. This gets the benefits of full scans (multiblock IO, cell offloading) but is smaller than the full table.
Larger DOP There's no magic number for the degree of parallelism (DOP). But in my experience the DOP sweet-spot is almost always larger than 4. This may improve performance but will use more resources.
Rewrite query? Re-writing the query may enable smart scan to process the join in the storage cells. Try changing
select * from SS where someID in
(select someID from SS where someDate is between date1 and date2)
to
select distinct ss1.*
from ss ss1
join ss ss2
on ss1.someID = ss2.someID
and ss2.someDate is between date1 and date2
This new version does extra work. The join returns more rows than necessary, and then they need to be made distinct. That extra work may be worth it if it means the join can happen in the storage cells. I can't find a great source for exactly what kind of processing can be offloaded, but at least some types of joins can be.

Related

Oracle in-memory column store is not improving SELECT query?

I have a SELECT statement
SELECT MIN(C_PRICE), MAX(C_PRICE)
FROM CAR;
I run the statement and create a processing plan to look at the cost.
-------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 3 | 12150 (1)| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 3 | | |
| 2 | TABLE ACCESS FULL| CAR | 1800K| 5273K| 12150 (1)| 00:00:01 |
-------------------------------------------------------------------------------
I created an inmemory to this table car after setting the inmemory size to be 200M.
ALTER TABLE CAR INMEMORY;
The result
----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 3 | 12150 (1)| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 3 | | |
| 2 | TABLE ACCESS INMEMORY FULL| CAR | 1800K| 5273K| 12150 (1)| 00:00:01 |
----------------------------------------------------------------------------------------
My question is why isn't the query improved after altering the table to be in the inmemory? The SELECT statement clearly shows that it is accessing the table via inmemory. I thought inmemory creation will improve the query processing thus reducing the cost?
Two things you should look at. One, is the table populated in the IM column store (i.e. v$im_segments)? Second, what was the time difference between the two queries and where was the time spent? SQL Monitor active reports are excellent for determining this.

what is significance of tempspc in oracle explain plan and the measure of this is M=MB and K=KB?

what is significance of tempspc in oracle explain plan and the measure of this is M=MB and K=KB?
is there any way we can reduce this tempspc utilization in oracle query
Please find sample plan
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
-------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 11966 | 3552K| | 623 (1)| 00:00:01 |
|* 1 | VIEW | | 11966 | 3552K| | 623 (1)| 00:00:01 |
|* 2 | WINDOW SORT PUSHED RANK | | 11966 | 1332K| 1968K| 623 (1)| 00:00:01 |
| 3 | TABLE ACCESS BY INDEX ROWID BATCHED| BAl_ACTIVITY | 11966 | 1332K| | 311 (1)| 00:00:01 |
|* 4 | INDEX RANGE SCAN | IDX_FIN_ACTVT_BAL_ID | 11966 | | | 37 (0)| 00:00:01 |
-------------------------------------------------------------------------------------------------------------------------------
I believe the metrics are expressed in bytes unless a unit specifyer is provided.
TEMP tablespace is used for large SQL operations that can't be completed in memory and need to spill to disk. For example:
Sorting
merging
parallel
To reduce temp:
You can increase the PGA to provide more memory for the SQL
operations and thus reduce the likelihood of spilling over to temp.
Rewrite or tune the query to use less expensive operations.

Join Two Big Tables Fast

after research via playing with the queries and doing online research, I am turning to you for input. Looking forward to your replies! I am trying to write this in a general way as I am looking for general ideas on how to do this, not for exact statements. If this question is received poorly, I am happy to rework it, let me know. Here we go:
I have two tables:
Table 1 has 10 MIO records. It has only one column:
Column A: A unique ID (a text e.g. '5uz1g09aksmsd')
Table 2 has 300 MIO records. It has two columns:
Column A: A unique ID (a text e.g. '5uz1g09aksmsd')
Column B: A number (e.g. 32.5432)
There are no indices etc. yet, not even a PK as both tables have just been created via CTAS. Both tables have been analysed.
I am now looking for a fast query (the prep-work e.g. index creation can take time, no problem) to create a third table that contains the 10 MIO rows of Table 1, and Column B of Table 2 if a match is found via Column A (for 99% a match will be found). To make it clearer, a simple CTAS could be:
create table3 as
select t1.a,
(select t2.b from table2 t2 where t2.a = t1.a and rownum = 1)
-- the rownum = 1 is to show Oracle that there can only be one match
from table1 t1;
This is not as fast as it can be (I hope). What are your ideas to make it faster? Creating indices? Which ones? What kind of join would you want to see in the execution plan? A hash join? I already found
create table nologging
parallel query execution and parallel table creation
But I am interested in the specifics of how the perfect execution plan for this would look given that we are allowed to alter the system (e.g. create an index). In particular I am asking for 11gR2 but 12c comments are also very welcome.
Start with the simplest possibility using this query
create table c as
select /*+ parallel(6) */ a.a, b.b
from a
left outer join b
on a.a = b.a
;
It will use a hash join, adjust parallel degree based on your hardware setup.
The expected execution plan is as follows
--------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | TQ |IN-OUT| PQ Distrib |
--------------------------------------------------------------------------------------------------------------------------
| 0 | CREATE TABLE STATEMENT | | 261M| 6238M| 56799 (1)| 00:03:48 | | | |
| 1 | PX COORDINATOR | | | | | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10002 | 261M| 6238M| 31984 (1)| 00:02:08 | Q1,02 | P->S | QC (RAND) |
| 3 | LOAD AS SELECT | C | | | | | Q1,02 | PCWP | |
|* 4 | HASH JOIN OUTER | | 261M| 6238M| 31984 (1)| 00:02:08 | Q1,02 | PCWP | |
| 5 | PX RECEIVE | | 7849K| 44M| 726 (1)| 00:00:03 | Q1,02 | PCWP | |
| 6 | PX SEND HASH | :TQ10000 | 7849K| 44M| 726 (1)| 00:00:03 | Q1,00 | P->P | HASH |
| 7 | PX BLOCK ITERATOR | | 7849K| 44M| 726 (1)| 00:00:03 | Q1,00 | PCWC | |
| 8 | TABLE ACCESS STORAGE FULL| A | 7849K| 44M| 726 (1)| 00:00:03 | Q1,00 | PCWP | |
| 9 | PX RECEIVE | | 261M| 4741M| 31149 (1)| 00:02:05 | Q1,02 | PCWP | |
| 10 | PX SEND HASH | :TQ10001 | 261M| 4741M| 31149 (1)| 00:02:05 | Q1,01 | P->P | HASH |
| 11 | PX BLOCK ITERATOR | | 261M| 4741M| 31149 (1)| 00:02:05 | Q1,01 | PCWC | |
| 12 | TABLE ACCESS STORAGE FULL| B | 261M| 4741M| 31149 (1)| 00:02:05 | Q1,01 | PCWP | |
--------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
4 - access("A"."A"="B"."A"(+))
My test with synthetic data of your size is 35 seconds.

How to optimize oracle query?

Following query is taking around 45 seconds in oracle 11g
select count(cap.ISHIGH),ms.SID,ms.NUM from CDetail cap,MData ms
where cap.MDataID_FK=ms.MDataID_PK and trunc(cap.CREATEDTIME) between trunc(sysdate-10) and trunc(sysdate)
group by ms.SID,ms.NUM ;
explain plan :
-------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
-------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 766K| 32M| | 94421 (1)| 00:18:54 |
| 1 | HASH GROUP BY | | 766K| 32M| 41M| 94421 (1)| 00:18:54 |
|* 2 | HASH JOIN | | 766K| 32M| 21M| 85716 (1)| 00:17:09 |
| 3 | VIEW | VW_GBC_5 | 766K| 13M| | 73348 (1)| 00:14:41 |
| 4 | HASH GROUP BY | | 766K| 13M| 98M| 73348 (1)| 00:14:41 |
|* 5 | FILTER | | | | | | |
| 6 | TABLE ACCESS BY INDEX ROWID| CDetail | 3217K| 58M| | 63738 (1)| 00:12:45 |
|* 7 | INDEX RANGE SCAN | IDX_CPCTYDTLTRNCCRTDTM | 3365K| | | 14679 (1)| 00:02:57 |
| 8 | TABLE ACCESS FULL | MData | 871K| 22M| | 9665 (1)| 00:01:56 |
-------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("ITEM_1"="MS"."MDataID_PK")
5 - filter(TRUNC(SYSDATE#!-10)<=TRUNC(SYSDATE#!))
7 - access(TRUNC(INTERNAL_FUNCTION("CREATEDTIME"))>=TRUNC(SYSDATE#!-10) AND
TRUNC(INTERNAL_FUNCTION("CREATEDTIME"))<=TRUNC(SYSDATE#!))
table MData contains around 900,000 rows and table CDetail contains 23,000,000 rows.
Should I introduce any new index or any other way to optimize the above query.
Edit 3. IDX_CPCTYDTLTRNCCRTDTM is a functional index on trunc(CREATEDTIME)
Edit :1
explain plan :for full table scan using hint /+full(Cdetail)/
---------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
---------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 780K| 33M| | 160K (2)| 00:32:01 |
| 1 | HASH GROUP BY | | 780K| 33M| 42M| 160K (2)| 00:32:01 |
|* 2 | HASH JOIN | | 780K| 33M| 22M| 151K (2)| 00:30:15 |
| 3 | VIEW | VW_GBC_5 | 780K| 13M| | 138K (2)| 00:27:46 |
| 4 | HASH GROUP BY | | 780K| 14M| 230M| 138K (2)| 00:27:46 |
|* 5 | FILTER | | | | | | |
|* 6 | TABLE ACCESS FULL| CDetail | 7521K| 136M| | 120K (2)| 00:24:02 |
| 7 | TABLE ACCESS FULL | MData | 890K| 22M| | 9666 (1)| 00:01:56 |
---------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("ITEM_1"="MS"."MDataID_PK")
5 - filter(TRUNC(SYSDATE#!-10)<=TRUNC(SYSDATE#!))
6 - filter(TRUNC(INTERNAL_FUNCTION("CREATEDTIME"))>=TRUNC(SYSDATE#!-10) AND
TRUNC(INTERNAL_FUNCTION("CREATEDTIME"))<=TRUNC(SYSDATE#!))
Thank you for sharing an explain plan; thats a good start. The thing with an explain plan however is that it gives you estimates, not actuals. If you can, can you get a SQL Monitor report? This will show you the actual cardinality and show you where time is being spent in the query.
The date filter is expecting about 3M rows (ID's 6 an 7)? Is that accurate?
What is the definition of the IDX_CPCTYDTLTRNCCRTDTM index? Does it happen to be function based?
Just to validate my thinking, can you add the following hint, run the query and get the explain plan again.
select /*+ full( cap ) */ ...

Optimize counts in SELECT with JOIN on Oracle

Hello all :) I have two tables that are about 30 millions rows each, and I'm seeking to improve performance when counts are performed.
Here is the query:
SELECT count(*)
FROM VEHICULE v
JOIN CLIENT c ON c.CL_ID = v.VE_CL_ID
WHERE v.VE_BRAND = 'MITSUBISHI'
AND c.CL_COUNTRY = 'SPAIN';
The foreign key is declared in the VEHICULE table
CONSTRAINT "VEHICULE_CLIENT_FK" FOREIGN KEY ("VE_CL_ID")
REFERENCES "MY_SCHEMA"."CLIENT" ("CL_ID") ENABLE
And there is an index on the foreign key:
CREATE INDEX "MY_SCHEMA"."VEHICULE_INDEX_CLIENT" ON "MY_SCHEMA"."VEHICULE" ("CL_ID")
There are indexes also on the columns used for the search criteria.
The requests can take up to 40 seconds. I have looked at bitmap joins indexes but I don't know if it will help, as bitmap joins are supposed to be for columns with low cardinalities. Is this the only type of index for joins? I'm totally at a loss at how I can improve the performance.
EDIT:
Here is what the SQL tuning advisor of SQL developer displays (execution plan)
The sql for this query is without AND c.CL_COUNTRY = 'SPAIN'
GENERAL INFORMATION SECTION
-------------------------------------------------------------------------------
Tuning Task Name : staName9168
Tuning Task Owner : USER
Tuning Task ID : 12125
Scope : COMPREHENSIVE
Time Limit(seconds): 1800
Completion Status : COMPLETED
Started at : 04/23/2013 15:44:35
Completed at : 04/23/2013 15:44:36
-------------------------------------------------------------------------------
There are no recommendations to improve the statement.
-------------------------------------------------------------------------------
EXPLAIN PLANS SECTION
-------------------------------------------------------------------------------
1- Original
-----------
Plan hash value: 3808155432
------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | TQ |IN-OUT| PQ Distrib |
------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 21 | 54011 (1)| 00:10:49 | | | |
| 1 | SORT AGGREGATE | | 1 | 21 | | | | | |
| 2 | PX COORDINATOR | | | | | | | | |
| 3 | PX SEND QC (RANDOM) | :TQ10001 | 1 | 21 | | | Q1,01 | P->S | QC (RAND) |
| 4 | SORT AGGREGATE | | 1 | 21 | | | Q1,01 | PCWP | |
|* 5 | HASH JOIN | | 475K| 9745K| 54011 (1)| 00:10:49 | Q1,01 | PCWP | |
| 6 | BUFFER SORT | | | | | | Q1,01 | PCWC | |
| 7 | PX RECEIVE | | 475K| 6497K| 32813 (1)| 00:06:34 | Q1,01 | PCWP | |
| 8 | PX SEND BROADCAST | :TQ10000 | 475K| 6497K| 32813 (1)| 00:06:34 | | S->P | BROADCAST |
|* 9 | TABLE ACCESS BY INDEX ROWID| VEHICULE | 475K| 6497K| 32813 (1)| 00:06:34 | | | |
|* 10 | INDEX RANGE SCAN | VEHICULE_INDEX_BRAND | 616K| | 1621 (2)| 00:00:20 | | | |
| 11 | PX BLOCK ITERATOR | | 20M| 138M| 21146 (1)| 00:04:14 | Q1,01 | PCWC | |
| 12 | TABLE ACCESS FULL | CLIENT | 20M| 138M| 21146 (1)| 00:04:14 | Q1,01 | PCWP | |
------------------------------------------------------------------------------------------------------------------------------------------
Query Block Name / Object Alias (identified by operation id):
-------------------------------------------------------------
1 - SEL$58A6D7F6
9 - SEL$58A6D7F6 / VEHICULE#SEL$1
10 - SEL$58A6D7F6 / VEHICULE#SEL$1
12 - SEL$58A6D7F6 / CLIENT#SEL$1
Predicate Information (identified by operation id):
---------------------------------------------------
5 - access("VE_CL_ID"="CL_ID")
9 - filter("VE_CL_ID" IS NOT NULL)
10 - access("VEHICULE"."VE_BRAND"='MITSUBISHI')
Column Projection Information (identified by operation id):
-----------------------------------------------------------
1 - (#keys=0) COUNT()[22]
2 - SYS_OP_MSR()[10]
3 - (#keys=0) SYS_OP_MSR()[10]
4 - (#keys=0) SYS_OP_MSR()[10]
5 - (#keys=1)
6 - (#keys=0) "VE_CL_ID"[NUMBER,22]
7 - "VE_CL_ID"[NUMBER,22]
8 - (#keys=0) "VE_CL_ID"[NUMBER,22]
9 - "VE_CL_ID"[NUMBER,22]
10 - "VEHICULE".ROWID[ROWID,10]
11 - "CL_ID"[NUMBER,22]
12 - "CL_ID"[NUMBER,22]
-------------------------------------------------------------------------------
Create composite indexes on client (cl_country, cl_id) and vehicule (ve_brand, ve_cl_id) (both in this order).
This way you could get rid of table access on both tables.
If you have but a few countries and brands possible you could also partition the indexes by country and brand so that INDEX FAST FULL SCAN could be used instead of INDEX RANGE SCAN.
You could also consider creating a cluster on client.id which would make the vehicle and client data to be stored in same or nearby data blocks, thus improving I/O.

Resources