I have a query that uses the date index if I query for 1,2,3,4 or 5 days worth of data. For these queries data comes back in less than 10 seconds. For 6 days or above it decides to not use the index and query takes forever, if it comes back at all. If I try using an index help it doesn't seem to use the index. I tried doing a gather statistics on the index and on the table. Here is the query (9 days worth of data in this case):
select
r.range_text as duration_range,
nvl(count(c.call_duration),0) as calls,
nvl(SUM(call_duration),0) as total_duration
from
duration_ranges r
left join
big_table c
on c.call_duration >= r.range_lbound AND c.call_duration <= r.range_ubound
where calltimestamp_local >= to_date('20-02-2014 00:00:00','dd-MM-yyyy HH24:mi:ss')
and calltimestamp_local <= to_date('28-02-2014 23:59:59' ,'dd-MM-yyyy HH24:mi:ss')
and c.destinationnumber = 'sip:1000#company.com:5060;user=phone'
group by
r.range_text
order by
r.range_text;
Here is the explain plan without the index:
Without Index
SELECT STATEMENT ALL_ROWSCost: 827,605 Bytes: 1,344 Cardinality: 14
8 SORT GROUP BY Cost: 827,605 Bytes: 1,344 Cardinality: 14
7 MERGE JOIN Cost: 827,486 Bytes: 246,552,768 Cardinality: 2,568,258
2 SORT JOIN Cost: 4 Bytes: 308 Cardinality: 14
1 TABLE ACCESS FULL TABLE MYDB.DURATION_RANGES Cost: 3 Bytes: 308 Cardinality: 14
6 FILTER
5 SORT JOIN Cost: 827,471 Bytes: 14,164,118 Cardinality: 191,407
4 PARTITION RANGE ALL Cost: 824,134 Bytes: 14,164,118 Cardinality: 191,407 Partition #: 7 Partitions accessed #1 - #1653
3 TABLE ACCESS FULL TABLE MYDB.BIG_TABLE Cost: 824,134 Bytes: 14,164,118 Cardinality: 191,407 Partition #: 7 Partitions accessed #1 - #1653
With the index:
With Index
Plan
SELECT STATEMENT ALL_ROWSCost: 822,732 Bytes: 1,344 Cardinality: 14
8 SORT GROUP BY Cost: 822,732 Bytes: 1,344 Cardinality: 14
7 MERGE JOIN Cost: 822,635 Bytes: 205,460,736 Cardinality: 2,140,216
2 SORT JOIN Cost: 4 Bytes: 308 Cardinality: 14
1 TABLE ACCESS FULL TABLE MYDB.DURATION_RANGES Cost: 3 Bytes: 308 Cardinality: 14
6 FILTER
5 SORT JOIN Cost: 822,621 Bytes: 11,803,444 Cardinality: 159,506
4 TABLE ACCESS BY GLOBAL INDEX ROWID TABLE MYDB.BIG_TABLE Cost: 819,841 Bytes: 11,803,444 Cardinality: 159,506 Partition #: 7 Partition access computed by row location
3 INDEX RANGE SCAN INDEX MYDB.IDX_CDR_CALLTIMESTAMP_LOCAL Cost: 2,744 Cardinality: 850,691
Any idea why it wouldn't use the index if at 5 days worth of data it is coming back perfectly fast? It doesn't seem to be optimizing things since it comes back much much slower.
Maybe it works slower because of data and index wouldn't change much? You can easily check it by forcing Oracle to use index - specify INDEX hint and then compare times with index and without it for identical query.
You can also specify NOINDEX hint to compare times for 5 and fewer days.
Related
I have two queries which I need to join for business purpose to make them one-step process
SELECT
empid,
assest_x_id,
asset_y_id
FROM emp e
WHERE e.empid = 'SOME_UNIQUE_VAL';
result:
EMPID ASSEST_X_ID ASSET_Y_ID
======= ============ ===========
1234 abc pqr
-- Even though millions rows in table this will return 1 row always within milliseconds as it's using PK column with unique value.
Now there is another table for asset_values in separate DB current market price
(also a million rows)
SELECT
asset_id,
assest_type,
asset_current_price
FROM asset_values#x_db a
WHERE (asset_id, assest_type) IN (('abc', 'X'), ('pqr', 'Y'));
result:
asset_id asset_type assest_current_price
======== ========= =============
abc X 10000
pqr Y 5000
This will also return 2-3 rows always in few millisecs as Primary key is defined for combination of asset_id,asset_type values and there exists only 3 type of assets as X/Y/Z.
(Note: Not possible to normalize this table further data in business rules)
**Now to make a one step process query in script I tried to join these queries which can take empid from user and get all desired reults.
But now problem is that when I try to merge these two in single query like below runs for 15+ mins to give results**
SELECT
a.asset_id,
a.asset_type,
asset_current_price
FROM asset_values#x_db a, emp b
WHERE b.empid = 'SAME_UNIQUE_VAL'
AND (asset_id, asset_type) IN ((b.asset_x_id, 'X'), (b.asset_y_id, 'Y'));
Surprisingly Explain Plan is also good. (bytes:597 cost:2)
Can someone please give your expert advice on this?
SELECT STATEMENT ALL_ROWSCost: 6 Bytes: 690 Cardinality: 2
13 CONCATENATION
6 NESTED LOOPS Cost: 3 Bytes: 345 Cardinality: 1
3 PARTITION RANGE SINGLE Cost: 1 Bytes: 2,232 Cardinality: 9 Partition #: 3 Partitions accessed #1
2 TABLE ACCESS BY LOCAL INDEX ROWID TABLE MYPRDAOWN.EMP Object Instance: 2 Cost: 1 Bytes: 2,232 Cardinality: 9 Partition #: 4 Partitions accessed #1
1 INDEX RANGE SCAN INDEX MYPRDAOWN.EMP_7IX Cost: 1 Cardinality: 9 Partition #: 5 Partitions accessed #1
5 FILTER Cost: 1 Bytes: 97 Cardinality: 1
4 REMOTE REMOTE SERIAL_FROM_REMOTE ASSEST_VALUES XDB
12 NESTED LOOPS Cost: 3 Bytes: 345 Cardinality: 1
9 PARTITION RANGE SINGLE Cost: 1 Bytes: 2,232 Cardinality: 9 Partition #: 9 Partitions accessed #1
8 TABLE ACCESS BY LOCAL INDEX ROWID TABLE MYPRDAOWN.EMP Object Instance: 2 Cost: 1 Bytes: 2,232 Cardinality: 9 Partition #: 10 Partitions accessed #1
7 INDEX RANGE SCAN INDEX MYPRDAOWN.EMP_7IX Cost: 1 Cardinality: 9 Partition #: 11 Partitions accessed #1
11 FILTER Cost: 1 Bytes: 97 Cardinality: 1
10 REMOTE REMOTE SERIAL_FROM_REMOTE ASSEST_VALUES XDB
from http://docs.oracle.com/cd/B28359_01/server.111/b28274/optimops.htm#i49732:
Nested loop joins are useful when small subsets of data are being joined and if the join condition is an efficient way of accessing the second table.
Use hash joins which:
... are used for joining large data sets. The optimizer uses the smaller of two tables or data sources to build a hash table on the join key in memory. It then scans the larger table, probing the hash table to find the joined rows.
to implement it use hint
SELECT /*+ use_hash(a,b) */ a.asset_id,
a.asset_type,
asset_current_price
FROM asset_values#x_db a,
emp b
WHERE b.empid = 'SAME_UNIQUE_VAL'
AND (asset_id, asset_type) IN ((b.asset_x_id, 'X'), (b.asset_y_id, 'Y'));
I have a table which holds more then 2 million records, I am trying to update a table using following query
UPDATE toc T
SET RANK =
65535
- (SELECT COUNT (*)
FROM toc T2
WHERE S_KEY LIKE '00010001%'
AND A_ID IS NOT NULL
AND T2.TARGET = T.TARGET
AND T2.RANK > T.RANK)
WHERE S_KEY LIKE '00010001%' AND A_ID IS NOT NULL
Usually this query tooks 5 mins to update 50000 rows in our staging db which is a exact replica of production db but in our production db it is taking 6 hours to execute...
I tried Oracle advisory to select the correct execution plan but nothing is working...
Plan
UPDATE STATEMENT ALL_ROWSCost: 329,471
6 UPDATE TT.TOC
2 TABLE ACCESS BY INDEX ROWID TABLE TT.TOC Cost: 5 Bytes: 4,173,236 Cardinality: 54,911
1 INDEX SKIP SCAN INDEX TT.DATASTAT_SORTKEY_IDX Cost: 4 Cardinality: 1
5 SORT AGGREGATE Bytes: 76 Cardinality: 1
4 TABLE ACCESS BY INDEX ROWID TABLE TT.TOC Cost: 5 Bytes: 76 Cardinality: 1
3 INDEX SKIP SCAN INDEX TT.DATASTAT_SORTKEY_IDX Cost: 4 Cardinality: 1
I can see the following wait events
1,066 db file sequential read 10,267 0 3,993 0 6 39,933,580
1,066 db file scattered read 413 0 188 0 6 1,876,464
Any help will be greatly appreciated.
here is the current list of indexes
DSTAT_SKEY_IDX D_STATUS 1
DSTAT_SKEY_IDX S_KEY 2
IDX$$_165A0002 N_LABEL 1
S_KEY_IDX S_KEY 1
XAK1_TOC N_RELATIONSHIP 1
XAK2_TOC TARGET 1
XAK2_TOC N_LABEL 2
XAK2_TOC D_STATUS 3
XAK2_TOC A_ID 4
XIE1_TOC N_RELBASE 1
XIF4_TOC SOURCE_FILE_ID 1
XIF5_TOC A_ID 1
XPK_TOC N_ID 1
Atif
You're doing a skip scan where you supposedly want to do a range scan.
A range scan is only possible when the index columns are ordered by descending selectivity - in your case it seems that it should be S_KEY - TARGET - RANK
Update: rewriting the query in different order wouldn't make any difference. What matters is the sequence of the columns in the indexes of that table.
first show us the current index columns for that table:
select index_name, column_name, column_position from all_ind_columns where table_name = 'TOC'
then you could create a new index, e.g.
create index toc_i_s_key_target_rank on toc (s_key, target, rank) compress;
Below is the query in which I am using an aggregate function. The where clause is simple with an index on corpId and incoming_date .
If I simply fetch all rows/count the query takes lesser than a second. However when I use the aggregate function the query takes about 4 minutes.
I am using oracle 11i and the total rows the where clause retrieves is around 64000. The table and index statistics have also been gathered recently and there are no new rows added in the table.
Please suggest on improving the speed.
SELECT
sum(paid_amt) totalamount
FROM test_table e
WHERE e.corpId =6
AND e. incoming_date >= to_date('01-12-2012','dd-mm-yyyy')
AND e. incoming _date <= to_date('09-01-2013','dd-mm-yyyy')
Include paid_amt into the index:
CREATE INDEX
ix_testtable_cord_date_paid
ON test_table (corpId, incoming_date, paid_amt)
If you have an index on just (corpId, incoming_date) and try to test speed like this:
SELECT COUNT(*)
FROM test_table
WHERE e.corpId = 6
AND e.incoming_date >= to_date('01-12-2012','dd-mm-yyyy')
AND e.incoming_date <= to_date('09-01-2013','dd-mm-yyyy')
you not querying for anything outside the index so the query can be satisfied with an INDEX (RANGE SCAN) alone.
As soon as you add anything not in the index (paid_amt in your case), the query needs to to use additional TABLE ACCESS (BY INDEX ROWID) to retrieve the record from the table.
It's random lookups in a nested loop and it's slow, especially if your table records are large (have lots of fields or long fields).
The optimizer may even deem this access method less efficient that the FULL SCAN and use the latter instead.
I recreated this scenario like this...
CREATE TABLE test_table
( id integer
, corpId integer
, paid_amt number(10,2)
, incoming_date DATE );
ALTER TABLE test_table
add CONSTRAINT test_table_pk PRIMARY KEY (id);
create index test_table_nui_1 on test_table(corpId);
create index test_table_nui_2 on test_table(incoming_date);
create sequence test_table_seq;
insert into test_table
select test_table_seq.nextval
,MOD(test_table_seq.currval,6)
,MOD(test_table_seq.currval,10) + 1
,sysdate - MOD(test_table_seq.currval,200)
from all_objects, user_objects;
The cartesian join between all_objects and user_objects is just a hack to get a load of records inserted quickly. (657,000 rows in this case)
Running through the selects first all 657,000...
select sum(paid_amt)
from test_table;
Plan
SELECT STATEMENT ALL_ROWSCost: 621 Bytes: 13 Cardinality: 1
2 SORT AGGREGATE Bytes: 13 Cardinality: 1
1 TABLE ACCESS FULL TABLE DAVE.TEST_TABLE Cost: 621 Bytes: 9,923,914 Cardinality: 763,378
Then 109,650 for one corpId...
select sum(paid_amt)
from test_table
where corpId = 5;
Plan
SELECT STATEMENT ALL_ROWSCost: 265 Bytes: 26 Cardinality: 1
3 SORT AGGREGATE Bytes: 26 Cardinality: 1
2 TABLE ACCESS BY INDEX ROWID TABLE DAVE.TEST_TABLE Cost: 265 Bytes: 3,310,138 Cardinality: 127,313
1 INDEX RANGE SCAN INDEX DAVE.TEST_TABLE_NUI_1 Cost: 213 Cardinality: 3,054
And finally 20,836 rows restricting by date...
SELECT sum(paid_amt) totalamount
FROM test_table e
WHERE e.corpId = 5
AND e. incoming_date >= to_date('01-12-2012','dd-mm-yyyy')
AND e. incoming_date <= to_date('09-01-2013','dd-mm-yyyy')
Plan
SELECT STATEMENT ALL_ROWSCost: 265 Bytes: 35 Cardinality: 1
3 SORT AGGREGATE Bytes: 35 Cardinality: 1
2 TABLE ACCESS BY INDEX ROWID TABLE DAVE.TEST_TABLE Cost: 265 Bytes: 871,360 Cardinality: 24,896
1 INDEX RANGE SCAN INDEX DAVE.TEST_TABLE_NUI_1 Cost: 213 Cardinality: 3,054
All 3 queries were rapid (i.e. < 0.5 seconds)
An alternative would be to drop nui_1 and nui_2 and create a combined index on both columns. This then ran in 31ms on my db
create index test_table_nui_3 on test_table(corpId, incoming_date);
Plan
SELECT STATEMENT ALL_ROWSCost: 15 Bytes: 35 Cardinality: 1
3 SORT AGGREGATE Bytes: 35 Cardinality: 1
2 TABLE ACCESS BY INDEX ROWID TABLE DAVE.TEST_TABLE Cost: 15 Bytes: 871,360 Cardinality: 24,896
1 INDEX RANGE SCAN INDEX DAVE.TEST_TABLE_NUI_3 Cost: 3 Cardinality: 14
This suggests that the aggregate function is not the problem, but your indexing may be. The best thing to do would be to check your explain plans.
I have a question on joining tables in a different scenario. Please find the sample tables below.
Capacity of expected table row 3-5 should be repeated as table 2 does not have those fields.
could anyone please help to get expected table?
Table 1:
No ProjectID Capacity
1 514 4
2 418 10
3 418 30
4 401 40
5 502 41
Table2:
NO ProjectID Capacity1 Capacity2
1 514 4 10
2 418 10 20
Expected Table:
NO ProjectID Capacity1 Capacity2
1 514 4 10
2 418 10 20
3 418 30 30
4 401 40 40
5 502 41 41
1.Do left outer join
2.For the values not matching take them from table 1 with if condition.
select t1.no,t1.projectid,t1.capacity1,if(t2.capacity2 is null,t1.capacity,t2.capacity)
from table1 t1 left outer join table2 t2 on t1.no=t2.no
I think above query meets your requirement let me know if need any more help.
We are having a serious problem with a query that defies explanation. In SQL-plus or TOAD it runs in 1/2 sec, but when run from a C++ program via a Distributed Transaction it is taking 41 minutes. Until this week this has run 10,000 of times from the C++ code all under a second.
Nothing has changed in the DB or the code or the W2k servers running the code.
when running from the code it has very high db file sequential read over 1,000,000
when exact same statement is run from SQL plus the db file sequential read is 8
So same statement is doing 100,000x more work when run via code and DT than from sqlplus.
we did the following query to find what blocks are being read
SELECT p1 "file#", p2 "block#", p3 "class#"
FROM v$session_wait
WHERE event = 'db file sequential read'
and they are the tables used in the query. It is reading the tables over and over yet the explain plan indicates only 8 blocks should be read
both tables are ~10 gig in size
here is the statment and the explain plan
SELECT COUNT (*)
FROM student st, testinstance ti
WHERE st.dataset_id = :b2
AND st.student_id = ti.student_id
AND ti.testinstance_id > :b1
AND NOT EXISTS (
SELECT 1
FROM programscoringexclusion ex
WHERE ex.program_id = wfgeneral.getprogramid (:b3)
AND ex.testfamily_id = ti.testfamily_id
AND NVL (ex.test_level, NVL (ti.test_level, '*')) =
NVL (ti.test_level, '*')
AND NVL (ex.battery, NVL (ti.battery, '*')) =
NVL (ti.battery, '*')
AND NVL (ex.form, NVL (ti.form, '*')) = NVL (ti.form, '*'))
Plan
SELECT STATEMENT CHOOSECost: 2
9 SORT AGGREGATE Bytes: 43 Cardinality: 1
8 FILTER
5 NESTED LOOPS Cost: 2 Bytes: 43 Cardinality: 1
2 TABLE ACCESS BY INDEX ROWID TABLE BBOX.TESTINSTANCE Cost: 1 Bytes: 32 Cardinality: 1
1 INDEX RANGE SCAN INDEX (UNIQUE) BBOX.XXPK0TESTINSTANCE Cost: 1 Cardinality: 1
4 TABLE ACCESS BY INDEX ROWID TABLE BBOX.STUDENT Cost: 1 Bytes: 11 Cardinality: 1
3 INDEX UNIQUE SCAN INDEX (UNIQUE) BBOX.XXPK0STUDENT Cost: 1 Cardinality: 1
7 TABLE ACCESS BY INDEX ROWID TABLE BBOX.PROGRAMSCORINGEXCLUSION Cost: 1 Bytes: 37 Cardinality: 1
6 INDEX RANGE SCAN INDEX BBOX.XXIE1PROGRAMSCORINGEXCLUSION Cost: 1 Cardinality: 1
how can we see what the actual plan is for a statement when it is actually running? We can tell it is reading the tables aboves. Is it possible that the actual plan is different than the one we are seeing and is in fact doing some kind of Cartesian join or something wierd that takes it 40 mins to resolve? is there a way of determining that?
To find the actual plan used, you can query v$sql_plan with the sql_id. The easiest thing to do, is to put a comment in the query to make it unique, eg
select /* FROM C++ */ ....
and
select /* FROM SQLPLUS */ ....
Then run the query. By querying from v$sql you can find the SQL_ID of the query, eg:
select sql_id, sql_fulltext
from v$sql
where upper(sql_fulltext) like '%FROM C++%';
Then using that SQL_ID, you can query v$sql_plan to get the plan, or better, use the following query:
select * from table(dbms_xplan.diplay_cursor('SQL ID OBTAINED ABOVE'));