Recently, after upgrading our test env with a new version of application and introducing some new features I've stumbled upon some weird oracle behavior, which shows itself in query results that doesn't make sense and seem to be index dependant.
After doing some investigation I was able to create a minimal conditions to reproduce the issue.
I was experimenting on Oracle21 started in with docker:
docker run --restart always -d -p 1521:1521 -e ORACLE_PASSWORD=system --name oracle-21c-01 gvenzl/oracle-xe:21-slim
Then logging in as system user (system/system) I created another one with few permissions:
CREATE USER test PROFILE DEFAULT IDENTIFIED BY test ACCOUNT UNLOCK;
GRANT CONNECT TO test;
GRANT UNLIMITED TABLESPACE TO test;
GRANT CREATE TABLE TO test;
After that I logged in under this new user (test/test) and executed the following:
CREATE TABLE TEST_BUG
(
ID NUMBER(10) NOT NULL
CONSTRAINT PK_TEST_BUG PRIMARY KEY,
TENANT NUMBER(10) NOT NULL,
IDENTIFIER VARCHAR2(255) NOT NULL,
NAME VARCHAR2(255) NOT NULL
);
INSERT INTO TEST_BUG
VALUES (10, 1, 'IDENTIFIER', 'TESTBUG');
ALTER TABLE TEST_BUG
ADD CONSTRAINT UK_NAME_TENANT UNIQUE (NAME, TENANT);
ALTER TABLE TEST_BUG
ADD CONSTRAINT UK_IDENTIFIER_TENANT UNIQUE (IDENTIFIER, TENANT);
ALTER INDEX PK_TEST_BUG REBUILD;
As you can see here I'm just creating a very simple table with a couple of indexes, but the statements must be executed in this particular order (Also only with index rebuild on the last line I was able to reproduce the issue we're having on the actual environment, even though we do not do it anywhere in the update scripts).
With all that in mind, executing the following simle java code gives me weird results:
public class Main {
public static void main(String[] args) throws Exception {
Class.forName("oracle.jdbc.driver.OracleDriver");
try (Connection connection = DriverManager.getConnection("jdbc:oracle:thin:#localhost:1521:XE", "test", "test")) {
try (PreparedStatement preparedStatement = connection.prepareStatement(
"select id from test_bug where tenant = ? and name=? and id<>?"
)) {
preparedStatement.setInt(1, 1);
preparedStatement.setString(2, "TESTBUG");
preparedStatement.setInt(3, 10);
ResultSet resultSet = preparedStatement.executeQuery();
System.out.println(resultSet.next());
System.out.println(resultSet.getInt(1));
}
}
}
}
And results are:
true
10
Meaning that even though I've inserted a single record with id=10 and in the query asking specifically that id<>10, I still get that recoded as an output instead of getting nothing!
Couple more notes:
This only works with PreparedStatement, if I try to execute this query with sql developer - works just fine.
If I remove put id condition explicitly in query (like select id from test_bug where tenant = ? and name=? and id<>10) and remove preparedStatement.setInt(3, 10); - works also fine (no results)
If I disable PK index with ALTER INDEX PK_TEST_BUG UNUSABLE; - works also fine, no results, until I get index back with ALTER INDEX PK_TEST_BUG REBUILD;
For oracle driver I'm using here ojdbc11 21.9.0.0, but I've played with different versions - no difference.
Any ideas kindly appreciated!
This is an optimizer bug. If I run the java unchanged and do a SQL trace on it, you see the following the trace file
select id
from
test_bug where tenant = :1 and name=:2 and id != :3
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 1 0
Fetch 1 0.00 0.00 0 3 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 3 0.00 0.00 0 3 1 1
Misses in library cache during parse: 1
Misses in library cache during execute: 1
Optimizer mode: ALL_ROWS
Parsing user id: 109
Number of plan statistics captured: 1
Rows (1st) Rows (avg) Rows (max) Row Source Operation
---------- ---------- ---------- ---------------------------------------------------
1 1 1 TABLE ACCESS BY INDEX ROWID BATCHED TEST_BUG (cr=3 pr=0 pw=0 time=32 us starts=1 cost=0 size=155 card=1)
1 1 1 BITMAP CONVERSION TO ROWIDS (cr=2 pr=0 pw=0 time=25 us starts=1)
1 1 1 BITMAP AND (cr=2 pr=0 pw=0 time=23 us starts=1)
1 1 1 BITMAP CONVERSION FROM ROWIDS (cr=1 pr=0 pw=0 time=9 us starts=1)
1 1 1 INDEX RANGE SCAN UK_NAME_TENANT (cr=1 pr=0 pw=0 time=8 us starts=1 cost=0 size=0 card=1)(object id 100690)
1 1 1 BITMAP CONVERSION FROM ROWIDS (cr=1 pr=0 pw=0 time=13 us starts=1)
1 1 1 SORT ORDER BY (cr=1 pr=0 pw=0 time=11 us starts=1)
1 1 1 INDEX RANGE SCAN PK_TEST_BUG (cr=1 pr=0 pw=0 time=3 us starts=1 cost=0 size=0 card=1)(object id 100689)
with the Fetch line Rows column of 1 meaning we found a row. If I disable that plan from use by changing the statement to:
con.prepareStatement("select /*+ full(t) */ id from test_bug t where tenant = ? and name=? and id != ?");
then with the altered plan, we get the result you expect
select /*+ full(t) */ id
from
test_bug t where tenant = :1 and name=:2 and id != :3
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 1 0
Fetch 1 0.00 0.00 0 7 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 3 0.00 0.00 0 7 1 0
Misses in library cache during parse: 1
Misses in library cache during execute: 1
Optimizer mode: ALL_ROWS
Parsing user id: 109
Number of plan statistics captured: 1
Rows (1st) Rows (avg) Rows (max) Row Source Operation
---------- ---------- ---------- ---------------------------------------------------
0 0 0 TABLE ACCESS FULL TEST_BUG (cr=7 pr=0 pw=0 time=32 us starts=1 cost=3 size=155 card=1)
This is not to say that you should just put in a FULL hint and forget about it :-) but its worth logging an SR to see if that's an existing bug or if patches are available.
If you want a quick workaround, then:
select /*+ opt_param('_b_tree_bitmap_plans','false') */ id from test_bug where tenant = :b1 and name= :b2 and id<> :b3;
should take care of it.
I have a performance issue that I cannot understand in on of my queries. The query I am executing is the following:
SELECT UNIQUE GROUP_ID
FROM pos, TABLE (t_type(12984918, 12984919, 12984917))
WHERE pos.pos_id = COLUMN_VALUE AND GROUP_ID <> 0;
1 row returned in 99 ms
You might think that 83 ms is fine, but what really puzzles me is that removing UNIQUE makes the query a LOT faster even when there are only 3 duplicate values:
SELECT GROUP_ID
FROM pos, TABLE (t_type(12984918, 12984919, 12984917))
WHERE pos.pos_id = COLUMN_VALUE AND GROUP_ID <> 0;
3 rows returned in 0.048 ms
Is oracle really taking almost 100ms just to reduce a set of 3 elements to a single value? That seemed crazy to me, so I started doing some investigation and fetched the execution plan with TKPROF.
SELECT UNIQUE GROUP_ID
FROM pos, TABLE (t_type(12984918, 12984919, 12984917))
WHERE pos.pos_id = COLUMN_VALUE AND GROUP_ID <> 0
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 2 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 1 0.00 0.09 0 8 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 3 0.00 0.10 0 10 0 1
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 58
Number of plan statistics captured: 1
Rows (1st) Rows (avg) Rows (max) Row Source Operation
---------- ---------- ---------- ---------------------------------------------------
1 1 1 HASH UNIQUE (cr=8 pr=0 pw=0 time=99258 us cost=31 size=3892 card=139)
3 3 3 NESTED LOOPS (cr=8 pr=0 pw=0 time=84 us)
3 3 3 NESTED LOOPS (cr=5 pr=0 pw=0 time=59 us cost=30 size=3892 card=139)
3 3 3 COLLECTION ITERATOR CONSTRUCTOR FETCH (cr=0 pr=0 pw=0 time=7 us cost=29 size=16336 card=8168)
3 3 3 INDEX UNIQUE SCAN IU_POS_POS_ID (cr=5 pr=0 pw=0 time=31 us cost=0 size=0 card=1)(object id 20684)
3 3 3 TABLE ACCESS BY INDEX ROWID POS (cr=3 pr=0 pw=0 time=12 us cost=0 size=26 card=1)
Well, it really looks like Oracle is taking 99 milliseconds to reduce 3 identical values into a single one, using the HASH UNIQUE operation. That actually STILL seems crazy. That should barely be noticeable. So I starter looking for an alternative. It turns out that Oracle can also use the SORT UNIQUE operation to implement DISTINCT. I also found out that I can force oracle to not use HASH UNIQUE by doing:
ALTER SESSION SET "_gby_hash_aggregation_enabled" = true;
An now, my query runs in 0.48ms! That's a 2000x improvement! WTF! Here's the execution plan that shows it's using SORT UNIQUE:
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 2 0.00 0.00 0 4 0 0
Execute 2 0.00 0.00 0 0 0 0
Fetch 2 0.00 0.00 0 16 0 2
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 6 0.00 0.00 0 20 0 2
Misses in library cache during parse: 2
Optimizer mode: ALL_ROWS
Parsing user id: 58
Number of plan statistics captured: 2
Rows (1st) Rows (avg) Rows (max) Row Source Operation
---------- ---------- ---------- ---------------------------------------------------
1 1 1 SORT UNIQUE (cr=8 pr=0 pw=0 time=48 us cost=30 size=28 card=1)
3 3 3 NESTED LOOPS (cr=8 pr=0 pw=0 time=35 us)
3 3 3 NESTED LOOPS (cr=5 pr=0 pw=0 time=26 us cost=29 size=28 card=1)
3 3 3 COLLECTION ITERATOR CONSTRUCTOR FETCH (cr=0 pr=0 pw=0 time=2 us cost=29 size=6 card=3)
3 3 3 INDEX UNIQUE SCAN IU_POS_POS_ID (cr=5 pr=0 pw=0 time=17 us cost=0 size=0 card=1)(object id 20684)
3 3 3 TABLE ACCESS BY INDEX ROWID POS (cr=3 pr=0 pw=0 time=7 us cost=0 size=26 card=1)
********************************************************************************
OK, so now I'm thinking how can I force oracle to use SORT UNIQUE without touching the session parameters. Turns out I can just add an ORDER BY clause, and speedup my query.
SELECT UNIQUE GROUP_ID
FROM pos, TABLE (t_type(12984918, 12984919, 12984917))
WHERE pos.pos_id = COLUMN_VALUE AND GROUP_ID <> 0
ORDER BY GROUP_ID;
So basically I've solved my problem (if you can call that kind of hack "solving a problem"), but I'm still puzzled.
So here are the question:
Why is Oracle choosing HASH UNIQUE over SORT UNIQUE ?
Why is HASH UNIQUE so slow when there are only 3 records to hash?
Is there a better way to hint oracle to use SORT UNIQUE over HASH UNIQUE?
Why is Oracle choosing HASH UNIQUE over SORT UNIQUE?
Your first tkprof results show much higher (expected?) cardinality for the TABLE iterator. Oracle is optimizing for many more than three rows.
Why is HASH UNIQUE so slow when there are only 3 records to hash?
Hash unique will need to build a hash table no matter how many elements are being hashed. Presumably the size of the hash table will be smaller for fewer elements, but again the cardinality estimate is such that Oracle is going to build a bigger hash table.
Is there a better way to hint oracle to use SORT UNIQUE over HASH UNIQUE?
I would try hinting the cardinality on the table operator.
Is there a better way to hint oracle to use SORT UNIQUE over HASH
UNIQUE?
I´d check how are the table and index stats are gathered. They are used by oracle optimizer in order to build effective access plans.
Objective
Find out a way to visually capture the recursive call trees in the TKPROF output (SYS=YES specified).
Environment
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
Windows 7 64 bit on DELL Latitude core i7 2.8GHz 8G memory & SSD HDD
Background
I am tryng to understand the effect of indexing and the performance indications as in the question Oacle 11G - Performance effect of indexing at insert.
To further understand what is happening behind the scene for indexing, run a SQL trace on a create index statement.
Execution
Run trace on the index creation (SQL towards the bottom) and run tkprof with "sys=yes" option.
SQL> ALTER TABLE TBL2 ADD CONSTRAINT PK_TBL2_COL1 PRIMARY KEY(COL1) ;
Table altered.
Elapsed: 00:00:01.75
> trcsess clientid="CREATE_INDEX" output="report_createindex.trc" *.trc
> tkprof createindex.trc output=createindex.txt sys=yes
Question
I would like to know if there is a way to visucaly capture the call hierarchy like explain plan from the trace (.trc) file with tkprof or with other tools.
The generated report includes recursive calls such as "ALTER TABLE TBL2 ADD" inducing "INDEX BUILD UNIQUE", and probably further sys recursive calls down the path. I suppose the output does not reflect the call-hierarchy (parent first, childs following immediately after).
TKPROF OUTPUT
SQL ID: 2w9c2khpsfj4m
Plan Hash: 3219312727
CREATE UNIQUE INDEX "TRY"."PK_TBL2_COL1" on "TRY"."TBL2"("COL1") NOPARALLEL
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 1 0 0
Execute 1 0.63 0.84 2999 15565 3173 0
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.63 0.85 2999 15566 3173 0
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 106 (recursive depth: 1) <----------------- child?
Rows Row Source Operation
------- ---------------------------------------------------
1 INDEX BUILD UNIQUE PK_TBL2_COL1 (cr=15904 pr=3003 pw=2090 time=0 us)(object id 0)
1000000 SORT CREATE INDEX (cr=15486 pr=2997 pw=0 time=208370 us)
1000000 TABLE ACCESS FULL TBL2 (cr=15486 pr=2997 pw=0 time=245360 us cost=4413 size=5000000 card=1000000)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 575 0.00 0.08
db file scattered read 138 0.00 0.07
direct path write 1 0.00 0.00
********************************************************************************
SQL ID: 47f85g3cmftry
Plan Hash: 0
ALTER TABLE TBL2 ADD CONSTRAINT PK_TBL2_COL1 PRIMARY KEY(COL1)
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.20 0.64 15630 29477 3 0
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.20 0.64 15630 29477 3 0
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 106 <------------------------------------------ parent?
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
db file sequential read 4 0.00 0.00
db file scattered read 259 0.01 0.42
db file parallel read 2 0.00 0.00
SQL*Net message to client 1 0.00 0.00
SQL*Net message from client 1 0.00 0.00
********************************************************************************
SQL
CREATE TABLE TBL2 (
"COL1" NUMBER,
"COL2" VARCHAR2(100 BYTE)
-- CONSTRAINT "PK_TBL2_COL1" PRIMARY KEY ("COL1")
);
INSERT INTO TBL2 /*+ APPEND MONITOR GATHER_PLAN_STATISTICS CONNECTBY_INSERT */
SELECT LEVEL, rpad(TO_CHAR(LEVEL),100,'A') FROM DUAL CONNECT BY LEVEL <= 1000000
COMMIT;
---------------------------------------------------------------------------
-- Flush the buffers and clear the caches.
---------------------------------------------------------------------------
ALTER SYSTEM CHECKPOINT;
ALTER SYSTEM FLUSH SHARED_POOL;
ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM SWITCH LOGFILE;
---------------------------------------------------------------------------
-- Start monitoring
---------------------------------------------------------------------------
ALTER SESSION SET TRACEFILE_IDENTIFIER ='CREATE_INDEX';
ALTER SESSION SET TIMED_STATISTICS=true;
BEGIN
DBMS_SESSION.SET_IDENTIFIER('CREATE_INDEX');
DBMS_MONITOR.CLIENT_ID_TRACE_ENABLE('CREATE_INDEX', waits=>true, binds=>false);
DBMS_MONITOR.CLIENT_ID_STAT_ENABLE('CREATE_INDEX');
END;
/
---------------------------------------------------------------------------
-- Run the SQL to insert and monitor
---------------------------------------------------------------------------
SET PAGESIZE 200
SET LINESIZE 200
SET TIMING ON
SET ECHO ON
SET AUTOTRACE ON
ALTER TABLE TBL2 ADD CONSTRAINT PK_TBL2_COL1 PRIMARY KEY(COL1) ;
SET AUTOTRACE OFF
SET ECHO OFF
SET TIMING OFF
---------------------------------------------------------------------------
-- Stop monitoring.
---------------------------------------------------------------------------
BEGIN
DBMS_MONITOR.CLIENT_ID_TRACE_DISABLE('CREATE_INDEX');
DBMS_MONITOR.CLIENT_ID_STAT_DISABLE('CREATE_INDEX');
END;
/
References
Oracle 11GR2 document 21.4.4.5 Understanding Recursive Calls
Sometimes, to execute a SQL statement issued by a user, Oracle Database must issue additional statements. ... If recursive calls occur while the SQL Trace facility is enabled, then TKPROF produces statistics for the recursive SQL statements and marks them clearly as recursive SQL statements in the output file.
I recommend the traceanalyzer. You can download it from oracle support.
TRCANLZR (TRCA): SQL_TRACE/Event 10046 Trace File Analyzer - Tool for Interpreting Raw SQL Traces (Doc ID 224270.1)
The trace analyzer features a chapter "SQL Genealogy" which shows a tree like view of recursive SQL.
Once the traceanalyzer is installed, you invoke it with
#trcanlzr <name of tracefile>
You can even do this remotely from you client (copy the trcanlzr.sql script to your local SQLPATH). It will eventually copy the html-file containing the analysis to your client machine.
Based on the information from #Jan, run the TRCA. I understand that SQL Genealogy is the call hierarchy and breakdown, but please provide any suggestions/corrections.
SQL
SET AUTOTRACE OFF
SET ECHO ON
SET TIMING OFF
DROP TABLE TBL2 PURGE;
CREATE TABLE TBL2 (
"COL1" NUMBER,
"COL2" VARCHAR2(100 BYTE)
-- CONSTRAINT "PK_TBL2_COL1" PRIMARY KEY ("COL1")
);
INSERT INTO TBL2 /*+ APPEND MONITOR GATHER_PLAN_STATISTICS CONNECTBY_INSERT */
SELECT LEVEL, rpad(TO_CHAR(LEVEL),100,'A') FROM DUAL CONNECT BY LEVEL <= 1000000;
COMMIT;
---------------------------------------------------------------------------
-- Start monitoring
---------------------------------------------------------------------------
ALTER SESSION SET TRACEFILE_IDENTIFIER ='CREATE_INDEX';
ALTER SESSION SET TIMED_STATISTICS=TRUE;
alter session set events '10046 trace name context forever, level 8';
---------------------------------------------------------------------------
-- Run the SQL to insert and monitor
---------------------------------------------------------------------------
ALTER TABLE TBL2 ADD CONSTRAINT PK_TBL2_COL1 PRIMARY KEY(COL1) ;
---------------------------------------------------------------------------
-- Stop monitoring.
---------------------------------------------------------------------------
alter session set events '10046 trace name context off';
TRCA
#run/trcanlzr.sql nr_ora_6976_CREATE_INDEX.trc
TOP SQL (CPU)
SQL Genealogy
I use orasrp for analyze trace files. Its generate html-reports with hierarchy plans in easy to read form.
Also its easy to use (in console) -
orasrp in_trace.trc out_report.html
On a new job I have to figure out how some database reporting scripts are working.
There is one table that is giving me some trouble. I see in existing scripts that it is a partitioned table.
My problem is that whatever query I run on this table returns me "no rows selected".
Here are some details about my investigation in this table:
Table size estimate
SQL> select sum(bytes)/1024/1024 Megabytes from dba_segments where segment_name = 'PPREC';
MEGABYTES
----------
45.625
Partitions
There are a total of 730 partitions on date range.
SQL> select min(PARTITION_NAME),max(PARTITION_NAME) from dba_segments where segment_name = 'PPREC';
MIN(PARTITION_NAME) MAX(PARTITION_NAME)
------------------------------ ------------------------------
PART20110201 PART20130130
There are several tablespaces and partitions are allocated in them
SQL> select tablespace_name, count(partition_name) from dba_segments where segment_name = 'PPREC' group by tablespace_name;
TABLESPACE_NAME COUNT(PARTITION_NAME)
------------------------------ ---------------------
REC_DATA_01 281
REC_DATA_02 48
REC_DATA_03 70
REC_DATA_04 26
REC_DATA_05 44
REC_DATA_06 51
REC_DATA_07 13
REC_DATA_08 48
REC_DATA_09 32
REC_DATA_10 52
REC_DATA_11 35
REC_DATA_12 30
Additional query:
SQL> select * from dba_segments where segment_name='PPREC' and partition_name='PART20120912';
OWNER SEGMENT_NAME PARTITION_NAME SEGMENT_TYPE TABLESPACE_NAME HEADER_FILE HEADER_BLOCK BYTES BLOCKS EXTENTS
----- ------------ -------------- --------------- --------------- ----------- ------------ ----- ------ -------
HIST PPREC PART20120912 TABLE PARTITION REC_DATA_01 13 475315 65536 8 1
INITIAL_EXTENT NEXT_EXTENT MIN_EXTENTS MAX_EXTENTS PCT_INCREASE FREELISTS FREELIST_GROUPS RELATIVE_FNO BUFFER_POOL
-------------- ----------- ----------- ----------- ------------ --------- --------------- ------------ -----------
65536 1 2147483645 13 DEFAULT
Tabespace usage
Here is a space summary (composite of dba_tablespaces, dba_data_files, dba_segments, dba_free_space)
TABLESPACE_NAME TOTAL_MEGABYTES USED_MEGABYTES FREE_MEGABYTES
------------------------------ --------------- -------------- --------------
REC_01_INDX 30,700 250 30,449
REC_02_INDX 7,745 7 7,737
REC_03_INDX 22,692 15 22,677
REC_04_INDX 15,768 10 15,758
REC_05_INDX 25,884 16 25,868
REC_06_INDX 27,992 16 27,975
REC_07_INDX 17,600 10 17,590
REC_08_INDX 18,864 11 18,853
REC_09_INDX 19,700 12 19,687
REC_10_INDX 28,716 16 28,699
REC_DATA_01 102,718 561 102,156
REC_DATA_02 24,544 3,140 21,403
REC_DATA_03 72,710 4 72,704
REC_DATA_04 29,191 2 29,188
REC_DATA_05 42,696 3 42,692
REC_DATA_06 52,780 323 52,456
REC_DATA_07 16,536 1 16,534
REC_DATA_08 49,247 3 49,243
REC_DATA_09 30,848 2 30,845
REC_DATA_10 49,620 3 49,616
REC_DATA_11 40,616 2 40,613
REC_DATA_12 184,922 123,435 61,486
The tablespace usage seems to confirm that this table is not empty, in fact its last tablespace (REC_DATA_12) seems pretty busy.
Existing scripts
What I find puzzling is that there are some PL/SQL stored procedures that seem to work on that table and get data out of it.
An example of such a stored procedure is as follows:
procedure FIRST_REC as
vpartition varchar2(12);
begin
select 'PART'||To_char(sysdate,'YYYYMMDD') INTO vpartition FROM DUAL;
execute immediate
'MERGE INTO FIRST_REC_temp a
USING (SELECT bno, min(trdate) mintr,max(trdate) maxtr
FROM PPREC PARTITION ('||vpartition||') WHERE route_id IS NOT NULL AND trunc(trdate) <= trunc(sysdate-1)
GROUP BY bno) b
ON (a.bno=b.bno)
when matched then
update set a.last_tr = b.maxtr
when not matched then
insert (a.bno,a.last_tr,a.first_tr)
values (b.bno,b.maxtr,b.mintr)';
commit;
However if I try using the same syntax manually on the table, here is what I get:
SQL> select count(*) from PPREC PARTITION (PART20120912);
COUNT(*)
----------
0
I have tried a few random partitions and I always get the same 0 count.
Summary
- I see a table that seems to contain data (space used, tablespaces, data files)
- The table is partitioned (one partition per day over a period of 730 days ending end of January 2013)
- Scripts are extracting data from that table somehow
Question
- My queries using PARTITION are all returning me "no rows selected". What am I doing wrong? How could I find out how to extract data from this table?
I suppose it's possible that some other process might be deleting the data, but without visiting your site there's no way for anyone here to tell if that might be so.
I don't see in your post that you mentioned the name of the partitioning DATE column, but based on the SQL you posted I'll assume it's TRDATE - if this is not correct, change TRDATE in the statement below to be the partitioning column.
That said, give this a try:
SELECT COUNT(*)
FROM PPREC
WHERE TRDATE >= TO_DATE('01-SEP-2012 00:00:00', 'DD-MON-YYYY HH24:MI:SS')
This assumes you should have data in this table from September. If you find data, great. If you don't - well, Back In The Day (when men were men, women were women, and computers were water-cooled :-) we had a little saying about memory on IBM mainframes:
1. If you can see it, and it's there, it's Real.
2. If you can't see it, but it's there, it's Protected.
3. If you can see it, but it's not there, it's Virtual.
4. If you can't see it, and it's not there, it's GONE!
:-)
Use of the PARTITION clause should be reserved for situations where you are experiencing a performance problem (note: guessing about what is or is not going to be a performance problem is not allowed. Until you've got a performance problem you don't have a performance problem. Over the years I've found that software spends a lot of execution time in the darndest places :-), and the usual fixes (adding indexes, deleting unnecessary data, human sacrifice, etc) haven't worked. Basically, write your queries normally and trust the database to get it right. (In the general case - always write the simplest code - and do the simplest thing - that could possibly work. 99+ percent of the time it will be fine. That allows you to spend your optimization time on the less-than-one-percent cases where simple isn't good enough - and most of the software you write or design will be simple and easy to understand).
Share and enjoy.
Here's the tkprof output for a query that's running extremely slowly (WARNING: it's long :-) ):
SELECT mbr_comment_idn, mbr_crt_dt, mbr_data_source, mbr_dol_bl_rmo_ind, mbr_dxcg_ctl_member, mbr_employment_start_dt, mbr_employment_term_dt, mbr_entity_active, mbr_ethnicity_idn, mbr_general_health_status_code, mbr_hand_dominant_code, mbr_hgt_feet, mbr_hgt_inches, mbr_highest_edu_level, mbr_insd_addr_idn, mbr_insd_alt_id, mbr_insd_name, mbr_insd_ssn_tin, mbr_is_smoker, mbr_is_vip, mbr_lmbr_first_name, mbr_lmbr_last_name, mbr_marital_status_cd, mbr_mbr_birth_dt, mbr_mbr_death_dt, mbr_mbr_expired, mbr_mbr_first_name, mbr_mbr_gender_cd, mbr_mbr_idn, mbr_mbr_ins_type, mbr_mbr_isreadonly, mbr_mbr_last_name, mbr_mbr_middle_name, mbr_mbr_name, mbr_mbr_status_idn, mbr_mpi_id, mbr_preferred_am_pm, mbr_preferred_time, mbr_prv_innetwork, mbr_rep_addr_idn, mbr_rep_name, mbr_rp_mbr_id, mbr_same_mbr_ins, mbr_special_needs_cd, mbr_timezone, mbr_upd_dt, mbr_user_idn, mbr_wgt, mbr_work_status_idn
FROM (SELECT /*+ FIRST_ROWS(1) */ mbr_comment_idn, mbr_crt_dt, mbr_data_source, mbr_dol_bl_rmo_ind, mbr_dxcg_ctl_member, mbr_employment_start_dt, mbr_employment_term_dt, mbr_entity_active, mbr_ethnicity_idn, mbr_general_health_status_code, mbr_hand_dominant_code, mbr_hgt_feet, mbr_hgt_inches, mbr_highest_edu_level, mbr_insd_addr_idn, mbr_insd_alt_id, mbr_insd_name, mbr_insd_ssn_tin, mbr_is_smoker, mbr_is_vip, mbr_lmbr_first_name, mbr_lmbr_last_name, mbr_marital_status_cd, mbr_mbr_birth_dt, mbr_mbr_death_dt, mbr_mbr_expired, mbr_mbr_first_name, mbr_mbr_gender_cd, mbr_mbr_idn, mbr_mbr_ins_type, mbr_mbr_isreadonly, mbr_mbr_last_name, mbr_mbr_middle_name, mbr_mbr_name, mbr_mbr_status_idn, mbr_mpi_id, mbr_preferred_am_pm, mbr_preferred_time, mbr_prv_innetwork, mbr_rep_addr_idn, mbr_rep_name, mbr_rp_mbr_id, mbr_same_mbr_ins, mbr_special_needs_cd, mbr_timezone, mbr_upd_dt, mbr_user_idn, mbr_wgt, mbr_work_status_idn, ROWNUM AS ora_rn
FROM (SELECT mbr.comment_idn AS mbr_comment_idn, mbr.crt_dt AS mbr_crt_dt, mbr.data_source AS mbr_data_source, mbr.dol_bl_rmo_ind AS mbr_dol_bl_rmo_ind, mbr.dxcg_ctl_member AS mbr_dxcg_ctl_member, mbr.employment_start_dt AS mbr_employment_start_dt, mbr.employment_term_dt AS mbr_employment_term_dt, mbr.entity_active AS mbr_entity_active, mbr.ethnicity_idn AS mbr_ethnicity_idn, mbr.general_health_status_code AS mbr_general_health_status_code, mbr.hand_dominant_code AS mbr_hand_dominant_code, mbr.hgt_feet AS mbr_hgt_feet, mbr.hgt_inches AS mbr_hgt_inches, mbr.highest_edu_level AS mbr_highest_edu_level, mbr.insd_addr_idn AS mbr_insd_addr_idn, mbr.insd_alt_id AS mbr_insd_alt_id, mbr.insd_name AS mbr_insd_name, mbr.insd_ssn_tin AS mbr_insd_ssn_tin, mbr.is_smoker AS mbr_is_smoker, mbr.is_vip AS mbr_is_vip, mbr.lmbr_first_name AS mbr_lmbr_first_name, mbr.lmbr_last_name AS mbr_lmbr_last_name, mbr.marital_status_cd AS mbr_marital_status_cd, mbr.mbr_birth_dt AS mbr_mbr_birth_dt, mbr.mbr_death_dt AS mbr_mbr_death_dt, mbr.mbr_expired AS mbr_mbr_expired, mbr.mbr_first_name AS mbr_mbr_first_name, mbr.mbr_gender_cd AS mbr_mbr_gender_cd, mbr.mbr_idn AS mbr_mbr_idn, mbr.mbr_ins_type AS mbr_mbr_ins_type, mbr.mbr_isreadonly AS mbr_mbr_isreadonly, mbr.mbr_last_name AS mbr_mbr_last_name, mbr.mbr_middle_name AS mbr_mbr_middle_name, mbr.mbr_name AS mbr_mbr_name, mbr.mbr_status_idn AS mbr_mbr_status_idn, mbr.mpi_id AS mbr_mpi_id, mbr.preferred_am_pm AS mbr_preferred_am_pm, mbr.preferred_time AS mbr_preferred_time, mbr.prv_innetwork AS mbr_prv_innetwork, mbr.rep_addr_idn AS mbr_rep_addr_idn, mbr.rep_name AS mbr_rep_name, mbr.rp_mbr_id AS mbr_rp_mbr_id, mbr.same_mbr_ins AS mbr_same_mbr_ins, mbr.special_needs_cd AS mbr_special_needs_cd, mbr.timezone AS mbr_timezone, mbr.upd_dt AS mbr_upd_dt, mbr.user_idn AS mbr_user_idn, mbr.wgt AS mbr_wgt, mbr.work_status_idn AS mbr_work_status_idn
FROM mbr JOIN mbr_identfn ON mbr.mbr_idn = mbr_identfn.mbr_idn
WHERE mbr_identfn.mbr_idn = mbr.mbr_idn AND mbr_identfn.identfd_type = :identfd_type_1 AND mbr_identfn.identfd_number = :identfd_number_1 AND mbr_identfn.entity_active = :entity_active_1)
WHERE ROWNUM <= :ROWNUM_1)
WHERE ora_rn > :ora_rn_1
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 9936 0.46 0.49 0 0 0 0
Execute 9936 0.60 0.59 0 0 0 0
Fetch 9936 329.87 404.00 0 136966922 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 29808 330.94 405.09 0 136966922 0 0
Misses in library cache during parse: 0
Optimizer mode: FIRST_ROWS
Parsing user id: 36 (JIVA_DEV)
Rows Row Source Operation
------- ---------------------------------------------------
0 VIEW (cr=102 pr=0 pw=0 time=2180 us)
0 COUNT STOPKEY (cr=102 pr=0 pw=0 time=2163 us)
0 NESTED LOOPS (cr=102 pr=0 pw=0 time=2152 us)
0 INDEX SKIP SCAN IDX_MBR_IDENTFN (cr=102 pr=0 pw=0 time=2140 us)(object id 341053)
0 TABLE ACCESS BY INDEX ROWID MBR (cr=0 pr=0 pw=0 time=0 us)
0 INDEX UNIQUE SCAN PK_CLAIMANT (cr=0 pr=0 pw=0 time=0 us)(object id 334044)
Rows Execution Plan
------- ---------------------------------------------------
0 SELECT STATEMENT MODE: HINT: FIRST_ROWS
0 VIEW
0 COUNT (STOPKEY)
0 NESTED LOOPS
0 INDEX MODE: ANALYZED (SKIP SCAN) OF 'IDX_MBR_IDENTFN'
(INDEX (UNIQUE))
0 TABLE ACCESS MODE: ANALYZED (BY INDEX ROWID) OF 'MBR'
(TABLE)
0 INDEX MODE: ANALYZED (UNIQUE SCAN) OF 'PK_CLAIMANT'
(INDEX (UNIQUE))
********************************************************************************
Based on my reading of Oracle's documentation of skip scans, a skip scan is most useful when the first column of an index has a low number of unique values. The thing is that the first index of this column has a high number of uniques. So am I correct in assuming that a skip scan is the wrong thing to do here? Also, what kind of scan should it be doing? Should I do some more hinting for this query?
EDIT: I should also point out that the query's where clause uses the columns in IDX_MBR_IDENTFN and no columns other than what's in that index. So as far as I can tell, I'm not skipping any columns.
EDIT 2: I've done a few things to speed this query up. First of all, I removed the paging. As it turns out, this query only returns one row anyway. Secondly, I added a LEADING hint to make sure tables were being queried in the right order. Thirdly, I removed the duplicate mbr_idn predicate. Lastly, I made IDX_MBR_IDENTFN unique. Altogether, this makes a drastic performance improvement (although it's still the most expensive query I'm running):
SELECT /*+ LEADING (mbr_identfn, mbr) */ mbr.comment_idn AS mbr_comment_idn, mbr.crt_dt AS mbr_crt_dt, mbr.data_source AS mbr_data_source, mbr.dol_bl_rmo_ind AS mbr_dol_bl_rmo_ind, mbr.dxcg_ctl_member AS mbr_dxcg_ctl_member, mbr.employment_start_dt AS mbr_employment_start_dt, mbr.employment_term_dt AS mbr_employment_term_dt, mbr.entity_active AS mbr_entity_active, mbr.ethnicity_idn AS mbr_ethnicity_idn, mbr.general_health_status_code AS mbr_general_health_status_code, mbr.hand_dominant_code AS mbr_hand_dominant_code, mbr.hgt_feet AS mbr_hgt_feet, mbr.hgt_inches AS mbr_hgt_inches, mbr.highest_edu_level AS mbr_highest_edu_level, mbr.insd_addr_idn AS mbr_insd_addr_idn, mbr.insd_alt_id AS mbr_insd_alt_id, mbr.insd_name AS mbr_insd_name, mbr.insd_ssn_tin AS mbr_insd_ssn_tin, mbr.is_smoker AS mbr_is_smoker, mbr.is_vip AS mbr_is_vip, mbr.lmbr_first_name AS mbr_lmbr_first_name, mbr.lmbr_last_name AS mbr_lmbr_last_name, mbr.marital_status_cd AS mbr_marital_status_cd, mbr.mbr_birth_dt AS mbr_mbr_birth_dt, mbr.mbr_death_dt AS mbr_mbr_death_dt, mbr.mbr_expired AS mbr_mbr_expired, mbr.mbr_first_name AS mbr_mbr_first_name, mbr.mbr_gender_cd AS mbr_mbr_gender_cd, mbr.mbr_idn AS mbr_mbr_idn, mbr.mbr_ins_type AS mbr_mbr_ins_type, mbr.mbr_isreadonly AS mbr_mbr_isreadonly, mbr.mbr_last_name AS mbr_mbr_last_name, mbr.mbr_middle_name AS mbr_mbr_middle_name, mbr.mbr_name AS mbr_mbr_name, mbr.mbr_status_idn AS mbr_mbr_status_idn, mbr.mpi_id AS mbr_mpi_id, mbr.preferred_am_pm AS mbr_preferred_am_pm, mbr.preferred_time AS mbr_preferred_time, mbr.prv_innetwork AS mbr_prv_innetwork, mbr.rep_addr_idn AS mbr_rep_addr_idn, mbr.rep_name AS mbr_rep_name, mbr.rp_mbr_id AS mbr_rp_mbr_id, mbr.same_mbr_ins AS mbr_same_mbr_ins, mbr.special_needs_cd AS mbr_special_needs_cd, mbr.timezone AS mbr_timezone, mbr.upd_dt AS mbr_upd_dt, mbr.user_idn AS mbr_user_idn, mbr.wgt AS mbr_wgt, mbr.work_status_idn AS mbr_work_status_idn
FROM mbr JOIN mbr_identfn ON mbr.mbr_idn = mbr_identfn.mbr_idn
WHERE mbr_identfn.identfd_type = :identfd_type_1 AND mbr_identfn.identfd_number = :identfd_number_1 AND mbr_identfn.entity_active = :entity_active_1
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 10102 0.45 0.42 0 0 0 0
Execute 10102 0.44 0.52 0 0 0 0
Fetch 10102 1.60 1.81 0 218121 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 30306 2.50 2.75 0 218121 0 0
Misses in library cache during parse: 0
Optimizer mode: ALL_ROWS
Parsing user id: 36 (JIVA_DEV)
Rows Row Source Operation
------- ---------------------------------------------------
0 NESTED LOOPS (cr=3 pr=0 pw=0 time=96 us)
0 TABLE ACCESS BY INDEX ROWID MBR_IDENTFN (cr=3 pr=0 pw=0 time=88 us)
0 INDEX UNIQUE SCAN UK_CLM_IDFN (cr=3 pr=0 pw=0 time=77 us)(object id 334118)
0 TABLE ACCESS BY INDEX ROWID MBR (cr=0 pr=0 pw=0 time=0 us)
0 INDEX UNIQUE SCAN PK_CLAIMANT (cr=0 pr=0 pw=0 time=0 us)(object id 334044)
Rows Execution Plan
------- ---------------------------------------------------
0 SELECT STATEMENT MODE: ALL_ROWS
0 NESTED LOOPS
0 TABLE ACCESS MODE: ANALYZED (BY INDEX ROWID) OF
'MBR_IDENTFN' (TABLE)
0 INDEX MODE: ANALYZED (UNIQUE SCAN) OF 'UK_CLM_IDFN' (INDEX
(UNIQUE))
0 TABLE ACCESS MODE: ANALYZED (BY INDEX ROWID) OF 'MBR' (TABLE)
0 INDEX MODE: ANALYZED (UNIQUE SCAN) OF 'PK_CLAIMANT' (INDEX
(UNIQUE))
Index skip scan means, that the first column of the index is ignored. This costs performance since Oracle has read every item of the first column, and check if the second (or third, ...) column is what you searched for. This usually is faster than a full-table scan (depends on your query), but slower than a index range scan.
Try to create a separate index on the column that is part of IDX_MBR_IDENTFN and used in your query.
For example, if your_table looks like this:
id status
1 0
2 0
3 0
4 1
and you have a compound index on (id, status), the query Select * From your_table Where status = 1 is likely to use the index, but in order to find the correct rows, it has to read every row of the index (id 1 to 4) and check the status.
Update: The following index could improve performance a little further, but you will have to try if it really helps:
mbr_identfn( identfd_type, identfd_number, entity_active, mbr_idn )
This could even help to avoid the hint.
I would shift my focus away from the skip scan.
The tkprof snippet shows you that your first priority should be to reduce the number of times you issue this statement. Currently you are executing this statement 9936 times. And each execution takes only 405/9936 seconds. Reasonably fast. But not if you execute it 9936 times.
So this statement is almost sure inside a loop construct. In each iteration you provide a different input parameter set(:identfd_type_1, :identfd_number_1, :entity_active_1, :ROWNUM_1, :ora_rn_1). Rewrite this loop construct to have this statement execute once for the entire set, and your performance problem will probably a thing from the past. If not, please post the new tkprof output.
Regards,
Rob.
It would help if you identified what the columns in the indexes (PK_CLAIMANT and IDX_MBR_IDENTFN) are, and in what order.
I suspect it is a datetype issue. If, for example, mbr_identfn.identfd_type is the leading column of the index and is numeric, but your :identfd_type_1 is character variable (or vice versa) it becomes unusable. However if there are few types, then the index can be used with a skip scan.
You also specify the predicate "mbr.mbr_idn = mbr_identfn.mbr_idn" in both the where clause and the join clause.
To explain the skip scan ... this appears to be the relevant predicate portion of the query:
WHERE mbr_identfn.mbr_idn = mbr.mbr_idn
AND mbr_identfn.identfd_type = :identfd_type_1
AND mbr_identfn.identfd_number = :identfd_number_1
AND mbr_identfn.entity_active = :entity_active_1
If the execution start with MBR_IDENTFN, then we don't yet have a value for MBR_IDN to look up in the index; this means we can't do a unique or range scan. But we have values given (as bind variables) for the other three columns of the index, so we can do a skip scan. Oracle is choosing to do this in order to avoid accessing the base table at all, which seems sensible.
What is the primary key of MBR_IDENTFN? Is it MBR_IDN alone?
I am thinking you should have a separate index on MBR_IDENTFN with some or all of IDENTFD_TYPE, IDENTFD_NUMBER, and ENTITY_ACTIVE as the leading columns. This would allow a range or unique scan to be done instead of a skip scan.