My table MYTABLE contains 14 million record data
CREATE MYTABLE `MYTABLE` (
`COLUM1` int(10) unsigned NOT NULL DEFAULT '0',
`COLUM2` int(10) unsigned NOT NULL DEFAULT '0',
`COLUM4` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`COLUM3` char(1) NOT NULL,
UNIQUE KEY `COLUM2` (`COLUM2`,`COLUM1`),
KEY `COLUM1` (`COLUM1`)
) ENGINE=InnoDB
show processlist at a given time
| 318 1204333 | user | xx.xxx.xx.xxx:24396 | newjs | Query | 0 | query end | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('11285294','9765238',now()) |
| 318 1204334 | user | xx.xxx.xx.xxx:38671 | newjs | Query | 0 | Updating | UPCOLUM4 newjs.MYTABLE SET COLUM3='Y' WHERE COLUM1=11200140 AND COLUM2=10927439 |
| 318 1204335 | user | xx.xxx.xx.xxx:35259 | newjs | Query | 0 | query end | UPCOLUM4 newjs.MYTABLE SET COLUM3='Y' WHERE COLUM1=8667546 AND COLUM2=9731954 |
| 318 1204337 | user | xx.xxx.xx.xxx:24405 | newjs | Sleep | 0 | | NULL |
| 318 1204339 | user | xx.xxx.xx.xxx:24412 | newjs | Query | 0 | query end | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('11267957','11250673',now()) |
| 318 1204342 | user | xx.xxx.xx.xxx:35287 | newjs | Query | 0 | query end | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('11229209','9873622',now()) |
| 318 1204343 | user | xx.xxx.xx.xxx:38681 | newjs | Query | 0 | query end | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('10157460','10967184',now()) |
| 318 1204347 | user | xx.xxx.xx.xxx:24424 | newjs | Query | 0 | query end | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('11268751','9423745',now()) |
| 318 1204348 | user | xx.xxx.xx.xxx:35316 | newjs | Query | 0 | query end | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('11268611','9263111',now()) |
| 318 1204349 | user | xx.xxx.xx.xxx:35319 | newjs | Sleep | 0 | | NULL |
| 318 1204350 | user | xx.xxx.xx.xxx:35345 | newjs | Query | 0 | query end | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('9049736','9585434',now()) |
| 318 1204352 | user | xx.xxx.xx.xxx:35375 | newjs | Sleep | 0 | | NULL |
| 318 1204353 | user | xx.xxx.xx.xxx:35376 | newjs | Query | 0 | uptime | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('10578170','9843201',now()) |
| 318 1204354 | user | xx.xxx.xx.xxx:35379 | newjs | Query | 0 | query end | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('8445209','11055214',now()) |
| 318 1204355 | user | xx.xxx.xx.xxx:24485 | newjs | Query | 0 | uptime | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('9699909','11236269',now()) |
| 318 1204357 | user | xx.xxx.xx.xxx:35396 | newjs | Sleep | 0 | | NULL |
| 318 1204359 | user | xx.xxx.xx.xxx:24531 | newjs | Query | 0 | query end | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('8925641','7987794',now()) |
| 318 1204360 | user | xx.xxx.xx.xxx:24550 | newjs | Query | 0 | query end | REPLACE INTO MYTABLE (COLUM1,COLUM2,COLUM4) VALUES ('11242379','9740270',now()) |
| 318 1204362 | user | xx.xxx.xx.xxx:35464 | newjs | Sleep | 0 |
What is this query end ?
"The "END" connection state is used for writing binary log records, query cache operations, and other things.." MySQL
"Probably the most common reason for a thread to stay with either killed or query end for a longer period of time is waiting for a transaction rollback on InnoDB tables. This sometimes can take a lot of time to complete, especially when hundreds of thousands or millions of changes have to be removed"
"Actually, there can be two different cases. If KILL was issued, there would be killed in the process list. KILL QUERY doesn’t kill a connection, but rather it only stops a running query within a connection, so in that case query end text may appear instead." psce
Many people reporting that they have performance issues with "The End"-State.
Fixes:
Update to a newer Version of MySQL
reduce cache
Related
I have problem with Apache NiFi 1.12.1. For some unknown for me reason CaptureChangeMySQL returns many nulls. Basically, only columns which are int, return correct values. I'm new in a matter of using NiFi so I might miss some obvious thing in configuration.
I have following table:
create table inventory.abc
(
id int auto_increment
primary key,
first_name varchar(100) not null,
last_name varchar(100) not null,
age int not null
);
Processor config:
Bin logs settings:
mysql> show variables like '%bin%';
+--------------------------------------------+--------------------------------+
| Variable_name | Value |
+--------------------------------------------+--------------------------------+
| bind_address | * |
| binlog_cache_size | 32768 |
| binlog_checksum | CRC32 |
| binlog_direct_non_transactional_updates | OFF |
| binlog_error_action | ABORT_SERVER |
| binlog_format | ROW |
| binlog_group_commit_sync_delay | 0 |
| binlog_group_commit_sync_no_delay_count | 0 |
| binlog_gtid_simple_recovery | ON |
| binlog_max_flush_queue_time | 0 |
| binlog_order_commits | ON |
| binlog_row_image | FULL |
| binlog_rows_query_log_events | OFF |
| binlog_stmt_cache_size | 32768 |
| binlog_transaction_dependency_history_size | 25000 |
| binlog_transaction_dependency_tracking | COMMIT_ORDER |
| innodb_api_enable_binlog | OFF |
| innodb_locks_unsafe_for_binlog | OFF |
| log_bin | ON |
| log_bin_basename | /var/lib/mysql/mysql-bin |
| log_bin_index | /var/lib/mysql/mysql-bin.index |
| log_bin_trust_function_creators | OFF |
| log_bin_use_v1_row_events | OFF |
| log_statements_unsafe_for_binlog | ON |
| max_binlog_cache_size | 18446744073709547520 |
| max_binlog_size | 1073741824 |
| max_binlog_stmt_cache_size | 18446744073709547520 |
| sql_log_bin | ON |
| sync_binlog | 1 |
+--------------------------------------------+--------------------------------+
29 rows in set (0.00 sec)
And I get results like this:
Any idea why I get so many nulls in output? I thought it might be related to Distributed Map Cache Client but since this option is not mandatory I don't think that's a problem.
I'm newbie with proxysql, this my enviroment:
Centos 7 , proxysql-1.4.13 . I define 2 host group ids : host group id 2 for mysql server that can write , host group id 3 for mysql server that can read.
All queries that begin with insert, update, delete, alter should be routed to host group id 2.
All queries that begin with select should be routed to host group id 3.
So here are my rules :
Admin> select rule_id,active,digest,match_digest,destination_hostgroup,flagIN,flagOUT,next_query_flagIN,sticky_conn,apply from mysql_query_rules;
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
| rule_id | active | digest | match_digest | destination_hostgroup | flagIN | flagOUT | next_query_flagIN | sticky_conn | apply |
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
| 101 | 1 | NULL | ^insert | 2 | 0 | NULL | NULL | NULL | 1 |
| 102 | 1 | NULL | ^update | 2 | 0 | NULL | NULL | NULL | 1 |
| 103 | 1 | NULL | ^delete | 2 | 0 | NULL | NULL | NULL | 1 |
| 104 | 1 | NULL | ^alter | 2 | 0 | NULL | NULL | NULL | 1 |
| 105 | 1 | NULL | ^select | 3 | 0 | NULL | NULL | NULL | 1 |
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
And they works fine. By the way, are these rules ok ? Should I replace match_digest by match_pattern ?
However , my application has 1 feature that insert data into a table (create booking) then select (almost immediately) new data from that table .
So if select query fails (because new data has not replicated to host group id 3 yet), application feature will run wrong.
I want to route the select query right after insert query to host group id 2 so it will get new data successfully.
I read proxysql document https://github.com/sysown/proxysql/wiki/Main-(runtime)#mysql_query_rules and this discussion https://github.com/sysown/proxysql/pull/825 , I think this is solution for me ,isn't it ? I still don't understand about these flagIN,flagOUT,next_query_flagIN,sticky_conn stuff clearly but I will give it a try.
I know the insert query digest is 0xCDD6DB677604AFA7
The select query digest is 0x0DCD2E8ADF6A66CB
Then I add 2 new rules:
Admin> select rule_id,active,digest,match_digest,destination_hostgroup,flagIN,flagOUT,next_query_flagIN,sticky_conn,apply from mysql_query_rules;
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
| rule_id | active | digest | match_digest | destination_hostgroup | flagIN | flagOUT | next_query_flagIN | sticky_conn | apply |
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
| 1 | 1 | 0xCDD6DB677604AFA7 | NULL | 2 | 0 | NULL | 1 | 1 | 1 |
| 2 | 1 | 0x0DCD2E8ADF6A66CB | NULL | 2 | 1 | NULL | NULL | NULL | 1 |
| 101 | 1 | NULL | ^insert | 2 | 0 | NULL | NULL | NULL | 1 |
| 102 | 1 | NULL | ^update | 2 | 0 | NULL | NULL | NULL | 1 |
| 103 | 1 | NULL | ^delete | 2 | 0 | NULL | NULL | NULL | 1 |
| 104 | 1 | NULL | ^alter | 2 | 0 | NULL | NULL | NULL | 1 |
| 105 | 1 | NULL | ^select | 3 | 0 | NULL | NULL | NULL | 1 |
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
They work fine, the select query right after insert query is route to host group id 2 so it gets new data successfully and application feature runs ok.
But am I doing right ?
I'm confused because stats_mysql_query_rules result:
Before application feature run:
Admin> select * from stats_mysql_query_rules;
+---------+------+
| rule_id | hits |
+---------+------+
| 1 | 20 |
| 2 | 20 |
| 101 | 33 |
| 102 | 0 |
| 103 | 2 |
| 104 | 0 |
| 105 | 903 |
+---------+------+
After application feature run (10 bookings):
Admin> select * from stats_mysql_query_rules;
+---------+------+
| rule_id | hits |
+---------+------+
| 1 | 30 |
| 2 | 30 |
| 101 | 43 |
| 102 | 0 |
| 103 | 2 |
| 104 | 0 |
| 105 | 1313 |
+---------+------+
Why rule_id 101 hits rate increase from 33 --> 43 ? So rule_id 101 (match_digest ^insert) also match insert query in application feature ? Does it mean I'm doing wrong ?
Intro
I am writing a query to compare 2 different oracle database tables containing hashes.
I want to find which hashes from location 1, have not been migrated over to location 2.
I have three different queries, and have written explain plan for statements for them.
However, the results don't tell me that much.
Question
How do I find which is the most efficient and fastest?
Current Guess
My suspicion however is that the first query is the fastest since it makes a one-off use of the remote link. But this is just a guess that is not supported by actual results.
Code
--------------------------
EXPLAIN PLAN
SET statement_id = 'ex_plan1' FOR
select* from document doc left outer join migrated_document#V2_PROD migrated on doc.hash = migrated.document_hash AND migrated.document_hash is null ;
SELECT PLAN_TABLE_OUTPUT
FROM TABLE(DBMS_XPLAN.DISPLAY(NULL, 'ex_plan1','BASIC'));
---------------------------------------------------
| Id | Operation | Name |
---------------------------------------------------
| 0 | SELECT STATEMENT | |
| 1 | HASH JOIN RIGHT OUTER| |
| 2 | REMOTE | MIGRATED_DOCUMENT |
| 3 | TABLE ACCESS FULL | DOCUMENT |
---------------------------------------------------
--------------------------
EXPLAIN PLAN
SET statement_id = 'ex_plan2' FOR
select* from document doc where not exists( select 1 from migrated_document#V2_PROD migrated where migrated.document_hash = doc.HASH ) ;
SELECT PLAN_TABLE_OUTPUT
FROM TABLE(DBMS_XPLAN.DISPLAY(NULL, 'ex_plan2','BASIC'));
------------------------------------------------
| Id | Operation | Name |
------------------------------------------------
| 0 | SELECT STATEMENT | |
| 1 | FILTER | |
| 2 | TABLE ACCESS FULL| DOCUMENT |
| 3 | REMOTE | MIGRATED_DOCUMENT |
------------------------------------------------
--------------------------
EXPLAIN PLAN
SET statement_id = 'ex_plan3' FOR
select* from document doc where doc.hash not in ( select migrated.document_hash from migrated_document#V2_PROD migrated) ;
SELECT PLAN_TABLE_OUTPUT
FROM TABLE(DBMS_XPLAN.DISPLAY(NULL, 'ex_plan3','BASIC'));
------------------------------------------------
| Id | Operation | Name |
------------------------------------------------
| 0 | SELECT STATEMENT | |
| 1 | NESTED LOOPS ANTI | |
| 2 | TABLE ACCESS FULL| DOCUMENT |
| 3 | REMOTE | MIGRATED_DOCUMENT |
------------------------------------------------
--------------------------
Update
I updated the explain plan statements to get more results.
Due to the remote operation... could something cost less but be more slow?
If I am reading the data correctly it seems that option 2 is best.
But I still think option 1 is quicker.
-------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost | Inst |IN-OUT|
-------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 105K| 51M| 194 | | |
| 1 | HASH JOIN RIGHT OUTER| | 105K| 51M| 194 | | |
| 2 | REMOTE | MIGRATED_DOCUMENT | 1 | 275 | 2 | V2_MN~ | R->S |
| 3 | TABLE ACCESS FULL | DOCUMENT | 105K| 23M| 192 | | |
-------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost | Inst |IN-OUT|
----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 105K| 23M| 104K| | |
| 1 | FILTER | | | | | | |
| 2 | TABLE ACCESS FULL| DOCUMENT | 105K| 23M| 192 | | |
| 3 | REMOTE | MIGRATED_DOCUMENT | 1 | 50 | 1 | V2_MN~ | R->S |
----------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost | Inst |IN-OUT|
----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 105K| 29M| 526 | | |
| 1 | NESTED LOOPS ANTI | | 105K| 29M| 526 | | |
| 2 | TABLE ACCESS FULL| DOCUMENT | 105K| 23M| 192 | | |
| 3 | REMOTE | MIGRATED_DOCUMENT | 1 | 50 | 0 | V2_MN~ | R->S |
----------------------------------------------------------------------------------------
In a perfect world, you would choose the plan with the lowest cost. However I do not think your first query does what you want. It looks to me like no rows will join; you should be using a filter predicate rather than a join.
Instead of
select * from document doc
left outer join migrated_document#V2_PROD migrated
on doc.hash = migrated.document_hash
AND migrated.document_hash is null
it should be
select * from document doc
left outer join migrated_document#V2_PROD migrated
on doc.hash = migrated.document_hash
WHERE migrated.document_hash is null
I have a query which is a UNION query which joins multiple tables. All the stats are updated. The query used to run fine and completed within 30-40 seconds but in 12c it's taking insane amount of time(~10 mins to 5 mins)
I have captured the actual plan.
lan hash value: 1038754298
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads | OMem | 1Mem | O/1/M |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 50 |00:00:49.93 | 960K| 7 | | | |
| 1 | SORT ORDER BY | | 1 | 1791 | 50 |00:00:49.93 | 960K| 7 | 267K| 267K| 1/0/0|
| 2 | UNION-ALL | | 1 | | 2648 |00:00:49.93 | 960K| 7 | | | |
| 3 | NESTED LOOPS SEMI | | 1 | 1 | 2648 |00:00:42.36 | 498K| 0 | | | |
|* 4 | HASH JOIN | | 1 | 1589 | 9111 |00:00:42.22 | 469K| 0 | 7900K| 2299K| 1/0/0|
|* 5 | HASH JOIN | | 1 | 1140K| 50569 |00:00:19.50 | 342K| 0 | 8321K| 2379K| 1/0/0|
|* 6 | HASH JOIN | | 1 | 44335 | 59660 |00:00:16.01 | 252K| 0 | 1817K| 1817K| 1/0/0|
|* 7 | TABLE ACCESS FULL | PROMHEAD | 1 | 870 | 870 |00:00:00.01 | 602 | 0 | | | |
|* 8 | HASH JOIN | | 1 | 1168K| 172K|00:00:15.94 | 251K| 0 | 1148K| 1148K| 1/0/0|
|* 9 | TABLE ACCESS FULL | PRICE_BATCH_TRAN | 1 | 857 | 857 |00:00:00.01 | 315 | 0 | | | |
| 10 | TABLE ACCESS FULL | PROMSTORE | 1 | 36M| 36M|00:00:04.64 | 251K| 0 | | | |
|* 11 | TABLE ACCESS FULL | PROMSKU | 1 | 6665K| 6665K|00:00:01.49 | 90224 | 0 | | | |
| 12 | PARTITION RANGE ALL | | 1 | 37M| 37M|00:00:10.72 | 127K| 0 | | | |
| 13 | INDEX FAST FULL SCAN | PK_ITEM_ZONE_PRICE | 24 | 37M| 37M|00:00:04.75 | 127K| 0 | | | |
| 14 | PARTITION RANGE ITERATOR | | 9111 | 32376 | 2648 |00:00:00.13 | 28752 | 0 | | | |
| 15 | PARTITION HASH ITERATOR | | 9111 | 32376 | 2648 |00:00:00.11 | 28752 | 0 | | | |
|* 16 | TABLE ACCESS BY LOCAL INDEX ROWID BATCHED | FDT_PRICE_FUTURE | 9111 | 32376 | 2648 |00:00:00.09 | 28752 | 0 | | | |
|* 17 | INDEX RANGE SCAN | FDT_PRICE_FUTURE_PK | 9111 | 1 | 2668 |00:00:00.07 | 26089 | 0 | | | |
|* 18 | HASH JOIN | | 1 | 1790 | 0 |00:00:07.31 | 450K| 7 | 1599K| 1599K| 1/0/0|
| 19 | TABLE ACCESS FULL | PRICE_SUSP_HEAD | 1 | 2056 | 2056 |00:00:00.01 | 965 | 0 | | | |
|* 20 | HASH JOIN | | 1 | 1912 | 0 |00:00:07.30 | 449K| 7 | 39M| 8299K| 1/0/0|
| 21 | PART JOIN FILTER CREATE | :BF0000 | 1 | 745K| 745K|00:00:00.12 | 1682 | 0 | | | |
| 22 | TABLE ACCESS FULL | PRICE_ZONE_GROUP_STORE | 1 | 745K| 745K|00:00:00.03 | 1682 | 0 | | | |
|* 23 | HASH JOIN | | 1 | 496K| 0 |00:00:07.03 | 448K| 7 | 73M| 6638K| 1/0/0|
| 24 | PARTITION RANGE ALL | | 1 | 1138K| 1138K|00:00:01.70 | 154K| 7 | | | |
| 25 | PARTITION HASH ALL | | 26 | 1138K| 1138K|00:00:01.53 | 154K| 7 | | | |
| 26 | TABLE ACCESS BY LOCAL INDEX ROWID BATCHED| FDT_PRICE_FUTURE | 208 | 1138K| 1138K|00:00:01.35 | 154K| 7 | | | |
|* 27 | INDEX SKIP SCAN | FDT_PRICE_FUTURE_I1 | 208 | 1138K| 1138K|00:00:00.89 | 125K| 7 | | | |
|* 28 | HASH JOIN | | 1 | 25M| 0 |00:00:04.84 | 293K| 0 | 977K| 977K| 1/0/0|
|* 29 | HASH JOIN | | 1 | 252K| 1123 |00:00:03.33 | 90826 | 0 | 1344K| 1344K| 1/0/0|
|* 30 | TABLE ACCESS FULL | PROMHEAD | 1 | 870 | 870 |00:00:00.01 | 602 | 0 | | | |
|* 31 | TABLE ACCESS FULL | PROMSKU | 1 | 6665K| 6665K|00:00:01.47 | 90224 | 0 | | | |
| 32 | PARTITION RANGE JOIN-FILTER | | 1 | 6719K| 32778 |00:00:01.51 | 202K| 0 | | | |
|* 33 | TABLE ACCESS FULL | PRICE_SUSP_DETAIL | 24 | 6719K| 32778 |00:00:01.51 | 202K| 0 | | | |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
4 - access("IZP"."ITEM"="PSK"."SKU" AND "IZP"."ZONE_GROUP_ID"="PBT"."ZONE_GROUP_ID" AND "IZP"."ZONE_ID"="PBT"."NEW_ZONE_ID")
5 - access("PS"."PROMOTION"="PSK"."PROMOTION")
6 - access("PH"."PROMOTION"="PS"."PROMOTION")
7 - filter("PH"."STATUS"='A')
8 - access("PS"."STORE"="PBT"."STORE")
filter("PS"."START_DATE">"PBT"."EFFECTIVE_DATE")
9 - filter("PBT"."FDC_STATUS"='A')
11 - filter(("PSK"."CHANGE_TYPE"='MA' OR "PSK"."CHANGE_TYPE"='PC'))
16 - filter("PRICE_MOD_TYPE"='ZC')
17 - access("STORE"="PS"."STORE" AND "SKU"="PSK"."SKU" AND "START_DATE"<"PS"."START_DATE")
18 - access("PSH"."PRICE_CHANGE"="PSD"."PRICE_CHANGE")
filter("FDT"."START_DATE">"PSH"."ACTIVE_DATE")
20 - access("PSD"."ZONE_ID"="PZGS"."ZONE_ID" AND "PSD"."ZONE_GROUP_ID"="PZGS"."ZONE_GROUP_ID" AND "FDT"."STORE"="PZGS"."STORE")
23 - access("PSK"."PROMOTION"="FDT"."PRICE_MOD_NO" AND "PSD"."SKU"="FDT"."SKU")
27 - access("FDT"."PRICE_MOD_TYPE"='PR')
filter("FDT"."PRICE_MOD_TYPE"='PR')
28 - access("PSK"."SKU"="PSD"."SKU")
29 - access("PH"."PROMOTION"="PSK"."PROMOTION")
30 - filter("PH"."STATUS"='A')
31 - filter(("PSK"."CHANGE_TYPE"='MA' OR "PSK"."CHANGE_TYPE"='PC'))
33 - filter("PSD"."STATUS"='A')
Note
-----
- this is an adaptive plan
As you can see Oracle predicts the cardinality correctly in most cases. Can you suggest where should I start looking from.
SELECT
PBT.STORE ,
PSK.SKU ,
0 MULTI_UNITS ,
0 MULTI_UNIT_RETAIL ,
PS.START_DATE PS_START_DATE ,
PBT.EFFECTIVE_DATE PC_ACTIVE_DATE ,
PBT.ZONE_CHANGE PRICE_CHANGE ,
PH.PROMOTION PRICE_MOD_NO ,
'PR' PRICE_TYPE ,
'U' REC_TYPE
FROM PROMHEAD PH ,
PROMSTORE PS ,
PROMSKU PSK ,
PRICE_BATCH_TRAN PBT,
ITEM_ZONE_PRICE IZP
WHERE PH.PROMOTION =PS.PROMOTION
AND PS.PROMOTION =PSK.PROMOTION
AND PBT.FDC_STATUS ='A'
AND PH.STATUS ='A'
AND PS.STORE =PBT.STORE
AND PSK.CHANGE_TYPE IN('MA','PC')
AND IZP.ITEM =PSK.SKU
AND IZP.ZONE_GROUP_ID=PBT.ZONE_GROUP_ID
AND IZP.ZONE_ID =PBT.NEW_ZONE_ID
AND PS.START_DATE > PBT.EFFECTIVE_DATE
AND EXISTS
(
SELECT 1
FROM FDT_PRICE_FUTURE
WHERE SKU =PSK.SKU
AND STORE =PS.STORE
AND START_DATE < PS.START_DATE
AND PRICE_MOD_TYPE = 'ZC'
)
UNION ALL
SELECT FDT.STORE ,
FDT.SKU ,
0 MULTI_UNITS ,
0 MULTI_UNIT_RETAIL ,
FDT.START_DATE PS_START_DATE ,
PSH.ACTIVE_DATE PC_ACTIVE_DATE ,
PSD.PRICE_CHANGE PRICE_CHANGE ,
PH.PROMOTION PRICE_MOD_NO ,
'PR' PRICE_TYPE ,
'U' REC_TYPE
FROM FDT_PRICE_FUTURE FDT ,
PROMHEAD PH ,
PROMSKU PSK ,
PRICE_SUSP_DETAIL PSD,
PRICE_SUSP_HEAD PSH ,
PRICE_ZONE_GROUP_STORE PZGS
WHERE PH.PROMOTION =PSK.PROMOTION
AND PSK.PROMOTION =FDT.PRICE_MOD_NO
AND FDT.PRICE_MOD_TYPE='PR'
AND PH.STATUS ='A'
AND PSH.PRICE_CHANGE =PSD.PRICE_CHANGE
AND PSD.STATUS ='A'
AND PSD.ZONE_ID =PZGS.ZONE_ID
AND PSD.ZONE_GROUP_ID =PZGS.ZONE_GROUP_ID
AND FDT.STORE =PZGS.STORE
AND PSK.SKU =PSD.SKU
AND PSD.SKU =FDT.SKU
AND PSK.CHANGE_TYPE IN('MA','PC')
AND FDT.START_DATE > PSH.ACTIVE_DATE
ORDER BY 1,2,7;
Also, if Diagnostics and Tuning Pack licenses available, you can run the SQL Tuning Advisor on the SQL.
-- SQLTUNE.sql
-- Updated 04-Mar-2015 RDCornejo for spool error
-- takes sql_id,plan_hash_value, start and ent snap_id's as parameters
-- created sql tuning report on c:\temp
set echo off
set feedback off
set term off
set verify off
set head on
-- uncomment for debugging if neeed
-- set echo on term on
set termout on
accept SQL_ID char prompt 'Enter the sql_id to Tune? '
accept hash_value char prompt 'Enter the hash_value to Tune? '
accept BEGIN_SNAP char prompt 'Enter the begin snap_id to Tune? '
accept END_SNAP char prompt 'Enter the end snap_id to Tune? '
-- using bind variables:
variable sql_id_var varchar2(25);
variable plan_hash_value varchar2(25);
variable begin_snap_var varchar2(25);
variable end_snap_var varchar2(25);
variable task_name varchar2(100);
exec :sql_id_var := '&SQL_ID';
exec :plan_hash_value := '&hash_value';
exec :begin_snap_var := '&BEGIN_SNAP';
exec :end_snap_var := '&END_SNAP';
exec :task_name := :sql_id_var || '_' || :plan_hash_value || '_AWR';
column sql_id format a25
column plan_hash_value format 999999999999
column begin_snap format a10
column end_snap format a10
select :sql_id_var as sql_id
, :plan_hash_value as plan_hash_value
, :begin_snap_var as begin_snap
, :end_snap_var as end_snap
from dual;
-- get my prefered text in file name
column MY_TEXT new_val FILE_NAME noprint
Select instance_name||'_'|| :SQL_ID_VAR || '_' || :plan_hash_value|| '_' || :BEGIN_SNAP_VAR || '_' || :END_SNAP_VAR || '_' || to_char(sysdate, 'yyyy_mm_dd_hh24_mi') MY_TEXT
from
(select upper(substr(global_name, 1, instr(global_name,'.' )-1)) instance_name from global_name);
-- turn spooling on to capture any error and the report.
set serveroutput on
spool c:\temp\AWR_Tuning_Task_&FILE_NAME..txt
set linesize 210
set pagesize 9999
set long 10000000
set time on
set timing on
SET SERVEROUTPUT ON
prompt Drop existing task if exists. Ignore error if not
-- Drop existig tuning task; assuming I'm running new tasks and if one exists, it it no longer needed; can ignore error if no task exists yet
execute DBMS_SQLTUNE.drop_tuning_task (task_name => :task_name);
prompt Create Tuning Task
-- Tuning task created for specific a statement from the AWR. [SEE BELOW FOR EXAPLES USING OTHER METHODS OF IDENTIFYING THE SQL]
--If the TASK_NAME parameter is specified it's value is returned as the SQL tune task identifier.
--If ommitted a system generated name like "TASK_1478" is returned.
--If the SCOPE parameter is set to scope_limited the SQL profiling analysis is omitted.
--The TIME_LIMIT parameter simply restricts the time the optimizer can spend compiling the recommendations.
/**/
DECLARE
l_sql_tune_task_id VARCHAR2(100);
BEGIN
l_sql_tune_task_id := DBMS_SQLTUNE.create_tuning_task (
begin_snap => to_number(:BEGIN_SNAP_VAR) ,
end_snap => to_number(:END_SNAP_VAR) ,
sql_id => :sql_id_var,
plan_hash_value => to_number(:plan_hash_value),
scope => DBMS_SQLTUNE.scope_comprehensive,
time_limit => 1200, -- in seconds
task_name => :task_name,
description => 'Tuning task for statement '|| :sql_id_var || ' hash: ' || :plan_hash_value|| ' in AWR ' || :BEGIN_SNAP_VAR || ' - ' || :END_SNAP_VAR || ' .');
DBMS_OUTPUT.put_line('l_sql_tune_task_id: ' || l_sql_tune_task_id);
END;
/
prompt Execute Tuning Task
-- With the tuning task defined the next step is to execute it using the EXECUTE_TUNING_TASK procedure.
EXEC DBMS_SQLTUNE.execute_tuning_task(task_name => :task_name);
/**/
-- Once the tuning task has executed successfully the recommendations can be displayed using the REPORT_TUNING_TASK function.
set linesize 210
set pagesize 9999
set long 10000000
set time on
set timing on
SET SERVEROUTPUT ON
-- width of output should not need more than 200
column recommendations format a200
prompt Report Tuning Task
SELECT DBMS_SQLTUNE.report_tuning_task(:task_name) AS recommendations FROM dual;
spool off
I have a SQL statement that has performance issues.
Adding the following index and a SQL hint to use the index improves the performance 10 fold but I do not understand why.
BUS_ID is part of the primary key(T1.REF is the other part fo the key) and clustered index on the T1 table.
The T1 table has about 100,000 rows. BUS_ID has only 6 different values. Similarly the T1.STATUS column can only have a limited number of
possibilities and the majority of these(99%) will be the same value.
If I run the query without the hint(/*+ INDEX ( T1 T1_IDX1) NO_UNNEST */) it takes 5 seconds and with the hint it takes .5 seconds.
I don't understand how the index helps the subquery as T1.STATUS isn't used in any of the 'where' or 'join' clauses in the subquery.
What am I missing?
SELECT
/*+ NO_UNNEST */
t1.bus_id,
t1.ref,
t2.cust,
t3.cust_name,
t2.po_number,
t1.status_old,
t1.status,
t1.an_status
FROM t1
LEFT JOIN t2
ON t1.bus_id = t2.bus_id
AND t1.ref = t2.ref
JOIN t3
ON t3.cust = t2.cust
AND t3.bus_id = t2.bus_id
WHERE (
status IN ('A', 'B', 'C') AND status_old IN ('X', 'Y'))
AND EXISTS
( SELECT /*+ INDEX ( T1 T1_IDX1) NO_UNNEST */
*
FROM t1
WHERE ( EXISTS ( SELECT /*+ NO_UNNEST */
*
FROM t6
WHERE seq IN ( '0', '2' )
AND t1.bus_id = t6.bus_id)
OR (EXISTS
(SELECT /*+ NO_UNNEST */
*
FROM t6
WHERE seq = '1'
AND (an_status = 'Y'
OR
an_status = 'X')
AND t1.bus_id = t6.bus_id))
AND t2.ref = t1.ref))
AND USER IN ('FRED')
AND ( t2.status != '45'
AND t2.status != '20')
AND NOT EXISTS ( SELECT
/*+ NO_UNNEST */
*
FROM t4
WHERE EXISTS
(
SELECT
/*+ NO_UNNEST */
*
FROM t5
WHERE pd IN ( '1',
'0' )
AND appl = 'RYP'
AND appl_id IN ( 'RL100')
AND t4.id = t5.id)
AND t2.ref = p.ref
AND t2.bus_id = p.bus_id);
Edited to include Explain Plan and index.
Without Index hint
------------------------------------------------------|-------------------------------------
Operation | Options |Cost| # |Bytes | CPU Cost | IO COST
------------------------------------------------------|-------------------------------------
select statement | | 20 | 1 | 211 | 15534188 | 19 |
view | | 20 | 1 | 211 | 15534188 | 19 |
count | | | | | | |
view | | 20 | 1 | 198 | 15534188 | 19 |
sort | ORDER BY | 20 | 1 | 114 | 15534188 | 19 |
nested loops | | 7 | 1 | 114 | 62487 | 7 |
nested loops | | 7 | 1 | 114 | 62487 | 7 |
nested loops | | 6 | 1 | 84 | 53256 | 6 |
inlist iterator | | | | | | |
TABLE access t1 | INDEX ROWID | 4 | 1 | 29 | 36502 | 4 |
index-t1_idx#3 | RANGE SCAN | 3 | 1 | | 28686 | 3 |
TABLE access - t2 | INDEX ROWID | 2 | 1 | 55 | 16754 | 2 |
index t2_idx#0 | UNIQUE SCAN | 1 | 1 | | 9042 | 1 |
filter | | | | | | |
TABLE access-t1 | INDEX ROWID | 2 | 1 | 15 | 7433 | 2 |
TABLE access-t6 | INDEX ROWID | 3 | 1 | 4 | 23169 | 3 |
index-t6_idx#0 | UNIQUE RANGE SCAN | 1 | 3 | | 7721 | 1 |
filter | | | | | | |
TABLE access-t6 | INDEX ROWID | 2 | 2 | 8 | 15363 | 2 |
index-t6_idx#0 | UNIQUE RANGE SCAN | 1 | 3 | | 7521 | 1 |
index-t4_idx#1 | RANGE SCAN | 3 | 1 | 28 | 21584 | 3 |
inlist iterator | | | | | | |
index-t5_idx#1 | RANGE SCAN | 4 | 1 | 24 | 42929 | 4 |
index-t3_idx#0 | INDEX UNIQUE SCAN | 0 | 1 | | 1900 | 0 |
TABLE access-t3 | INDEX ROWID | 1 | 1 | 30 | 9231 | 1 |
--------------------------------------------------------------------------------------------
With Index hint
------------------------------------------------------|-------------------------------------
Operation | Options |Cost| # |Bytes | CPU Cost | IO COST
------------------------------------------------------|-------------------------------------
select statement | | 21 | 1 | 211 | 15549142 | 19 |
view | | 21 | 1 | 211 | 15549142 | 19 |
count | | | | | | |
view | | 21 | 1 | 198 | 15549142 | 19 |
sort | ORDER BY | 21 | 1 | 114 | 15549142 | 19 |
nested loops | | 7 | 1 | 114 | 62487 | 7 |
nested loops | | 7 | 1 | 114 | 62487 | 7 |
nested loops | | 6 | 1 | 84 | 53256 | 6 |
inlist iterator | | | | | | |
TABLE access t1 | INDEX ROWID | 4 | 1 | 29 | 36502 | 4 |
index-t1_idx#3 | RANGE SCAN | 3 | 1 | | 28686 | 3 |
TABLE access - t2 | INDEX ROWID | 2 | 1 | 55 | 16754 | 2 |
index t2_idx#0 | UNIQUE SCAN | 1 | 1 | | 9042 | 1 |
filter | | | | | | |
TABLE access-t1 | INDEX ROWID | 3 | 1 | 15 | 22387 | 2 |
index-t1_idx#1 | FULL SCAN | 2 |97k| | 14643 | |
TABLE access-t6 | INDEX ROWID | 3 | 1 | 4 | 23169 | 3 |
index-t6_idx#0 | UNIQUE RANGE SCAN | 1 | 3 | | 7721 | 1 |
filter | | | | | | |
TABLE access-t6 | INDEX ROWID | 2 | 2 | 8 | 15363 | 2 |
index-t6_idx#0 | UNIQUE RANGE SCAN | 1 | 3 | | 7521 | 1 |
index-t4_idx#1 | RANGE SCAN | 3 | 1 | 28 | 21584 | 3 |
inlist iterator | | | | | | |
index-t5_idx#1 | RANGE SCAN | 4 | 1 | 24 | 42929 | 4 |
index-t3_idx#0 | INDEX UNIQUE SCAN | 0 | 1 | | 1900 | 0 |
TABLE access-t3 | INDEX ROWID | 1 | 1 | 30 | 9231 | 1 |
--------------------------------------------------------------------------------------------
Table Index
CREATE INDEX T1_IDX#1 ON T1 (BUS_ID, STATUS)