There is a query that takes more than two minutes. that query's result are 960000 row.
so i use hint. then it takes just 30 seconds. so i applied query in mybatis and run application. but that query takes more over two minute. so i copied query in log and paste local developer. and excute query. but it takes just 30 seconds.
i don't know........
environment : spring, mybatis, oracle
SELECT *
FROM (
SELECT /*+ USE_HASH(a b c) */ ROW_NUMBER() OVER(ORDER BY A.TRANS_DT DESC, A.TRANS_TM DESC) AS RNUM
,A.PAY_METHOD
,B.BRAND_NM
,A.I_MID
,C.TRANS_DT
,C.TRANS_TM
,C.TRANS_DT || C.TRANS_TM AS TRANS_DTM
,C.VACCT_VALID_DT
,C.VACCT_VALID_TM
,C.VACCT_VALID_DT||C.VACCT_VALID_TM AS VACCT_VALID_DTM
,NVL(C.DEPOSIT_DT, ' ') DEPOSIT_DT
,NVL(C.DEPOSIT_TM, ' ') DEPOSIT_TM
,NVL(C.DEPOSIT_DT, ' ')||NVL(C.DEPOSIT_TM, ' ') AS DEPOSIT_DTM
,NVL(C.DEPOSIT_AMT, 0) AS AMT
,NVL(A.AMT, 0) AS INPUT_AMT
,A.BANK_CD
,IONPAY.UF_GET_BANK_NAME(A.BANK_CD) AS BANK_CD_NM
,C.VACCT_NO
,A.BILLING_NM
,A.REFERENCE_NO
,A.TXID
,A.STATUS
,NVL(A.STATUS, ' ') AS INPUT_STATUS
,IONPAY.UF_GET_TRANS_VACCT_STATUS_NAME(C.STATUS) AS INPUT_STATUS_NAME
,C.STATUS AS TRANS_STATUS
,IONPAY.UF_GET_TRANS_VACCT_STA_NAME_2(C.STATUS, C.MATCH_CL, C.VACCT_VALID_DT, C.VACCT_VALID_TM) AS STATUS_NAME
,A.ACQU_STATUS
,A.CANCEL_DT||A.CANCEL_TM AS REVERSAL_DATE
,NVL((SELECT DESC2
FROM TB_CODE
WHERE CODE_CL = 'CHNL'
AND CODE1 = C.BANK_CD
AND CODE2 = C.CHANNEL_TYPE), ' ') AS channel
FROM TB_TRANS_HISTORY A, TB_BO_MER_MGMT B, TB_VACCT_TRANS C
WHERE A.I_MID = B.I_MID
AND A.TXID = C.TXID
AND A.I_MID = C.I_MID
AND B.I_MID = C.I_MID
AND A.PAY_METHOD IN ('02')
AND A.TRANS_DT BETWEEN '20161016' AND '20161116'
AND A.TRANS_DT||A.TRANS_TM BETWEEN '2016101600%3A0000' AND '2016111624%3A0000'
AND B.TAX_NO != 'NICEPAY'
AND C.SIMULATION_FLG = '0'
AND C.STATUS IN ('0', '1', '2', '3', '4')
) TBL
WHERE RNUM BETWEEN 210000 AND 220000
I have already met such situation: the time difference is not in SQL query execution, but on fetching results.
Oracle JDBC driver default fetch size is 10, that means the ResultSet is fed 10 rows by 10 rows, which make a lot of round-trips to DB when there are 1 million records. The Fetch size must be increased.
With PostgreSql, the default fetch size is unlimited: the ResultSet is fed with the whole result (or until OutOfMemory). The Fetch size could need to be decreased.
To specify the fetch size value in Mybatis:
Use fetchSize attribute in XML select statement:
<select id="listItems" fetchSize="300">SELECT ...</select>
Or use fetchSize option in annotations:
#Select("SELECT ...")
#Options(fetchSize=300)
A value in range 200-500 is often a good compromise.
Related
So I'm trying to build a function that returns the records of items that are included in some client subscription.
So I've been building up the following:
2 types:
CREATE OR REPLACE TYPE PGM_ROW AS OBJECT
(
pID NUMBER(10),
pName VARCHAR2(300)
);
CREATE OR REPLACE TYPE PGM_TAB AS TABLE OF PGM_ROW;
1 function:
CREATE OR REPLACE FUNCTION FLOGIN (USER_ID NUMBER) RETURN PGM_TAB
AS
SELECTED_PGM PGM_TAB;
BEGIN
FOR RESTRICTION
IN ( SELECT (SELECT LISTAGG (ID_CHANNEL, ',')
WITHIN GROUP (ORDER BY ID_CHANNEL)
FROM (SELECT DISTINCT CHA2.ID_CHANNEL
FROM CHANNELS_ACCESSES CHA2
JOIN CHANNELS CH2
ON CH2.ID = CHA2.ID_CHANNEL
WHERE CHA2.ID_ACCESS = CMPA.ID_ACCESS
AND CH2.ID_CHANNELS_GROUP = CG.ID))
AS channels,
(SELECT LISTAGG (ID_SUBGENRE, ',')
WITHIN GROUP (ORDER BY ID_SUBGENRE)
FROM (SELECT DISTINCT SGA2.ID_SUBGENRE
FROM SUBGENRES_ACCESSES SGA2
JOIN CHANNELS_ACCESSES CHA2
ON CHA2.ID_ACCESS = SGA2.ID_ACCESS
JOIN CHANNELS CH2
ON CH2.ID = CHA2.ID_CHANNEL
WHERE SGA2.ID_ACCESS = CMPA.ID_ACCESS
AND CH2.ID_CHANNELS_GROUP = CG.ID))
AS subgenres,
CG.NAME,
A.BEGIN_DATE,
A.END_DATE,
CMP.PREVIEW_ACCESS
FROM USERS U
JOIN COMPANIES_ACCESSES CMPA
ON U.ID_COMPANY = CMPA.ID_COMPANY
JOIN COMPANIES CMP ON CMP.ID = CMPA.ID_COMPANY
JOIN ACCESSES A ON A.ID = CMPA.ID_ACCESS
JOIN CHANNELS_ACCESSES CHA
ON CHA.ID_ACCESS = CMPA.ID_ACCESS
JOIN SUBGENRES_ACCESSES SGA
ON SGA.ID_ACCESS = CMPA.ID_ACCESS
JOIN CHANNELS CH ON CH.ID = CHA.ID_CHANNEL
JOIN CHANNELS_GROUPS CG ON CG.ID = CH.ID_CHANNELS_GROUP
WHERE U.ID = USER_ID
GROUP BY CG.NAME,
A.BEGIN_DATE,
A.END_DATE,
CMPA.ID_ACCESS,
CG.ID,
CMP.PREVIEW_ACCESS)
LOOP
SELECT PFT.ID_PROGRAM, PFT.LOCAL_TITLE
BULK COLLECT INTO SELECTED_PGM
FROM PROGRAMS_FT PFT
WHERE PFT.ID_CHANNEL IN
( SELECT TO_NUMBER (
REGEXP_SUBSTR (RESTRICTION.CHANNELS,
'[^,]+',
1,
ROWNUM))
FROM DUAL
CONNECT BY LEVEL <=
TO_NUMBER (
REGEXP_COUNT (RESTRICTION.CHANNELS,
'[^,]+')))
AND PFT.ID_SUBGENRE IN
( SELECT TO_NUMBER (
REGEXP_SUBSTR (RESTRICTION.SUBGENRES,
'[^,]+',
1,
ROWNUM))
FROM DUAL
CONNECT BY LEVEL <=
TO_NUMBER (
REGEXP_COUNT (RESTRICTION.SUBGENRES,
'[^,]+')))
AND (PFT.LAUNCH_DATE BETWEEN RESTRICTION.BEGIN_DATE
AND RESTRICTION.END_DATE);
END LOOP;
RETURN SELECTED_PGM;
END FLOGIN;
I expect the function tu return a table with 2 columns containing all the records from table PROGRAMS_FT that are included in the user access.
For some reason, I'm getting compilation warning ORA-000947.
My understanding of the error code is that it occurs when the values inserted does not match the type of the object receiving the values, and I can't see how this can be the case here.
You're selecting two scalar values and trying to put them into an object. That doesn't happen automatically, you need to convert them to an object:
...
LOOP
SELECT PGM_ROW(PFT.ID_PROGRAM, PFT.LOCAL_TITLE)
BULK COLLECT INTO SELECTED_PGM
FROM PROGRAMS_FT PFT
...
(It's an unhelpful quirk of PL/SQL that it says 'not enough values' rather than 'too many values', as you might expect when you try to put two things into one; I'm sure I came up with a fairly convincing explanation/excuse for that once but it escapes me at the moment...)
I'm not sure your loop makes sense though. Assuming your cursor query returns multiple rows, each time around the loop you're replacing the contents of the SELECTED_PGM collection - you might think you are appending to it, but that's not how it works. So you will end up returning a collection based only on the final iteration of the loop.
Aggregating and then splitting the data seems like a lot of work too. You could maybe use collections for those; but you can probably get rid of the cursor and loop and combine the cursor query with the inner query, which would be more efficient and would allow you to do a single bulk-collect for all the combined data.
I am trying to select the total number of each value in column paxtype with the following letters m,f,i,c but my error is sql not ended properly
(select b.PAXTYPE from xxxx b, xxx a)
(case b.PAXTYPE
when 'M' then count('M')
when 'F' then count('F')
when 'I' then count('I')
when 'C' then count('C')
END)
where a.date_key=to_char(b.FLIGHTDATE,'RRRRMMDD')
and a.FLTNUM_KEY= trim(substr(b.flightnumber,3))
and a.origin=b.frm
and a.destination=b.too
--and a.date_key=20170801
--and fightnumber = '100'
and trim(a.cancelled) is null
and rownum = 1
)
Firstly You are not using the correct sql syntax. Also, there are several other problems with your query.
Table names should come after columns in select .
You need to use group by since you need count
There is no need for CASE block
why is ROWNUM = 1 required?
This query should work fine for your requirement.
SELECT b.PAXTYPE, COUNT (b.PAXTYPE)
FROM xxxx b, xxx a
WHERE a.date_key = TO_CHAR (b.FLIGHTDATE, 'RRRRMMDD')
AND a.FLTNUM_KEY = TRIM (SUBSTR (b.flightnumber, 3))
AND a.origin = b.frm
AND a.destination = b.too
--and a.date_key=20170801
--and fightnumber = '100'
AND TRIM (a.cancelled) IS NULL
--and rownum = 1 # Why was it required?
GROUP BY b.PAXTYPE;
Additionally, If you need only the counts for M,F,I,C then add AND b.PAXTYPE IN ('M','F','I','C') before group by
Refer Table WORKQUEUELOG with columns (ID,QueueTypeId,CreateDateTime,WorkId,InQ,OutQ)
There is an index available for (QueueTypeId,CreateDateTime,Id)
When trying to read the first X number of records with following query it is going for a full table search.
OPEN a_CursorHandle FOR
SELECT * FROM (
SELECT * FROM WORKQUEUELOG tbl
WHERE EX_CLS_WQLOG.FILTER(
tbl.QueueTypeId,
tbl.CreateDateTime,
NULL) = 1
AND EX_CLS_WQLOG.VALIDATION(
tbl.ID ,
tbl.WorkId ,
tbl.InQ ) = 1
ORDER BY tbl.QueueTypeId ASC,tbl.CreateDateTime ASC,tbl.Id ASC)
WHERE ROWNUM <= a_MaxCount;
We changed the query as following and started using the index for reading.
ie Validaiton function first and filter function as second.
OPEN a_CursorHandle FOR
SELECT * FROM (
SELECT * FROM WORKQUEUELOG tbl
WHERE EX_CLS_WQLOG.VALIDATION(
tbl.ID ,
tbl.WorkId,
tbl.InQ ) = 1
AND EX_CLS_WQLOG.FILTER(
tbl.QueueTypeId,
tbl.CreateDateTime,
NULL) = 1
ORDER BY tbl.QueueTypeId ASC,tbl.CreateDateTime ASC,tbl.Id ASC)
WHERE ROWNUM <= a_MaxCount;
What is the order of processing the SQL statement in oracle .? left to right or right to left.
From the above example this seems to be right to left . is this a consistent behavior. Is there any other efficient way to write this query.?
I have 3 tables
1. BIG (~6 Million Records, indexed on ID1 and some other columns , not partitioned , on DB instance 1)
2. VBIG (~6 Billion Records, indexed on ID2 and some other columns , partition on DATE field, on DB instance 2)
3. VVBIG (> VBIG by 10-15% , indexed on ID1, ID2 and some other columns, partition on DATE field, on DB instance 2)
For a given DATE and few other filter conditions, I am using data from these 3 table to run some processing. I have to decide between the 2 queries.
select /*+ ORDERED */
column1, column2
from
BIG, VBIG, VVBIG
where
BIG.ID1 = VVBIG.ID1 and
VBIG.ID2 = VVBIG.ID2 and
VBIG.DATE = VVBIG.DATE and
VBIG.DATE = '1-Jan-2015' and
BIG.CL1 = 'XYZ' and
VVBIG.CL1 = 'ABC'
OR
select /*+ ORDERED */
column1, column2
from
(select /*+ parallel */ from BIG
where BIG.CL1 = 'XYZ'),
(select /*+ parallel */ from VBIG
where VBIG.DATE = '1-Jan-2015'),
(select /*+ parallel */ from VVBIG
where VVBIG.DATE = '1-Jan-2015' and VVBIG.CL1 = 'ABC')
where
BIG.ID1 = VVBIG.ID1 and
VBIG.ID2 = VVBIG.ID2 and
VBIG.DATE = VVBIG.DATE and
Not sure if oracle is playing tricks, or if it is the distributed DB architecture, but my explain plan changes randomly.
My tests with synthetic data shows better performance with option#2.
Is there a way I can rest assured that this would be the correct choice?
Also my Performance DBA suggested of using
/*+ use_hash( BIG, VBIB, VVBIG ) full(BIG) full(VBIG) full(VVBIG) */
instead of the ORDERED hint. Would it be advisable as I am getting a CARTESIAN JOIN MERGE with his suggested change.
Overall Task :- I need to retrieve data from 45 fields in system A and dump that data into a temp table which is then picked up by a unix process which produces an xml data file to be imported into system B.
Specific Question : What would be the best way of retrieving the data to be written into the 45 fields. Majority of the data is independent and can't be retrieved using a single statement. The way i currently retrieve this data is as follows (example below)
My temp tables hold the affected properties ID that i need to extract data for. i.e PROP_LIST_TEMP and ASSOC_PROP_TEMP.
SELECT SUBSTR (pro.pro_propref, 1, 25) UPRN,
(SELECT SUBSTR (adr_building, 1, 100)
FROM addresses, address_usages
WHERE aus_adr_refno = adr_refno
AND aus_aut_far_code = 'PHYSICAL'
AND aus_aut_fao_code = 'PRO'
AND (aus_end_date IS NULL OR aus_end_date > SYSDATE)
AND aus_pro_refno = pro.pro_refno)
BUILDING_NAME,
(SELECT CASE
WHEN (adr_street_number like 'BLOC%'
OR adr_street_number like '%-%'
OR adr_street_number like '%/%')
THEN NULL
ELSE regexp_replace (adr_street_number, '[^[:digit:]]+')
END
FROM addresses, address_usages
WHERE aus_adr_refno = adr_refno
AND aus_aut_far_code = 'PHYSICAL'
AND aus_aut_fao_code = 'PRO'
AND (aus_end_date IS NULL OR aus_end_date > SYSDATE)
AND aus_pro_refno = pro.pro_refno)
STREET_NUMBER,
(SELECT CASE
WHEN (adr_street_number like 'BLOC%'
OR adr_street_number like '%-%'
OR adr_street_number like '%/%')
THEN SUBSTR (adr_street_number, 1, 20)
ELSE REGEXP_REPLACE (adr_street_number, '[^[:alpha:]]+', '')
END
FROM addresses, address_usages
WHERE aus_adr_refno = adr_refno
AND aus_aut_far_code = 'PHYSICAL'
AND aus_aut_fao_code = 'PRO'
AND (aus_end_date IS NULL OR aus_end_date > SYSDATE)
AND aus_pro_refno = pro.pro_refno)
STREET_NUMBER_SUFFIX,
(SELECT SUBSTR (ptv_pty_code, 1, 3)
FROM prop_type_values
WHERE ptv_refno = pro.pro_hou_ptv_refno)
HOUSE_TYPE
FROM properties pro
WHERE pro_refno IN (select * from PIMSS_PROP_LIST_TEMP
UNION
select * from PIMSS_ASSOC_PROP_TEMP)
AND pro.pro_hou_hrv_hot_code IN
(SELECT frv_code
FROM first_ref_values
WHERE frv_frd_domain IN ('ASS_OWN_REF')
AND frv_current_ind = 'Y');
Since the where clauses of the subqueries in the select statement are identical, you could simply pull that out into the where clause, like so:
SELECT SUBSTR (pro.pro_propref, 1, 25) UPRN,
SUBSTR (addr.adr_building, 1, 100) BUILDING_NAME,
CASE WHEN (addr.adr_street_number like 'BLOC%'
OR addr.adr_street_number like '%-%'
OR addr.adr_street_number like '%/%')
THEN NULL
ELSE regexp_replace (addr.adr_street_number, '[^[:digit:]]+')
END STREET_NUMBER,
CASE WHEN (addr.adr_street_number like 'BLOC%'
OR addr.adr_street_number like '%-%'
OR addr.adr_street_number like '%/%')
THEN SUBSTR (addr.adr_street_number, 1, 20)
ELSE REGEXP_REPLACE (addr.adr_street_number, '[^[:alpha:]]+', '')
END STREET_NUMBER_SUFFIX,
(SELECT SUBSTR (ptv_pty_code, 1, 3)
FROM prop_type_values
WHERE ptv_refno = pro.pro_hou_ptv_refno) HOUSE_TYPE
FROM properties pro,
(select adr_building,
adr_street_number
FROM addresses, address_usages
WHERE aus_adr_refno = adr_refno
AND aus_aut_far_code = 'PHYSICAL'
AND aus_aut_fao_code = 'PRO'
AND (aus_end_date IS NULL OR aus_end_date > SYSDATE)) addr
WHERE pro.pro_refno = aus_pro_refno
and pro_refno IN (select * from PIMSS_PROP_LIST_TEMP
UNION
select * from PIMSS_ASSOC_PROP_TEMP)
AND pro.pro_hou_hrv_hot_code IN (SELECT frv_code
FROM first_ref_values
WHERE frv_frd_domain IN ('ASS_OWN_REF')
AND frv_current_ind = 'Y');
You might possibly need an outer join if there's a chance that no rows could be returned from the addr subquery.