I have a list of 50 thousand receipt Ids (hard coded values). i want to apply these 50 thousand Ids in where condition or join operation. I have used below 'with' clause to create a temp table to collect those 50 thousand Ids. Then I used this temp table in join query for filtering.
with temp_receiptIds(receiptId)
as
(
select 'M0000001' from dual
union
select 'M0000002' from dual
union
select 'M0000003' from dual
union
select 'M0000004' from dual
..
..
...
union
select 'M0049999' from dual
union
select 'M0050000' from dual
)
select sal.receiptId, prd.product_name, prd.product_price, sal.sales_date, sal.seller_name
from product prd
join sales sal on prd.product_id=sal.product_id
join temp_receiptIds tmp on tmp.receiptId=sal.receiptId
Whenever I run the above select join query to extract data as requested by business people, it takes about 8 minutes to fetch result in the production server.
Is my above approach correct? Are there any simpler approach than this by considering best performance in the production server.
Please note, every second , the production database is used by customer. since production db is very busy, can I run this query in production db directly, will it cause slow performance in the customer using website which calls this production db in every second. Correct answers would be greatly appreciated! Thanks
Why wouldn't you store those receiptIDs into a table?
create table receiptids as
with temp_receiptIds(receiptId)
as
(
select 'M0000001' from dual
union all --> "union ALL" instead of "union"
...
)
select * from temp_receiptids;
Index it:
create index i1recid on receiptids (receiptIdD);
See how that query now behaves.
If you - for some reason - can't do that, see whether UNION ALL within the CTE does any good. For 50.000 rows, it could make a difference.
Related
All the Oracles out here,
I have an Oracle PL/SQL procedure but very small data that can run on the query. I suspect that when the data gets large, the query might start performing back. Are there ways in which I can check for performance and take corrective measure even before the data build up? If I wait for the data buildup, it might get too late.
Do you have any general & practical suggestions for me? Searching the internet did not get me anything convincing.
Better to build yourself some test data to get an idea of how things will perform. Its easy to get started, eg
create table MY_TEST as select * from all_objects;
gives you approx 50,000 rows typically. You can scale that easily with
create table MY_TEST as select a.* from all_objects a ,
( select 1 from dual connect by level <= 10);
Now you have 500,000 rows
create table MY_TEST as select a.* from all_objects a ,
( select 1 from dual connect by level <= 10000);
Now you have 500,000,000 rows!
If you want unique values per row, then add rownum eg
create table MY_TEST as select rownum r, a.* from all_objects a ,
( select 1 from dual connect by level <= 10000);
If you want (say) 100,000 distinct values in a column then TRUNC or MOD. You can also use DBMS_RANDOM to generate random numbers, strings etc.
Also check out Morten's test data generator
https://github.com/morten-egan/testdata_ninja
for some domain specific data, and also the Oracle sample schemas on github which can also be scaled using techniques above.
https://github.com/oracle/db-sample-schemas
My understanding as per standard practice is that HAVING is to be used along with GROUP BY for filtering conditions, while WHERE is supposed to be used for general row-wise filtering conditions.
However, there are online discussions with mixed conclusions as to whether use HAVING as a superset of WHERE clause. That is, whether it can be used even without GROUP BY in which case it works as a WHERE clause.
I want to understand what is the industry practice in using HAVING clause across Oracle, Microsoft SQL server, MySQL, PostGreSQL and other tools.
A funny thing I observed when executing this query:
SELECT *
FROM SH.SALES
WHERE amount_sold > 1000
HAVING amount_sold < 2000;
It gives an error when executing in Oracle SQL developer desktop whereas runs successfully in Oracle SQL developer web.
This is a great question AND puzzle!
Oracle SQL Developer Web is provided via Oracle REST Data Services (ORDS). There is a RESTful Web Service used to execute 'ad hoc' SQL statements and scripts.
Instead of bringing back all the rows from a query in a single call, we page them. And instead of holding a resultset open and process running, we stick to the RESTful way, and do all the work on a single call and response.
How do we make this happen?
Well, when you type in that query from your question and execute it, on the back end, that's not actually what gets executed.
We wrap that query with another SELECT, and use the ROW_NUMBER() OVER analytic function call. This allows us to 'window' the query results, in this case between rows 1 and 26, or the the first 25 rows of that query, your query.
SELECT *
FROM (
SELECT Q_.*,
ROW_NUMBER() OVER(
ORDER BY 1
) RN___
FROM (
select *
from sh.sales
where amount_sold > 1000
having amount_sold < 2000
) Q_
)
WHERE RN___ BETWEEN :1 AND :2
Ok, but so what?
Well, Optimizer figures out this query can still run, even if the having clause isn't appropriate.
The optimizer is always free to re-arrange a query before searching for best execution plans.
In this case, a 10053 trace shows that a query such as below that came from SQL Dev Web (I'm using EMP but the same applies for any table)
SELECT *
FROM (
SELECT Q_.*,
ROW_NUMBER() OVER(
ORDER BY 1
) RN___
FROM (
SELECT *
FROM emp
WHERE sal > 1000
HAVING sal < 2000
) Q_
)
WHERE RN___ BETWEEN :1 AND :2
got internally transformed to the following before being optimized for plans.
SELECT
subq.EMPNO EMPNO,
subq.ENAME ENAME,
subq.JOB JOB,
subq.MGR MGR,
subq.HIREDATE HIREDATE,
subq.SAL SAL,subq.COMM COMM,
subq.DEPTNO DEPTNO,
subq.RN___ RN___
FROM
(SELECT
EMP.EMPNO EMPNO,
EMP.ENAME ENAME,
EMP.JOB JOB,EMP.MGR MGR,
EMP.HIREDATE HIREDATE,
EMP.SAL SAL,
EMP.COMM COMM,
EMP.DEPTNO DEPTNO,
ROW_NUMBER() OVER ( ORDER BY NULL ) RN___
FROM EMP EMP
WHERE EMP.SAL>1000 AND TO_NUMBER(:B1)>=TO_NUMBER(:B2)
) subq
WHERE subq.RN___>=TO_NUMBER(:B3)
AND subq.RN___<=TO_NUMBER(:B4)
Notice the HAVING has been transformed/optimized out of the query, which lets it pass through onto the execution phase.
Major 👏 to #connor-mcdonald of AskTom fame for helping me parse this out.
And so that's why it works in SQL Developer Web, but NOT in SQL Developer Desktop, where the query is executed exactly as written.
Dear all experts.
I have IOT having 7 million records in oracle database, eventually iot use for fast access primary key but in my case, when i select primary key column it takes 5-4 seconds for select single column.
My query is:
Select Emp_Refno from Emp_master where Rownum =1 order
by Emp_Refno asc;
I have also used Sql Tunning Advisor for optimize it and also get index suggest ion from SQL Tunning Advisor and also applied it, But in explain plan not seen this index and it takes same time after it.
I'm curious if the following query has the same execution time:
select * from (select Emp_Refno from Emp_master order by Emp_Refno asc) where rownum = 1
This is how I usually write top-n queries for Oracle.
In my company's application there is a query in oracle using parallel execution (configured to 4 servers), it wasn't me who built it, but the developer put it that way for performance.
The query makes joins between views and tables and the weirdest thing is: sometimes it returns 11k results (incorrect), sometimes 27k results (correct).
After much research I found out that if I removed this parallel thing, it always returns the correct number: 27k. And if I increase the number of server to 6 or 7, it always returns the incorrect number: 11k.
The layout of the query is like this:
SELECT /*+ PARALLEL(NAME, 4) */ * FROM(
SELECT DISTINCT COLUMNS
FROM VIEW
JOIN TABLE1 ON (....)
JOIN TABLE2 ON (....)
JOIN TABLE3 ON (....)
ORDER BY 3
) NAME
Anyone has any idea why? I don't know much about this subject.
In my program there are a lot of situation when i need to get additional information about knowing ids. So i have list of ids, which length may be very long (for example 100000 elements in it).
How i can use this list and transfer in oracle for getting sql without using temp tables?
No i try to use collection:
CREATE TYPE TEST_VARRAY IS VARRAY(5000) OF NUMBER(18);
SELECT G.ID, G.NAME FROM ANY_TABLE G
WHERE G.ID IN
(
SELECT COLUMN_VALUE FROM TABLE(
NEW TEST_VARRAY
(0,1,2,3... and so on ...,995,996,997,998,999)
)
);
there are 1000 numbers. And when I try execute this query the error ORA-00907: missing right parenthesis tips is appeared! But if i delete first 0 (so we have 999 numbers) the sql is executed ok.
What is problem here?
There is a limit in Oracle IN clause.
A comma-delimited list of expressions can contain no more than 1000
expressions. A comma-delimited list of sets of expressions can contain
any number of sets, but each set can contain no more than 1000
expressions.
Read here or here or here
In my opinion, you are misusing collections, at least I am not sure something like you did is good.
As far as I understand you generate this query before run, so what is the problem to do like that?
with ids as (select /*+ materialize */ 1 id from dual union all
select 2 from dual union all
select 3 from dual union all
select 4 from dual union all
/* repeat with the ids you need */
select 9999 from dual)
select *
from yourTable, ids
where yourTable.id = ids.id;
And that's it! Without any limitations, with pure SQL only. I have added materialize hint to ensure it is not performance relevant, but I think it can be skipped.
No temporary tables, no collections, nothing to create and support. Just SQL.
If you will put ids out of with into from clause it will work in any RDBMS (I guess).