I'm trying to make my query run as quickly as possible but i'm struggling to get it under 5 seconds.
I think it's because i'm referencing two linked databases
Here's my query
select column2, column3, column4
from table1#dev
where column1 in (
select distinct column2
from table2#dev
where column3 > 0
)
order by column1
Is there a way to optimise this query any more?
I've tried using join but it seems to make the query run longer
Thanks in advance
EDIT
From further investigation the DRIVING_SITE makes it run very quick like this
select /*+ DRIVING_SITE(table1) */ t1.column2, t1.column3, t1.column4
from table1#dev t1, table2#dev t2
WHERE t2.column3 > 0
But as soon as I add the distinct column2 in it makes it run really slow
First, no need for distinct. The query can be written as:
select *
from table1#dev
where column1 in (
select column2
from table2#dev
where column3 > 0
)
order by column1
Second, there are (at least) two more ways to write it. Either with JOIN:
select t1.*
from table1#dev t1
join table2#dev t2
where t2.column2 = t1.column1
and t2.column3 > 0
group by
t1.id, t1.column1, ...
order by t1.column1
or (my preference) with EXISTS:
select t1.*
from table1#dev t1
where exists
( select *
from table2#dev
where t2.column2 = t1.column1
and t2.column3 > 0
)
order by column1
In any case, you should check the execution plans for all of them.
I would expect performance to be best if you have an index on table1.column1 and for table2, either an index on column2 or a composite index on (column3, column2)
I agree with Shannon above , but are you able to create a view on the dev server ?
Also select * is a bit naughty - it is better to name the fields you really want. For very large datasets that will give you a performance improvement too.
Am I missing something in believing that this will work?
select t1.*
from table1 t1, table2 t2
where t1.column1 = t2.column2(+)
and t2.column3 > 0;
Related
I have a requirement to do matching of few attributes one by one. I'm looking to avoid multiple select statements. Below is the example.
Table1
Col1|Price|Brand|size
-----------------------
A|10$|BRAND1|SIZE1
B|10$|BRAND1|SIZE1
C|30$|BRAND2|SIZE2
D|40$|BRAND2|SIZE4
Table2
Col1|Col2|Col3
--------------
B|XYZ|PQR
C|ZZZ|YYY
Table3
Col1|COL2|COL3|LIKECOL1|Price|brand|size
-----------------------------------------
B|XYZ|PQR|A|10$|BRAND1|SIZE1
C|ZZZ|YYY|D|NULL|BRAND2|NULL
In table3, I need to insert data from table2 by checking below conditions.
Find a match for record in table2, if Brand and size, Price match
If no match found, then try just Brand, Size
still no match found, try brand only
In the above example, for the first record in table2, found match with all the 3 attributes and so inserted into table3 and second record, record 'D' is matching but only 'Brand'.
All I can think of is writing 3 different insert statements like below into an oracle pl/sql block.
insert into table3
select from tab2
where all 3 attributes are matching;
insert into table3
select from tab2
where brand and price are matching
and not exists in table3 (not exists is to avoid
inserting the same record which was already
inserted with all 3 attributes matched);
insert into table3
select from tab2
where Brand is matching and not exists in table3;
Can anyone please suggest a better way to achieve it in any better way avoiding multiple times selecting from table2.
This is a case for OUTER APPLY.
OUTER APPLY is a type of lateral join that allows you join on dynamic views that refer to tables appearing earlier in your FROM clause. With that ability, you can define a dynamic view that finds all the matches, sorts them by the pecking order you've specified, and then use FETCH FIRST 1 ROW ONLY to only include the 1st one in the results.
Using OUTER APPLY means that if there is no match, you will still get the table B record -- just with all the match columns null. If you don't want that, you can change OUTER APPLY to CROSS APPLY.
Here is a working example (with step by step comments), shamelessly stealing the table creation scripts from Michael Piankov's answer:
create table Table1 (Col1,Price,Brand,size1)
as select 'A','10','BRAND1','SIZE1' from dual union all
select 'B','10','BRAND1','SIZE1' from dual union all
select 'C','30','BRAND2','SIZE2' from dual union all
select 'D','40','BRAND2','SIZE4'from dual
create table Table2(Col1,Col2,Col3)
as select 'B','XYZ','PQR' from dual union all
select'C','ZZZ','YYY' from dual;
-- INSERT INTO table3
SELECT t2.col1, t2.col2, t2.col3,
t1.col1 likecol1,
decode(t1.price,t1_template.price,t1_template.price, null) price,
decode(t1.brand,t1_template.brand,t1_template.brand, null) brand,
decode(t1.size1,t1_template.size1,t1_template.size1, null) size1
FROM
-- Start with table2
table2 t2
-- Get the row from table1 matching on col1... this is our search template
inner join table1 t1_template on
t1_template.col1 = t2.col1
-- Get the best match from table1 for our search
-- template, excluding the search template itself
outer apply (
SELECT * FROM table1 t1
WHERE 1=1
-- Exclude search template itself
and t1.col1 != t2.col1
-- All matches include BRAND
and t1.brand = t1_template.brand
-- order by match strength based on price and size
order by case when t1.price = t1_template.price and t1.size1 = t1_template.size1 THEN 1
when t1.size1 = t1_template.size1 THEN 2
else 3 END
-- Only get the best match for each row in T2
FETCH FIRST 1 ROW ONLY) t1;
Unfortunately is not clear what do you mean when say match. What is you expectation if there is more then one match?
Should it be only first matching or it will generate all available pairs?
Regarding you question how to avoid multiple inserts there is more then one way:
You could use multitable insert with INSERT first and condition.
You could join table1 to self and get all pairs and filter results in where condition
You could use analytical function
I suppose there is another ways. But why you would like to avoid 3 simple inserts. Its easy to read and maintain. And may be
There is example with analytical function next:
create table Table1 (Col1,Price,Brand,size1)
as select 'A','10','BRAND1','SIZE1' from dual union all
select 'B','10','BRAND1','SIZE1' from dual union all
select 'C','30','BRAND2','SIZE2' from dual union all
select 'D','40','BRAND2','SIZE4'from dual
create table Table2(Col1,Col2,Col3)
as select 'B','XYZ','PQR' from dual union all
select'C','ZZZ','YYY' from dual
with s as (
select Col1,Price,Brand,size1,
count(*) over(partition by Price,Brand,size1 ) as match3,
count(*) over(partition by Price,Brand ) as match2,
count(*) over(partition by Brand ) as match1,
lead(Col1) over(partition by Price,Brand,size1 order by Col1) as like3,
lead(Col1) over(partition by Price,Brand order by Col1) as like2,
lead(Col1) over(partition by Brand order by Col1) as like1,
lag(Col1) over(partition by Price,Brand,size1 order by Col1) as like_desc3,
lag(Col1) over(partition by Price,Brand order by Col1) as like_desc2,
lag(Col1) over(partition by Brand order by Col1) as like_desc1
from Table1 t )
select t.Col1,t.Col2,t.Col3, coalesce(s.like3, like_desc3, s.like1, like_desc1, s.like1, like_desc1),
case when match3 > 1 then size1 end as size1,
case when match1 > 1 then Brand end as Brand,
case when match2 > 1 then Price end as Price
from table2 t
left join s on s.Col1 = t.Col1
COL1 COL2 COL3 LIKE_COL SIZE1 BRAND PRICE
B XYZ PQR A SIZE1 BRAND1 10
C ZZZ YYY D - BRAND2 -
My query result is a union of several queries. I am facing the below error when I use WITH clause within a union. Any ideas why?
select column1 from TABLE_A
union
with abcd as (select * from TABLE_B)
select column2 from TABLE_A A, abcd
where abcd.m_reference = A.m_reference
ORA-32034: unsupported use of WITH clause
32034. 00000 - "unsupported use of WITH clause"
*Cause: Inproper use of WITH clause because one of the following two reasons
1. nesting of WITH clause within WITH clause not supported yet
2. For a set query, WITH clause can't be specified for a branch.
3. WITH clause can't sepecified within parentheses.
*Action: correct query and retry
Encapsulate your WITH statement in a dummy select.
select column1 from TABLE_A
union
select * from (
with abcd as (select * from TABLE_B)
select column2 from TABLE_A A, abcd
where abcd.m_reference = A.m_reference
)
Just define the CTE first, before the actual UNION query. Then use it as you would a regular table:
with abcd as (select * from TABLE_B)
select column1 from TABLE_A
union
select column2
from TABLE_A A
inner join abcd
on abcd.m_reference = A.m_reference
You can use multiple CTE as follows:
with cte1 AS (...),
cte2 AS (...)
select * from ...
Encapsulating it is the way to go if you have multiple WITHs; for example I just had to do this monstrosity to quickly pull in data from ID numbers from an Excel sheet
select * from (
with childvendor as (
select vendornumber, name From vendor where vendornumber = '0000800727'
)
select
v.vendornumber as parentvendor,
v.name as parentname,
cv.vendornumber as childvendor,
cv.name as childname
From
vendor v, childvendor cv
where
v.vendornumber = '0000800004'
)
UNION ALL
select * from (
with childvendor as (
select vendornumber, name From vendor where vendornumber = '0000800042'
)
select
v.vendornumber as parentvendor,
v.name as parentname,
cv.vendornumber as childvendor,
cv.name as childname
From
vendor v, childvendor cv
where
v.vendornumber = '0000800035'
)
And so on
This question is wrong. I had some very big misunderstanding about how union works. I am reading about it now.
edit 04.12.2016
If you are still intersted, you can go here
Selecting the right column
I have something like this
with table3 as
(
select t1.c1, t1.c2...
from table1 t1
union all
select t2.c1, t2.c2...
from table2 t2
)select * from table3
I need to insert all rows from above in another table
insert into table4 t4
(
t4.c1, t4.c2...
)
select t3.c1, t3.c2...
from table3 t3
My question is, will this insert work. I have clumns in table 1 and 2 named the same, will I need to reference them somehow differently?
Do I need to write it like this?
insert into table4 t4
(
t4.c1, t4.c2...
)
select t3.t1.c1, t3.t1.c2, t3.t2.c1...
from table3 t3
with is part of select statement. You can insert result of select and you can use with in this select. Maybe syntax is not the most intuitive but this should work:
insert into table4
with table3 as
(
select t1.c1, t1.c2...
from table1 t1
union all
select t2.c1, t2.c2...
from table2 t2
) select * from table3;
And no you don't need (even can't) use double aliases.
No alias needed
if the column match you could simply use insert select
insert into table4
( select t1.c1, t1.c2...
from table1 t1
union all
select t2.c1, t2.c2...
from table2 t2)
otherwise you should declare the column name
insert insert into table4(c1, c2... )
( select t1.c1, t1.c2...
from table1 t1
union all
select t2.c1, t2.c2...
from table2 t2)
Assuming that you needto use that UNION ALL, instead of single insert-as-select statements to insert into another table, you can try to use different aliases for columns from different tables:
with table1 as
(
select t2.name as t2_name,
t2.address as t2_address,
t2.age as t2_age,
null as t3_name,
null as t3_address,
null as t3_age,
from table2 t2
union all
select null,
null,
null,
t3.name,
t3.address,
t3.age
from table3 t3
)
I have a table with three columns.
I query that table with IN clause.
select column1 from table1 where column1 in (1,2,3) order by column2, column3
The table1 contains only values 1 and 2 in column1. I want to return the not available value also in my result, and that should be sorted in the bottom.
example data
column1 column 2 column 3
1 100 11
2 101 50
output, the not available values should be in the last.
column1 column 2 column 3
1 100 11
2 101 50
3 null null
I tried with subquery with NVL, like select nvl((select.. in(1,2,3)),null) from dual, due to IN Clause, I am getting single row subquery returns more than one row issue, which is expected.
Also tried with the union but nothing works. Great if any help. Thanks
I think you can do it with a union all:
select column1 from table1 where column1 in (1,2,3) order by column2, column3
union all
select null from table1 where column1 not in (1,2,3) order by column2, column3
If you can't take 1,2,3 values from another table you can try with:
with t1 as (
select col1,col2,col3
from tab1
where cod_flusso in ('1','2','3')),
t2 as (
select '1' as col1,null,null
from dual
union
select '2',null,null
from dual
union
select '3',null,null
from dual)
select t2.col1,col2,col3
from t2
left outer join t1
on t1.col1= t2.col1
It's better if you can store 1,2,3 values in a second table, then use left outer join.
Here is my query,
SELECT ID As Col1,
(
SELECT VID FROM TABLE2 t
WHERE (a.ID=t.ID or a.ID=t.ID2)
AND t.STARTDTE =
(
SELECT MAX(tt.STARTDTE)
FROM TABLE2 tt
WHERE (a.ID=tt.ID or a.ID=tt.ID2) AND tt.STARTDTE < SYSDATE
)
) As Col2
FROM TABLE1 a
Table1 has 48850 records and Table2 has 15944098 records.
I have separate indexes in TABLE2 on ID,ID & STARTDTE, STARTDTE, ID, ID2 & STARTDTE.
The query is still too slow. How can this be improved? Please help.
I'm guessing that the OR in inner queries is messing up with the optimizer's ability to use indexes. Also I wouldn't recommend a solution that would scan all of TABLE2 given its size.
This is why in this case I would suggest using a function that will efficiently retrieve the information you are looking for (2 index scan per call):
CREATE OR REPLACE FUNCTION getvid(p_id table1.id%TYPE)
RETURN table2.vid%TYPE IS
l_result table2.vid%TYPE;
BEGIN
SELECT vid
INTO l_result
FROM (SELECT vid, startdte
FROM (SELECT vid, startdte
FROM table2 t
WHERE t.id = p_id
AND t.startdte < SYSDATE
ORDER BY t.startdte DESC)
WHERE rownum = 1
UNION ALL
SELECT vid, startdte
FROM (SELECT vid, startdte
FROM table2 t
WHERE t.id2 = p_id
AND t.startdte < SYSDATE
ORDER BY t.startdte DESC)
WHERE rownum = 1
ORDER BY startdte DESC)
WHERE rownum = 1;
RETURN l_result;
END;
Your SQL would become:
SELECT ID As Col1,
getvid(a.id) vid
FROM TABLE1 a
Make sure you have indexes on both table2(id, startdte DESC) and table2(id2, startdte DESC). The order of the index is very important.
Possibly try the following, though untested.
WITH max_times AS
(SELECT a.ID, MAX(t.STARTDTE) AS Startdte
FROM TABLE1 a, TABLE2 t
WHERE (a.ID=t.ID OR a.ID=t.ID2)
AND t.STARTDTE < SYSDATE
GROUP BY a.ID)
SELECT b.ID As Col1, tt.VID
FROM TABLE1 b
LEFT OUTER JOIN max_times mt
ON (b.ID = mt.ID)
LEFT OUTER JOIN TABLE2 tt
ON ((mt.ID=tt.ID OR mt.ID=tt.ID2)
AND mt.startdte = tt.startdte)
You can look at analytic functions to avoid having to hit the second table twice. Something like this might work:
SELECT id AS col1, vid
FROM (
SELECT t1.id, t2.vid, RANK() OVER (PARTITION BY t1.id ORDER BY
CASE WHEN t2.startdte < TRUNC(SYSDATE) THEN t2.startdte ELSE null END
NULLS LAST) AS rn
FROM table1 t1
JOIN table2 t2 ON t2.id IN (t1.ID, t1.ID2)
)
WHERE rn = 1;
The inner select gets the id and vid values from the two tables with a simple join on id or id2. The rank function calculates a ranking for each matching row in the second table based on the startdte. It's complicated a bit by you wanting to filter on that date, so I've used a case to effectively ignore any dates today or later by changing the evaluated value to null, and in this instance that means the order by in the over clause needs nulls last so they're ignored.
I'd suggest you run the inner select on its own first - maybe with just a couple of id values for brevity - to see what its doing, and what ranks are being allocated.
The outer query is then just picking the top-ranked result for each id.
You may still get duplicates though; if table2 has more than one row for an id with the same startdte they'll get the same rank, but then you may have had that situation before. You may need to add more fields to the order by to break ties in a way that makes sens to you.
But this is largely speculation without being able to see where your existing query is actually slow.