---------------
MASTER TABLE
---------------
DATA_KEY NUMBER
TEXT VARCHAR2(2000)
ORDER_NO NUMBER
---------------
DETAIL TABLE
---------------
DATA_KEY NUMBER
SIMILAR_DATA_KEY NUMBER
DISTANCE_COUNT NUMBER
---------------
INSERT QUERY
---------------
INSERT INTO DETAIL
(
SELECT DATA_KEY, SIMILAR_DATA_KEY, DISTANCE_COUNT
FROM
(
SELECT A.DATA_KEY AS DATA_KEY, B.DATA_KEY AS SIMILAR_DATA_KEY,
UTL_MATCH.EDIT_DISTANCE(A.TEXT, B.TEXT) AS DISTANCE_COUNT
FROM
(SELECT DATA_KEY, TEXT, ORDER_NO FROM MASTER) A
INNER JOIN
(SELECT DATA_KEY, TEXT, ORDER_NO FROM MASTER) B
ON (A.ORDER_NO < B.ORDER_NO)
)
WHERE DISTANCE_COUNT <= 5
)
I need compare MASTER table TEXT field with other TEXT field.
indexes are not exist.
master table 90,000 rows.
ORDER_NO field is for avoid duplicated compare. (1 .. 90000)
=============================================================
A.ORDER_NO < B.ORDER_NO
------------------------------------------
1, 1 <- exclude
1, 2 <- join
1, 3 <- join
1, 4 <- join
..
2, 1 <- exclude
2, 2 <- exclude
2, 3 <- join
2, 4 <- join
...
3, 1 <- exclude
3, 2 <- exclude
3, 3 <- exclude
3, 4 <- join
1. NOT need compare 1 and 1
2. Need compare 1 and 2
3. NOT need compare 2 and 1 (because, duplicate 2.)
so, for decrease compare count...
=============================================================
Slow zone is (WHERE DISTANCE_COUNT <= 5) ?
Slow zone is comparing rows (90000*89999/2) ?
Query elapse time is 7 days.
6,000 rows inserted to DETAIL table.
How to speed up?
I'm sorry for poor English...
Related
I have a pretty lengthy SQL query which I'm going to run on Oracle via hibernate. It consists of two nested selects. In the first select statement, a number of sums are calculated, but in one of them I want to filter the results using unique ids.
SELECT ...
SUM(NVL(CASE WHEN SECOND_STATUS= 50 OR SECOND_STATUS IS NULL THEN RECEIVE_AMOUNT END, 0) +
NVL(CASE WHEN FIRST_STATUS = 1010 THEN AMOUNT END, 0) +
NVL(CASE WHEN FIRST_STATUS = 1030 THEN AMOUNT END, 0) -
NVL(CASE WHEN FIRST_STATUS = 1010 AND (SECOND_STATUS= 50 OR SECOND_STATUS IS NULL) THEN RECEIVE_AMOUNT END, 0)) TOTAL, ...
And at the end:
... FROM (SELECT s.*, p.* FROM FIRST_TABLE s
JOIN SECOND_TABLE p ON s.ID = p.FIRST_ID
In one of the lines that start with NVL (second line actually), I want to add a distinct clause that sums the amounts only if first table ids are unique. But I don't know if this is possible or not. If yes, how would it be?
Assume such setup
select * from first;
ID AMOUNT
---------- ----------
1 10
2 20
select * from second;
SECOND_ID FIRST_ID AMOUNT2
---------- ---------- ----------
1 1 100
2 1 100
3 2 100
After the join you get the total sum of both amounts too high because the amount from the first table is duplicated.
select *
from first
join second on first.id = second.first_id;
ID AMOUNT SECOND_ID FIRST_ID AMOUNT2
---------- ---------- ---------- ---------- ----------
1 10 1 1 100
1 10 2 1 100
2 20 3 2 100
You must add a row_number that identifies the first occurence in the parent table and consider in the AMOUNT only the first row and resets it to NULL in the duplicated rows.
select ID,
case when row_number() over (partition by id order by second_id) = 1 then AMOUNT end as AMOUNT,
SECOND_ID, FIRST_ID, AMOUNT2
from first
join second on first.id = second.first_id;
ID AMOUNT SECOND_ID FIRST_ID AMOUNT2
---------- ---------- ---------- ---------- ----------
1 10 1 1 100
1 2 1 100
2 20 3 2 100
Now you can safely sum in a separate subquery
with tab as (
select ID,
case when row_number() over (partition by id order by second_id) = 1 then AMOUNT end as AMOUNT,
SECOND_ID, FIRST_ID, AMOUNT2
from first
join second on first.id = second.first_id
)
select id, sum(nvl(amount,0) + nvl(amount2,0))
from tab
group by id
;
ID SUM(NVL(AMOUNT,0)+NVL(AMOUNT2,0))
---------- ---------------------------------
1 210
2 120
Note also that this is an answer to your question. Some will argue that in your situation you should first aggregate and than join. This will also resolve your problem possible more elegantly.
I have a data-set in which there are duplicate IDs in the first column. I'm hoping to obtain a single row of data for each ID based on the second column's value. The data looks like so:
ID Info_Source Prior?
A 1 Y
A 3 N
A 2 Y
B 1 N
B 1 N
B 2 Y
C 2 N
C 3 Y
C 1 N
Specifically the criteria would call for prioritizing based on the second column's value (3 highest priority; then 1; and lastly 2): if the 'Info_Source' column has a value of 3, return that row; if there is no 3 in the second column for a given ID, look for a 1 and if found return that row; and finally if there is no 3 or 1 associated with the ID, search for 2 and return that row for the ID.
The desired results would be a single row for each ID, and the resulting data would be:
ID Info_Source Prior?
A 3 N
B 1 N
C 3 Y
row_number() over() usually solves these needs nicely and efficiently e.g.
select ID, Info_Source, Prior
from (
select ID, Info_Source, Prior
, row_number() over(partition by id order by Info_source DESC) as rn
)
where rn = 1
For prioritizing the second column's value (3 ; then 1, then 2) use a case expression to alter the raw value into an order that you need.
select ID, Info_Source, Prior
from (
select ID, Info_Source, Prior
, row_number() over(partition by id
order by case when Info_source = 3 then 3
when Infor_source = 1 then 2
else 1 end DESC) as rn
)
where rn = 1
I try to generate list of products for printing labels, but all of my attempt fail (with connect by level)!
My table:
CREATE TABLE LABELS
(
PRODUCT VARCHAR2(8 BYTE),
Q_ROWS NUMBER
);
Information in the table:
INSERT INTO LABELS (PRODUCT, Q_ROWS) VALUES('D', 3);
INSERT INTO LABELS (PRODUCT, Q_ROWS) VALUES('A', 1);
INSERT INTO LABELS (PRODUCT, Q_ROWS) VALUES('C', 4);
INSERT INTO LABELS (PRODUCT, Q_ROWS) VALUES('B', 2);
Expected Result in a oracle select
PRODUCT
A
B
B
C
C
C
C
D
D
D
Results: (1 row for A, 2 rows for B, 4 rows to C and 3 rows to D)
Can someone help me?
Use LEVEL to get a "table" that counts from 1 to the maximum number of rows:
SELECT LEVEL AS LabelNum
FROM DUAL
CONNECT BY LEVEL <= (SELECT MAX(Q_Rows) FROM Labels)
This will give you the following table:
LabelNum
--------
1
2
3
4
Next, join this to your LABELS table where LabelNum <= Q_Rows. Here's the whole query:
WITH Mult AS (
SELECT LEVEL AS LabelNum
FROM DUAL
CONNECT BY LEVEL <= (SELECT MAX(Q_Rows) FROM Labels)
)
SELECT Product
FROM Labels
INNER JOIN Mult ON LabelNum <= Q_Rows
ORDER BY Product, LabelNum
There's a working SQLFiddle here.
Finally, good job including the create/populate scripts :)
Another approach, using model clause:
select product
from labels
model
partition by (product)
dimension by (1 as indx)
measures(q_rows)
rules(
q_rows[for indx from 1 to q_rows[1] increment 1] = q_rows[1]
)
order by product
result:
PRODUCT
----------
A
B
B
C
C
C
C
D
D
D
SQLFiddle Demo
I have 3 tables:
two with hierarchical structures
(like "dimensions" of recursive type of hierarchy);
one with summing data (like "facts" with X column).
They are here:
DIM1 (ID1, PARENT2, NAME1)
DIM2 (ID2, PARENT2, NAME2)
FACTS (ID1, ID2, X)
Example of DIM1 table:
-- 1 0 DIM1
---- 2 1 DIM1-A
------ 3 2 DIM1-A-A
-------- 4 3 DIM1-A-A-A
-------- 5 3 DIM1-A-A-B
------ 6 2 DIM1-A-B
-------- 7 6 DIM1-A-B-A
-------- 8 6 DIM1-A-B-B
------ 9 2 DIM1-A-C
---- 10 1 DIM1-B
------ 11 10 DIM1-B-C
------ 12 10 DIM1-B-D
---- 13 1 DIM1-C
Example of DIM2 table:
-- 1 0 DIM2
---- 2 1 DIM2-A
------ 3 2 DIM2-A-A
-------- 4 3 DIM2-A-A-A
-------- 5 3 DIM2-A-A-B
-------- 6 3 DIM2-A-B-C
------ 7 2 DIM2-A-B
---- 8 1 DIM2-B
---- 9 1 DIM2-C
Example of FACTS table:
1 1 100
1 2 30
1 3 500
-- ................
13 9 200
And I would like to create the only SELECT where I will specify the parent for DIM1 (for example ID1=2 for DIM1-A) and parent for DIM2 (for example ID2=2 for DIM2-A) and SELECT will generate a report like this:
Name_of_1 Name_of_2 Sum_of_X
--------- --------- ----------
DIM1-A-A DIM2-A-A (some sum)
DIM1-A-A DIM2-A-B (some sum)
DIM1-A-B DIM2-A-A (some sum)
DIM1-A-B DIM2-A-B (some sum)
DIM1-A-C DIM2-A-A (some sum)
DIM1-A-C DIM2-A-B (some sum)
I would like to use CONNECT BY phrase, START WITH phrase, SUM phrase, GROUP BY phrase, and OUTER or INNER (?) JOIN. I need no other extensions of Oracle 10.2.
In other words: only with "classic" SQL and
only Oracle extensions for hierarchy queries.
Is it possible?
I tried some experiments with question in
Mixing together Connect by, inner join and sum with Oracle
(where is a very nice solution but only for one
dimension table ("Tasks"), but I need to JOIN two dimension tables to one facts table), but I was not successful.
"Some sum" is not very descriptive, so I don't see why do you need CONNECT BY at all.
SELECT dim1.name, dim2.name, x
FROM (
SELECT id1, id2, SUM(x) AS x
FROM facts
GROUP BY
id1, id2
) f
JOIN dim1
ON dim1.id = f.id1
JOIN dim2
ON dim2.id = f.id2
I think what you're trying to do is get the sum of the value in the facts table for all of the children of the specified rows grouped by the topmost children. This would mean that in your example above, the results for the first row would be the sum any intersections of (DIM1-A-A, DIM1-A-A-A, DIM1-A-A-B) and (DIM2-A-A, DIM2-A-A-A, DIM2-A-A-B, DIM3-A-A-C) found in the FACTS table. With that assumption, I have come to the following solution:
SELECT root_name1, root_name2, SUM(X)
FROM ( SELECT CONNECT_BY_ROOT(name1) AS root_name,
id1
FROM dim1
CONNECT BY parent1 = PRIOR id1
START WITH parent1 = 2) d1
CROSS JOIN
( SELECT CONNECT_BY_ROOT(name2) AS root_name,
id2
FROM dim2
CONNECT BY parent2 = PRIOR id2
START WITH parent2 = 2) d2
LEFT OUTER JOIN
facts
ON d1.id1 = facts.id1
AND d2.id2 = facts.id2
GROUP BY root_name1, root_name2
(This also assumes that the columns of FACTS are named ID1, ID2, and X.)
I have a table with these columns
create table demo (
ID integer not null,
INDEX integer not null,
DATA varchar2(10)
)
Example data:
ID INDEX DATA
1 1 A
1 3 B
2 1 C
2 1 D
2 5 E
I want:
ID INDEX DATA
1 1 A
1 2 B -- 3 -> 2
2 1 C or D -- 1 -> 1 or 2
2 2 D or C -- 1 -> 2 or 1
2 3 E -- 5 -> 3 (closing the gap)
Is there a way to renumber the INDEX per ID using a single SQL update? Note that the order of items should be preserved (i.e. the item "E" should still be after the items "C" and "D" but the order of "C" and "D" doesn't really matter.
The goal is to be able to create a primary key over ID and INDEX.
If that matters, I'm on Oracle 10g.
MERGE
INTO demo d
USING (
SELECT rowid AS rid, ROW_NUMBER() OVER (PARTITION BY id ORDER BY index) AS rn
FROM demo
) d2
ON (d.rowid = d2.rid)
WHEN MATCHED THEN
UPDATE
SET index = rn