I am working on analyzing huge set of data over a year. The approach is to pick the data one day at a time with the help of a cursor and keep on feeding another table with whole year data :-
declare
i_start_date date := date '2019-04-01';
i_end_date date := date '2019-04-02';
begin
for cur_r in (select a.id, b.status
from table1 a join table2 b on a.msg_id = b.msg_id
where b.t_date between i_start_date and i_end_date
)
loop
insert into test_table (id, status)
values (cur_r.id, cur_r.status);
end loop;
end;
/
Could you please help me run this cursor in a PL/SQL block for the whole year with error handling (e.g:- if data is already there for Apr 01 it should not be inserted again in the table creating no duplicates)
Something like below:-
declare
i_start_date date := date '2019-01-01'; --start date set
i_end_date date := date '2019-12-31'; --end date set
begin
for i_start_date<=i_end_date --condition to fetch data & insert
(for cur_r in (select a.id, b.status
from table1 a join table2 b on a.msg_id = b.msg_id
where b.t_date = i_start_date
)
loop
insert into test_table (id, status)
values (cur_r.id, cur_r.status);
end loop;)
i_start_date+1 -- increment start date
end;
/
Thanks,
Why do you even need pl/sql?
insert into test_table (id,
status
)
values (select a.id,
b.status
from table1 a
join table2 b on a.msg_id = b.msg_id
where b.t_date between date '2019-04-01
and date '2019-04-02'
and b.t_date not in (select t_date
from status)
;
But beware in your comparison of DATEs (which I have simply replicated) that oracle DATE always includes a time component, and the above comparison will truncate your supplied dates to midnight. Thus, a row with b.t_date = to_date('2019-04-02 09:10:11','yyyy-mm-dd') will not be selected.
If you have a Primary Key with the date value you can handle the exception with dup_val_on_index and then use a return.
BEGIN
...
EXCEPTION
WHEN DUP_VAL_ON_INDEX THEN
...
RETURN;
END;
Or you can use a MERGE to command when to insert or not.
MERGE INTO TEST_TABLE T
USING CUR_R C
ON (C.DATE = T.DATE)
WHEN NOT MATCHED THEN
INSERT (id, status)
values (cur_r.id, cur_r.status);
You can directly use insert into <table> select ... statement as
SQL> insert into test_table
select a.id, b.status
from table1 a
join table2 b
on a.msg_id = b.msg_id
where b.t_date >= trunc(sysdate) - interval '1' year
and not exists ( select 0 from test_table t where t.id = a.id );
SQL> commit;
through use of b.t_date >= trunc(sysdate) - interval '1' year starting from the one year before to the current day.
If you need to start with a certain date such as date'2019-04-01' and scan for upcoming one year period,
then use b.t_date between date'2019-04-01' and date'2019-04-01' + interval '1' year - 1
and exclude the already existing data within the test_table through
not exists ( select 0 from test_table t where t.id = a.id ) considering those id columns are unique or primary keys in their respective tables.
Related
There are around 120k records in the database, and based on a few functions I calculate scores for all the records, weekly I have to update the table with new records and respective scores.
Below is a procedure that I am using to merge data into the table:
create or replace procedure scorecalc
AS
score1 number;
score2 number;
score3 number;
CURSOR cur IS
SELECT Id_number from tableA;
r_num cur%ROWTYPE;
BEGIN
--OPEN cur;
FOR r_num IN cur
LOOP
select functionA(r_num.id_number),functionb(r_num.id_number),functionc(r_num.id_number) into score1, score2,score3 from dual;
Merge into scores A USING
(Select
r_num.id_number as ID, score1 as scorea, score2 as scoreb, score3 as scorec, TO_DATE(sysdate, 'DD/MM/YYYY') as scoredate
FROM DUAL) B
ON ( A.ID = B.ID and A.scoredate = B.scoredate)
WHEN NOT MATCHED THEN
INSERT (
ID, scorea, scoreb, scorec, scoredate)
VALUES (
B.ID, B.scorea, B.scoreb, B.scorec,B.scoredate)
WHEN MATCHED THEN
UPDATE SET
A.scorea = B.scorea,
A.scoreb = B.scoreb,
A.scorec = B.scorec;
COMMIT;
END LOOP;
END;
whereas functionA/ B/ C has complex queries, joins in it to calculate the score.
Please suggest me any way to improve the performance because currently with this snippet of code I am only able to insert some 2k records in 1 hour? Can I use parallel DML here?
Thank you!
Why are you doing this in a procedure? This could all be done via DML:
MERGE INTO scores a USING
(SELECT ta.id_number AS ID,
functionA(ta.id) AS scoreA,
functionB(ta.id) AS scoreB,
functionC(ta.id) AS scoreC,
TO_DATE(sysdate, 'DD/MM/YYYY') as scoredate
FROM tableA ta) b
ON (a.id = b.id AND a.scoredate = b.scoredate)
WHEN MATCHED THEN UPDATE SET
a.scorea = b.scorea,
a.scoreb = b.scoreb,
a.scorec = b.scorec
WHEN NOT MATCHED THEN INSERT (ID, scorea, scoreb, scorec, scoredate)
VALUES (B.ID, B.scorea, B.scoreb, B.scorec,B.scoredate);
If you want to try using PARALLEL hint after that, feel free. But you should definitely get rid of that cursor and stop doing "Slow-by-slow" processing.
Something I've had some success with has been inserting from a select statement. It is pretty performant as it doesn't involve row by row inserting.
In your case, I'm thinking it would be something like:
INSERT INTO table (ID, scorea, scoreb, scorec, scoredate)
SELECT functionA(id_number), functionB(id_number), functionC(id_number)
FROM tableA
An example of this can be found at the link below:
https://docs.oracle.com/cd/B12037_01/appdev.101/b10807/13_elems025.htm
To schedule it just put the statement #Del in a procedure block;
create or replace procedure Saturday_Night_Merge is
begin
<Put the merge statement here>
end Saturday_Night_Merge;
I need to write a PL/SQL that sums up the value from 2 tables and update the value to another table.
I have the following tables: ONLINE_SALES, STORE_SALES, TOTAL_SALES
Assume the tables are structured like this:
ONLINE_SALES: OS_ID, STORE_ID, SEQ, ITEM_NAME, PRICE, PURCHASED_DATE
STORE_SALES: SS_ID, STORE_ID, SEQ, ITEM_NAME, PRICE, PURCHASED_DATE
TOTAL_SALES: STORE_ID, YEAR, TOTAL_INCOME
I want to write a PL/SQL that runs monthly and sums up the income (values in PRICE field) made in the month from both ONLINE_SALES and STORE_SALES of each store (identified by STORE_ID) and add the value to record in TOTAL_SALES with relative year.
My idea is to first filter the record by PURCHASED_DATE from both table with a SELECT, loop through all selected rows and sum to a variable and at last update the result with an UPDATE. But I am stuck in the first step since I found that I cannot use SELECT and only SELECT INTO is available.
Any ideas on how such PL/SQL can be written?
This solution assumes your stored procedure takes a DATE which it uses to identify the month and the year which is being totalled. Perhaps your assignment is expecting a different form of input? No worries. The deriving the values of year and date range is separate from the main process, so it is easy to swap in some different logic.
The loop uses an explicit cursor with the FOR UPDATE syntax. This locks the TOTAL_SALES table, which means you are guaranteed to be able to update all the rows.
create or replace calc_total_sales
( p_month in date )
is
cursor c_tot_sales (p_year number) is
select * from total_sales
where year = p_year
for update of total_income;
c_tot_sales c_tot_sales%rowtype;
l_os_sales number;
l_ss_sales number;
l_year number;
1_first_day date;
1_last_day date;
begin
l_year := to_number( to_char( p_month, 'yyyy') );
l_first_day := trunc( p_month, 'mm');
l_last_day := last_day( p_month);
open c_tot_sales( l_year );
loop
fetch c_tot_sales into r_tot_sales;
exit when c_tot_sales%not found;
select sum(price)
into l_os_sales
from online_sales
where store_id = r_tot_sales.store_id
and purchased_date >= l_first_day
and purchased_date <= l_last_day;
select sum(price)
into l_ss_sales
from store_sales
where store_id = r_tot_sales.store_id
and purchased_date >= l_first_day
and purchased_date <= l_last_day;
update total_sales
set total_income = total_income + nvl(l_ss_sales,0) + nvl(l_os_sales,0)
where current of c_total_sales;
end loop;
close c_tot_sales;
commit;
end;
/
I have a function, which will get greatest of three dates from the table.
create or replace FUNCTION fn_max_date_val(
pi_user_id IN number)
RETURN DATE
IS
l_modified_dt DATE;
l_mod1_dt DATE;
l_mod2_dt DATE;
ret_user_id DATE;
BEGIN
SELECT MAX(last_modified_dt)
INTO l_modified_dt
FROM table1
WHERE id = pi_user_id;
-- this table contains a million records
SELECT nvl(MAX(last_modified_ts),sysdate-90)
INTO l_mod1_dt
FROM table2
WHERE table2_id=pi_user_id;
-- this table contains clob data, 800 000 records, the table 3 does not have user_id and has to fetched from table 2, as shown below
SELECT nvl(MAX(last_modified_dt),sysdate-90)
INTO l_mod2_dt
FROM table3
WHERE table2_id IN
(SELECT id FROM table2 WHERE table2_id=pi_user_id
);
execute immediate 'select greatest('''||l_modified_dt||''','''||l_mod1_dt||''','''||l_mod2_dt||''') from dual' into ret_user_id;
RETURN ret_user_id;
EXCEPTION
WHEN OTHERS THEN
return SYSDATE;
END;
this function works perfectly fine and executes within a second.
-- random user_id , just to test the functionality
SELECT fn_max_date_val(100) as max_date FROM DUAL
MAX_DATE
--------
27-02-14
For reference purpose i have used the table name as table1,table2 and table3 but my business case is similar to what i stated below.
I need to get the details of the table1 along with the highest modified date among the three tables.
I did something like this.
SELECT a.id,a.name,a.value,fn_max_date_val(id) as max_date
FROM table1 a where status_id ='Active';
The above query execute perfectly fine and got result in millisecods. But the problem came when i tried to use order by.
SELECT a.id,a.name,a.value,a.status_id,last_modified_dt,fn_max_date_val(id) as max_date
FROM table1 where status_id ='Active' a
order by status_id desc,last_modified_dt desc ;
-- It took almost 300 seconds to complete
I tried using index also all the values of the status_id and last_modified, but no luck. Can this be done in a right way?
How about if your query is like this?
select a.*, fn_max_date_val(id) as max_date
from
(SELECT a.id,a.name,a.value,a.status_id,last_modified_dt
FROM table1 where status_id ='Active' a
order by status_id desc,last_modified_dt desc) a;
What if you don't use the function and do something like this:
SELECT a.id,a.name,a.value,a.status_id,last_modified_dt x.max_date
FROM table1 a
(
select max(max_date) as max_date
from (
SELECT MAX(last_modified_dt) as max_date
FROM table1 t1
WHERE t1.id = a.id
union
SELECT nvl(MAX(last_modified_ts),sysdate-90) as max_date
FROM table2 t2
WHERE t2.table2_id=a.id
...
) y
) x
where a.status_id ='Active'
order by status_id desc,last_modified_dt desc;
Syntax might contain errors, but something like that + the third table in the derived table too.
This is a simple example of what I need to do. In fact I want to Update value in one table by coping it from another table by using cursors in plsql.
I take table f and table b as two examples:
f=
1|Thom
2|Bob
3|Steven
5|Arthur
b=
7|Nataly
9|Alfred
, where I need to insert the b 's tow lines in first f two lines:
create table f (a number, b varchar2(10));
insert into f values (1,'Thom');
insert into f values (2,'Bob');
insert into f values (3,'Steven');
insert into f values (5,'Arthur');
commit;
create table b (c number, d varchar2(10));
insert into b values (7,'Nataly');
insert into b values (9,'Alfred');
commit;
create or replace procedure wco as
cursor c_f is
select a,b from f for update;
v_a f.a%type;
v_b f.b%type;
cursor c_b is
select c,d from b;
v_c b.c%type;
v_d b.d%type;
begin
open c_f;
open c_b
loop
fetch c_f into v_a, v_b;
exit when c_f%ROWCOUNT=c_b%RROWCOUNT;
update f set a=v_c and b=v_d where current of c_f;
end loop;
close c_d:
close c_f;
end;
/
exec wco;
select * from f;
drop table f;
The expected result (What i hope to have):
7|Nataly
9|Alfred
3|Steven
5|Arthur
But what I have now (as a result)is:
1|Thom
2|Bob
3|Steven
5|Arthur
How do I resolve this problem, I am a beginner with PLSQL, I would be very grateful if you could help me please.
Your procedure compiles, but with errors;
In order to see compilation errors
run show errors; after compilation.
Change open c_b to open c_b; (missing semicolon at the end)
Change close c_d: to close c_b; (correct name is c_b and semicolon instead of colon)
Change exit when c_f%ROWCOUNT=c_b%RROWCOUNT; to exit when c_f%ROWCOUNT=c_b%ROWCOUNT; (wrong %RROWCOUNT;)
update f set a=v_c and b=v_d where current of c_f; This is wrong SQL syntax.
Should be update table_name set column=value, column=value;
It is important to provide descriptive names to your variables, so it would be easier to understand your logic.
As I understood you want to copy only second two rows from source table.
create table source_table (
id number,
name varchar2(10));
/
insert into source_table values (1,'Thom');
insert into source_table values (2,'Bob');
insert into source_table values (3,'Steven');
insert into source_table values (5,'Arthur');
/
create table target_table (
id number,
name varchar2(10));
/
insert into target_table values (7,'Nataly');
insert into target_table values (9,'Alfred');
/
create or replace procedure copy_tables
AS
begin
FOR source_row IN (select id, name
from source_table
offset 2 rows)
LOOP
insert into target_table
values(source_row.id, source_row.name);
END LOOP;
end;
/
exec copy_tables;
/
select id, name from target_table;
/
You don't need PL/SQL for this; it can be achieved in a single MERGE statement:
merge into f tgt
using (select coalesce(b1.id, f1.id) id,
coalesce(b1.name, f1.name) name,
f1.f_rowid
from (select id,
name,
rowid f_rowid,
row_number() over (order by id) rn
from f) f1
full outer join (select id,
name,
row_number() over (order by id) rn
from b) b1
on f1.rn = b1.rn) src
on (tgt.rowid = src.f_rowid)
when matched then
update set tgt.id = src.id,
tgt.name = src.name
where tgt.id != src.id
or tgt.name != src.name
when not matched then
insert (tgt.id, tgt.name)
values (src.id, src.name);
See https://livesql.oracle.com/apex/livesql/file/content_FBPR7YCLFVWO7NGDXTLSP1R97.html for the test details (two examples; the target table has more and has fewer rows than the source table).
If you need to, you could add the above insert into a procedure, e.g.:
create procedure populate_target_table as
begin
merge into f tgt
using (select coalesce(b1.id, f1.id) id,
coalesce(b1.name, f1.name) name,
f1.f_rowid
from (select id,
name,
rowid f_rowid,
row_number() over (order by id) rn
from f) f1
full outer join (select id,
name,
row_number() over (order by id) rn
from b) b1
on f1.rn = b1.rn) src
on (tgt.rowid = src.f_rowid)
when matched then
update set tgt.id = src.id,
tgt.name = src.name
where tgt.id != src.id
or tgt.name != src.name
when not matched then
insert (tgt.id, tgt.name)
values (src.id, src.name);
end;
/
What is the easiest way to INSERT a row if it doesn't exist, in PL/SQL (oracle)?
I want something like:
IF NOT EXISTS (SELECT * FROM table WHERE name = 'jonny') THEN
INSERT INTO table VALUES ("jonny", null);
END IF;
But it's not working.
Note: this table has 2 fields, say, name and age. But only name is PK.
INSERT INTO table
SELECT 'jonny', NULL
FROM dual -- Not Oracle? No need for dual, drop that line
WHERE NOT EXISTS (SELECT NULL -- canonical way, but you can select
-- anything as EXISTS only checks existence
FROM table
WHERE name = 'jonny'
)
Assuming you are on 10g, you can also use the MERGE statement. This allows you to insert the row if it doesn't exist and ignore the row if it does exist. People tend to think of MERGE when they want to do an "upsert" (INSERT if the row doesn't exist and UPDATE if the row does exist) but the UPDATE part is optional now so it can also be used here.
SQL> create table foo (
2 name varchar2(10) primary key,
3 age number
4 );
Table created.
SQL> ed
Wrote file afiedt.buf
1 merge into foo a
2 using (select 'johnny' name, null age from dual) b
3 on (a.name = b.name)
4 when not matched then
5 insert( name, age)
6* values( b.name, b.age)
SQL> /
1 row merged.
SQL> /
0 rows merged.
SQL> select * from foo;
NAME AGE
---------- ----------
johnny
If name is a PK, then just insert and catch the error. The reason to do this rather than any check is that it will work even with multiple clients inserting at the same time. If you check and then insert, you have to hold a lock during that time, or expect the error anyway.
The code for this would be something like
BEGIN
INSERT INTO table( name, age )
VALUES( 'johnny', null );
EXCEPTION
WHEN dup_val_on_index
THEN
NULL; -- Intentionally ignore duplicates
END;
I found the examples a bit tricky to follow for the situation where you want to ensure a row exists in the destination table (especially when you have two columns as the primary key), but the primary key might not exist there at all so there's nothing to select.
This is what worked for me:
MERGE INTO table1 D
USING (
-- These are the row(s) you want to insert.
SELECT
'val1' AS FIELD_A,
'val2' AS FIELD_B
FROM DUAL
) S ON (
-- This is the criteria to find the above row(s) in the
-- destination table. S refers to the rows in the SELECT
-- statement above, D refers to the destination table.
D.FIELD_A = S.FIELD_A
AND D.FIELD_B = S.FIELD_B
)
-- This is the INSERT statement to run for each row that
-- doesn't exist in the destination table.
WHEN NOT MATCHED THEN INSERT (
FIELD_A,
FIELD_B,
FIELD_C
) VALUES (
S.FIELD_A,
S.FIELD_B,
'val3'
)
The key points are:
The SELECT statement inside the USING block must always return rows. If there are no rows returned from this query, no rows will be inserted or updated. Here I select from DUAL so there will always be exactly one row.
The ON condition is what sets the criteria for matching rows. If ON does not have a match then the INSERT statement is run.
You can also add a WHEN MATCHED THEN UPDATE clause if you want more control over the updates too.
Using parts of #benoit answer, I will use this:
DECLARE
varTmp NUMBER:=0;
BEGIN
-- checks
SELECT nvl((SELECT 1 FROM table WHERE name = 'john'), 0) INTO varTmp FROM dual;
-- insert
IF (varTmp = 1) THEN
INSERT INTO table (john, null)
END IF;
END;
Sorry for I don't use any full given answer, but I need IF check because my code is much more complex than this example table with name and age fields. I need a very clear code. Well thanks, I learned a lot! I'll accept #benoit answer.
In addition to the perfect and valid answers given so far, there is also the ignore_row_on_dupkey_index hint you might want to use:
create table tq84_a (
name varchar2 (20) primary key,
age number
);
insert /*+ ignore_row_on_dupkey_index(tq84_a(name)) */ into tq84_a values ('Johnny', 77);
insert /*+ ignore_row_on_dupkey_index(tq84_a(name)) */ into tq84_a values ('Pete' , 28);
insert /*+ ignore_row_on_dupkey_index(tq84_a(name)) */ into tq84_a values ('Sue' , 35);
insert /*+ ignore_row_on_dupkey_index(tq84_a(name)) */ into tq84_a values ('Johnny', null);
select * from tq84_a;
The hint is described on Tahiti.
you can use this syntax:
INSERT INTO table_name ( name, age )
select 'jonny', 18 from dual
where not exists(select 1 from table_name where name = 'jonny');
if its open an pop for asking as "enter substitution variable" then use this before the above queries:
set define off;
INSERT INTO table_name ( name, age )
select 'jonny', 18 from dual
where not exists(select 1 from table_name where name = 'jonny');
You should use Merge:
For example:
MERGE INTO employees e
USING (SELECT * FROM hr_records WHERE start_date > ADD_MONTHS(SYSDATE, -1)) h
ON (e.id = h.emp_id)
WHEN MATCHED THEN
UPDATE SET e.address = h.address
WHEN NOT MATCHED THEN
INSERT (id, address)
VALUES (h.emp_id, h.address);
or
MERGE INTO employees e
USING hr_records h
ON (e.id = h.emp_id)
WHEN MATCHED THEN
UPDATE SET e.address = h.address
WHEN NOT MATCHED THEN
INSERT (id, address)
VALUES (h.emp_id, h.address);
https://oracle-base.com/articles/9i/merge-statement
CTE and only CTE :-)
just throw out extra stuff. Here is almost complete and verbose form for all cases of life. And you can use any concise form.
INSERT INTO reports r
(r.id, r.name, r.key, r.param)
--
-- Invoke this script from "WITH" to the end (";")
-- to debug and see prepared values.
WITH
-- Some new data to add.
newData AS(
SELECT 'Name 1' name, 'key_new_1' key FROM DUAL
UNION SELECT 'Name 2' NAME, 'key_new_2' key FROM DUAL
UNION SELECT 'Name 3' NAME, 'key_new_3' key FROM DUAL
),
-- Any single row for copying with each new row from "newData",
-- if you will of course.
copyData AS(
SELECT r.*
FROM reports r
WHERE r.key = 'key_existing'
-- ! Prevent more than one row to return.
AND FALSE -- do something here for than!
),
-- Last used ID from the "reports" table (it depends on your case).
-- (not going to work with concurrent transactions)
maxId AS (SELECT MAX(id) AS id FROM reports),
--
-- Some construction of all data for insertion.
SELECT maxId.id + ROWNUM, newData.name, newData.key, copyData.param
FROM copyData
-- matrix multiplication :)
-- (or a recursion if you're imperative coder)
CROSS JOIN newData
CROSS JOIN maxId
--
-- Let's prevent re-insertion.
WHERE NOT EXISTS (
SELECT 1 FROM reports rs
WHERE rs.name IN(
SELECT name FROM newData
));
I call it "IF NOT EXISTS" on steroids. So, this helps me and I mostly do so.