Another PL/SQL refactoring question!
I have several cursors that are of the general simplified form:
cursor_1 is
with X as (select col1, col2 from TAB where col1 = '1'),
Y as (select col1, col2 from TAB where col2 = '3'),
/*main select*/
select count(X.col1), ...
from X inner join Y on...
group by rollup (X.col1, ...
cursor_2 is
with X as (select col1, col2 from TAB where col1 = '7' and col2 = '9' and col3 = 'TEST'),
Y as (select col1, col2 from TAB where col3 = '6'),
/*main select*/
select count(X.col1), ...
from X inner join Y on...
group by rollup (X.col1, ...
cursor_2 is
with X as (select col1, col2 from TAB where col1 IS NULL ),
Y as (select col1, col2 from TAB where col2 IS NOT NULL ),
/*main select*/
select count(X.col1), ...
from X inner join Y on...
group by rollup (X.col1, ...
...
begin
for r in cursor_1 loop
print_report_results(r);
end loop;
for r in cursor_2 loop
print_report_results(r);
end loop;
...
end;
Basically, all of these cursors (there's more than 3) are the same summary/reporting queries. The difference is in the factored subqueries. There are always 2 factored subqueries, "X" and "Y", and they always select the same columns to feed into the main reporting query.
The problem is that the main reporting query is VERY large, about 70 lines. This itself isn't so bad, but it was copy-pasted for ALL of the reporting queries (I think there's over a dozen).
Since the only difference is in the factored subqueries (and they all return the same columns, it's really just a difference in the tables they select from and their conditions) I was hoping to find a way to refactor all this so that there is ONE query for the giant report and smaller ones for the various factored subqueries so that when changes are made to the way the report is done, I only have to do it in one place, not a dozen. Not to mention a much easier-to-navigate (and read) file!
I just don't know how to properly refactor something like this. I was thinking pipelined functions? I'm not sure they're appropriate for this though, or if there's a simpler way...
On the other hand, I also wonder if performance would be significantly worse by splitting out the reporting query. Performance (speed) is an issue for this system. I'd rather not introduce changes for developer convenience if it adds significant execution time.
I guess what I'd ultimately like is something that looks sort of like this (I'm just not sure how to do this so that it will actually compile):
cursor main_report_cursor (in_X, in_Y) is
with X as (select * from in_X),
Y as (select * from in_Y)
/*main select*/
select count(X.col1), ...
from X inner join Y on...
group by rollup (X.col1, ...
cursor x_1 is
select col1, col2 from TAB where col1 = '1';
cursor y_1 is
select col1, col2 from TAB where col2 = '3'
...
begin
for r in main_report_cursor(x_1,y_1) loop
print_report_results(r);
end loop;
for r in main_report_cursor(x_2,y_2) loop
print_report_results(r);
end loop;
...
(Using Oracle 10g)
Use a pipelined function. For example:
drop table my_tab;
create table my_tab
(
col1 number,
col2 varchar2(10),
col3 char(1)
);
insert into my_tab values (1, 'One', 'X');
insert into my_tab values (1, 'One', 'Y');
insert into my_tab values (2, 'Two', 'X');
insert into my_tab values (2, 'Two', 'Y');
insert into my_tab values (3, 'Three', 'X');
insert into my_tab values (4, 'Four', 'Y');
commit;
-- define types
create or replace package refcur_pkg is
--type people_tab is table of people%rowtype;
type my_subquery_tab is table of my_tab%rowtype;
end refcur_pkg;
Create the function pipelined
-- create pipelined function
create or replace function get_tab_data(p_cur_num in number, p_cur_type in char)
return REFCUR_PKG.my_subquery_tab pipelined
IS
v_ret REFCUR_PKG.my_subquery_tab;
begin
if (p_cur_num = 1) then
if (upper(p_cur_type) = 'X') then
for rec in (select * from my_tab where col1=1 and col3='X')
loop
pipe row(rec);
end loop;
elsif (upper(p_cur_type) = 'Y') then
for rec in (select * from my_tab where col1=1 and col3='Y')
loop
pipe row(rec);
end loop;
else
return;
end if;
elsif (p_cur_num = 2) then
if (upper(p_cur_type) = 'X') then
for rec in (select * from my_tab where col1=2 and col3='X')
loop
pipe row(rec);
end loop;
elsif (upper(p_cur_type) = 'Y') then
for rec in (select * from my_tab where col1=2 and col3='Y')
loop
pipe row(rec);
end loop;
else
return;
end if;
end if;
return;
end;
MAIN procedure example
-- main procedure/usage
declare
cursor sel_cur1 is
with X as (select * from table(get_tab_data(1, 'x'))),
Y as (select * from table(get_tab_data(1, 'y')))
select X.col1, Y.col2 from X,Y where X.col1 = Y.col1;
begin
for rec in sel_cur1
loop
dbms_output.put_line(rec.col1 || ',' || rec.col2);
end loop;
end;
All of your various subqueries are reduced to a call to a single pipelined function, which determines the rows to return.
EDIT:
To combine all needed types and functions into 1 procedure, and also to use variables for subquery function parameters, I'm adding the following example:
create or replace procedure my_pipe
IS
-- define types
type my_subquery_tab is table of my_tab%rowtype;
type ref_cur_t is ref cursor;
v_ref_cur ref_cur_t;
-- define vars
v_with_sql varchar2(4000);
v_main_sql varchar2(32767);
v_x1 number;
v_x2 char;
v_y1 number;
v_y2 char;
v_col1 my_tab.col1%type;
v_col2 my_tab.col2%type;
-- define local functions/procs
function get_tab_data(p_cur_num in number, p_cur_type in char)
return my_subquery_tab pipelined
IS
v_ret my_subquery_tab;
begin
if (p_cur_num = 1) then
if (upper(p_cur_type) = 'X') then
for rec in (select * from my_tab where col1=1 and col3='X')
loop
pipe row(rec);
end loop;
elsif (upper(p_cur_type) = 'Y') then
for rec in (select * from my_tab where col1=1 and col3='Y')
loop
pipe row(rec);
end loop;
else
return;
end if;
elsif (p_cur_num = 2) then
if (upper(p_cur_type) = 'X') then
for rec in (select * from my_tab where col1=2 and col3='X')
loop
pipe row(rec);
end loop;
elsif (upper(p_cur_type) = 'Y') then
for rec in (select * from my_tab where col1=2 and col3='Y')
loop
pipe row(rec);
end loop;
else
return;
end if;
end if;
return;
end;
BEGIN
---------------------------------
-- Setup SQL for cursors
---------------------------------
-- this will have different parameter values for subqueries
v_with_sql := q'{
with X as (select * from table(get_tab_data(:x1, :x2))),
Y as (select * from table(get_tab_data(:y1, :y2)))
}';
-- this will stay the same for all cursors
v_main_sql := q'{
select X.col1, Y.col2 from X,Y where X.col1 = Y.col1
}';
---------------------------------
-- set initial subquery parameters
---------------------------------
v_x1 := 1;
v_x2 := 'x';
v_y1 := 1;
v_y2 := 'y';
open v_ref_cur for v_with_sql || v_main_sql using v_x1, v_x2, v_y1, v_y2;
loop
fetch v_ref_cur into v_col1, v_col2;
exit when v_ref_cur%notfound;
dbms_output.put_line(v_col1 || ',' || v_col2);
end loop;
close v_ref_cur;
---------------------------------
-- change subquery parameters
---------------------------------
v_x1 := 2;
v_x2 := 'x';
v_y1 := 2;
v_y2 := 'y';
open v_ref_cur for v_with_sql || v_main_sql using v_x1, v_x2, v_y1, v_y2;
loop
fetch v_ref_cur into v_col1, v_col2;
exit when v_ref_cur%notfound;
dbms_output.put_line(v_col1 || ',' || v_col2);
end loop;
close v_ref_cur;
end;
Note the benefit now is that even if you have many different cursors, you only need to define the main query and subquery SQL once. After that, you're just changing variables.
Cheers
--Create views that will be replaced by common table expressions later.
--The column names have to be the same, the actual content doesn't matter.
create or replace view x as select 'wrong' col1, 'wrong' col2 from dual;
create or replace view y as select 'wrong' col1, 'wrong' col2 from dual;
--Put the repetitive logic in one view
create or replace view main_select as
select count(x.col1) total, x.col2
from X inner join Y on x.col1 = y.col1
group by rollup (x.col1);
--Just querying the view produces the wrong results
select * from main_select;
--But when you add the common table expressions X and Y they override
--the dummy views and produce the real results.
declare
cursor cursor_1 is
with X as (select 'right' col1, 'right' col2 from dual),
Y as (select 'right' col1, 'right' col2 from dual)
select total, col2 from main_select;
--... repeat for each cursor, just replace X and Y as necessary
begin
for r in cursor_1 loop
dbms_output.put_line(r.col2);
end loop;
null;
end;
/
This solution is a little weirder than the pipelined approach, and requires 3 new objects for the views, but it will probably run faster
since there is less context switching between SQL and PL/SQL.
One possibility you could consider is using 2 Global Temporary Tables (GTTs) for X and Y. Then you just need one cursor, but you have to clear and re-populate the 2 GTTs several times - and if data volumes are large you may want to get optimiser stats on the GTTs each time too.
This is the sort of thing I mean:
cursor_gtt is
select count(X.col1), ...
from GTT_X inner join GTT_Y on...
group by rollup (X.col1, ...
begin
insert into gtt_x select col1, col2 from TAB where col1 = '1';
insert into gtt_y select col1, col2 from TAB where col2 = '3';
-- maybe get stats for gtt_x and gtt_y here
for r in cursor_gtt loop
print_report_results(r);
end loop;
delete gtt_x;
delete gtt_y;
insert into gtt_x select col1, col2 from TAB where col1 = '7' and col2 = '9' and col3 = 'TEST';
insert into gtt_y select col1, col2 from TAB where col3 = '6'
-- maybe get stats for gtt_x and gtt_y here
for r in cursor_gtt loop
print_report_results(r);
end loop;
...
end;
So the same 2 GTTs are re-populated and the same cursor is used each time.
What about creating a view for the main query? That pretties up your code and centralizes the main query to boot.
Related
I am trying to insert the result of a UNION into a table .While I cannot reproduce the original query here, due to it being in a restricted environment, it's structure looks similar to this:
WITH temp(X,Y,Z) AS(
SELECT....
)
SELECT X,Y,Z from TEMP --PART A
UNION
SELECT 'A','B','C' FROM DUAL;--PART B
Part A of the query returns about 1000 records, while Part B is just a single record.
When I wrap this whole query inside a procedure, such that the result of the select statements are inserted into a table, I see that only Part B, a single record gets inserted into the target table.Here is what it loos like when I wrap it inside the procedure.
CREATE OR REPLACE PROCEDURE PROC_INSERT
AS
BEGIN
INSERT INTO TABLENAME(COLUMN1,COLUMN2....)
WITH temp(X,Y,Z) AS(
SELECT....
)
SELECT X,Y,Z from TEMP
UNION
SELECT 'A','B','C' FROM DUAL;
COMMIT;
END;
I tried using both UNION and UNION ALL, but somehow I don't see to get the Part A of the result set inserted into the table.
What could be the possible reasons for such a scenario to occur? I tried to replicate the same ,but failed.
You can do something like this
DECLARE
CURSOR CUR IS SELECT X, Y, Z FROM OG_TABLE UNION SELECT 'A', 'B', 'C' FROM DUAL;
BEGIN
FOR REC IN CUR
LOOP
INSERT INTO NEWTABLE (COL1, COL2, COL3) VALUES (REC.X, REC.Y, REC.Z);
END LOOP;
END;
Alternative Optimized code
DECLARE
LIM PLS_INTEGER := 100;
CURSOR CUR IS SELECT X, Y, Z FROM OG_TABLE UNION SELECT 'A', 'B', 'C'
FROM DUAL;
TYPE OG_REC IS RECORD( X OG_TABLE.X%TYPE, Y OG_TABLE.Y%TYPE, Z
OG_TABLE%TYPE);
TYPE OGT IS TABLE OF OG_REC;
OG_TAB OGT;
BEGIN
OPEN CUR;
LOOP
FETCH CUR BULK COLLECT INTO OG_TAB LIMIT LIM;
BEGIN
FORALL i IN 1..OG_TAB.COUNT SAVE EXCEPTIONS
INSERT INTO NEWTABLE (COL1, COL2, COL3) VALUES (OG_TAB(i).X,
OG_TAB(i).Y, OG_TAB(i).Z);
EXCEPTION
WHEN OTHERS THEN
NULL;
END;
EXIT WHEN CUR%NOTFOUND;
END LOOP;
COMMIT;
CLOSE CUR;
END;
I have cursor it selects from TableA then Fetch Loop that inserts into TableB.
I want to check if the value already exists in the TableB.
If it exists then I want to skip the insert.
create or replace
PROCEDURE DAILY_RPT (
v_start IN DATE,
v_end IN DATE)
IS
ao_out_no out_pair.out_no%type;
cursor get is
SELECT ao_out_no from tableA;
BEGIN
open get;
LOOP
fetch get into ao_out_no;
EXIT WHEN get%NOTFOUND;
if (ao_out_no = (select out_no from TableA where out_no = ao_out_no) THEN
--DO NOTHING
else
INSERT INTO TABLEB(OUT_NO) VALUES (ao_out_no);
end if;
END LOOP;
close get;
END;
I used IF CONDITION however, I used variable into if condition & I am getting below.
PLS-00405: subquery not allowed in this context
if (ao_out_no = (select out_no from TableA where out_no = ao_out_no) THEN
You don't need cursor or PL/SQL at all:
INSERT INTO TABLEB(OUT_NO)
SELECT ao_out_no
FROM tableA ta
WHERE ... -- filtering rows
AND NOT EXISTS (SELECT * From TableB tb WHERE tb.OUT_NO = ta.ao_out_no);
Use the following :
for i in (
select out_no from TableA where out_no
)
loop
if i.out_no = ao_out_no
then
-- DO NOTHING
else
...
or
create a new variable named x, and then assign a value to it by
select out_no into x from TableA where out_no = ao_out_no;
and check returning value for x.
With corrected syntax, it would be something like this:
create or replace procedure daily_rpt
( v_start in date
, v_end in date )
as
begin
for r in (
select ao_out_no, 0 as exists_check
from tablea
)
loop
select count(*) into exists_check
from tablea
where out_no = r.ao_out_no
and rownum = 1;
if r.exists_check > 0 then
--DO NOTHING
else
insert into tableb (out_no) values (r.ao_out_no);
end if;
end loop;
end;
However, it's inefficient to query all of the rows and then do an additional lookup for each row to decide whether you want to use it, as SQL can do that kind of thing for you. So version 2 might be something like:
create or replace procedure daily_rpt
( v_start in date
, v_end in date )
as
begin
for r in (
select ao_out_no
from tablea
where not exists
( select count(*)
from tablea
where out_no = r.ao_out_no
and rownum = 1 )
)
loop
insert into tableb (out_no) values (r.ao_out_no);
end loop;
end;
at which point you might replace the whole loop with an insert ... where not exists (...) statement.
Can I use "execute immediate" with for cycle?
I need of generate a combinations of values in pl/sql.
About this I'm thinking of call a series of nested dynamics for cycle.
Example:
column 1 have this values: A,B
column 2 have this values: C,D,E
I would like to generate this combination:
AC / AD/ AE/ BC / BD /BE
I'm thinking to obtain this like:
for i in 1..count.column1
for j in 1..count.column2
dbms_output.put_line(column1.value(i)||'-'||column1.value(j));
end loop;
end loop;
Due to I don't know the number of column (variable),
Can I use a execute immediate?
declare
sql_stmt varchar2(200);
begin
sql_stmt := 'for i in 1..count.column1 for j in 1..count.column2 dbms_output.put_line(column1.value(i)||'-'||column1.value(j)); end loop; end loop';
execute immediate sql_stmt;
end;
But I have error ORA-06512.
How can i make this? :)
Thank you in advance for your suggestion!
If you really want to do this slowly in PL/SQL using nested loops, you don't need anything to be dynamic. You'd just want
for i in (select distinct column1
from your_table)
loop
for j in (select distinct column2
from your_table)
loop
dbms_output.put_line( i.column1 || j.column2 );
end loop;
end loop;
In the vast majority of cases, though, you'd be better off doing this in SQL
with col1 as (
select distinct col1 val
from your_table
),
col2 as (
select distinct col2 val
from your_table
)
select col1.val || col2.val
from col1
cross join col2
I think you can easily get the combinations using this select statement.
SELECT DISTINCT t1.col1, t2.col2
FROM table_name t1, table_name t2;
Is it possible to do something like this in pl/sql for bulk insert using FORALL?
TYPE c_type1 IS RECORD
(
column1 table1.column1%TYPE,
column2 table1.column2%TYPE,
client table2.client%TYPE
);
type1 c_type1;
CURSOR cur_t IS select * BULK COLLECT INTO recs from table3 ;
begin
FOR recs IN cur_t
LOOP
SELECT * INTO type1 FROM (select a.column1, a.column2,imm.client
...
from table1 a, table2 imm
WHERE
a.column1 = recs.column1
) WHERE ROWNUM=1;
INSERT INTO table2 values (recs.column1,type1.column2);
...
P.S : There are more 80 columns to be inserted.
Your question is not pretty clear but looking at your code I have the following. Check if this is what you were looking for.
declare
CURSOR cur_t IS
select t3.column1 , t1.column2
from table3 t3
inner join table1 t1
on t3.column1 = t1.column1;
type var_cur is table of cur_t%rowtype;
var var_cur;
begin
open cur_t;
LOOP
FETCH cur_t bulk collect into var limit 100;
EXIT WHEN cur_t%NOTFOUND;
FORALL i IN 1 .. var.count SAVE EXCEPTIONS
INSERT INTO TABLE2
VALUES var(i);
END LOOP;
CLOSE distinctUserIdCursor;
COMMIT;
EXCEPTION
WHEN OTHERS THEN
dbms_output.put_line('Error in Insertion of record' || '~~~~' || SQLERRM);
FOR indx IN 1 .. SQL%BULK_EXCEPTIONS.COUNT
LOOP
DBMS_OUTPUT.put_line (SQL%BULK_EXCEPTIONS (indx).ERROR_INDEX|| ': '
|| SQL%BULK_EXCEPTIONS (indx).ERROR_CODE);
END LOOP;
end;
I am trying to read values from two column1, column2 from table1 using cursor. Then I want to pass these values to another cursor or select into statement
so my PL/Sql script will use the values of these two columns to get data from another table called table2
Is this possible? And what's the best and fastest way to do something like that?
Thanks :)
Yes, it's possible to pass cursor values into variables. Just use fetch <cursor_name> into <variable_list> to get one more row from a cursor. After that you can use the variables in where clause of some select into statement. E.g.,
declare
cursor c1 is select col1, col2 from table1;
l_col1 table1.col1%type;
l_col2 table1.col2%type;
l_col3 table2.col3%type;
begin
open c1;
loop
fetch c1 into l_col1, l_col2;
exit when c1%notfound;
select col3
into l_col3
from table2 t
where t.col1 = l_col1 --Assuming there is exactly one row in table2
and t.col2 = l_col2; --satisfying these conditions
end loop;
close c1;
end;
If you use an implicit cursor, then it's even simpler:
declare
l_col3 table2.col3%type;
begin
for i in (select col1, col2 from table1)
loop
select col3
into l_col3
from table2 t
where t.col1 = i.col1 --Assuming there is exactly one row in table2
and t.col2 = i.col2; --satisfying these conditions
end loop;
end;
In these examples, it's more efficient to use a subquery
begin
for i in (select t1.col1
, t1.col2
, (select t2.col3
from table2 t2
where t2.col1 = t1.col1 --Assuming there is atmost one such
and t2.col2 = t1.col2 --row in table2
) col3
from table1 t1)
loop
...
end loop;
end;