Add elements one at a time from PL/SQL variable into collection variable? - oracle

I have a routine written in T-SQL for SQL Server. We are migrating to Oracle so I am trying to port it to PL/SQL. Here is the T-SQL routine (simplified); note the use of the table-valued variable which, in Oracle, will become a "nested table" type PL/SQL variable. The main thrust of my question is on the best ways of working with such "collection" objects within PL/SQL. Several operations in the ported code (second code sample, below) are quite awkward, where they seemed a lot easier in the SQL Server original:
DECLARE #MyValueCollection TABLE( value VARCHAR(4000) );
DECLARE #valueForThisRow VARCHAR(4000);
DECLARE #dataItem1Val INT, #dataItem2Val INT, #dataItem3Val INT, #dataItem4Val INT;
DECLARE theCursor CURSOR FAST_FORWARD FOR
SELECT DataItem1, DataItem2, DataItem3, DataItem4 FROM DataTable;
OPEN theCursor;
FETCH NEXT FROM theCursor INTO #dataItem1Val, #dataItem2Val, #dataItem3Val, #dataItem4Val;
WHILE ##FETCH_STATUS = 0
BEGIN
-- About 50 lines of logic that evaluates #dataItem1Val, #dataItem2Val, #dataItem3Val, #dataItem4Val and constructs #valueForThisRow
SET #valueForThisRow = 'whatever';
-- !!! This is the row that seems to have no natural Oracle equivalent
INSERT INTO #MyValueCollection VALUES(#valueForThisRow);
FETCH NEXT FROM theCursor INTO #dataItem1Val, #dataItem2Val, #dataItem3Val, #dataItem4Val;
END;
CLOSE theCursor;
DEALLOCATE theCursor;
-- !!! output all the results; this also seems harder than it needs to be in Oracle
SELECT * FROM #MyValueCollection;
I have been able to port pretty much everything, but in two places (see comments in the code), the logic is a lot more complex than the old SQL Server way, and I wonder if there might be, in Oracle, some more graceful way that is eluding me:
set serveroutput on; -- needed for DBMS_OUTPUT; see below
DECLARE
TYPE StringList IS TABLE OF VARCHAR2(4000);
myValueCollection StringList;
dummyTempCollection StringList; -- needed for my kludge; see below
valueForThisRow VARCHAR2(4000);
BEGIN
-- build all the sql statements
FOR c IN (
SELECT DataItem1, DataItem2, DataItem3, DataItem4 FROM DataTable;
)
LOOP
-- About 50 lines of logic that evaluates c.DataItem1, c.DataItem2, c.DataItem3, c.DataItem4 and constructs valueForThisRow
valueForThisRow := 'whatever';
-- This seems way harder than it should be; I would rather not need an extra dummy collection
SELECT valueForThisRow BULK COLLECT INTO dummyTempCollection FROM dual; -- overwrites content of dummy temp
myValueCollection := myValueCollection MULTISET UNION dummyTempCollection; -- merges into main collection
END LOOP;
-- output all the results... again, there's no shorter/easier/more-compact/single-line equivalent?
IF myValueCollection.COUNT > 0
THEN
FOR indx IN myValueCollection.FIRST .. myValueCollection.LAST
LOOP
DBMS_OUTPUT.PUT_LINE(myValueCollection(indx));
END LOOP;
END IF;
END;
/
Thanks in advance for any help!

Personally, I'd take the "50 lines of logic", move it into a function that you call in your SQL statement, and then do a simple BULK COLLECT to load the data into your local collection.
Assuming that you really want to load data element-by-element into the collection, you can simplify the code that loads the collection
DECLARE
TYPE StringList IS TABLE OF VARCHAR2(4000);
myValueCollection StringList := StringList();
valueForThisRow VARCHAR2(4000);
BEGIN
-- build all the sql statements
FOR c IN (
SELECT DataItem1, DataItem2, DataItem3, DataItem4 FROM DataTable;
)
LOOP
-- About 50 lines of logic that evaluates c.DataItem1, c.DataItem2, c.DataItem3, c.DataItem4 and constructs valueForThisRow
valueForThisRow := 'whatever';
myValueCollection.extend();
myValueCollection( myValueCollection.count ) := valueForThisRow;
END LOOP;
-- output all the results... again, there's no shorter/easier/more-compact/single-line equivalent?
IF myValueCollection.COUNT > 0
THEN
FOR indx IN myValueCollection.FIRST .. myValueCollection.LAST
LOOP
DBMS_OUTPUT.PUT_LINE(myValueCollection(indx));
END LOOP;
END IF;
END;
/
If you declare the collection as an associative array, you could avoid calling extend to increase the size of the collection. If you know the number of elements that you are going to load into the collection, you could pass that to a single extend call outside the loop. Potentially, you can also eliminate the valueForThisRow local variable and just operate on elements in the collection.
As for the code that processes the collection, what is it that you are really trying to do? It would be highly unusual for production code to write to dbms_output and expect that anyone will see the output during normal processing. That will influence the way that you would write that code. Assuming that your intention is really to just call dbms_output, knowing that will generally send the data into the ether
FOR indx IN 1 .. myValueCollection.count
LOOP
dbms_output.put_line( myValueCollection(indx) );
END LOOP;
This works when you have a dense collection (all indexes between 1 and the count of the collection exist and have values). If you might have a sparse collection, you would want to use FIRST, NEXT, and LAST in a loop but that's a bit more code.

Related

How to remove data from or empty a cursor in PLSQL Oracle Procedure?

I want to fetch some rows from a table into a cursor and after doing my desired work,I want to empty the cursor so that I can use the cursor again. the whole process will be inside an loop. I have searched for this many times but I am not getting my desired post. I think I have made myself clear what do I want. In short, I want to treat my cursor as an ArrayList in Java.
As others have stated you need to fetch your cursor into a collection (or don't use an explicit cursor and BULK COLLECT directly with INTO (see below). Once you have it in a collection you can iterate through and delete entries that meet some requirement. PL/SQL collections have first and next functions that work together like an Iterator in Java so you can delete entries as you loop through them.
DECLARE
-- table definition
TYPE t_obj_tab IS TABLE OF all_objects%ROWTYPE;
v_obj_tab t_obj_tab;
-- local variables
v_index INTEGER;
BEGIN
-- get some data into a collection
SELECT *
BULK COLLECT
INTO v_obj_tab
FROM all_objects
WHERE rownum <= 500;
-- log the count for demonstration
dbms_output.put_line(v_obj_tab.count);
-- get the index of the first element in the collection
v_index := v_obj_tab.first;
-- loop through the collection
WHILE v_index IS NOT NULL
LOOP
-- delete objects with some criteria
IF v_obj_tab(v_index).object_name LIKE '%E%' THEN
v_obj_tab.delete(v_index);
END IF;
-- get next element of collection
v_index := v_obj_tab.next(v_index);
END LOOP;
-- log the count to see if any were deleted
dbms_output.put_line(v_obj_tab.count);
END;
/
-- 500
-- 206
You should be aware that pulling a large amount of objects into a collection will use a proportional amount of memory. PL/SQL collections should aim to be small when possible (personally I like 500 rows for bulk processing, but you should benchmark). It would be advisable to, instead of pulling data and filtering it procedurally, use SQL to filter the results to only return the rows you are interested in (as others also mentioned).
I think you were also asking how to process the same set of data multiple times. If you want to do that you can make a second collection variable, set it to the first (which makes a copy) and keep that as the original set. When you are done processing you can use that to get the original rows and do some different processing to them.
DECLARE
-- table definition
TYPE t_obj_tab IS TABLE OF all_objects%ROWTYPE;
v_obj_tab t_obj_tab;
v_obj_tab_2 t_obj_tab;
-- local variables
v_index INTEGER;
BEGIN
-- get some data into a collection
SELECT *
BULK COLLECT
INTO v_obj_tab
FROM all_objects
WHERE rownum <= 500;
-- copy!
v_obj_tab_2 := v_obj_tab;
-- log the count for demonstration
dbms_output.put_line(v_obj_tab.count);
-- get the index of the first element in the collection
v_index := v_obj_tab.first;
-- loop through the collection
WHILE v_index IS NOT NULL
LOOP
-- delete objects with some criteria
IF v_obj_tab(v_index).object_name LIKE '%E%' THEN
v_obj_tab.delete(v_index);
END IF;
-- get next element of collection
v_index := v_obj_tab.next(v_index);
END LOOP;
-- log the count to see if any were deleted
dbms_output.put_line(v_obj_tab.count);
dbms_output.put_line(v_obj_tab_2.count);
END;
/
-- 500
-- 206
-- 500 (copy count)

cannot mix between single row and multi-row (BULK) in INTO list.ERROR in proc

CREATE OR REPLACE PROCEDURE RDBSTAGE.ATCHMNT_ERR_FILEID AUTHID CURRENT_USER
IS
CURSOR cv_atchtab IS
SELECT * FROM ATTACHMENT_ERROR;
I_ATCHMNT_ERR cv_atchtab%ROWTYPE;
V_FILE_ID VARCHAR2(40);
BEGIN
OPEN cv_atchtab;
LOOP
FETCH cv_atchtab BULK COLLECT INTO I_ATCHMNT_ERR;
EXIT WHEN cv_atchtab%NOTFOUND;
FOR i IN 1..I_ATCHMNT_ERR.COUNT
LOOP
SELECT FILE_ID BULK COLLECT
INTO V_FILE_ID
FROM ATTACHMENT_CLAIM t1
WHERE t1.CLAIM_TCN_ID=I_ATCHMNT_ERR(i).CLAIM_TCN_ID;
UPDATE ATTACHMENT_ERROR
SET FILE_ID = V_FILE_ID
WHERE t1.CLAIM_TCN_ID=I_ATCHMNT_ERR.CLAIM_TCN_ID;
END LOOP;
END LOOP;
CLOSE cv_atchtab;
END;
END ATCHMNT_ERR_FILEID;
/
SHOW ERRORS
Procedure ATCHMNT_ERR_FILEID compiled
Errors: check compiler log Errors for PROCEDURE
RDBSTAGE.ATCHMNT_ERR_FILEID:
LINE/COL ERROR
11/40 PLS-00497: cannot mix between single row and multi-row (BULK) in INTO list
14/5 PL/SQL: Statement ignored
14/31 PLS-00302: component 'COUNT' must be declared
PLS-00497: cannot mix between single row and multi-row (BULK) in INTO list
BULK COLLECT INTO is the syntax for populating a PL/SQL collection from a query. But your code is populating a scalar single row variable.
14/31 PLS-00302: component 'COUNT' must be declared
I_ATCHMNT_ERR.COUNT is invalid because count() only applies to collections I_ATCHMNT_ERR is scalar.
To fix this you need to define and use collection types. Something like this:
CREATE OR REPLACE PROCEDURE ATCHMNT_ERR_FILEID
IS
CURSOR cv_atchtab IS
SELECT * FROM ATTACHMENT_ERROR;
type ATCHMNT_ERR_nt is table of cv_atchtab%ROWTYPE;
I_ATCHMNT_ERR ATCHMNT_ERR_nt;
V_FILE_ID VARCHAR2(40);
BEGIN
OPEN cv_atchtab;
LOOP
FETCH cv_atchtab BULK COLLECT INTO I_ATCHMNT_ERR; -- collection type
EXIT WHEN I_ATCHMNT_ERR.COUNT = 0; -- changed this
FOR i IN 1..I_ATCHMNT_ERR.COUNT
LOOP
SELECT FILE_ID
INTO V_FILE_ID -- scalar type
FROM ATTACHMENT_CLAIM t1
WHERE t1.CLAIM_TCN_ID = I_ATCHMNT_ERR(i).CLAIM_TCN_ID;
UPDATE ATTACHMENT_ERROR t2 -- changed this
SET FILE_ID = V_FILE_ID
WHERE t2.CLAIM_TCN_ID = I_ATCHMNT_ERR(i).CLAIM_TCN_ID; -- changed this
END LOOP;
END LOOP;
CLOSE cv_atchtab;
END ATCHMNT_ERR_FILEID;
Here is a db<>fiddle demo of the above working against my guess of the data model.
The Oracle documentation is comprehensive, online and free. The PL/SQL Guide has a whole chapter on Collections and Records which I suggest you read. Find it here.
As an aside, nested loops with single row statements like this are usually a red flag in PL/SQL. They are pretty inefficient and slow. SQL is a set-based language, and we should always try to solve problems using SQL whenever possible, and ideally in one set-based statement. If this code is intended for production (rather than being a homework assignment) you should definitely consider re-writing it in a more performative fashion.

PLSQL IMPLICIT CURSOR No Data Found After CURSOR

I have a Main cursor that is working fine.
declare
v_firm_id number;
amount number;
v_total_sum TABLE_TEMP.TOTAL_SUM%TYPE;
CURSOR MT_CURSOR IS
SELECT firm_id FROM t_firm;
BEGIN
OPEN MT_CURSOR;
LOOP
FETCH MT_CURSOR INTO v_firm_id;
EXIT WHEN MT_CURSOR%NOTFOUND;
DBMS_OUTPUT.PUT_LINE(to_char(sysdate, 'mi:ss') ||'--- '|| v_firm_id)
INSERT INTO TABLE_TEMP(TOTAL_SUM) VALUES(v_firm_id)
COMMIT;
END LOOP;
DBMS_LOCK.SLEEP(20);
BEGIN
FOR loop_emp IN
(SELECT TOTAL_SUM INTO v_total_sum FROM TABLE_TEMP)
LOOP
dbms_output.put_line(to_char(sysdate, 'mi:ss') ||'--- '|| v_total_sum || '-TEST--');
END LOOP loop_emp;
END;
end;
Everything Works fine except dbms_output.put_line(v_total_sum || '---');
I do not get any data there. I get the correct number of rows. which it inserted.
The problem is the cursor FOR loop has a redundant into clause which it appears the compiler silently ignores, and so v_total_sum is never used.
Try this:
begin
for r in (
select firm_id from t_firm
)
loop
insert into table_temp (total_sum) values (r.firm_id);
end loop;
dbms_lock.sleep(20);
for r in (
select total_sum from table_temp
)
loop
dbms_output.put_line(r.total_sum || '---');
end loop;
commit;
end;
If this had been a stored procedure rather than an anonymous block and you had PL/SQL compiler warnings enabled with alter session set plsql_warnings = 'ENABLE:ALL'; (or the equivalent preference setting in your IDE) then you would have seen:
PLW-05016: INTO clause should not be specified here
I also moved the commit to the end so you only commit once.
To summarise the comments below, the Cursor FOR loop construction declares, opens, fetches and closes the cursor for you, and is potentially faster because it fetches in batches of 100 (or similar - I haven't tested in recent versions). Simpler code has less chance of bugs and is easier to maintain in the future, for example if you need to add a column to the cursor.
Note the original version had:
for loop_emp in (...)
loop
...
end loop loop_emp;
This is misleading because loop_emp is the name of the record, not the cursor or the loop. The compiler is ignoring the text after end loop although really it should at least warn you. If you wanted to name the loop, you would use a label like <<LOOP_EMP>> above it. (I always name my loop records r, similar to the i you often see used in numeric loops.)

Copy REF CURSOR to a temporary table

Is there a quick way to flush REF CURSOR into a temporary table in oracle
CURSOR size will vary between 5K to 100K rows
currently I am using a for loop, but its very slow
There is no way to change the OUT parameter for SOME_FUNCTION_CALL function.
DECLARE
v_return SYS_REFCURSOR;
VAR_A varchar2(100);
VAR_B varchar2(100);
BEGIN
v_return := SOME_FUNCTION_CALL();
LOOP
FETCH v_return into VAR_A,VAR_B
EXIT WHEN v_return%NOTFOUND
INSERT INTO temp_table(a,b) values (VAR_A,VAR_B);
END LOOP;
CLOSE v_return;
END LOOP;
Probably not.
You could potentially make the code more efficient by doing a BULK COLLECT into local collections rather than doing row-by-row fetches. Something like
DECLARE
TYPE vc100_tbl IS TABLE OF VARCHAR2(100);
l_return SYS_REFCURSOR;
l_var_a_tbl vc100_tbl;
l_var_b_tbl vc100_tbl;
BEGIN
l_return := some_function_call();
LOOP
FETCH l_return
BULK COLLECT INTO l_var_a_tbl, l_var_b_tbl
LIMIT 100;
EXIT WHEN l_var_a_tbl.count = 0;
FORALL i IN 1 .. l_var_a_tbl.count
INSERT INTO temp_table( a, b )
VALUES( l_var_a_tbl(i), l_var_b_tbl(i) );
END LOOP;
close l_return;
END;
That will reduce the number of context shifts taking place between the SQL and PL/SQL engines. But depending on your definition of "very slow", it seems unlikely that context shifts are your most pressing problem. Were I to guess, I'd wager that if your code is "very slow" for most definitions of that term, the problem is likely that the query that is being used to open the cursor in some_function is slow and that most of your time is being spent executing that query in order to generate the rows that you want to fetch. If that's the case, you'd want to optimize the query.
Architecturally, this approach also seems rather problematic. Even if you can't change the definition of the function, can't you do something to factor out the query the function is executing so that you can use the same code without calling the function? For example, could you not factor out the query the function calls into a view and then call that view from your code?

Oracle: Fastest Way in PL/SQL to See if Value Exists: List, VARRAY, or Temp Table

UPDATE View the edits if you care to see the long original question. This is the clearer short version of the question...
I need to see if GroupA (not always GroupA, this changes each loop iteration) exists in a [list,varray,temp table, whatever] of 200 or so groups. How I store those 200 groups is totally in my control. But I want to store them in a construct that lends itself to the FASTEST "existence" checking because I will have to check this list MANY times within a loop against different values (not always GroupA). So whats fastest in PL/SQL, checking a list...
IF 'GroupA' IN ('GroupA','GroupB') THEN...
or checking a VARRAY using MEMBER OF...
IF 'GroupA' MEMBER OF myGroups THEN
or checking a VARRAY this way...
FOR i IN myGroups.FIRST .. myGroups.LAST
LOOP
IF myGroups(i) = 'GroupA' THEN
v_found := TRUE;
EXIT;
END IF;
END LOOP;
or checking associative arrays...
will test this tomorrow
UPDATE: FINAL RESULTS OF TESTING FROM EVERYONE'S SUGGESTIONS
Thanks all.
I ran these tests, looped 10 million times and the commas separated string using a LIKE seemed to be the fastest so I guess the points have to go to #Brian McGinity (the times are in the comments below). But since the times were all so close it probably doesn't matter which method I go with. I think I'll go with the VARRAY MEMBER OF method since I can load the array with a single line of code (bulk collect) instead of having to loop a cursor to build a string (thanks #Wernfried for bringing MEMBER OF to my attention)...
comma separated list, example: ,GroupA,GroupB,GroupC,...around 200 groups... (list made by looping a cursor)
FOR i IN 1 .. 10000000 loop
if myGroups like '%,NONE,%' then
z:=z+1;
end if;
end loop;
--690msec
same commas separated list (list made by looping a cursor)...
FOR i IN 1 .. 10000000 loop
if instr(myGroups, ',NONE,') > 0 then
z:=z+1;
end if;
end loop;
--818msec
varray, same 200 groups (varray made by bulk collect)...
FOR i IN 1 .. 10000000 loop
IF 'NONE' MEMBER of myGroups THEN
z:=z+1;
end if;
end loop;
--780msec
associative array method suggested by #Yaroslav Shabalin (assoc. array made by looping a cursor)...
FOR i IN 1 .. 10000000 loop
if (a_values('NONE') = 1) then
z:=z+1;
end if;
end loop;
--851msec
Is myGroup a varray? If it is a string try something like:
select 1
from dual
where 'abc,NONE,def' like '%,NONE,%'
It is hard to follow the constraints you're working under... If at all possible, do everything inside of sql and it will be faster.
Update:
So if you're already in a plsql unit and wanted to stay in a plsql unit then the logic above would go something like this:
declare
gp varchar2(200) := 'abc,def,NONE,higlmn,op';
begin
if ','||gp||',' like '%,NONE,%' then
dbms_output.put_line('y');
else
dbms_output.put_line('n');
end if;
end;
if this itself is in a loop then, make the list once as:
declare
gp varchar2(200) := 'abc,def,NONE,higlmn,op';
gp2 varchar2(200) := ',' || gp || ',';
begin
if g2 like '%,NONE,%' then
dbms_output.put_line('y');
else
dbms_output.put_line('n');
end if;
end;
Also try instr which is probably faster than like:
declare
gp varchar2(200) := ',abc,def,NONE,hig,';
begin
if instr(gp, ',NONE,') > 0 then
dbms_output.put_line('y');
else
dbms_output.put_line('n');
end if;
end;
I have no idea if this faster than the other solutions mentioned (it stands a good chance), it is something else to try.
I did not get your full question but perhaps this function helps you:
MEMBER Condition
WHERE 'groupA' MEMBER of myGroups
Have you considered using associative arrays also formerly known as "index-by tables"? Associative arrays indexed by string are optimized for efficient lookup by implicitly using the B*-tree organization of the values. It is PL/SQL equivalent to hash tables in other programming languages.
For example if you define an array as:
type t_values is table of number index by varchar2(20);
Then assign GroupA etc. to keys and 1 to each respective value:
a_values t_values;
for c_cursor in (select ...)
loop
a_value(c_cursor.group_name) := 1;
end loop;
When you try to access the value for non-existent index, you will get null. Whereas for any real index you have 1 returned;
(a_value('GroupA') = 1) => TRUE
(a_value('Some_not_existent_index') IS NULL) => TRUE

Resources