Strange behaviour of BULK COLLECT - oracle

I tinkered together the following PL/SQL BULK-COLLECT which works astonishingly fast for updates on huge tables (>50.000.000). The only problem is, that it does not perform the updates of the remaining < 5000 rows per table. 5000 is the given limit for the FETCH instruction:
DECLARE
-- source table cursor (only columns to be updated)
CURSOR base_table_cur IS
select a.rowid, TARGET_COLUMN from TARGET_TABLE a
where TARGET_COLUMN is null;
TYPE base_type IS
TABLE OF base_table_cur%rowtype INDEX BY PLS_INTEGER;
base_tab base_type;
-- new data
CURSOR new_data_cur IS
select a.rowid,
coalesce(b.SOURCE_COLUMN, 'FILL_VALUE'||a.JOIN_COLUMN) TARGET_COLUMN from TARGET_TABLE a
left outer join SOURCE_TABLE b
on a.JOIN_COLUMN=b.JOIN_COLUMN
where a.TARGET_COLUMN is null;
TYPE new_data_type IS TABLE OF new_data_cur%rowtype INDEX BY PLS_INTEGER;
new_data_tab new_data_type;
TYPE row_id_type IS TABLE OF ROWID INDEX BY PLS_INTEGER;
row_id_tab row_id_type;
TYPE rt_update_cols IS RECORD (
TARGET_COLUMN TARGET_TABLE.TARGET_COLUMN%TYPE
);
TYPE update_cols_type IS
TABLE OF rt_update_cols INDEX BY PLS_INTEGER;
update_cols_tab update_cols_type;
dml_errors EXCEPTION;
PRAGMA exception_init ( dml_errors,-24381 );
BEGIN
OPEN base_table_cur;
OPEN new_data_cur;
LOOP
FETCH base_table_cur BULK COLLECT INTO base_tab LIMIT 5000;
IF base_table_cur%notfound THEN
DBMS_OUTPUT.PUT_LINE('Nothing to update. Exiting.');
EXIT;
END IF;
FETCH new_data_cur BULK COLLECT INTO new_data_tab LIMIT 5000;
FOR i IN base_tab.first..base_tab.last LOOP
row_id_tab(i) := new_data_tab(i).rowid;
update_cols_tab(i).TARGET_COLUMN := new_data_tab(i).TARGET_COLUMN;
END LOOP;
FORALL i IN base_tab.first..base_tab.last SAVE EXCEPTIONS
UPDATE (SELECT TARGET_COLUMN FROM TARGET_TABLE)
SET row = update_cols_tab(i)
WHERE ROWID = row_id_tab(i);
COMMIT;
EXIT WHEN base_tab.count < 5000; -- changing to 1 didn't help!
END LOOP;
COMMIT;
CLOSE base_table_cur;
CLOSE new_data_cur;
EXCEPTION
WHEN dml_errors THEN
FOR i IN 1..SQL%bulk_exceptions.count LOOP
dbms_output.put_line('Some error occured');
END LOOP;
END;
Where is my mistake? It looks correct to me though.

The problem is this line:
IF base_table_cur%notfound THEN
The cursor meets %NOTFOUND when the number of records found is less than the LIMIT value. So if the last fetch is not exactly 5000 those records won't be processed.
It's a common gotcha for people using BULK COLLECT ... LIMIT for the first time. The solution is to change the exit condition to
EXIT when base_tab.count() = 0;
"I need to ensure, that the base_table_cur is not empty and exit if it is. I'l get an error if it is empty"
The new_data_cur cursor includes the table which is selected in base_table_cur cursor. So I don't think you need the two loops. You need a simple test to see whether the first cursor returns something, then just loop round the second cursor.
I'm not entirely clear on your logic, so I have changed as little as possible to demonstrate the sort of structure I think you need. However, the UPDATE statement looks a little odd, so you may still run into issues.
OPEN base_table_cur;
FETCH base_table_cur BULK COLLECT INTO base_tab LIMIT 1;
if base_table_tab.count = 0 then
DBMS_OUTPUT.PUT_LINE('Nothing to update. Exiting.');
else
OPEN new_data_cur;
LOOP
FETCH new_data_cur BULK COLLECT INTO new_data_tab LIMIT 5000;
exit when new_data_tab.count() = 0;
FOR i IN base_tab.first..base_tab.last LOOP
row_id_tab(i) := new_data_tab(i).rowid;
update_cols_tab(i).TARGET_COLUMN := new_data_tab(i).TARGET_COLUMN;
END LOOP;
FORALL i IN base_tab.first..base_tab.last SAVE EXCEPTIONS
UPDATE (SELECT TARGET_COLUMN FROM TARGET_TABLE)
SET row = update_cols_tab(i)
WHERE ROWID = row_id_tab(i);
END LOOP;
CLOSE new_data_cur;
end if;
COMMIT;
CLOSE base_table_cur;

Related

No data found in oracle

I have a procedure in oracle that looks like this
create or replace procedure check_display_stock as
id_brg number(20);
rowcount number (20);
begin
-- 1. insert datas from display into temp_display
insert into temp_display
select id_barang,stok,min_stok
from display
where stok <= min_stok;
--2. select the number of datas in temp_display
select count(rownum)
into rowcount
from temp_display;
while(rowcount != 0)
loop
-- Error: no data found
select id_barang
into id_brg
from temp_display
where rownum = 1;
--just another procedure to do other things
insert_spb(id_brg);
delete from temp_display where rownum = 1;
end if;
end loop;
end check_display_stock;
An error occurs when I tried to select into that says no data found.
I don't understand why this happened.
You never decrement rowcount so you will end up deleting the rows in temp_display one-by-one and then keep going (potentially forever) and on the next iteration after you have already emptied the table you will try to select id_barang into id_brg ... and it will fail as you have already emptied the table.
Instead, you could use BULK COLLECT INTO or a CURSOR to bypass the temporary table:
create or replace procedure check_display_stock
as
begin
FOR cur IN ( select id_barang,
stok,
min_stok
from display
where stok <= min_stok
)
LOOP
insert_spb(cur.id_barang);
END LOOP;
END;
/
db<>fiddle here

Oracle PL/SQL how do you output how many inserts have been made in a FORALL statement

What's the best way of getting and outputting how many rows have been inserted in the FORALL statement I have below. I've seen the SQL%BULK_ROWCOUNT but I'm not sure how that would work in the below statement.
is it
DBMS_OUTPUT.('rows inserted '||SQL%BULK_ROWCOUNT||'');
Does the above need to go in another FORALL statement? For the code below how would I achieve this?
DECLARE
TYPE t_arc_act_plus_trigger1 IS TABLE OF arc_act_plus_triggers1%ROWTYPE;
v_arc_act_plus_triggers1 t_arc_act_plus_trigger1;
CURSOR c_arc_act_plus_triggers1 IS
SELECT /*+ PARALLEL */ apt.*
FROM act_plus_triggers1 apt
WHERE NOT EXISTS
(SELECT 1
FROM act_plus_triggers_copy1 aptc
WHERE aptc.surr_id = apt.surr_id)
AND apt.status IN ('EXT', 'EXP');
BEGIN
OPEN c_arc_act_plus_triggers1;
LOOP
FETCH c_arc_act_plus_triggers1 BULK COLLECT INTO v_arc_act_plus_triggers1 LIMIT 10000; -- limit to 10k to avoid out of memory
FORALL i IN 1..v_arc_act_plus_triggers1.COUNT
INSERT /*+ APPEND_VALUES */ INTO arc_act_plus_triggers1 values v_arc_act_plus_triggers1(i);
Com0932.get_parameter ('ACT_ARCHIVE_TRIGGER_STOP_YN',l_STOP_PROGRAM_YN);
IF l_STOP_PROGRAM_YN = 'Y' THEN
p_location('insert_into_arc_act_plus - STOP_PROGRAM_YN flag = '||l_STOP_PROGRAM_YN||' so ROLLBACK');
ROLLBACK;
EXIT;
END IF;
-- **************************************************
-- Output how many records have been inserted here???
-- **************************************************
-- commit after every 10000 records into arc_act_plus_triggers1
COMMIT;
EXIT WHEN c_arc_act_plus_triggers1%NOTFOUND;
END LOOP;
CLOSE c_arc_act_plus_triggers1;
END;
I haven't checked as I have nothing to test against so please forgive any 'missing semi-colon type errors' and I'm afraid I'm not in a position to performance check this.
Your code seems to select which rows to insert to the archive table based on there non-existence in the archive. Therefore simply use an INSERT based on a SELECT limited by a suitable ROWNUM value. Once you commit then the next time round the loop it wont try getting already archived rows as you just committed them.
I think this should be as quick if not quicker than bulkifying the inserts with the advantage that its simpler - Occams Razor and all that.
DECLARE
l_commit_count NUMBER := 10000;
l_rows_copied NUMBER := 0;
BEGIN
DBMS_OUTPUT.PUT_LINE('Started at '||TO_DATE(SYSDATE, 'DD_MON_YYY HH24:MI:SS');
LOOP
INSERT /*+APPEND */
INTO c_arc_act_plus_triggers1
SELECT /*+ PARALLEL */ apt.*
FROM act_plus_triggers1 apt
WHERE NOT EXISTS
(SELECT 1
FROM act_plus_triggers_copy1 aptc
WHERE aptc.surr_id = apt.surr_id)
AND apt.status IN ('EXT', 'EXP')
AND rownum < l_commit_count;
COMMIT;
l_rows := l_rows + SQL%ROWCOUNT;
EXIT WHEN SQL%ROWCOUNT < 1;
END LOOP
DBMS_OUTPUT.PUT_LINE('Finished at '||TO_DATE(SYSDATE, 'DD_MON_YYY HH24:MI:SS');
DBMS_OUTPUT.PUT_LINE(TO_CHAR(l_rows)||' rows copied to the archive table');
END;

Sequence Number is not allowed in Explicit Cursors

I can't use a sequence in a Explicit Cursors.
Error: PL/SQL:ORA-02287: Sequence number not allowed here
Code:
CURSOR c_service_consumer IS
select ops$KLI.SVC_SEQ.Nextval,
adr_cli_id_s,
null,
svc_srv_id_s
....
I tried to use like this:
CURSOR c_service_consumer IS
select SELECT ops$KLI.SVC_SEQ.Nextval FROM DUAL,
adr_cli_id_s,
null,
svc_srv_id_s
And is the same error.
This is the Cursor:
TYPE l_service_consumer_row IS TABLE OF TA_MAIN.SERVICE_CONSUMER%ROWTYPE;
l_service_consumer l_service_consumer_row;
CURSOR c_service_consumer IS
select ops$KLI.SVC_SEQ.Nextval,
a.adr_cli_id_s,
svc_prp_id_s,
svc_srv_id_s
from ops$luan.be_1304_grain_adr a
left outer join ta_main.clients c on (a.adr_cli_id_s = c.cli_id_s);
BEGIN
OPEN c_service_consumer;
FETCH c_service_consumer BULK COLLECT INTO l_service_consumer;
CLOSE c_service_consumer;
/*Insert the collection of values*/
FORALL i IN INDICES OF l_service_consumer SAVE EXCEPTIONS
INSERT INTO TA_MAIN.SERVICE_CONSUMER
VALUES l_service_consumer(i);
Hope this helps (not tested)
TYPE l_service_consumer_row IS TABLE OF TA_MAIN.SERVICE_CONSUMER%ROWTYPE
INDEX BY BINARY_INTEGER;
l_service_consumer l_service_consumer_row;
fetch_size NUMBER := 5000; -- scale the value
CURSOR c_service_consumer IS
select -1 id,
a.adr_cli_id_s,
svc_prp_id_s,
svc_srv_id_s
from ops$luan.be_1304_grain_adr a
left outer join ta_main.clients c on (a.adr_cli_id_s = c.cli_id_s);
BEGIN
OPEN c_service_consumer;
loop
FETCH c_service_consumer BULK COLLECT INTO l_service_consumer LIMIT fetch_size;
FOR i IN 1 .. l_service_consumer.COUNT LOOP
l_service_consumer(i).id = ops$KLI.SVC_SEQ.Nextval;
END LOOP;
/*Insert the collection of values*/
FORALL i IN INDICES OF l_service_consumer SAVE EXCEPTIONS
INSERT INTO TA_MAIN.SERVICE_CONSUMER
VALUES l_service_consumer(i);
EXIT WHEN c_service_consumer%NOTFOUND;
END LOOP;
CLOSE c_service_consumer;
End;

How to find number of rows in cursor

I would like to find the number of rows in a cursor. Is there a keyword that can help? Using COUNT, we have to write a query. Any help will be greatly appreciated.
The cursor_variable.%ROWCOUNT is the solution. But its value will be 0 if you check it after opening. You need to loop through all the records, to get the total row count. Example below:
DECLARE
cur sys_refcursor;
cur_rec YOUR_TABLE%rowtype;
BEGIN
OPEN cur FOR
SELECT * FROM YOUR_TABLE;
dbms_output.put_line(cur%rowcount);--returning 0
LOOP
FETCH cur INTO cur_rec;
EXIT WHEN cur%notfound;
dbms_output.put_line(cur%rowcount);--will return row number beginning with 1
dbms_output.put_line(cur_rec.SOME_COLUMN);
END LOOP;
dbms_output.put_line('Total Rows: ' || cur%rowcount);--here you will get total row count
END;
/
You must open the cursor and then fetch and count every row. Nothing else will work.
You can also use BULK COLLECT so that a LOOP is not needed,
DECLARE
CURSOR c
IS SELECT *
FROM employee;
TYPE emp_tab IS TABLE OF c%ROWTYPE INDEX BY BINARY_INTEGER;
v_emp_tab emp_tab;
BEGIN
OPEN c;
FETCH c BULK COLLECT INTO v_emp_tab;
DBMS_OUTPUT.PUT_LINE(v_emp_tab.COUNT);
CLOSE c;
END;
/
Edit: changed employee%ROWTYPE to c%ROWTYPE
You can use following simple single line code to print cursor count
dbms_output.put_line(TO_CHAR(cur%rowcount));
This should work for you
DECLARE
CURSOR get_data_ IS
SELECT *
FROM table_abc_
WHERE owner = user_; -- your query
counter_ NUMBER:= 0;
BEGIN
FOR data_ IN get_data_ LOOP
counter_ := counter_ + 1;
END LOOP;
dbms_output.put_line (counter_);
END;
DECLARE #STRVALUE NVARCHAR(MAX),
#CREATEDDATE DATETIME,
#STANTANCEVALUE NVARCHAR(MAX),
#COUNT INT=0,
#JOBCODE NVARCHAR(50)='JOB00123654',
#DATE DATETIME=GETDATE(),
#NAME NVARCHAR(50)='Ramkumar',
#JOBID INT;
CREATE TABLE #TempContentSplitValue (ITEMS NVARCHAR(200))
SELECT #JOBID = i.Id FROM JobHeader_TBL i WHERE Id=1201;
IF EXISTS (SELECT 1 FROM JobHeader_TBL WHERE Id=#JOBID)
BEGIN
SELECT #STRVALUE= Description from ContentTemplate_TBL where Id=1
INSERT INTO #TempContentSplitValue SELECT * FROM dbo.split(#STRVALUE, '_')
SET #STRVALUE=''
DECLARE db_contentcursor CURSOR FOR SELECT ITEMS FROM #TempContentSplitValue
OPEN db_contentcursor
FETCH NEXT FROM db_contentcursor
INTO #STANTANCEVALUE
WHILE (##FETCH_STATUS = 0)
BEGIN
SET #STRVALUE += #STANTANCEVALUE + 'JOB00123654'
SET #COUNT += 1
SELECT #COUNT
FETCH NEXT FROM db_contentcursor INTO #STANTANCEVALUE
END
CLOSE db_contentcursor
DEALLOCATE db_contentcursor
DROP TABLE #TempContentSplitValue
SELECT #STRVALUE
END
Here I am trying to count the total number of customers with age greater than 25. So store the result in the cursor first. Then count the size of the cursor inside the function or in the main begin itself.
DECLARE
cname customer24.cust_name%type;
count1 integer :=0;
CURSOR MORETHAN is
SELECT cust_name
FROM customer24
where age>25;
BEGIN
OPEN MORETHAN;
LOOP
FETCH MORETHAN into cname;
count1:=count1+1;
EXIT WHEN MORETHAN%notfound;
END LOOP;
-- dbms_output.put_line(count1);
dbms_output.put_line(MORETHAN%ROWCOUNT);
CLOSE MORETHAN;
END;
There is a possible work around that may be useful/needed because of the overhead of accessing a database server over a network (e.g., when using Ajax calls)
Consider this:
CURSOR c_data IS
SELECT per_first_name , null my_person_count
FROM person
UNION
SELECT null as per_first_name , count( distinct per_id ) as my_person_count
FROM person
order by my_person_count ;
The first row fetched has the count of records. One MUST add specific columns fetched (the use of the * does not work), and one can add additional filters.
Try this:
print(len(list(cursor)))
I always read that people loop through results. Why not using a count(*)?
An example from my production code:
PROCEDURE DeleteStuff___(paras_ IN Parameters_Type_Rec)
IS
CURSOR findEntries_ IS
select * from MyTable
where order_no = paras_.order_no;
counter_ NUMBER;
CURSOR findEntries_count_ IS
SELECT COUNT(*) from MyTable
where order_no = paras_.order_no;
BEGIN
OPEN findEntries_count_;
FETCH findEntries_count_ INTO counter_;
CLOSE findEntries_count_;
dbms_output.put_line('total records found: '||counter_);
IF (counter_ = 0) THEN
-- log and leave procedure
RETURN;
END IF;
FOR order_rec_ IN findEntries_ LOOP
EXIT WHEN findEntries_%NOTFOUND OR findEntries_%NOTFOUND IS NULL;
-- do stuff - i.e. delete a record.
API_Package.Delete(order_rec_);
END LOOP;
END DeleteStuff___;
If the query is small, that is my prefered way.
In this example, I just want to know (and log) how many entries I'll delete.
p.s. Ignore the three underlines. In IFS, this is used when you want private procedures or functions.
You can’t have cursor count at start. For that you need to fetch complete cursor; that is the way get cursor count.
declare
cursor c2 is select * from dept;
var c2%rowtype;
i number :=0;
begin
open c2;
loop
fetch c2 into var;
exit when c2%NOTFOUND;
i: = i+1;
end loop;
close c2;
dbms_output.put_line('total records in cursor'||i);
end;
You can use %ROWCOUNT attribute of a cursor.
e.g:
DECLARE
CURSOR lcCursor IS
SELECT *
FROM DUAL;
BEGIN
OPEN lcCursor ;
DBMS_OUTPUT.PUT_LINE(lcCursor%ROWCOUNT);
CLOSE lcCursor ;
END;

How can I find the number of records in an Oracle PL/SQL cursor?

Here's my cursor:
CURSOR C1 IS SELECT * FROM MY_TABLE WHERE SALARY < 50000 FOR UPDATE;
I immediately open the cursor in order to lock these records for the duration of my procedure.
I want to raise an application error in the event that there are < 2 records in my cursor. Using the C1%ROWCOUNT property fails because it only counts the number which have been fetched thus far.
What is the best pattern for this use case? Do I need to create a dummy MY_TABLE%ROWTYPE variable and then loop through the cursor to fetch them out and keep a count, or is there a simpler way? If this is the way to do it, will fetching all rows in my cursor implicitly close it, thus unlocking those rows, or will it stay open until I explicitly close it even if I've fetched them all?
I need to make sure the cursor stays open for a variety of other tasks beyond this count.
NB: i just reread you question.. and you want to fail if there is ONLY 1 record..
i'll post a new update in a moment..
lets start here..
From Oracle® Database PL/SQL User's Guide and Reference
10g Release 2 (10.2)
Part Number B14261-01
reference
All rows are locked when you open the cursor, not as they are fetched. The rows are unlocked when you commit or roll back the transaction. Since the rows are no longer locked, you cannot fetch from a FOR UPDATE cursor after a commit.
so you do not need to worry about the records unlocking.
so try this..
declare
CURSOR mytable_cur IS SELECT * FROM MY_TABLE WHERE SALARY < 50000 FOR UPDATE;
TYPE mytable_tt IS TABLE OF mytable_cur %ROWTYPE
INDEX BY PLS_INTEGER;
l_my_table_recs mytable_tt;
l_totalcount NUMBER;
begin
OPEN mytable_cur ;
l_totalcount := 0;
LOOP
FETCH mytable_cur
BULK COLLECT INTO l_my_table_recs LIMIT 100;
l_totalcount := l_totalcount + NVL(l_my_table_recs.COUNT,0);
--this is the check for only 1 row..
EXIT WHEN l_totalcount < 2;
FOR indx IN 1 .. l_my_table_recs.COUNT
LOOP
--process each record.. via l_my_table_recs (indx)
END LOOP;
EXIT WHEN mytable_cur%NOTFOUND;
END LOOP;
CLOSE mytable_cur ;
end;
ALTERNATE ANSWER
I read you answer backwards to start and thought you wanted to exit if there was MORE then 1 row.. not exactly one.. so here is my previous answer.
2 simple ways to check for ONLY 1 record.
Option 1 - Explicit Fetchs
declare
CURSOR C1 IS SELECT * FROM MY_TABLE WHERE SALARY < 50000 FOR UPDATE;
l_my_table_rec C1%rowtype;
l_my_table_rec2 C1%rowtype;
begin
open C1;
fetch c1 into l_my_table_rec;
if c1%NOTFOUND then
--no data found
end if;
fetch c1 into l_my_table_rec2;
if c1%FOUND THEN
--i have more then 1 row
end if;
close c1;
-- processing logic
end;
I hope you get the idea.
Option 2 - Exception Catching
declare
CURSOR C1 IS SELECT * FROM MY_TABLE WHERE SALARY < 50000 FOR UPDATE;
l_my_table_rec C1%rowtype;
begin
begin
select *
from my_table
into l_my_table_rec
where salary < 50000
for update;
exception
when too_many_rows then
-- handle the exception where more than one row is returned
when no_data_found then
-- handle the exception where no rows are returned
when others then raise;
end;
-- processing logic
end;
Additionally
Remember: with an explicit cursor.. you can %TYPE your variable off the cursor record rather then the original table.
this is especially useful when you have joins in your query.
Also, rememebr you can update the rows in the table with an
UPDATE table_name
SET set_clause
WHERE CURRENT OF cursor_name;
type statement, but I that will only work if you haven't 'fetched' the 2nd row..
for some more information about cursor FOR loops.. try
Here
If you're looking to fail whenver you have more than 1 row returned, try this:
declare
l_my_table_rec my_table%rowtype;
begin
begin
select *
from my_table
into l_my_table_rec
where salary < 50000
for update;
exception
when too_many_rows then
-- handle the exception where more than one row is returned
when no_data_found then
-- handle the exception where no rows are returned
when others then raise;
end;
-- processing logic
end;
If this is the way to do it, will
fetching all rows in my cursor
implicitly close it, thus unlocking
those rows
The locks will be present for the duration of the transaction (ie until you do a commit or rollback) irrespective of when (or whether) you close the cursor.
I'd go for
declare
CURSOR C1 IS SELECT * FROM MY_TABLE WHERE SALARY < 50000 FOR UPDATE;;
v_1 c1%rowtype;
v_cnt number;
begin
open c_1;
select count(*) into v_cnt FROM MY_TABLE WHERE SALARY < 50000 and rownum < 3;
if v_cnt < 2 then
raise_application_error(-20001,'...');
end if;
--other processing
close c_1;
end;
There's a very small chance that, between the time the cursor is opened (locking rows) and the select count, someone inserts one or more rows into the table with a salary under 50000. In that case the application error would be raised but the cursor would only process the rows present when the cursor was opened. If that is a worry, at the end do another check on c_1%rowcount and, if that problem was experienced, you'd need to rollback to a savepoint.
Create a savepoint before you iterate through the cursor and then use a partial rollback when you find there are < 2 records returned.
You can start transaction and check if SELECT COUNT(*) MY_TABLE WHERE SALARY < 50000 greater than 1.

Resources