ORA-01555 error when updating 200 million rows with BULK COLLECT - oracle

I have the following PL/SQL code, which updates one column of each row in a table with about 200 million rows. I use BULK COLLECT to repeatedly fetch 150,000 rows from the table and update the rows. I do a commit after 50,000 updates.
DECLARE
CURSOR jobs_cursor IS
SELECT e.ID, e.PTI, e.CAT, e.JOBNAME, e.JOBDATE, e.WORK_DESCRIPTION
FROM JOB e
WHERE length(e.WORK_DESCRIPTION) > 1000;
TYPE JOBS_TYPE IS TABLE OF jobs_cursor%ROWTYPE;
v_jobs JOBS_TYPE;
fetch_jobs_limit PLS_INTEGER := 150000;
trimmed_work_description VARCHAR2(2000 CHAR);
sub_string_work_description_left VARCHAR2(1000 CHAR);
sub_string_work_description_right VARCHAR2(1000 CHAR);
update_counter NUMBER := 0;
commit_counter NUMBER := 50000;
BEGIN
OPEN jobs_cursor;
LOOP
FETCH jobs_cursor BULK COLLECT INTO v_jobs LIMIT fetch_jobs_limit;
EXIT WHEN v_jobs.COUNT = 0;
FOR idx IN 1..v_jobs.COUNT
LOOP
trimmed_work_description := ' ';
IF v_jobs(idx).WORK_DESCRIPTION IS NOT NULL THEN
trimmed_work_description := TRIM(TRAILING ' ' FROM v_jobs(idx).WORK_DESCRIPTION);
END IF;
IF length(trimmed_work_description) <= 1000 THEN
UPDATE JOBS j SET j.WORK_DESCRIPTION = trimmed_work_description WHERE j.ID = v_jobs(idx).ID;
update_counter := update_counter + 1;
IF mod(update_counter, commit_counter) = 0 THEN
COMMIT;
update_counter := 0;
END IF;
CONTINUE;
ELSIF length(trimmed_work_description) > 1000 THEN
sub_string_work_description_left := SUBSTR(trimmed_work_description, 1, 1000);
sub_string_work_description_right := SUBSTR(trimmed_work_description, 1001, 2000);
END IF;
UPDATE JOBS j SET j.WORK_DESCRIPTION = sub_string_work_description_left WHERE j.ID = v_jobs(idx).ID;
INSERT INTO JOBS j VALUES ("SEQUENCE_JOBS".NEXTVAL, j.PTI, j.CAT, j.JOBNAME, j.JOBDATE, sub_string_work_description_right);
update_counter := update_counter + 1;
IF mod(update_counter, commit_counter) = 0 THEN
COMMIT;
update_counter := 0;
END IF;
END LOOP;
END LOOP;
COMMIT;
CLOSE jobs_cursor;
END;
The code runs for several hours, but then Oracle raises an ORA-01555 - Snapshot too old - Rollback segment number 14 with name xxxx too small.
Could you please tell me what is wrong with my PL/SQL? I already did the Google research and found some threads saying that this error could be avoided by expanding the UNDO table space, however this is not an option in my case. Thus, I need to modify the PL/SQL code.

On first view I don't see any reason why you make the update in a loop, it should be possible with single statements. Would be similar to this (not verified/tested)
update JOBS j SET
WORK_DESCRIPTION = SUBSTR(TRIM(TRAILING ' ' FROM WORK_DESCRIPTION), 1, 1000)
WHERE length(WORK_DESCRIPTION) > 1000;
INSERT INTO JOBS
SELECT SEQUENCE_JOBS.NEXTVAL, j.PTI, j.CAT, j.JOBNAME, j.JOBDATE,
SUBSTR(WORK_DESCRIPTION, 1001, 2000)
FROM JOBS j
WHERE length(TRIM(TRAILING ' ' FROM WORK_DESCRIPTION)) > 1000;

Related

ORA-01555: snapshot too old error when deleting with transactions

I hope someone can explain me why I'm seeing ORA-01555. I have a function and a procedure to perform a cleanup in a huge table:
-- Small cleanup in separate transaction.
FUNCTION clean_single(ts_until IN TIMESTAMP, datapoint_id IN NUMBER) RETURN NUMBER IS
PRAGMA AUTONOMOUS_TRANSACTION;
BEGIN
DELETE FROM VALUES WHERE DATAPOINT_ID = datapoint_id AND TS < ts_until;
COMMIT;
RETURN sql%rowcount;
END clean_single;
PROCEDURE prc_clean IS
count_all_deleted_vals NUMBER(10) := 0;
BEGIN
BEGIN
FOR dps IN (
SELECT x AS dpid, y as tsUntil FROM Z where some conditions
LOOP
count_all_deleted_vals := count_all_deleted_vals + clean_single(dps.tsUntil, dps.dpid);
END LOOP;
DBMS_OUTPUT.PUT_LINE('Removed ' || count_all_deleted_vals || ' values');
END;
END prc_clean;
The idea is to run prc_clean() from a job and after the relevant datapoint ids are selected, the deletion per datapoint-id is done in a single transaction to avoid having one huge transaction.
But when I run this it runs for a while and then fails with ORA-01555.
In detail i do not understand why this is happening. Why does the PRAGMA AUTONOMOUS_TRANSACTION; in the function not prevent this?
What can I do to prevent it?
As far as I can tell, cause of ORA-01555 is often committing within a loop - and that's exactly what you're doing.
Skip the function altogether (doing DML in it is usually wrong anyway) and use only procedure, e.g.
PROCEDURE prc_clean
IS
count_all_deleted_vals NUMBER (10) := 0;
BEGIN
FOR dps IN (SELECT x AS dpid, y AS tsUntil
FROM Z
WHERE some conditions)
LOOP
DELETE FROM VALUES
WHERE DATAPOINT_ID = dps.dpid
AND TS < dps.tsuntil;
count_all_deleted_vals := count_all_deleted_vals + SQL%ROWCOUNT;
END LOOP;
COMMIT;
DBMS_OUTPUT.PUT_LINE ('Removed ' || count_all_deleted_vals || ' values');
END;

Inserting values into a created table with while loop

I'm working on a while loop exercise on oracle. I have created a table with two columns.
What I want to do is; inserting values into first column with a sequence of from 1 to 1 million(1,2,3,4,5....1000000).
I've tried
DECLARE
a int := 0;
BEGIN
WHILE a < 1000000 LOOP
a := a + 1;
END LOOP;
END;
insert into Schema_name.table_name
(column_1)
values('a')
P.S: I'm working on Toad 12.9
Would you like to give a hand to me for this?
Just insert values(a), when you write 'a' you insert the character 'a' and not the variable a
DECLARE
a int := 0;
BEGIN
WHILE a < 1000000 LOOP
a := a + 1;
insert into Schema_name.table_name
(column_1)
values(a);
END LOOP;
END;

Create Serial Number with a String in oracle

I am using the following code in oracle on create record trigger to create a serial number. I want my serial number like LM0001 . Using LM000 STRING WITH NUMBER and concat both to generate the serial number. I am getting error ora-06502. Need help where i am doing wrong. Where should i modify the code
DECLARE
VCAN varchar2(40);
L number (10);
BEGIN
SELECT MAX(SL_NO)INTO VCAN FROM LAND_MANAGEMENT;
IF VCAN > 0 THEN
L := VCAN + 1;
:LAND_MANAGEMENT.SL_NO := 'LM000' || L;
ELSE
:LAND_MANAGEMENT.SL_NO := 'LM000' || 1;
END IF;
EXCEPTION
WHEN NO_DATA_FOUND THEN
:LAND_MANAGEMENT.SL_NO := 'LM000'|| 1;
END;
Solved the problem using sequence
create a sequence
CREATE SEQUENCE LMANAGEMENT_SEQ
START WITH 21
MAXVALUE 9999999999999999999999999999
MINVALUE 0
NOCYCLE
CACHE 20
NOORDER;
sql query in when-create-record trigger
select 'LM' || to_char(lmanagement_seq.nextval, '000000') INTO
:land_management.sl_no
from dual;

given two scripts for deleting records in table(6 million rows) want to know which script will be better and why?

I have table with 6 million records I am running archival script to delete around 5 million records.
My script1 will do the deletion but the DBA said my script1 getting into more buffer gets and
he recommended the approach which is script2. I am confused how come script2 is better than script1.
So please review the scripts and reply which is the best approach and why.
script1 :
PROCEDURE archival_charging_txn(p_no_hrs IN NUMBER,
p_error_code OUT NUMBER,
p_error_msg OUT VARCHAR2) IS
v_sysdate DATE := SYSDATE - p_no_hrs / 24;
TYPE t_txn_id IS TABLE OF scg_charging_txn.txn_id%TYPE INDEX BY PLS_INTEGER;
v_txn_id t_txn_id;
CURSOR c IS
SELECT txn_id FROM scg_charging_txn WHERE req_time < v_sysdate; /* non unique index */
BEGIN
OPEN c;
LOOP
FETCH c BULK COLLECT
INTO v_txn_id LIMIT 10000;
IF v_txn_id.COUNT > 0 THEN
FORALL i IN v_txn_id.FIRST .. v_txn_id.LAST
DELETE FROM scg_charging_txn WHERE txn_id = v_txn_id(i); /* Primary key based */
END IF;
COMMIT;
EXIT WHEN c%NOTFOUND;
END LOOP;
CLOSE c;
COMMIT;
p_error_code := 0;
EXCEPTION
WHEN OTHERS THEN
p_error_code := 1;
p_error_msg := substr(SQLERRM, 1, 200);
END archival_charging_txn;
script 2:
PROCEDURE archival_charging_txn_W(p_no_hrs IN NUMBER,
p_error_code OUT NUMBER,
p_error_msg OUT VARCHAR2) IS
BEGIN
DELETE FROM scg_charging_txn WHERE req_time < SYSDATE - p_no_hrs / 24;
COMMIT;
p_error_code := 0;
EXCEPTION
WHEN OTHERS THEN
p_error_code := 1;
p_error_msg := substr(SQLERRM, 1, 200);
END archival_charging_txn_W;
the 1st script reads the table entries using the cursor, therefore more buffer gets, while the 2nd script just deletes the table entries.
I would prefer the 2nd script. in case you get troubles with locking the table for too log time, make a LOOP UNTIL SQL%NOTFOUND with DELETE ... AND ROWNUM <= 10000; COMMIT; inside.
If you're deleting 5 million of 6 million rows, performance will suck.
You're better off doing CTAS.
Something like:
create table new_scg_charging_txn nologging as select * from scg_charging_txn WHERE req_time >= SYSDATE - p_no_hrs / 24;
If downtime is unacceptable, you may be able to do something similar, but wrap it with DBMS_REDEFINITION.

Changing a Procedure to a View in Oracle

Not sure if this is possible, but I'm trying to see if I can convert this procedure to become a view because we've been having trouble with drives not populating the table when the procedure is run.
I'm trying to understand someone else's code and because of the cursors, I'm not even sure we can change this procedure to a view.
----------------------------------------------------------------------
--This Procedure will interface drive information on a nightly basis--
----------------------------------------------------------------------
Procedure HEMA_DRIVE_AUTO IS
v_start_date DATE := trunc(sysdate) -30;
v_end_date DATE := trunc(sysdate);
v_delete_stats_dt DATE := trunc(sysdate)-120;
v_total_registration_count NUMBER;
v_total_performed_count NUMBER;
v_total_collected_count NUMBER;
v_total_deferred_count NUMBER;
v_total_qns_count NUMBER;
v_existing_drive NUMBER;
v_existing_performed NUMBER;
v_maph_drive NUMBER;
--This Cursor will collect the initial data
cursor c_drive_info is
select dr.drive_id, dr.Start_time, dr.vehicle_id
from drives dr
--where dr.drive_id in(1605606);
where trunc(dr.start_time) between v_start_date and v_end_date;
--This Cursor will be used to decode the Donation Types
cursor c_procedure_codes is
select * from hema_donation_type_map hdt
where hdt.mobiles = 1 order by procedure_code_id;
--This Cursor will define the intentions but exclude theraputics inthe mapping
cursor c_intention is
select rsa_motivation_id,hema_intent_id from hema_intent_map
where rsa_motivation_id <> 4 order by rsa_motivation_id;
BEGIN
-- delete records older then 4 months
delete from hema_nightly h where trunc(h.drive_date) < v_delete_stats_dt;
commit;
FOR cur_drive IN c_drive_info LOOP
delete from hema_nightly where drive_id = cur_drive.drive_id;
commit;
-- Loop by motivation/intention
FOR cur_intent in c_intention LOOP
-- Loop to get the procedure code data
FOR cur_proc_code IN c_procedure_codes LOOP
v_total_registration_count := 0;
v_total_performed_count := 0;
v_total_collected_count := 0;
v_total_deferred_count := 0;
v_total_qns_count := 0;
v_maph_drive := 0;
-- get the count for all other procedures
select count(1)
into v_total_registration_count
from registration r
where r.drive_id = cur_drive.drive_id
and r.donation_type_id = cur_proc_code.donation_type_id
and r.motivation_id = cur_intent.rsa_motivation_id;
--get the deferral count
select count(unique(r.registration_id))
into v_total_deferred_count
from registration r
where r.drive_id = cur_drive.drive_id
and r.donation_type_id = cur_proc_code.donation_type_id
and r.motivation_id = cur_intent.rsa_motivation_id
and r.step_completed < 12
and exists (select rsc.registration_id
from reg_steps_completed rsc
where rsc.registration_id = r.registration_id
and rsc.collection_step_id = 99);
-- QNS count
select count(unique(r.registration_id))
into v_total_qns_count
from registration r
where r.drive_id = cur_drive.drive_id
and r.step_completed < 12
and not exists (select rsc.registration_id
from reg_steps_completed rsc
where rsc.registration_id = r.registration_id
and rsc.collection_step_id = 99)
and r.donation_type_id = cur_proc_code.donation_type_id
and r.motivation_id = cur_intent.rsa_motivation_id;
-- performed count is the difference between total registrations and total deferrals.
v_total_performed_count := v_total_registration_count -
(v_total_deferred_count +
v_total_qns_count);
-- not calulatind yield so keep count the same
v_total_collected_count := v_total_performed_count;
-- does this drive exist
select count(drive_id)
into v_existing_drive
from hema_nightly
where drive_id = cur_drive.drive_id
and procedure_id = cur_proc_code.procedure_code_id
and intent = cur_intent.hema_intent_id;
-- Is this an aph vehicle?
select count(vehicle_id)
into v_maph_drive
from vehicles
where veh_drive_type_uid = 2
and vehicle_id = cur_drive.vehicle_id;
if v_existing_drive > 0 then
update hema_nightly
set performed = performed + v_total_performed_count,
collected = collected + v_total_collected_count,
registered = registered + v_total_registration_count,
deferrals = deferrals + v_total_deferred_count,
qns = qns + v_total_qns_count,
drive_date = cur_drive.start_time,
mod_date = sysdate,
intent = cur_intent.hema_intent_id,
aph = v_maph_drive
where drive_id = cur_drive.drive_id
and procedure_id = cur_proc_code.procedure_code_id
and intent = cur_intent.hema_intent_id;
commit;
elsif v_existing_drive = 0 and v_total_registration_count > 0 then
insert into hema_nightly
(drive_id,
procedure_id,
performed,
collected,
registered,
deferrals,
qns,
drive_date,
mod_date,
intent,
aph)
values
(cur_drive.drive_id,
cur_proc_code.procedure_code_id,
v_total_performed_count,
v_total_collected_count,
v_total_registration_count,
v_total_deferred_count,
v_total_qns_count,
trunc(cur_drive.start_time),
sysdate,
cur_intent.hema_intent_id,
v_maph_drive);
commit;
end if;
v_existing_drive := 0;
end loop;
end loop;
end loop;
end hema_drive_auto;
Views don't perform DML (insert, update, delete) and they don't manage transactions with COMMIT and ROLLBACK; they only select and retrieve data.

Resources