ORA-01555: snapshot too old error when deleting with transactions - oracle

I hope someone can explain me why I'm seeing ORA-01555. I have a function and a procedure to perform a cleanup in a huge table:
-- Small cleanup in separate transaction.
FUNCTION clean_single(ts_until IN TIMESTAMP, datapoint_id IN NUMBER) RETURN NUMBER IS
PRAGMA AUTONOMOUS_TRANSACTION;
BEGIN
DELETE FROM VALUES WHERE DATAPOINT_ID = datapoint_id AND TS < ts_until;
COMMIT;
RETURN sql%rowcount;
END clean_single;
PROCEDURE prc_clean IS
count_all_deleted_vals NUMBER(10) := 0;
BEGIN
BEGIN
FOR dps IN (
SELECT x AS dpid, y as tsUntil FROM Z where some conditions
LOOP
count_all_deleted_vals := count_all_deleted_vals + clean_single(dps.tsUntil, dps.dpid);
END LOOP;
DBMS_OUTPUT.PUT_LINE('Removed ' || count_all_deleted_vals || ' values');
END;
END prc_clean;
The idea is to run prc_clean() from a job and after the relevant datapoint ids are selected, the deletion per datapoint-id is done in a single transaction to avoid having one huge transaction.
But when I run this it runs for a while and then fails with ORA-01555.
In detail i do not understand why this is happening. Why does the PRAGMA AUTONOMOUS_TRANSACTION; in the function not prevent this?
What can I do to prevent it?

As far as I can tell, cause of ORA-01555 is often committing within a loop - and that's exactly what you're doing.
Skip the function altogether (doing DML in it is usually wrong anyway) and use only procedure, e.g.
PROCEDURE prc_clean
IS
count_all_deleted_vals NUMBER (10) := 0;
BEGIN
FOR dps IN (SELECT x AS dpid, y AS tsUntil
FROM Z
WHERE some conditions)
LOOP
DELETE FROM VALUES
WHERE DATAPOINT_ID = dps.dpid
AND TS < dps.tsuntil;
count_all_deleted_vals := count_all_deleted_vals + SQL%ROWCOUNT;
END LOOP;
COMMIT;
DBMS_OUTPUT.PUT_LINE ('Removed ' || count_all_deleted_vals || ' values');
END;

Related

Ignore lines that causes errors

I have a big Oracle script with thousands of package call inside a BEGIN - END;
Is there a way to ignore the lines that causes error and continue executing the next lines? Some sort of "On Error Resume Next" in vb.
If you have only one BEGIN END section, then you can use EXCEPTION WHEN OTHERS THEN NULL.
SQL> declare
v_var pls_integer;
begin
select 1 into v_var from dual;
-- now error
select 'A' into v_var from dual;
exception when others then null;
end;
SQL> /
PL/SQL procedure successfully completed.
SQL> declare
v_var pls_integer;
begin
select 1 into v_var from dual;
-- now error
select 'A' into v_var from dual;
--exception when others then null;
end;
/
declare
*
ERROR at line 1:
ORA-06502: PL/SQL: numeric or value error: character to number conversion error
ORA-06512: at line 6
SQL>
The whole concept of "ignore errors" is a bug, and a lie if any errors occur. That is not to say you cannot trap errors and continue processing, just that you MUST handle the errors. For example, assume the use case: "Data has been loaded into a stage table from multiple .csv files. Now load into the tables A and Table B according to ....".
create procedure
Load_Tables_A_B_from_Stage(process_message out varchar2)
is
Begin
For rec in (select * from stage)
loop
begin
insert into table_a (col1, col2)
values (rec.col_a1, col_a2);
insert into table_b (col1, col2)
values (rec.col_b1, col_b2);
exception
when others then null;
end;
end loop;
process_message := 'Load Tables A,B Complete';
end ;
Now suppose a user created the a .csv file entered "n/a" in numeric columns where there was no value or the value was unknown. The result of this all too common occurrence is all such rows were not loaded, but you have no way to know that until the user complains their data was not loaded even though you told them it was. Further you have no way of determining the problem.
A much better approach is to "capture and report".
create procedure
Load_Tables_A_B_from_Stage(process_message out varchar2)
is
load_error_occurred boolean := False;
Begin
For rec in (select * from stage)
loop
begin
insert into table_a (col1, col2)
values (rec.col_a1, rec.col_a2);
exception
when others then
log_load_error('Load_Tables_A_B_from_Stage', stage_id, sqlerrm);
load_error_occurred := True;
end;
begin
insert into table_b (col1, col2)
values (rec.col_b1, rec.col_b2);
exception
when others then
log_load_error('Load_Tables_A_B_from_Stage', stage_id, sqlerrm);
load_error_occurred := True;
end;
end loop;
if load_error_occurred then
process_message := 'Load Tables A,B Complete: Error(s) Detected';
else
process_message := 'Load Tables A,B Complete: Successful No Error(s)';
end if;
end Load_Tables_A_B_from_Stage ;
Now you have informed the user of the actual status, and where you are contacted you can readily identify the issue.
User here is used in the most general sense. It could mean a calling routine instead of an individual. Point is you do not have to terminate your process due to errors but DO NOT ignore them.
I don't think there is any magic one-liner that will solve this.
As others have, use a editor to automate the wrapping of each call within a BEGIN-EXCEPTION-END block might be quicker/easier.
But, if feel a little adventurous, or try this strategy:
Let's assume you have this:
BEGIN
proc1;
proc2;
proc3;
.
.
.
proc1000;
END;
You could try this (untested, uncompiled but might give you an idea of what to try):
DECLARE
l_progress NUMBER := 0;
l_proc_no NUMBER := 0;
e_proc_err EXCEPTION;
-- A 'runner' procedure than manegrs the counters and runs/skips dpending on these vals
PROCEDURE run_proc ( pname IN VARCHAR2 ) IS
BEGIN
l_proc_no := l_proc_no + 1;
IF l_proc_no >= l_progress
THEN
-- log 'Running pname'
EXECUTE IMMEDIATE 'BEGIN ' || pname || '; END;' ;
l_progress := l_progress + 1;
ELSE
-- log 'Skipping pname'
END IF;
EXCEPTION
WHEN OTHERS THEN
-- log 'Error in pname'
l_progress := l_progress + 1;
RAISE e_proc_err;
END;
BEGIN
l_progress := 0;
<<start>>
l_proc_no := 0;
run_proc ( 'proc1' );
run_proc ( 'proc2' );
run_proc ( 'proc3' );
.
.
run_proc ( 'proc1000' );
EXCEPTION
WHEN e_proc_err THEN
GOTO start;
WHEN OTHERS THEN
RAISE;
END;
The idea here is to add a 'runner' procedure to execute each procedure dynamically and log the run, skip, error.
We maintain a global count of the current process number (l_proc_no) and overall count of steps executed (l_progress).
When an error occurs we log it, raise it and let it fall into the outer blocks EXCEPTION handler where it will restart via an (evil) GOTO.
The GOTO is placed such that the overall execution count is unchanged but the process number is reset to 0.
Now when the run_proc is called it sees that l_progress is greater than l_proc_no, and skips it.
Why is this better than simply wrapping a BEGIN EXCEPTION END around each call?
It might not be, but you make a smaller change to each line of code, and you standardise the logging around each call more neatly.
The danger is a potential infinite loop which is why I specify e_proc_err to denote errors within the called procedures. But it might need tweaking to make it robust.

How to handle cursor exception when the select query returns "zero" records

How to handle cursor exception when the select query returns "zero" records
I have a cursor in a procedure, and after cursor initialization I'm iterating through the cursor to access the data from it.
But the problem is when the cursor select query returns 0 records then it throws exception
ORA-06531: Reference to uninitialized collection.
How to handle this exception?
---procedure code
create or replace PROCEDURE BIQ_SECURITY_REPORT
(out_chr_err_code OUT VARCHAR2,
out_chr_err_msg OUT VARCHAR2,
out_security_tab OUT return_security_arr_result ,
)
IS
l_chr_srcstage VARCHAR2 (200);
lrec return_security_report;
CURSOR cur_security_data IS
SELECT
"ID" "requestId",
"ROOM" "room",
"FIRST_NAME" "FIRST_NAME",
"LAST_NAME" "LAST_NAME",
FROM
"BI_REQUEST_CATERING_ACTIVITY" ;
TYPE rec_security_data IS TABLE OF cur_security_data%ROWTYPE
INDEX BY PLS_INTEGER;
l_cur_security_data rec_security_data;
begin
OPEN cur_security_data;
LOOP
FETCH cur_security_data
BULK COLLECT INTO l_cur_security_data
LIMIT 1000;
EXIT WHEN l_cur_security_data.COUNT = 0;
lrec := return_security_report();
out_security_tab := return_security_arr_result(return_security_report());
out_security_tab.delete;
FOR i IN 1 .. l_cur_security_data.COUNT
LOOP
BEGIN
l_num_counter := l_num_counter + 1;
lrec := return_security_report();
lrec.requestid := l_cur_security_data(i).requestId ; lrec.room := l_cur_security_data(i).room ; lrec.firstName := l_cur_security_data(i).firstName ;
IF l_num_counter > 1
THEN
out_security_tab.extend();
out_security_tab(l_num_counter) := return_security_report();
ELSE
out_security_tab := return_security_arr_result(return_security_report());
END IF;
out_security_tab(l_num_counter) := lrec;
EXCEPTION
WHEN OTHERS
THEN
DBMS_OUTPUT.PUT_LINE('Error occurred : ' || SQLERRM);
END;
END LOOP;
END LOOP;
EXCEPTION
WHEN OTHERS
THEN
DBMS_OUTPUT.PUT_LINE ('HERE INSIIDE OTHERS' || SQLERRM);
END;
Can you please explain how handle it.
You must be using out_security_tab, which is an output parameter in some other code where the procedure is called.
In your procedure, If cursor returns zero rows then the loop will not be executed and your code will not even initialize the out_security_tab which will lead to the error that you are facing.
There is a simple way to avoid:
initialize out_security_tab outside the loop -- which will definitely initialize it
You can create one out variable containing details as Y or N based on if cursor rows count -- Not recommended
Cheers!!

Recuperate Non inserted rows Oracle

I have 10.000 rows to insert into a table.
For an Insert With FORALL in oracle...
FORALL x IN TABLE_NAME.First .. TABLE_NAME.Last
INSERT
INTO TABLE_NAME VALUES
(
TABLE_NAME(x).VAL1,
TABLE_NAME(x).VAL2,
TABLE_NAME(x).VAL3,
TABLE_NAME(x).VAL4,
TABLE_NAME(x).VAL5
);
How can I recover the data from rows that doesnt insert into table due violation of constrainst in order to insert those items in a non-typed table of rejected items?
You can use the save exceptions clause of the forall statement to collect the error(s), and then use the sql%bulk_exceptions implicit cursor attribute to see what actually happened. There's an example in that documentation, but in your case you can do (using a made up table and data):
create table your_table (val1 number primary key, val2 number, val3 number, val4 number, val5 number);
declare
type l_table_type is table of your_table%rowtype;
l_table l_table_type := l_table_type();
dml_errors exception;
pragma exception_init(dml_errors, -24381);
begin
l_table.extend;
l_table(l_table.count).val1 := 1;
l_table(l_table.count).val2 := 1.2;
l_table(l_table.count).val3 := 1.3;
l_table(l_table.count).val4 := 1.4;
l_table(l_table.count).val5 := 1.5;
l_table.extend;
l_table(l_table.count).val1 := 2;
l_table(l_table.count).val2 := 2.2;
l_table(l_table.count).val3 := 2.3;
l_table(l_table.count).val4 := 2.4;
l_table(l_table.count).val5 := 2.5;
l_table.extend;
l_table(l_table.count).val1 := 1;
l_table(l_table.count).val2 := 3.2;
l_table(l_table.count).val3 := 3.3;
l_table(l_table.count).val4 := 3.4;
l_table(l_table.count).val5 := 3.5;
forall x in l_table.first .. l_table.last save exceptions
insert
into your_table values
(
l_table(x).val1,
l_table(x).val2,
l_table(x).val3,
l_table(x).val4,
l_table(x).val5
);
exception
when dml_errors then
for i in 1..sql%bulk_exceptions.count loop
dbms_output.put_line('Index ' || sql%bulk_exceptions(i).error_index
|| ' error ' || -sql%bulk_exceptions(i).error_code);
dbms_output.put_line(' val1: ' || l_table(sql%bulk_exceptions(i).error_index).val1);
dbms_output.put_line(' val2: ' || l_table(sql%bulk_exceptions(i).error_index).val2);
dbms_output.put_line(' val3: ' || l_table(sql%bulk_exceptions(i).error_index).val3);
dbms_output.put_line(' val4: ' || l_table(sql%bulk_exceptions(i).error_index).val4);
dbms_output.put_line(' val5: ' || l_table(sql%bulk_exceptions(i).error_index).val5);
end loop;
end;
/
That produces the output:
PL/SQL procedure successfully completed.
Index 3 error -1
val1: 1
val2: 3.2
val3: 3.3
val4: 3.4
val5: 3.5
The first collection element with val1 set to one was inserted successfully; the second one got the unique constraint exception and was not - but it was put into the bulk exception mechanism instead of causing the entire statement to fail.
You can then decide whether to raise an exception (or re-raise), or immediately roll back (possibly to a savepoint); or commit the inserts that did not error.
You can also insert the same values into your table of rejected items, but you'd need to be a bit careful if you're rolling back other changes (presumably you wouldn't be in that scenario though).
You can't directly use another forall to do that - referring to l_table(sql%bulk_exceptions(i).error_index).val1 inside the values() clause throws ORA-00911 because of the % character - so you'd either have to do individual inserts inside the for loop, or copy the values to another collection and bulk-insert from that. Unless you are expecting a lot of rejections individual inserts might be good enough.

ORA-01007 "variable not in select list" from dbms_sql.column_value call

I am trying to use dynamic SQL to sample all the data in a schema with a pattern:
DECLARE
xsql varchar2(5000);
c NUMBER;
d NUMBER;
col_cnt INTEGER;
f BOOLEAN;
rec_tab DBMS_SQL.DESC_TAB;
col_num NUMBER;
varvar varchar2(500);
PROCEDURE print_rec(rec in DBMS_SQL.DESC_REC) IS
BEGIN
DBMS_OUTPUT.ENABLE(1000000);
DBMS_OUTPUT.NEW_LINE;
DBMS_OUTPUT.PUT_LINE('col_type = '
|| rec.col_type);
DBMS_OUTPUT.PUT_LINE('col_maxlen = '
|| rec.col_max_len);
DBMS_OUTPUT.PUT_LINE('col_name = '
|| rec.col_name);
DBMS_OUTPUT.PUT_LINE('col_name_len = '
|| rec.col_name_len);
DBMS_OUTPUT.PUT_LINE('col_schema_name = '
|| rec.col_schema_name);
DBMS_OUTPUT.PUT_LINE('col_schema_name_len = '
|| rec.col_schema_name_len);
DBMS_OUTPUT.PUT_LINE('col_precision = '
|| rec.col_precision);
DBMS_OUTPUT.PUT_LINE('col_scale = '
|| rec.col_scale);
DBMS_OUTPUT.PUT('col_null_ok = ');
IF (rec.col_null_ok) THEN
DBMS_OUTPUT.PUT_LINE('true');
ELSE
DBMS_OUTPUT.PUT_LINE('false');
END IF;
END;
BEGIN
c := DBMS_SQL.OPEN_CURSOR;
xsql:='
WITH got_r_num AS
(
SELECT e.* -- or whatever columns you want
, ROW_NUMBER () OVER (ORDER BY dbms_random.value) AS r_num
FROM dba_tab_columns e
)
SELECT * -- or list all columns except r_num
FROM got_r_num
WHERE r_num <= 10';
DBMS_SQL.PARSE(c, xsql, DBMS_SQL.NATIVE);
d := DBMS_SQL.EXECUTE(c);
DBMS_SQL.DESCRIBE_COLUMNS(c, col_cnt, rec_tab);
LOOP
IF DBMS_SQL.FETCH_ROWS(c)>0 THEN
NULL;
-- get column values of the row
DBMS_SQL.COLUMN_VALUE(c, 2, varvar);
--dbms_output.put_line('varvar=');
--DBMS_SQL.COLUMN_VALUE(source_cursor, 2, name_var);
--DBMS_SQL.COLUMN_VALUE(source_cursor, 3, birthdate_var);
-- Bind the row into the cursor that inserts into the destination table. You
-- could alter this example to require the use of dynamic SQL by inserting an
-- if condition before the bind.
--DBMS_SQL.BIND_VARIABLE(destination_cursor, ':id_bind', id_var);
--DBMS_SQL.BIND_VARIABLE(destination_cursor, ':name_bind', name_var);
--DBMS_SQL.BIND_VARIABLE(destination_cursor, ':birthdate_bind',
--birthdate_var);
--ignore := DBMS_SQL.EXECUTE(destination_cursor);
--ELSE
-- No more rows to copy:
--EXIT;
END IF;
END LOOP;
--EXIT WHEN d != 10;
--END LOOP;
col_num := rec_tab.first;
IF (col_num IS NOT NULL) THEN
LOOP
print_rec(rec_tab(col_num));
col_num := rec_tab.next(col_num);
EXIT WHEN (col_num IS NULL);
END LOOP;
END IF;
DBMS_SQL.CLOSE_CURSOR(c);
END;
/
When I run that it gives me this error from the line with the dbms_sql.column_value call:
ORA-01007: variable not in select list
If I comment out that dbms_sql.column_value call it still errors but now with:
ORA-01002: fetch out of sequence
What am I doing wrong?
You have two problems in the code you posted. Firstly you have skipped part of the execution flow because you haven't called the DEFINE_COLUMN procedure. That is what is causing the ORA-01007 error, as the dynamic SQL processing hasn't been told about the select list columns via that call. For your current code you only need to define column 2, but assuming you will actually want to refer to the others you can define them in a loop. To treat them all as string for display you could do:
...
DBMS_SQL.PARSE(c, xsql, DBMS_SQL.NATIVE);
d := DBMS_SQL.EXECUTE(c);
DBMS_SQL.DESCRIBE_COLUMNS(c, col_cnt, rec_tab);
FOR i IN 1..col_cnt
LOOP
-- dbms_output.put_line('col_name is ' || rec_tab(i).col_name);
DBMS_SQL.DEFINE_COLUMN(c, i, varvar, 500);
END LOOP;
LOOP
IF DBMS_SQL.FETCH_ROWS(c)>0 THEN
...
If you want to do anything that needs to treat the variables as the right types you could have a local variable of each type and use the data type from the rec_tab information you already have from describe_columns to use the appropriately typed variable for each column.
The second problem, which you were hitting when you commented the column_value call, is still there once that definbe issue has been fixed. Your loop doesn't ever exit, so after you fetch the last row from the cursor you do a further invalid fetch, which throws ORA-01002. You have the code to avoid that already but it's commented out:
...
LOOP
IF DBMS_SQL.FETCH_ROWS(c)>0 THEN
-- get column values of the row
DBMS_SQL.COLUMN_VALUE(c, 2, varvar);
...
ELSE
-- No more rows to copy:
EXIT;
END IF;
END LOOP;
...
With those two changes your code runs, and dumps the view structure:
PL/SQL procedure successfully completed.
col_type = 1
col_maxlen = 30
col_name = OWNER
col_name_len = 5
col_schema_name =
col_schema_name_len = 0
col_precision = 0
col_scale = 0
col_null_ok = false
col_type = 1
col_maxlen = 30
col_name = TABLE_NAME
...
To those who find this question when accessing Oracle through ODP.NET, as I did:
We started getting this error whenever we would add column to an existing table in our application. I'm not sure what all the conditions were to make it fail, but ours were:
Run a SELECT * FROM "table".
Include a ROWNUM restriction in the WHERE clause (WHERE ROWNUM < 10).
Run that through the ODP.NET dataReader.GetSchemaTable() call.
Running unrestricted queries or running queries directly on Oracle SQL Developer did not seem to cause the error.
I've hit some pretty weird stuff in the past with Oracle connection pooling, so I eventually thought that could be the problem. The solution was to restart the web service to force all the connections to be fully dropped and recreated.
The theory is that the ODP.NET connection from the connection pool still had no idea the column existed on the table, but the column was returned by the database.

given two scripts for deleting records in table(6 million rows) want to know which script will be better and why?

I have table with 6 million records I am running archival script to delete around 5 million records.
My script1 will do the deletion but the DBA said my script1 getting into more buffer gets and
he recommended the approach which is script2. I am confused how come script2 is better than script1.
So please review the scripts and reply which is the best approach and why.
script1 :
PROCEDURE archival_charging_txn(p_no_hrs IN NUMBER,
p_error_code OUT NUMBER,
p_error_msg OUT VARCHAR2) IS
v_sysdate DATE := SYSDATE - p_no_hrs / 24;
TYPE t_txn_id IS TABLE OF scg_charging_txn.txn_id%TYPE INDEX BY PLS_INTEGER;
v_txn_id t_txn_id;
CURSOR c IS
SELECT txn_id FROM scg_charging_txn WHERE req_time < v_sysdate; /* non unique index */
BEGIN
OPEN c;
LOOP
FETCH c BULK COLLECT
INTO v_txn_id LIMIT 10000;
IF v_txn_id.COUNT > 0 THEN
FORALL i IN v_txn_id.FIRST .. v_txn_id.LAST
DELETE FROM scg_charging_txn WHERE txn_id = v_txn_id(i); /* Primary key based */
END IF;
COMMIT;
EXIT WHEN c%NOTFOUND;
END LOOP;
CLOSE c;
COMMIT;
p_error_code := 0;
EXCEPTION
WHEN OTHERS THEN
p_error_code := 1;
p_error_msg := substr(SQLERRM, 1, 200);
END archival_charging_txn;
script 2:
PROCEDURE archival_charging_txn_W(p_no_hrs IN NUMBER,
p_error_code OUT NUMBER,
p_error_msg OUT VARCHAR2) IS
BEGIN
DELETE FROM scg_charging_txn WHERE req_time < SYSDATE - p_no_hrs / 24;
COMMIT;
p_error_code := 0;
EXCEPTION
WHEN OTHERS THEN
p_error_code := 1;
p_error_msg := substr(SQLERRM, 1, 200);
END archival_charging_txn_W;
the 1st script reads the table entries using the cursor, therefore more buffer gets, while the 2nd script just deletes the table entries.
I would prefer the 2nd script. in case you get troubles with locking the table for too log time, make a LOOP UNTIL SQL%NOTFOUND with DELETE ... AND ROWNUM <= 10000; COMMIT; inside.
If you're deleting 5 million of 6 million rows, performance will suck.
You're better off doing CTAS.
Something like:
create table new_scg_charging_txn nologging as select * from scg_charging_txn WHERE req_time >= SYSDATE - p_no_hrs / 24;
If downtime is unacceptable, you may be able to do something similar, but wrap it with DBMS_REDEFINITION.

Resources