I am handling millions of rows and it takes hours, so I want some feedback to give me an idea about the status of the process. It won't be possible to have real time feedback from a stored procedure since the output will be available only after the complete execution. Any solution?
There are a couple of solutions.
One is to write to a log of some description. You can use a file (writing out with UTL_FILE) or a table (using autonomous transactions so that the records are visible in another session without affecting the main transaction).
The other solution is to write to the V$SESSION_LONGOPS view using DBMS_APPLICATION_INFO.SET_SESSION_LONGOPS. Find out more
I think logging is always a good idea with long-running background procedures. If something goes wrong your logs are the only source of info you have.
You could have a ReportStatus procedure that writes to a different table (one you can select from while your procedure is running).
It would need a PRAGMA AUTONOMOUS_TRANSACTION so it can commit independently of your main procedure.
Example:
CREATE OR REPLACE PROCEDURE ReportStatus(status NUMBER)
AS
PRAGMA AUTONOMOUS_TRANSACTION;
BEGIN
INSERT INTO StatusTable VALUES(SYSDATE, status);
COMMIT;
END ReportStatus;
/
One more way to get feedback from running PL/SQL block is to use inter-session communication. In order to use this solution you need at least two sessions.
In audit (first) session:
set head off arrays 1 pages 0 feedback off timing off
select column_value notification from table (notificationReceive);
It's going to be stuck there until first message arrives.
As the next step execute your PL/SQL block in the another (second) session:
begin
notificationSend ('START: '||'myBlockName');
/* UPDATE table1 SET col1 = value ... WHERE some condition;
*/ dbms_lock.sleep(3);
notificationSend ('Updated 1000000 rows'); -- sql%rowcount instead of literal
/* INSERT INTO table2 (val1, SYSDATE, v_user_id);
*/ dbms_lock.sleep(2);
notificationSend ('Inserted 1000000 rows');
notificationSend ('ENDTX');
end;
/
Now, you can go back to the first session and see the real-time feedback from running PL/SQL block:
11:13:39: START: myBlockName
11:13:42: Updated 1000000 rows
11:13:44: Inserted 1000000 rows
Below the code of almost one line functions:
create or replace procedure notificationSend (message varchar2) is
pragma autonomous_transaction;
begin
dbms_alert.signal ('feedback$_queue', message);
commit;
end;
/
create or replace function notificationReceive return sys.odciVarchar2List pipelined is
pragma autonomous_transaction;
message varchar2(1800) := 'NONE';
status number := -1;
begin
dbms_alert.register('feedback$_queue');
<<reading>> loop
dbms_alert.waitone('feedback$_queue', message, status);
if status = 0 and message != 'ENDTX' then
pipe row (to_char (sysdate, 'hh:mi:ss')||': '||message);
pipe row (null); -- dummy row for prefetch in sqlplus
else
exit reading;
end if;
end loop reading;
dbms_alert.remove('feedback$_queue');
return;
end;
/
Tested with releases 11.2.0.4.0, 12.2.0.1.0.
Related
I need some help in writing Oracle PL/SQL procedure that should do the following:
the procedure is called from a trigger after an update of the field in one table with the input parameter B-block or D-activate (this is already done)
the procedure should first open one cursor that will catch the account numbers of a client and open a loop that will process account by account
this one account should be forwarded to another loop that will catch card numbers of that client for that account (second cursor) and when into this loop, the card number should be used as an input parameter for a stored procedure that is called to block/unblock this card - this stored procedure already exists I just need to call it
the procedure don't need to return any parameters, the idea is just to block/activate card number of a client with the already written stored procedure for that
Should I write a package for this or just a procedure? And how can I write one loop in another?
I just realized that i can do this without cursors in a procedure. For simple example:
create or replace procedure blokiraj_proc (core_cust_id varchar2, kyc_blocked varchar2) as
type NumberArray is Array(100) of test_racuni.foracid%type;
type StringArray is Array (1000) of test_kartice.card_num%type;
accnt NumberArray;
card_number StringArray;
begin
select foracid bulk collect into accnt from test_racuni where cif_id = core_cust_id;
for i in accnt.first..accnt.last
loop
select card_num bulk collect into card_number from test_kartice where rbs_acct_num = accnt(i);
dbms_output.enable (100000);
dbms_output.put_line (accnt(i));
for j in 1..card_number.count
loop
dbms_output.put_line (card_number(j));
blokiraj_karticu (card_number(j));
end loop;
end loop;
end;
Is this a better approach then the curssors? And why is dbms_output not printing anything when i trigger the procedure?
As #EdStevens indicated you cannot avoid processing cursors. But you can avoid the looping structure of cursor within cursor. And the implicit open and close cursor for the inner one. The queries have combine into a simple JOIN then bulk collect into a single collection.
For this I created a RECORD to contain both the account number and card number; then a collection of that record. The cursor is then bulk collected into the collection. Your initial code allows for up to 100000 cards to be processed, and while I am a fan of bulk collect (when needed) I am not a fan of filling memory, therefore I limit the number of rows bulk collect gathers of each fetch. This unfortunately introduces a loop-within-loop construct, but the penalty is not near as great as cursor-within-cursor construct. The following is the result.
create or replace procedure blokiraj_proc (core_cust_id varchar2) as
type acct_card_r
is record(
acct_num test_kartice.rbs_acct_num%type
, card_num test_kartice.card_num%type
);
type acct_card_array is table of acct_card_r;
acct_card_list acct_card_array;
k_acct_card_buffer_limit constant integer := 997;
cursor c_acct_card(c_cust_id varchar2) is
select r.foracid
, k.card_num
from test_racuni r
left join test_kartice k
on (k.rbs_acct_num = r.foracid)
where r.cif_id = c_cust_id
order by r.foracid
, k.card_num;
begin
dbms_output.enable (buffer_size => null); -- enable dbms_output with size unlimited
open c_acct_card(core_cust_id);
loop
fetch c_acct_card
bulk collect
into acct_card_list
limit k_acct_card_buffer_limit;
for i in 1 .. acct_card_list.count
loop
dbms_output.put (acct_card_list(i).acct_num || ' ==> ');
if acct_card_list(i).card_num is not null
then
dbms_output.put_line (acct_card_list(i).card_num);
blokiraj_karticu (acct_card_list(i).card_num);
else
dbms_output.put_line ('No card for this account');
end if;
end loop;
-- exit buffer fetch when current buffeer is not full. As that means all rows
-- from cursor have been fetched/processed.
exit when acct_card_list.count < k_acct_card_buffer_limit;
end loop;
close c_acct_card;
end blokiraj_proc;
Well this is just another approach. If it's better for you, great. I also want to repeat and expand Ed Stevens warning of running this from a trigger. If either of the tables here is the table on which the trigger fired you will still get a mutating table exception - you cannot just hide it behind a procedure. And even if not its a lot of looping for trigger.
i'm facing a problem with a stored procedure, the thing is i'm trying to find if the transaction number in the temporary table is already in the final table, if not it will insert the record, if it's in the final table, it's going to a log_error table, here's my SP
BEGIN
DECLARE
date temporary_table.transfer_date%TYPE;
auth temporary_table.auth_code%TYPE;
transac_num temporary_table.transaction_number%TYPE;
card temporary_table.card_number%TYPE;
amount temporary_table.amount%TYPE;
num_trx_search NUMBER;
counter NUMBER;
sid1 NUMBER;
sid2 NUMBER;
loopcounter NUMBER;
BEGIN
cod_error := 0;
warning := 'execution';
OPEN vocursor FOR
SELECT transfer_date,
auth_code,
transaction_number,
card_number,
amount
FROM temporary_table order by id;
prfcursor := vocursor;
OPEN ntxcursor FOR
SELECT transaction_number FROM final_table order by id;
trxcursor := ntxcursor;
LOOP
FETCH prfcursor INTO date, auth, transac_num, card, amount;
EXIT WHEN prfcursor%NOTFOUND;
FETCH trxcursor INTO num_trx_search;
dbms_output.Put_line('NumTrx: ' || num_trx);
begin
-- i need to check if the transaction number from the temporary table is already in the
--final table
FOR loopcounter IN (Select id from final_table where transaction_number = transac_num)
LOOP
DBMS_OUTPUT.PUT_LINE(loopcounter.sid);
END LOOP;
dbms_output.Put_line('num_trx_search: ' || num_trx_search);
dbms_output.Put_line('counter: ' || counter);
exception
WHEN NO_DATA_FOUND THEN
dbms_output.Put_line('No Data found');
end;
EXIT WHEN trxcursor%NOTFOUND;
--just for testing and debuging
counter := 1;
IF(counter > 0) THEN
--inserts into log error table
ELSE
--inserts into final table
END IF;
END LOOP;
dbms_output.Put_line( 'end loop' );
CLOSE trxcursor;
CLOSE prfcursor;
dbms_output.Put_line( 'end cursor' );
END;
The thing is, it's getting all the results, for each record in the temporary, should get just one if the transaction number matches.
NumTrx is the transaction number in the temporary table.
I'm a noob in plsql, thanks
You could achieve the same thing by trying to insert the records from the temporary table into the final table and use the LOG ERRORS INTO clause to push those records that are already in final into another table.
INSERT INTO final_table final
SELECT transfer_date,
auth_code,
transaction_number,
card_number,
amount
FROM temporary_table
LOG ERRORS INTO ERR$_final_table
The query above assumes that final_table and temporary_table have the same structure. If they are different you will need to adjust the query slightly. Generally you should try to do as much of what you want in a single SQL rather than writing lots of procedural code to achieve the same thing. It is usually quicker and in this case appears to be much simpler.
For the set up of the ERR$ table I suggest you look at the Oracle docs under DML ERROR LOGGING.
If you do wish to do a row-by-row (slow) update then I would suggest using implicit cursor for loops instead simply for readability. Also I don't think the ORDER BY on each cursor is going to do anything except slow your code down.
I am working on an application and made the decision that all the queries would be procedures. I hope to have gains in performance and ease of maintenance by doing it this way. Our DBA's have also expressed interest in having it done this way.
I have an HR table where operations are performed on it each night, and any changes are recorded in a secondary table. We are not doing auditing, these change records are kept until the next run and show users the changes that have happened.
To keep my question shorter I have reduced the number of columns in HR.
The HR table ID, GROUP_NAME, and GROUP_LEVEL. The Drill table has ID and TYPEVALUE.
CREATE OR REPLACE PROCEDURE DOCSADM.DRILL_RECORD_POSITION (
RECORD_TYPE IN VARCHAR2,
OUT_ID OUT VARCHAR2,
OUT_GROUP_NAME OUT VARCHAR2,
OUT_GROUP_LEVEL OUT VARCHAR2
) AS
BEGIN
SELECT HR.ID, HR.GROUP_NAME, HR.GROUP_LEVEL
INTO OUT_ID, OUT_GROUP_NAME, OUT_GROUP_LEVEL
FROM HR_POSITION HR JOIN DRILL_POSITION DP ON (HR.ID = DP.ID) WHERE DP.TYPEVALUE = RECORD_TYPE;
END DRILL_RECORD_POSITION;
The procedure compiles without issue. Before doing all the work in the application to link to the procedure and extract the values which in this case will eventually be displayed in a view or webpage, I wanted to have a quick little script that would call the procedure and then display the results so I can verify in Oracle.
Loops
BEGIN
for t in (DRILL_RECORD_POSITION('D', V1,V5,V6))
loop
--dbms_output.put_line(t.V1 || t.V5 || t.V6);
dbms_output.put_line(t.OUT_ID);
end loop;
END;
/
CURSORS
DECLARE
V1 HR_POSITION.ID%TYPE;
V5 HR_POSITION.GROUP_NAME%TYPE;
V6 HR_POSITION.GROUP_LEVEL%TYPE;
CURSOR T_CUR IS DRILL_RECORD_POSITION('D', V1,V5,V6);
BEGIN
OPEN T_CUR;
DBMS_OUTPUT.PUTLINE('START');
LOOP
FETCH T_CUR INTO V1,V5,V6;
EXIT WHEN T_CUR%NOTFOUND;
DBMS_OUTPUT.PUTLINE(V1||V5||V6);
END LOOP;
CLOSE T_CUR;
END;
FOR LOOPS
DECLARE
V1 HR_POSITION.POSITION_ID%TYPE;
V5 HR_POSITION.GROUP_NAME%TYPE;
V6 HR_POSITION.GROUP_LEVEL%TYPE;
BEGIN
DBMS_OUTPUT.PUTLINE('START');
FOR INDEX IN (DRILL_RECORD_POSITION('D', V1,V5,V6))
LOOP
--DBMS_OUTPUT.PUTLINE(INDEX.ID);
DBMS_OUTPUT.PUTLINE(INDEX.V1||INDEX.V5||INDEX.V6);
END LOOP;
END;
Note: I edited the column names out and shorted some when transferring here so I might have made a few mistakes.
All the articles I have seen online show me how to display from within the original procedure or by using views, cursors, records. Unless I am wrong, Eclipse wont have any problems using the information in the current form which is why I am passing it that way. So I am not interested in changing the procedure and would like to work with it as is, since thats how the application will be doing it.
As this is the first of the stored procedures I am doing for the application, instead of using adhoc queries from the application, I dont have any existing examples to work from, which is why I believe the results will work fine, because it should be the same format the adhoc ones use.
Update:
In one of the comments, I was pointed to what should have been a solution. This was confirmed by another solution that was under it.
I keep getting the error
ORA-01422: exact fetch returns more than requested number of rows
So Im returning multiple rows, but that is my expectation and what is happening. I just cant seem to figure out how to display the results.
To test the procedure you showed, you would do something like:
declare
l_id hr_position.id%type;
l_group_name hr_position.group_name%type;
l_group_level hr_position.group_level%type;
begin
drill_record_position('D', l_id, l_group_name, l_group_level);
dbms_output.put_line(l_id ||':'|| l_group_name ||':'|| l_group_level);
end;
/
But that - or more specifically, your procedure - only works if there is exactly one row in the query's result set for the passed-in value type. It seems you're expecting multiple rows back (which would get too-many-rows), but there could also be non (which would get no-data-found).
So really it seems like your question should be about how to write your procedure so it works with one of the retrieval/test methods you tried.
If your procedure needs to return multiple rows then it can use a ref cursor, e.g.:
create or replace procedure drill_record_position (
p_record_type in varchar2,
p_ref_cursor out sys_refcursor
)
as
begin
open p_ref_cursor for
select hr.id, hr.group_name, hr.group_level
from hr_position hr
join drill_position dp
on hr.id = dp.id
where dp.typevalue = p_record_type;
end drill_record_position;
/
which you could then test with something like:
declare
l_ref_cursor sys_refcursor;
l_id hr_position.id%type;
l_group_name hr_position.group_name%type;
l_group_level hr_position.group_level%type;
begin
drill_record_position('D', l_ref_cursor);
loop
fetch l_ref_cursor into l_id, l_group_name, l_group_level;
exit when l_ref_cursor%notfound;
dbms_output.put_line(l_id ||':'|| l_group_name ||':'|| l_group_level);
end loop;
close l_ref_cursor;
end;
/
You can also do that as a function, which might be easier to work with from your application:
-- drop procedure drill_record_position;
create or replace function drill_record_position (p_record_type in varchar2)
return sys_refcursor as
l_ref_cursor sys_refcursor;
begin
open l_ref_cursor for
select hr.id, hr.group_name, hr.group_level
from hr_position hr
join drill_position dp
on hr.id = dp.id
where dp.typevalue = p_record_type;
return l_ref_cursor;
end drill_record_position;
/
declare
l_ref_cursor sys_refcursor;
l_id hr_position.id%type;
l_group_name hr_position.group_name%type;
l_group_level hr_position.group_level%type;
begin
l_ref_cursor := drill_record_position('D');
loop
fetch l_ref_cursor into l_id, l_group_name, l_group_level;
exit when l_ref_cursor%notfound;
dbms_output.put_line(l_id ||':'|| l_group_name ||':'|| l_group_level);
end loop;
close l_ref_cursor;
end;
/
You coudl also do this with collections and a pipelined function, which is more work to set up:
create type t_drill_obj as object (
-- use your real data types...
id number,
group_name varchar2(10),
group_level number
)
/
create type t_drill_tab as table of t_drill_obj
/
create or replace function drill_record_position (p_record_type in varchar2)
return t_drill_tab pipelined as
begin
for l_row in (
select t_drill_obj(hr.id, hr.group_name, hr.group_level) as obj
from hr_position hr
join drill_position dp
on hr.id = dp.id
where dp.typevalue = p_record_type
)
loop
pipe row (l_row.obj);
end loop;
return;
end drill_record_position;
/
but you could call it as part of another query, and even join tot he result if you needed to:
select * from table(drill_record_position('D'));
I'm coding a complex PLSQL block (complex for me hahaha) to insert rows using information from the FOR LOOP CURSOR and add parameters to insert using a stored procedure. The current problem is there are around 200 rows to be inserted but when a simple row fail to insert all rows inserted broke and oracle execute a ROLLBACK command (I think so). So... How could I handle exceptions to insert succefully all rounds I can and when any rows fail show it in screen? Thanks
FOR i IN c_mig_saldos LOOP
IF i.tipo_comprobante = 'P' THEN -- Nota de debito (positivo)
v_cmp_p.prn_codigo := 'VIV';
v_cmp_p.tcm_codigo := 'NRA';
v_cmp_p.cmp_fecha_emision := TRUNC(SYSDATE);
v_cmp_p.cmp_fecha_contable := TRUNC(SYSDATE);
v_cmp_p.cmp_observacion := 'GENERACION AUTOMATICA DE SALDOS';
v_cmp_p.cli_codigo := i.cli_codigo;
v_tab_dco_p(1).cnc_codigo := 'VIA';
v_tab_dco_p(1).dco_precio_unitario := i.total_final;
v_tab_dco_p(1).dco_cantidad := 1;
v_tab_dco_p(1).dco_importe := i.total_final;
-- Insert a new row using stored procedure but when a itereted fail, no rows has inserted in table
PKG_COMPROBANTES.PRC_INSERTAR_COMPROBANTE(v_cmp_p, v_tab_dco_p, v_tab_pgc_p, v_tab_apl_p, v_tab_mar_p);
COMMIT;
END IF;
END LOOP;
-- Insert a new row using stored procedure but when a itereted fail, no rows has inserted in table
begin
PKG_COMPROBANTES.PRC_INSERTAR_COMPROBANTE(v_cmp_p, v_tab_dco_p, v_tab_pgc_p, v_tab_apl_p, v_tab_mar_p);
exception
when others then --this could be made more specific but you didn't say what type of error you were getting
-- Log to a table so you can export failed inserts later.
-- Make sure log table cols are large enough to store everything that can possibly be inserted here...
ErrorLogProc(the, cols, you, want, to, see, and, SQLERRM);
end;
In the ErrorLogProc() I'd recommend a couple things, here is a snippet of some things I do in my error logging proc that you may find helpful (it's just a few snippets, not intended to fully work, but you should get the idea)...
oname varchar2(100);
pname varchar2(100);
lnumb varchar2(100);
callr varchar2(100);
g1B CHAR(2) := chr(13)||chr(10);
PRAGMA AUTONOMOUS_TRANSACTION; --important!
begin
owa_util.who_called_me(oname, pname, lnumb, callr);
--TRIM AND FORMAT FOR ERRORLOG
lv_errLogText := 'Package: '||pname||' // Version: '||version_in||' // Line Number: '||lnumb||' // Error: ';
lv_string1 := mid(errStr_in,1,4000-Length(lv_errLogText));
lv_errLogText := lv_errLogText||lv_string1;
lv_errLogText := lv_errLogText||g1B||'Error Backtrace: '||replace(dbms_utility.format_error_backtrace,'ORA-', g1b||'ORA-');
insertIntoYourErrorLogTable(lv_errLogText, and, whatever, else, you, need);
commit;
To keep this simple, since there's not enough information to know the what and why of the question, this will kick out some text information about failures as desired.
SQL> set serveroutput on
Then here's an anonymous PL/SQL block:
BEGIN
FOR i IN c_mig_saldos
LOOP
-- not relevant
BEGIN
PKG_COMPROBANTES.PRC_INSERTAR_COMPROBANTE(v_cmp_p, v_tab_dco_p, v_tab_pgc_p, v_tab_apl_p, v_tab_mar_p);
EXCEPTION
-- The goal of this is to ignore but output information about your failures
WHEN OTHERS THEN DBMS_OUTPUT.PUT_LINE('whatever you want about v_cmp_p, v_tab_dco_p, v_tab_pgc_p, v_tab_apl_p, v_tab_mar_p or SQLERRM/SQLCODE or the error stack - not enough info in the question');
END;
END LOOP;
END;
/
Note: I have removed the commit from the execution.
Then if you like what you see...
SQL> commit;
Ideally, if I knew more about why the insert failures were expected to occur and what I wanted to do about them, I would not insert them in the first place.
Agree with comment that more information is needed, but a couple of things to consider:
Does this need to be done as a loop - if you can write your query as a select statement, then you can do a simple insert without the need for PLSQL, which would be simpler and likely quicker (set based SQL vs row-by-row PLSQL)
You say a ROLLBACK is occuring - you have a commit inside your IF statement, so any records which make it into your IF statement and succesfully make it through your insert procedure will be committed i.e. they will not be rolled back; You should consider if some records you think are being rolled back are actually not making it into the IF statement at all
Can you provide example data, or an example of the error you are receiving?
I am making an oracle function that sync values between two tables and My intention is to show a string that show how many rows affected and displays it when user execute the function.
my function creation is like this
CREATE OR REPLACE FUNCTION WELTESADMIN.DUM_MST_SYNC(PROJNAME IN VARCHAR2)
RETURN NUMBER IS NUM_ROWS NUMBER;
BEGIN
MERGE INTO MASTER_DRAWING DST
USING (SELECT WEIGHT_DUM, HEAD_MARK_DUM FROM DUMMY_MASTER_DRAWING) SRC
ON (DST.HEAD_MARK = SRC.HEAD_MARK_DUM)
WHEN MATCHED THEN UPDATE SET DST.WEIGHT = SRC.WEIGHT_DUM
WHERE DST.PROJECT_NAME = SRC.PROJECT_NAME_DUM
AND DST.PROJECT_NAME = PROJNAME;
dbms_output.put_line( sql%rowcount || ' rows merged' );
END;
if i execute the begin part in the TOAD or SQL Developer i can see how many rows affected. My target is to collect this function into a procedure and when user wants to sync tables they just need to run the procedure with PROJNAME value supplied for specific project.
Please help me on how to improve this code,
Best regards
You can use SQL%ROWCOUNT to get the number of rows affected by the MERGE. Add the following statement in your code immediately after the MERGE :
dbms_output.put_line( sql%rowcount || ' rows merged' );
To return this value, declare a NUMBER variable and assign the sql%rowcount value to it. And then return that value. Something like :
Function
.......
Return NUMBER
.......
num_rows NUMBER;
......
Begin
Merge....
num_rows := SQL%ROWCOUNT;
RETURN num_rows;
END;
And, you don't need a procedure to execute the function. You can execute in SQL :
select function(project_name)
from dual
/
Update Since OP is trying to use DML inside a function, need to make it autonomous transaction to be able to perform DML without raising the ORA-14551.
You could use the directive pragma autonomous_transaction. This will run the function into an independent transaction that will be able to perform DML without raising the ORA-14551. However, remember, the results of the DML will be committed outside of the scope of the parent transaction. If you have only single transaction, you could use the workaround that I suggested. Add, PRAGMA AUTONOMOUS_TRANSACTION; immediately after the RETURN statement before BEGIN.
CREATE OR REPLACE
FUNCTION WELTESADMIN.DUM_MST_SYNC(
PROJNAME IN VARCHAR2)
RETURN NUMBER
IS
num_rows NUMBER;
PRAGMA AUTONOMOUS_TRANSACTION;
BEGIN
MERGE INTO MASTER_DRAWING DST USING
(SELECT WEIGHT_DUM, HEAD_MARK_DUM FROM DUMMY_MASTER_DRAWING
) SRC ON (DST.HEAD_MARK = SRC.HEAD_MARK_DUM)
WHEN MATCHED THEN
UPDATE
SET DST.WEIGHT = SRC.WEIGHT_DUM
WHERE DST.PROJECT_NAME = SRC.PROJECT_NAME_DUM
AND DST.PROJECT_NAME = PROJNAME;
num_rows := SQL%ROWCOUNT;
COMMIT;
RETURN num_rows;
END;
/