I have very simple pl\sql code.
In this code I'm printing the index of the loop and wait 1 second before each print.
My problem is that I want this output to be used like a live log, when the dbms_output.put_line procedure is invoked and print - I want to see the output immediately.
In the current code - only after it finished (5 seconds..), it prints all the output in one shot...
set serveroutput on
set echo on
begin
for i in 1..5
loop
dba_maint.pkg_utils.sp_sleep(1);
dbms_output.put_line(i);
end loop;
end;
/
No way, you can't. It is displayed when PL/SQL procedure has finished.
If you want to create a live log,
create a table
a sequence
an autonomous transaction procedure which would
insert a row into that table
using a sequence (so that you'd know how to order rows)
possibly a timestamp (so that you'd know how long certain step took)
commit (which won't affect main transaction as - remember - procedure is an autonomous transaction one)
Then put calls to the logging procedure into your long-time-run PL/SQL procedure, run it, and let it work. In another session, query the log table to view progress.
Related
I have an SQL script that is called from within a shell script and takes a long time to run. It currently contains dbms_output.put_line statements at various points. The output from these print statements appear in the log files, but only once the script has completed.
Is there any way to ensure that the output appears in the log file as the script is running?
Not really. The way DBMS_OUTPUT works is this: Your PL/SQL block executes on the database server with no interaction with the client. So when you call PUT_LINE, it is just putting that text into a buffer in memory on the server. When your PL/SQL block completes, control is returned to the client (I'm assuming SQLPlus in this case); at that point the client gets the text out of the buffer by calling GET_LINE, and displays it.
So the only way you can make the output appear in the log file more frequently is to break up a large PL/SQL block into multiple smaller blocks, so control is returned to the client more often. This may not be practical depending on what your code is doing.
Other alternatives are to use UTL_FILE to write to a text file, which can be flushed whenever you like, or use an autonomous-transaction procedure to insert debug statements into a database table and commit after each one.
If it is possible to you, you should replace the calls to dbms_output.put_line by your own function.
Here is the code for this function WRITE_LOG -- if you want to have the ability to choose between 2 logging solutions:
write logs to a table in an autonomous transaction
CREATE OR REPLACE PROCEDURE to_dbg_table(p_log varchar2)
-- table mode:
-- requires
-- CREATE TABLE dbg (u varchar2(200) --- username
-- , d timestamp --- date
-- , l varchar2(4000) --- log
-- );
AS
pragma autonomous_transaction;
BEGIN
insert into dbg(u, d, l) values (user, sysdate, p_log);
commit;
END to_dbg_table;
/
or write directly to the DB server that hosts your database
This uses the Oracle directory TMP_DIR
CREATE OR REPLACE PROCEDURE to_dbg_file(p_fname varchar2, p_log varchar2)
-- file mode:
-- requires
--- CREATE OR REPLACE DIRECTORY TMP_DIR as '/directory/where/oracle/can/write/on/DB_server/';
AS
l_file utl_file.file_type;
BEGIN
l_file := utl_file.fopen('TMP_DIR', p_fname, 'A');
utl_file.put_line(l_file, p_log);
utl_file.fflush(l_file);
utl_file.fclose(l_file);
END to_dbg_file;
/
WRITE_LOG
Then the WRITE_LOG procedure which can switch between the 2 uses, or be deactivated to avoid performances loss (g_DEBUG:=FALSE).
CREATE OR REPLACE PROCEDURE write_log(p_log varchar2) AS
-- g_DEBUG can be set as a package variable defaulted to FALSE
-- then change it when debugging is required
g_DEBUG boolean := true;
-- the log file name can be set with several methods...
g_logfname varchar2(32767) := 'my_output.log';
-- choose between 2 logging solutions:
-- file mode:
g_TYPE varchar2(7):= 'file';
-- table mode:
--g_TYPE varchar2(7):= 'table';
-----------------------------------------------------------------
BEGIN
if g_DEBUG then
if g_TYPE='file' then
to_dbg_file(g_logfname, p_log);
elsif g_TYPE='table' then
to_dbg_table(p_log);
end if;
end if;
END write_log;
/
And here is how to test the above:
1) Launch this (file mode) from your SQLPLUS:
BEGIN
write_log('this is a test');
for i in 1..100 loop
DBMS_LOCK.sleep(1);
write_log('iter=' || i);
end loop;
write_log('test complete');
END;
/
2) on the database server, open a shell and
tail -f -n500 /directory/where/oracle/can/write/on/DB_server/my_output.log
Two alternatives:
You can insert your logging details in a logging table by using an autonomous transaction. You can query this logging table in another SQLPLUS/Toad/sql developer etc... session. You have to use an autonomous transaction to make it possible to commit your logging without interfering the transaction handling in your main sql script.
Another alternative is to use a pipelined function that returns your logging information. See here for an example: http://berxblog.blogspot.com/2009/01/pipelined-function-vs-dbmsoutput.html When you use a pipelined function you don't have to use another SQLPLUS/Toad/sql developer etc... session.
the buffer of DBMS_OUTPUT is read when the procedure DBMS_OUTPUT.get_line is called. If your client application is SQL*Plus, it means it will only get flushed once the procedure finishes.
You can apply the method described in this SO to write the DBMS_OUTPUT buffer to a file.
Set session metadata MODULE and/or ACTION using dbms_application_info().
Monitor with OEM, for example:
Module: ArchiveData
Action: xxx of xxxx
If you have access to system shell from PL/SQL environment you can call netcat:
BEGIN RUN_SHELL('echo "'||p_msg||'" | nc '||p_host||' '||p_port||' -w 5'); END;
p_msg - is a log message
v_host is a host running python script that reads data from socket on port v_port.
I used this design when I wrote aplogr for real-time shell and pl/sql logs monitoring.
I recently started working on a number of large Oracle PL/SQL stored procedures with Toad for Oracle. Number of these procedures updates and inserts stuff into tables. My question is, is there a way to "safely" execute PL/SQL procedures without permanently modifying any of the tables ? Also, how do I safely modify and execute stored procedures for experimentation without actually making changes to the database ?
Doesn't matter if you have Toad or SQ*Plus or anything really - it's all about the code.
First - does your program have any commits or rollbacks IN the stored procedures?
Second - does your program do any DDL work: create a table? That will do an implicit COMMIT. Mind you, if your program calls another program and THAT program has a COMMIT or DDL - you're COMMITTED as its' all in one session.
Third - when you go to execute your stored procedure, does your anonymous block have a COMMIT or ROLLBACK there?
Your tool comes into play for the third bit. Inspect the code behind the 'execute' button.
In SQL Developer (similar to Toad in this regard)...
In this case my SP has a commit in the code - so barring an exception before that line...it's a permanent change.
In the generated anonymous block, there's a ROLLBACK, but it's commented out. When you hit the execute button in your GUI, look at the code there. Change it if necessary.
You can create a copy of your database, then play there. Other thing is, you can create a copy of the procedures/functions, packages and tables involve and play with it.
Let's you have this procedure,
CREATE PROCEDURE proc1
IS
BEGIN
INSERT INTO table1
(col1, col2)
VALUES
('actual data', 'hello');
UPDATE table2
SET col1 = 'actual'
WHERE col2 = 1;
COMMIT;
END;
You will create new procedure with same logic inside it.
CREATE PROCEDURE proc1_test
IS
BEGIN
INSERT INTO table1_test
(col1, col2)
VALUES ('test', 'hello');
UPDATE table2_test
SET col1 = 'test2'
WHERE col2 = 1;
COMMIT;
END;
/
Doing this, it will let you compare your actual data to your test data.
You can put rollback at the end of the procedure and comment any Commits/DDL statements. Also you need to be careful for the Pragma statements if any.
I'm using Ref Cursor as output parameter for PLSQL Procedure. I need to maintain the exact start and end time of proc in log table.
The dummy code below:
Procedure(P1 IN NUMBER, P_REF_CUR OUT SYS_REFCURSOR)
IS
V_TS TIMESTAMP;
BEGIN
V_TS := SYSTIMESTAMP;
<Business logic here to generate SELECT query for Ref Cursor...>;
OPEN P_REF_CUR FOR <SELECT QUERY>;
INSERT INTO LOG_TABLE(ID, STR_TIME,END_TIME,..) VALUES
(1,V_TS,SYSTIMESTAMP,...);
END;
The select query for Ref Cursor sometimes takes 2-3 mins to execute but in log table I see the difference between STR_TIME and END_TIME as only few seconds.
How can I capture the total time taken by procedure including the query execution time?
Once your procedure hands the ref cursor back to the calling process, it has no way of knowing what will happen to it. The caller may never even fetch all of the rows from the cursor. It’s for the caller to log what happens next.
You may try to split this procedure into two packaged procedures, and apply set timing on :
SQL> create or replace package myPkg is
procedure pr1(P1 IN NUMBER);
procedure pr2(P_REF_CUR OUT SYS_REFCURSOR);
end;
/
SQL> create or replace package body myPkg is
v_ts timestamp;
procedure pr1(P1 IN NUMBER) is
begin
v_ts := SYSTIMESTAMP;
<Business logic here to generate SELECT query for Ref Cursor...>;
end;
procedure pr2(P_REF_CUR OUT SYS_REFCURSOR) is
begin
open P_REF_CUR for <SELECT QUERY>;
insert into log_table(ID, STR_TIME,END_TIME,..) values(1,V_TS,SYSTIMESTAMP,...);
end;
end;
/
SQL> set timing on;
SQL> var v_p1 number:=107;
SQL> var v_rc refcursor;
SQL> exec myPkg.pr1( :v_p1 );
PL/SQL procedure successfully completed
Executed in 152,25 seconds
SQL> exec myPkg.pr2( :v_rc );
PL/SQL procedure successfully completed
Executed in 12,34 seconds
SQL> print v_rc;
You can't tell from inside the procedure. The OPEN FOR statement:
... associates a cursor variable with a query, allocates database resources to process the query, identifies the result set, and positions the cursor before the first row of the result set.
All you can time in your procedure is how long it takes to generate the query text and how long it takes to open the cursor. The procedure then ends, and the caller takes over the OUT ref cursor. You can't see anything about what happens to the cursor from here.
The caller than (presumably) fetches the data, which is taking the bulk of the time; but may also be doing other processing. You need the caller to log the time between it calling your procedure and when it closes the ref cursor when it's finished with it - but that will still include any additional processing it does, so you can't separate out how much is actually from the cursor query processing and fetching.
If that is close enough then you could potentially have a second procedure that closes the cursor and logs the time, if you don't want that the caller to have to worry about it. You could have the 'open' cursor record the start time in a session variable (making the package stateful) and have the 'close' procedure retrieve that and insert the logging record; or have the 'open' do the initial insert into the logging table with a null end time, and then have the 'close' update that record with the actual end time. But again, it's only approximate.
if you really want to do it all in that procedure then you would have to do all the query processing within it, which probably means bulk collecting the cursor into a collection and using that collection type as the OUT parameter, adjusting your caller to iterate over that instead of the cursor. That has more memory overhead too of course, so may not be practical.
hi i'm newb in pl/sql :), this is for educational pruposes only.
the schama Dispatching including a table named Employes.and PRF schema that include a table named ZONE.
Dispatching : Employes(num_emp number,name nvarchar2,design_unit varchar2,design_zone varchar2)
and PRF: ZONE(num_zone number,design_zone varchar2,number_of_units number).
the problema is writing a pl/sql procedure to populate ZONE table from Employes table. this is my procedure :
create or replace procedure zoneD as
cursor cur is select design_zone,design_unit from dispatching.employes group by design_zone,design_unit;
varzone cur%rowtype;
begin
open cur;
fetch cur into varzone;loop
exit when cur%notfound;
insert into zone(num_zone,design_zone,nbr_of_unit) values (num_zone.nextval,varzone.design_zone,0);
update zone set nbr_of_unit =( select count(design_unit) from dispatching.employes);
end loop;
close cur;
end zoneD;
the unit is a town , each zone contains many units. in a simple way the prob the procedure does not insert the data i dont know if it is the right way to do that. (sorry about my english :)).
It seems that you are connected as PRF and want to fetch values that belong to DISPATCHING user. In order to do that, DISPATCHING has to grant (at least) SELECT on its EMPLOYEES table to PRF:
-- connect as DISPATCHING
grant select on employees to prf;
A procedure (as you're practicing PL/SQL) should utilize a cursor FOR loop as it is much easier to maintain than a loop which uses explicitly declared cursor (as you don't need to declare it as well as variable(s) you need to store its values into), open it, worry when to exit the loop and - finally - close it. Cursor FOR loop does all of that for you (OK, except writing a SELECT statement which is just the same as the one you'd use while declaring an explicit cursor).
-- connect as PRF
create or replace procedure zoned as
begin
-- cursor FOR loop - you can select both DESIGN_ZONE and count number of
-- units so that you wouldn't have to update that value separately
for cur_r in (select e.design_zone, count(*) number_of_units
from dispatching.employees e -- naming the owner which granted SELECT on its table to PRF user
group by e.design_zone
)
loop
insert into zone (num_zone, design_zone, number_of_units)
values (num_zone.nextval, cur_r.design_zone, cur_r.number_of_units);
end loop;
end;
/
That should do it (unless I made a typo).
Finally, a suggestion, if I may: do format your code properly. The one you posted is a mess difficult to read - no indentation, too long lines (break them!), and it contains only several lines. Imagine what happens when you have thousands of lines of code - who do you expect to debug it? Just a month or two after you've done with that code, you'll forget what you did and why (so comment it), and - if it is unformatted - you'll get a headache. Today's GUI tools offer automatic formatting, so - use it. Otherwise, there are free online formatters, such as Instant SQL Formatter.
Here's a piece of Oracle code I'm trying to adapt. I've abbreviated all the details:
declare
begin
loop
--do stuff to populate a global temporary table. I'll call it 'TempTable'
end loop;
end;
/
Select * from TempTable
Right now, this query runs fine provided I run it in two steps. First I run the program at the top, then I run the select * to get the results.
Is it possible to combine the two pieces so that I can populate the global temp table and retrieve the results all in one step?
Thanks in advance!
Well, for me it depends on how I would see the steps. You are doing a PL/SQL and SQL command. I would rather type in those into a file, and run them in one command (if that could called as a single step for you)...
Something like
file.sql
begin
loop
--do stuff to populate a global temporary table. I'll call it 'TempTable'
end loop;
end;
/
Select *
from TempTable
/
And run it as:
prompt> sqlplus /#db #file.sql
If you give us more details like how you populate the GTT, perhaps we might find a way to do it in a single step.
Yes, but it's not trivial.
create global temporary table my_gtt
( ... )
on commit preserve rows;
create or replace type my_gtt_rowtype as object
( [columns definition] )
/
create or replace type my_gtt_tabtype as table of my_gtt_rowtype
/
create or replace function pipe_rows_from_gtt
return my_gtt_tabtype
pipelined
is
pragma autonomous_transaction;
type rc_type is refcursor;
my_rc rc_type;
my_output_rec my_gtt_rectype := my_gtt_rectype ([nulls for each attribute]);
begin
delete from my_gtt;
insert into my_gtt ...
commit;
open my_rc for select * from my_gtt;
loop
fetch my_rc into my_output_rec.attribute1, my_output_rec.attribute1, etc;
exit when my_rc%notfound;
pipe_row (my_output_rec);
end loop;
close my_rc;
return;
end;
/
I don't know it the autonomous transaction pragma is required - but I suspect it is, otherwise it'll throw errors about functions performing DML.
We use code like this to have reporting engines which can't perform procedural logic build the global temporary tables they use (and reuse) in various subreports.
In oracle, an extra table to store intermediate results is very seldom needed. It might help to make things easier to understand. When you are able to write SQL to fill the intermediate table, you can certainly query the rows in a single step without having to waste time by filling a GTT. If you are using pl/sql to populate the GTT, see if this can be corrected to be pure SQL. That will almost certainly give you a performance benefit.