I have an SQL script that is called from within a shell script and takes a long time to run. It currently contains dbms_output.put_line statements at various points. The output from these print statements appear in the log files, but only once the script has completed.
Is there any way to ensure that the output appears in the log file as the script is running?
Not really. The way DBMS_OUTPUT works is this: Your PL/SQL block executes on the database server with no interaction with the client. So when you call PUT_LINE, it is just putting that text into a buffer in memory on the server. When your PL/SQL block completes, control is returned to the client (I'm assuming SQLPlus in this case); at that point the client gets the text out of the buffer by calling GET_LINE, and displays it.
So the only way you can make the output appear in the log file more frequently is to break up a large PL/SQL block into multiple smaller blocks, so control is returned to the client more often. This may not be practical depending on what your code is doing.
Other alternatives are to use UTL_FILE to write to a text file, which can be flushed whenever you like, or use an autonomous-transaction procedure to insert debug statements into a database table and commit after each one.
If it is possible to you, you should replace the calls to dbms_output.put_line by your own function.
Here is the code for this function WRITE_LOG -- if you want to have the ability to choose between 2 logging solutions:
write logs to a table in an autonomous transaction
CREATE OR REPLACE PROCEDURE to_dbg_table(p_log varchar2)
-- table mode:
-- requires
-- CREATE TABLE dbg (u varchar2(200) --- username
-- , d timestamp --- date
-- , l varchar2(4000) --- log
-- );
AS
pragma autonomous_transaction;
BEGIN
insert into dbg(u, d, l) values (user, sysdate, p_log);
commit;
END to_dbg_table;
/
or write directly to the DB server that hosts your database
This uses the Oracle directory TMP_DIR
CREATE OR REPLACE PROCEDURE to_dbg_file(p_fname varchar2, p_log varchar2)
-- file mode:
-- requires
--- CREATE OR REPLACE DIRECTORY TMP_DIR as '/directory/where/oracle/can/write/on/DB_server/';
AS
l_file utl_file.file_type;
BEGIN
l_file := utl_file.fopen('TMP_DIR', p_fname, 'A');
utl_file.put_line(l_file, p_log);
utl_file.fflush(l_file);
utl_file.fclose(l_file);
END to_dbg_file;
/
WRITE_LOG
Then the WRITE_LOG procedure which can switch between the 2 uses, or be deactivated to avoid performances loss (g_DEBUG:=FALSE).
CREATE OR REPLACE PROCEDURE write_log(p_log varchar2) AS
-- g_DEBUG can be set as a package variable defaulted to FALSE
-- then change it when debugging is required
g_DEBUG boolean := true;
-- the log file name can be set with several methods...
g_logfname varchar2(32767) := 'my_output.log';
-- choose between 2 logging solutions:
-- file mode:
g_TYPE varchar2(7):= 'file';
-- table mode:
--g_TYPE varchar2(7):= 'table';
-----------------------------------------------------------------
BEGIN
if g_DEBUG then
if g_TYPE='file' then
to_dbg_file(g_logfname, p_log);
elsif g_TYPE='table' then
to_dbg_table(p_log);
end if;
end if;
END write_log;
/
And here is how to test the above:
1) Launch this (file mode) from your SQLPLUS:
BEGIN
write_log('this is a test');
for i in 1..100 loop
DBMS_LOCK.sleep(1);
write_log('iter=' || i);
end loop;
write_log('test complete');
END;
/
2) on the database server, open a shell and
tail -f -n500 /directory/where/oracle/can/write/on/DB_server/my_output.log
Two alternatives:
You can insert your logging details in a logging table by using an autonomous transaction. You can query this logging table in another SQLPLUS/Toad/sql developer etc... session. You have to use an autonomous transaction to make it possible to commit your logging without interfering the transaction handling in your main sql script.
Another alternative is to use a pipelined function that returns your logging information. See here for an example: http://berxblog.blogspot.com/2009/01/pipelined-function-vs-dbmsoutput.html When you use a pipelined function you don't have to use another SQLPLUS/Toad/sql developer etc... session.
the buffer of DBMS_OUTPUT is read when the procedure DBMS_OUTPUT.get_line is called. If your client application is SQL*Plus, it means it will only get flushed once the procedure finishes.
You can apply the method described in this SO to write the DBMS_OUTPUT buffer to a file.
Set session metadata MODULE and/or ACTION using dbms_application_info().
Monitor with OEM, for example:
Module: ArchiveData
Action: xxx of xxxx
If you have access to system shell from PL/SQL environment you can call netcat:
BEGIN RUN_SHELL('echo "'||p_msg||'" | nc '||p_host||' '||p_port||' -w 5'); END;
p_msg - is a log message
v_host is a host running python script that reads data from socket on port v_port.
I used this design when I wrote aplogr for real-time shell and pl/sql logs monitoring.
Related
Actually I am creating stored procedure with substitution, while trying to compile the procedure, I get the popup to enter the substitution values in compiling itself, Instead of getting popup while execution.
Please share me your idea to compile the procedure without asking the substitution
In SQL*Plus or Oracle SQL Developer, you'd SET DEFINE OFF. You tagged the question with PL/SQL Developer tag (which is a tool I don't use), but - see if this helps.
However: I'd suggest you not to do it that way. If you're creating a stored procedure, then use its parameters, don't ask for substitution variables. Something like this:
SQL> set serveroutput on
SQL> create or replace procedure p_test (par_deptno in dept.deptno%type) is
2 begin
3 dbms_output.put_line('Department ' || par_deptno);
4 end;
5 /
Procedure created.
SQL> exec p_test(10);
Department 10
PL/SQL procedure successfully completed.
You can now reuse such a procedure, passing any parameter value you want.
The way you're doing it now:
SQL> create or replace procedure p_test is
2 begin
3 dbms_output.put_line('Department ' || &par_deptno);
4 end;
5 /
Enter value for par_deptno: 25
old 3: dbms_output.put_line('Department ' || &par_deptno);
new 3: dbms_output.put_line('Department ' || 25);
Procedure created.
SQL> exec p_test
Department 25
PL/SQL procedure successfully completed.
SQL>
you can run the procedure many times, but it'll always display (i.e. use) the same value, throughout that session. Once you exit and log in again, procedure will always use the same value.
You cannot! A substitution variable acts like a find-replace operation in the client application at the time the statement is run; the database does NOT see the substitution variable as the client application you are using will have already performed the find-replace operation. Which, for a procedure, would be at the time the CREATE PROCEDURE statement is sent from the client to the database to be compiled. It is NOT an operation that the database performs.
If you try it in a client that does not support substitution variables (or in a client that does support it after turning off substitution variables using, for example, the command SET DEFINE OFF in the client application) then you will get a compilation error. db<>fiddle
If you want to use substitution variables then use them in the anonymous block when you call the procedure.
An example procedure would take parameters and have no substitution variable:
CREATE PROCEDURE procedure_name(
p_value IN NUMBER
)
IS
BEGIN
-- Do something
DBMS_OUTPUT.PUT_LINE( p_value );
END;
/
Then when you want to execute the procedure you can use a substitution variable in the calling block:
BEGIN
procedure_name( &value );
END;
/
I have very simple pl\sql code.
In this code I'm printing the index of the loop and wait 1 second before each print.
My problem is that I want this output to be used like a live log, when the dbms_output.put_line procedure is invoked and print - I want to see the output immediately.
In the current code - only after it finished (5 seconds..), it prints all the output in one shot...
set serveroutput on
set echo on
begin
for i in 1..5
loop
dba_maint.pkg_utils.sp_sleep(1);
dbms_output.put_line(i);
end loop;
end;
/
No way, you can't. It is displayed when PL/SQL procedure has finished.
If you want to create a live log,
create a table
a sequence
an autonomous transaction procedure which would
insert a row into that table
using a sequence (so that you'd know how to order rows)
possibly a timestamp (so that you'd know how long certain step took)
commit (which won't affect main transaction as - remember - procedure is an autonomous transaction one)
Then put calls to the logging procedure into your long-time-run PL/SQL procedure, run it, and let it work. In another session, query the log table to view progress.
I have PL/SQL Block, it query from a Table function and I use a cursor to process it record by record, have some business logic and finally write the Qualified records into the file.
Number of records to be processed is upto 1 Million. And the total processing speed is roughly 10000 records per minute.(After testing with few chunks of data)
Now, that I need to indicate the processing status in a different environment, JSP.
DECLARE
vSFile utl_file.file_type;
vNewLine VARCHAR2(200);
my_cursor IS SELECT * FROM MYTABLE;
my_details my_cursor%rowtype;
BEGIN
vSFile := utl_file.fopen('ORALOAD', file_name,'r');
IF utl_file.is_open(vSFile) THEN
utl_file.get_line(vSFile, vNewLine);
OPEN my_cursor;
LOOP
FETCH my_cursor INTO my_details;
EXIT WHEN sll_cur%NOTFOUND;
-- Do processing
utl_file.putf(logfile,'%s ',my_details);
-- A info tht record completed!
END LOOP;
CLOSE logfile;
CLOSE my_cursor;
END IF;
EXCEPTION
WHEN OTHERS THEN
--Error handling
END;
/
The log information written here, is not available until the completion of the process. So, I am unable to track, how far it is completed.
Can someone please assist me on this?
To write data to the file you should use the FFLUSH procedure. For instance:
OPEN my_cursor;
LOOP
FETCH my_cursor INTO my_details;
EXIT WHEN sll_cur%NOTFOUND;
-- Do processing
utl_file.putf(logfile,'%s ',my_details);
-- Call the FFLUSH proc here..And contents are available immediately.
utl_file.FFLUSH(logfile);
END LOOP;
From the documentation:
FFLUSH physically writes pending data to the file identified by the
file handle. Normally, data being written to a file is buffered. The
FFLUSH procedure forces the buffered data to be written to the file.
The data must be terminated with a newline character.
Flushing is useful when the file must be read while still open. For
example, debugging messages can be flushed to the file so that they
can be read immediately.
There is more information on UTL_FILE in the Oracle docs.
I am using Oracle/MyBatis and trying to debug a stored procedure with an enormous amount of parameters. Inside the stored procedure I get a ORA-01438: value larger than specified precision allowed for this column
So my initial approach would be to do like dbms_output.put_line in the stored procedure to try to see what the values are right before the offending statement. Without MyBatis, I would ordinarily open up a sqlplus script and type set serveroutput on and then run my stored procedure at some later point to see all the debug messages come out. With MyBatis, I cannot figure out how (if possible) to get these debug statements.
I have the ibatis and sql debuggers set for DEBUG and I use log4j to log everything for my Tomcat 6 application.
The DBMS_OUTPUT package has a few other procedures that you could use. DBMS_OUTPUT.ENABLE functions much like the SQL*Plus command set serveroutput on in that it allocates a buffer for DBMS_OUTPUT.PUT_LINE to write to. DBMS_OUTPUT.GET_LINE can be used to fetch the data written to that buffer by previous calls to DBMS_OUTPUT.PUT_LINE. So it should be possible to call the ENABLE function, call the procedure which writes a number of lines to the buffer, and then call GET_LINE (or GET_LINES) to fetch the data that was written to the DBMS_OUTPUT buffer and write that data to your logs.
It may be simpler, however, to redirect the logging to an Oracle database table rather than trying to use DBMS_OUTPUT. One common approach is to create your own package that has a switch to determine whether to write to DBMS_OUTPUT or whether to write to a table. Something like
CREATE OR REPLACE PACKAGE p
AS
procedure l( p_str IN VARCHAR2 );
END;
CREATE OR REPLACE PACKAGE BODY p
AS
g_destination INTEGER;
g_destination_table CONSTANT INTEGER := 1;
g_destination_dbms_out CONSTANT INTEGER := 2;
PROCEDURE l( p_str IN VARCHAR2 )
AS
BEGIN
IF( g_destination = g_destination_dbms_out )
THEN
dbms_output.put_line( p_str );
ELSE
INSERT INTO log_table ...
END IF;
END;
BEGIN
g_destination := <<determine which constant to set it to. This
may involve querying a `SETTINGS` table, looking
at the environment, or something else>>
END;
END;
I have a following oracle stored procedure
CREATE OR REPLACE
PROCEDURE getRejectedReasons
(
p_cursor IN OUT SYS_REFCURSOR)
AS
BEGIN
OPEN p_cursor FOR SELECT * FROM reasons_for_rejection;
END;
However, when I run this stored procedure in sql-developer then I dont see anything. I just see something like this:
Connecting to the database oracleLocal.
Process exited.
Disconnecting from the database oracleLocal.
I'm coming from MS sql server and am used to seeing actual results when running a stored procedure like this. Is this stored procedure not returning results because I am using a cursor??
The stored procedure is returning something it's just you aren't doing anything with the results.
You can do this simply by running the following script in SQLDeveloper:
VARIABLE csr REFCURSOR;
EXEC getRejectedReasons(:csr); -- the colon identifies the parameter as a variable
PRINT csr;
Another method is to fetch each row and do some sort of processing:
DECLARE
-- sys_refcursor is weakly typed
refcsr SYS_REFCURSOR;
-- define a record so we can reference the fields
rej_rec Reasons_for_Rejection%ROWTYPE;
BEGIN
getRejectedReasons(refcsr);
-- loop through the results
LOOP
-- gets one row at a time
FETCH refcsr INTO rej_rec;
-- if the fetch doesn't find any more rows exit the loop
EXIT WHEN refcsr%NOTFOUND;
-- Do something here.
-- For example : DBMS_OUTPUT.PUT_LINE(rej_rec.reason_desc);
END LOOP;
END;
You opened the cursor. You didn't select anything from it, update it, or advance it.
All open does, effectively, to select the matching rows into temporary memory, so you can advance the cursor row by row. Which you didn't do.
One of the differences between Oracle and SQL Server is that the latter returns result sets naturally. I'd use a function, by the way.
In Oracle, functions typically return a single element. Cursors came later.
There's some documentation online that will help you understand the use of refcursor bind variables. Here's one such for SQL*Plus:
http://download.oracle.com/docs/cd/B19306_01/server.102/b14357/ch5.htm#sthref1122
I think in SQL Developer you can do the same thing with autoprint on, although I haven't tested that.
Found a blog that also discusses something similar:
http://vadimtropashko.wordpress.com/cursors/
ETA: Ok. Ignore what I wrote. Listen to someone else. Apparently it's wrong, as I got down voted.
What tpdi said is correct. You have to do something with the cursor after you declare it.
Here's an example using two cursors in nested loops
PROCEDURE update_insert_tree (exid_in IN NUMBER, outvar_out OUT VARCHAR2)
IS
nxtid NUMBER;
phaseid NUMBER;
rowcounter1 NUMBER;
BEGIN
rowcounter1 := 0;
outvar_out := 0;
FOR acur IN (SELECT dept_exercise_id, phase
FROM ep_dept_exercise
WHERE exercise_id = exid_in)
LOOP
<<dept_loop>>
FOR thecur IN (SELECT document_name, thelevel, sortnum, type_flag,
ex_save_id
FROM ep_exercise_save
WHERE exercise_id = exid_in)
LOOP
phaseid := acur.phase;
IF phaseid = 0
THEN
phaseid := 10;
UPDATE ep_dept_exercise
SET phase = 10
WHERE dept_exercise_id = acur.dept_exercise_id;
END IF;
<<doc_loop>>