I am using Oracle/MyBatis and trying to debug a stored procedure with an enormous amount of parameters. Inside the stored procedure I get a ORA-01438: value larger than specified precision allowed for this column
So my initial approach would be to do like dbms_output.put_line in the stored procedure to try to see what the values are right before the offending statement. Without MyBatis, I would ordinarily open up a sqlplus script and type set serveroutput on and then run my stored procedure at some later point to see all the debug messages come out. With MyBatis, I cannot figure out how (if possible) to get these debug statements.
I have the ibatis and sql debuggers set for DEBUG and I use log4j to log everything for my Tomcat 6 application.
The DBMS_OUTPUT package has a few other procedures that you could use. DBMS_OUTPUT.ENABLE functions much like the SQL*Plus command set serveroutput on in that it allocates a buffer for DBMS_OUTPUT.PUT_LINE to write to. DBMS_OUTPUT.GET_LINE can be used to fetch the data written to that buffer by previous calls to DBMS_OUTPUT.PUT_LINE. So it should be possible to call the ENABLE function, call the procedure which writes a number of lines to the buffer, and then call GET_LINE (or GET_LINES) to fetch the data that was written to the DBMS_OUTPUT buffer and write that data to your logs.
It may be simpler, however, to redirect the logging to an Oracle database table rather than trying to use DBMS_OUTPUT. One common approach is to create your own package that has a switch to determine whether to write to DBMS_OUTPUT or whether to write to a table. Something like
CREATE OR REPLACE PACKAGE p
AS
procedure l( p_str IN VARCHAR2 );
END;
CREATE OR REPLACE PACKAGE BODY p
AS
g_destination INTEGER;
g_destination_table CONSTANT INTEGER := 1;
g_destination_dbms_out CONSTANT INTEGER := 2;
PROCEDURE l( p_str IN VARCHAR2 )
AS
BEGIN
IF( g_destination = g_destination_dbms_out )
THEN
dbms_output.put_line( p_str );
ELSE
INSERT INTO log_table ...
END IF;
END;
BEGIN
g_destination := <<determine which constant to set it to. This
may involve querying a `SETTINGS` table, looking
at the environment, or something else>>
END;
END;
Related
I have an SQL script that is called from within a shell script and takes a long time to run. It currently contains dbms_output.put_line statements at various points. The output from these print statements appear in the log files, but only once the script has completed.
Is there any way to ensure that the output appears in the log file as the script is running?
Not really. The way DBMS_OUTPUT works is this: Your PL/SQL block executes on the database server with no interaction with the client. So when you call PUT_LINE, it is just putting that text into a buffer in memory on the server. When your PL/SQL block completes, control is returned to the client (I'm assuming SQLPlus in this case); at that point the client gets the text out of the buffer by calling GET_LINE, and displays it.
So the only way you can make the output appear in the log file more frequently is to break up a large PL/SQL block into multiple smaller blocks, so control is returned to the client more often. This may not be practical depending on what your code is doing.
Other alternatives are to use UTL_FILE to write to a text file, which can be flushed whenever you like, or use an autonomous-transaction procedure to insert debug statements into a database table and commit after each one.
If it is possible to you, you should replace the calls to dbms_output.put_line by your own function.
Here is the code for this function WRITE_LOG -- if you want to have the ability to choose between 2 logging solutions:
write logs to a table in an autonomous transaction
CREATE OR REPLACE PROCEDURE to_dbg_table(p_log varchar2)
-- table mode:
-- requires
-- CREATE TABLE dbg (u varchar2(200) --- username
-- , d timestamp --- date
-- , l varchar2(4000) --- log
-- );
AS
pragma autonomous_transaction;
BEGIN
insert into dbg(u, d, l) values (user, sysdate, p_log);
commit;
END to_dbg_table;
/
or write directly to the DB server that hosts your database
This uses the Oracle directory TMP_DIR
CREATE OR REPLACE PROCEDURE to_dbg_file(p_fname varchar2, p_log varchar2)
-- file mode:
-- requires
--- CREATE OR REPLACE DIRECTORY TMP_DIR as '/directory/where/oracle/can/write/on/DB_server/';
AS
l_file utl_file.file_type;
BEGIN
l_file := utl_file.fopen('TMP_DIR', p_fname, 'A');
utl_file.put_line(l_file, p_log);
utl_file.fflush(l_file);
utl_file.fclose(l_file);
END to_dbg_file;
/
WRITE_LOG
Then the WRITE_LOG procedure which can switch between the 2 uses, or be deactivated to avoid performances loss (g_DEBUG:=FALSE).
CREATE OR REPLACE PROCEDURE write_log(p_log varchar2) AS
-- g_DEBUG can be set as a package variable defaulted to FALSE
-- then change it when debugging is required
g_DEBUG boolean := true;
-- the log file name can be set with several methods...
g_logfname varchar2(32767) := 'my_output.log';
-- choose between 2 logging solutions:
-- file mode:
g_TYPE varchar2(7):= 'file';
-- table mode:
--g_TYPE varchar2(7):= 'table';
-----------------------------------------------------------------
BEGIN
if g_DEBUG then
if g_TYPE='file' then
to_dbg_file(g_logfname, p_log);
elsif g_TYPE='table' then
to_dbg_table(p_log);
end if;
end if;
END write_log;
/
And here is how to test the above:
1) Launch this (file mode) from your SQLPLUS:
BEGIN
write_log('this is a test');
for i in 1..100 loop
DBMS_LOCK.sleep(1);
write_log('iter=' || i);
end loop;
write_log('test complete');
END;
/
2) on the database server, open a shell and
tail -f -n500 /directory/where/oracle/can/write/on/DB_server/my_output.log
Two alternatives:
You can insert your logging details in a logging table by using an autonomous transaction. You can query this logging table in another SQLPLUS/Toad/sql developer etc... session. You have to use an autonomous transaction to make it possible to commit your logging without interfering the transaction handling in your main sql script.
Another alternative is to use a pipelined function that returns your logging information. See here for an example: http://berxblog.blogspot.com/2009/01/pipelined-function-vs-dbmsoutput.html When you use a pipelined function you don't have to use another SQLPLUS/Toad/sql developer etc... session.
the buffer of DBMS_OUTPUT is read when the procedure DBMS_OUTPUT.get_line is called. If your client application is SQL*Plus, it means it will only get flushed once the procedure finishes.
You can apply the method described in this SO to write the DBMS_OUTPUT buffer to a file.
Set session metadata MODULE and/or ACTION using dbms_application_info().
Monitor with OEM, for example:
Module: ArchiveData
Action: xxx of xxxx
If you have access to system shell from PL/SQL environment you can call netcat:
BEGIN RUN_SHELL('echo "'||p_msg||'" | nc '||p_host||' '||p_port||' -w 5'); END;
p_msg - is a log message
v_host is a host running python script that reads data from socket on port v_port.
I used this design when I wrote aplogr for real-time shell and pl/sql logs monitoring.
I'm using Ref Cursor as output parameter for PLSQL Procedure. I need to maintain the exact start and end time of proc in log table.
The dummy code below:
Procedure(P1 IN NUMBER, P_REF_CUR OUT SYS_REFCURSOR)
IS
V_TS TIMESTAMP;
BEGIN
V_TS := SYSTIMESTAMP;
<Business logic here to generate SELECT query for Ref Cursor...>;
OPEN P_REF_CUR FOR <SELECT QUERY>;
INSERT INTO LOG_TABLE(ID, STR_TIME,END_TIME,..) VALUES
(1,V_TS,SYSTIMESTAMP,...);
END;
The select query for Ref Cursor sometimes takes 2-3 mins to execute but in log table I see the difference between STR_TIME and END_TIME as only few seconds.
How can I capture the total time taken by procedure including the query execution time?
Once your procedure hands the ref cursor back to the calling process, it has no way of knowing what will happen to it. The caller may never even fetch all of the rows from the cursor. It’s for the caller to log what happens next.
You may try to split this procedure into two packaged procedures, and apply set timing on :
SQL> create or replace package myPkg is
procedure pr1(P1 IN NUMBER);
procedure pr2(P_REF_CUR OUT SYS_REFCURSOR);
end;
/
SQL> create or replace package body myPkg is
v_ts timestamp;
procedure pr1(P1 IN NUMBER) is
begin
v_ts := SYSTIMESTAMP;
<Business logic here to generate SELECT query for Ref Cursor...>;
end;
procedure pr2(P_REF_CUR OUT SYS_REFCURSOR) is
begin
open P_REF_CUR for <SELECT QUERY>;
insert into log_table(ID, STR_TIME,END_TIME,..) values(1,V_TS,SYSTIMESTAMP,...);
end;
end;
/
SQL> set timing on;
SQL> var v_p1 number:=107;
SQL> var v_rc refcursor;
SQL> exec myPkg.pr1( :v_p1 );
PL/SQL procedure successfully completed
Executed in 152,25 seconds
SQL> exec myPkg.pr2( :v_rc );
PL/SQL procedure successfully completed
Executed in 12,34 seconds
SQL> print v_rc;
You can't tell from inside the procedure. The OPEN FOR statement:
... associates a cursor variable with a query, allocates database resources to process the query, identifies the result set, and positions the cursor before the first row of the result set.
All you can time in your procedure is how long it takes to generate the query text and how long it takes to open the cursor. The procedure then ends, and the caller takes over the OUT ref cursor. You can't see anything about what happens to the cursor from here.
The caller than (presumably) fetches the data, which is taking the bulk of the time; but may also be doing other processing. You need the caller to log the time between it calling your procedure and when it closes the ref cursor when it's finished with it - but that will still include any additional processing it does, so you can't separate out how much is actually from the cursor query processing and fetching.
If that is close enough then you could potentially have a second procedure that closes the cursor and logs the time, if you don't want that the caller to have to worry about it. You could have the 'open' cursor record the start time in a session variable (making the package stateful) and have the 'close' procedure retrieve that and insert the logging record; or have the 'open' do the initial insert into the logging table with a null end time, and then have the 'close' update that record with the actual end time. But again, it's only approximate.
if you really want to do it all in that procedure then you would have to do all the query processing within it, which probably means bulk collecting the cursor into a collection and using that collection type as the OUT parameter, adjusting your caller to iterate over that instead of the cursor. That has more memory overhead too of course, so may not be practical.
I have a large source data set (a few million rows) that requires complex processing, resulting in much larger amount of data, which should be then offloaded and stored as files. The storage requires dividing up resulting data based on certain parameters, namely N source rows that meet certain criteria.
Since it's possible to compute the said parameters within PL/SQL, it was decided that the most efficient way would be to create a package, specify a spec-level cursor for source rows in it, then write a procedure that would partially consume the opened cursor until the criteria is meet and fill temporary tables with resulting data, which would then be offloaded, and the procedure would be called again, repeating until there's no more source rows. PL/SQL basically looks like this:
create or replace PACKAGE BODY generator as
cursor glob_cur_body(cNo number) is
select *
from source_table
where no = cNo
order by conditions;
procedure close_cur is
begin
if glob_cur_body%isopen then
close glob_cur_body;
end if;
end close_cur;
procedure open_cur(pNo number) is
begin
close_cur;
open glob_cur_body(pNo);
end open_cur;
function consume_cur return varchar2 is
v source_table%rowtype;
part_id varchar2(100);
begin
fetch glob_cur_body into v;
if glob_cur_body%notfound then
return null;
end if;
--Clear temporary tables
--Do the processing until criteria is meet of there's no more rows
--Fill the temporary tables and part_id
return part_id;
end consume_cur;
end generator;
And the consumer is doing the following (in pseudocode)
generator.open_cur;
part_id = generator.consume;
while ( part_id != null )
{
//offload data from temp tables
part_id = generator.consume;
}
generator.close_cur;
It's working fine, but unfortunately there's one problem: a spec-level cursor makes the package stateful, meaning that its recompilation results in ORA-04068 for sessions that already accessed it before. It makes maintenance cumbersome, because there's a lot more to the package besides said functions, and it's actively used for unrelated purposes.
So, I want to get rid of the spec-level cursor, but I'm not sure if that's possible. Some ideas I've already discarded:
Re-opening the cursor and skipping N rows: terrible performance, unreliable because affected by any changes to data made between opens
Fetching the source cursor into plsql table: size too large.
Filling up the entire unload tables at once, splitting them later: size too large, subpar performance.
Opening the cursor as refcursor and storing refcursor variable in a dedicated package: impossible, as pl/sql doesn't allow sys_refcursor variables at spec levels
Having open_cur procedures return refcursor, storing it in the offloader, and then somehow passing it to consume_cur: looked viable, but the offloader is in Java, and JDBC doesn't allow binding of SYS_REFCURSOR parameters.
Changing consume_cur to pipelined function: could have worked, but oracle buffers pipelined rows, meaning it would execute multiple times when fetching data from it row-by-row. Also counterintuitive.
Only other idea I've had so far is to make a dedicated package storing said cursor, having open and close procedures and get_cursor returning refcursor; then call get_cursor from generator.consume_cur. That would make the dedicated package (which is unlikely to change) stateful and main package stateless. However, it seems like a half-baked patch rather than a problem solution. Is there a more decent way of achieving what I need? Perhaps changing the logic completely without affecting performance and storage limits too much.
I have a problem to understand your question. But I can provide clarification for your ideas.
Opening the cursor as refcursor and storing refcursor variable in a
dedicated package: impossible, as pl/sql doesn't allow sys_refcursor
variables at spec levels
The workaround with dbms_sql.
create table test_rows as (select level rr from dual connect by level <= 100);
create or replace package cursor_ctx is
ctx_number integer;
end;
declare
p_cursor sys_refcursor;
begin
open p_cursor for 'select rr from test_rows';
cursor_ctx.ctx_number := DBMS_SQL.TO_CURSOR_NUMBER(p_cursor);
end;
This part consuming is data from the cursor.
declare
p_cursor sys_refcursor;
type l_number is table of number;
v_numbers l_number;
begin
if DBMS_SQL.IS_OPEN(cursor_ctx.ctx_number) then
p_cursor := DBMS_SQL.TO_REFCURSOR( cursor_ctx.ctx_number);
fetch p_cursor bulk collect into v_numbers limit 10;
if v_numbers.count < 10 then
dbms_output.put_line('No more data, close cursor');
close p_cursor;
cursor_ctx.ctx_number := null;
else
cursor_ctx.ctx_number := DBMS_SQL.TO_CURSOR_NUMBER(p_cursor);
end if;
for i in nvl(v_numbers.first,1) .. nvl(v_numbers.last,-1) loop
dbms_output.put_line(v_numbers(i));
end loop;
else
dbms_output.put_line('Null or cursor close ');
end if;
end;
Pipelined function has future to split input cursor into chunk. Parallel Enabled Pipelined Table Functions
JDBC allows using sys_refcursor as an output parameter. sys_refcursor = ResultSet.
I'm quite new to PL/SQL, and am using Oracle SQL Developer to write a procedure which uses a sequence to generate a primary key for some existing data, to write into another DB.
The code in question is under NDA.. Essentially I have the following:
create or replace
PROCEDURE Generate_Data
(
output IN VARCHAR2
)
AS
-- Variables here --
CURSOR myCursor IS
SELECT data1, data2
FROM table;
CREATE SEQUENCE mySequence <-- error on this line
START WITH 0
INCREMENT BY 1;
BEGIN
LOOP
-- snip --
It raises the error PLS-00103, saying it encountered the symbol CREATE when expecting on of the following: begin, function, package, pragma, procedure, ...
I've been following the example at:
http://www.techonthenet.com/oracle/sequences.php
The reason you're getting this error is that you're trying to perform DDL, in this case creating a sequence, within PL/SQL. It is possible to do this, but you must use execute immediate.
As Alex says, you also wouldn't be able to do this in the declare section. It would look something like this:
begin
execute immediate 'CREATE SEQUENCE mySequence
START WITH 0
INCREMENT BY 1';
end;
However, as Padmarag also says, it's highly unlikely that you want to do this within PL/SQL. It would be more normal to create a sequence outside and then reference this later. More generally speaking, performing DDL inside a PL/SQL block is a bad idea; there should be no need for you to do it.
You don't mention what version of Oracle you're using. From 11g the ways in which you could access sequences got extended. If you're using 11g then you can access the sequence by creating a variable and assigning the next value in the sequence, .nextval, to this variable:
declare
l_seq number;
begin
loop
-- For each loop l_seq will be incremented.
l_seq := mysequence.nextval;
-- snip
end;
If you're before 11g you must (outside of DML) use a select statement in order to get the next value:
declare
l_seq number;
begin
loop
-- For each loop l_seq will be incremented.
select mysequence.nextval into l_seq from dual;
-- snip
end;
Please bear in mind that a sequence is meant to be a persistent object in the database. There is no need to drop and re-create it each time you want to use it. If you were to run your script, then re-run it the sequence would happily keep increasing the returned value.
Further Reading
About sequences
Using sequences
You can't create sequence in the DECLARE block of procedure. Move it after BEGIN. It's arguable if it makes sense, though. You probably need to create it outside your procedure in the first place.
Update
Actually, if you truly want it inside BEGIN/END use following:
EXECUTE IMMEDIATE 'CREATE SEQUENCE mySequence START WITH 0 INCREMENT BY 1';
You'd need to create the sequence before using it.
And in the PL/SQL code use
-- Variables here --1
v_seq_val number;
BEGIN
Select mySequence.nextval from dual into v_seq_val
In general SQL is for DDL(Data Definition Language) and PL/SQL is for DML(Data Manipulation Language) and logic.
If you wanted you could do Create from PL/SQL, but I think that's not what you want over here.
I have a following oracle stored procedure
CREATE OR REPLACE
PROCEDURE getRejectedReasons
(
p_cursor IN OUT SYS_REFCURSOR)
AS
BEGIN
OPEN p_cursor FOR SELECT * FROM reasons_for_rejection;
END;
However, when I run this stored procedure in sql-developer then I dont see anything. I just see something like this:
Connecting to the database oracleLocal.
Process exited.
Disconnecting from the database oracleLocal.
I'm coming from MS sql server and am used to seeing actual results when running a stored procedure like this. Is this stored procedure not returning results because I am using a cursor??
The stored procedure is returning something it's just you aren't doing anything with the results.
You can do this simply by running the following script in SQLDeveloper:
VARIABLE csr REFCURSOR;
EXEC getRejectedReasons(:csr); -- the colon identifies the parameter as a variable
PRINT csr;
Another method is to fetch each row and do some sort of processing:
DECLARE
-- sys_refcursor is weakly typed
refcsr SYS_REFCURSOR;
-- define a record so we can reference the fields
rej_rec Reasons_for_Rejection%ROWTYPE;
BEGIN
getRejectedReasons(refcsr);
-- loop through the results
LOOP
-- gets one row at a time
FETCH refcsr INTO rej_rec;
-- if the fetch doesn't find any more rows exit the loop
EXIT WHEN refcsr%NOTFOUND;
-- Do something here.
-- For example : DBMS_OUTPUT.PUT_LINE(rej_rec.reason_desc);
END LOOP;
END;
You opened the cursor. You didn't select anything from it, update it, or advance it.
All open does, effectively, to select the matching rows into temporary memory, so you can advance the cursor row by row. Which you didn't do.
One of the differences between Oracle and SQL Server is that the latter returns result sets naturally. I'd use a function, by the way.
In Oracle, functions typically return a single element. Cursors came later.
There's some documentation online that will help you understand the use of refcursor bind variables. Here's one such for SQL*Plus:
http://download.oracle.com/docs/cd/B19306_01/server.102/b14357/ch5.htm#sthref1122
I think in SQL Developer you can do the same thing with autoprint on, although I haven't tested that.
Found a blog that also discusses something similar:
http://vadimtropashko.wordpress.com/cursors/
ETA: Ok. Ignore what I wrote. Listen to someone else. Apparently it's wrong, as I got down voted.
What tpdi said is correct. You have to do something with the cursor after you declare it.
Here's an example using two cursors in nested loops
PROCEDURE update_insert_tree (exid_in IN NUMBER, outvar_out OUT VARCHAR2)
IS
nxtid NUMBER;
phaseid NUMBER;
rowcounter1 NUMBER;
BEGIN
rowcounter1 := 0;
outvar_out := 0;
FOR acur IN (SELECT dept_exercise_id, phase
FROM ep_dept_exercise
WHERE exercise_id = exid_in)
LOOP
<<dept_loop>>
FOR thecur IN (SELECT document_name, thelevel, sortnum, type_flag,
ex_save_id
FROM ep_exercise_save
WHERE exercise_id = exid_in)
LOOP
phaseid := acur.phase;
IF phaseid = 0
THEN
phaseid := 10;
UPDATE ep_dept_exercise
SET phase = 10
WHERE dept_exercise_id = acur.dept_exercise_id;
END IF;
<<doc_loop>>