... before anyone marks this as a duplicate ... i have checked Is there any way to flush output from PL/SQL in Oracle?
and am looking for more information on how to actually do what that answer referenced ... so hold it.
I absolutely need to use DBMS_OUTPUT due to various reasons. UTL_FILE won't help.
For the benefit of this question, let's say i'm generating a flat file.
declare
function printerFunction(someArray) <---- this function swallows an array and dumps it to
is begin
for each element of some array ... dbms_output.put_line(element)
end printerFunction;
begin
for row in some_cursor loop
printerFunction(someArrayData);
end loop;
end
The aforementioned code block is essentially the jist of the matter... nothing special.
Dave Costa mentions something along the lines of "break up a large PL/SQL block into multiple smaller blocks" ... the question is how.
I understand I have a nested loop structure and this is most likely the reason behind the output buffer not flushing itself, but can't think of a way to keep some_cursor open or for all intents and purposes switch back and forth between two code blocks outside of that loop .
dbms_output.enable(null) is kind of a dumb idea in this case as well. Ideally i'd like to essentially flush out the stuff in the buffer to my sqlDeveloper scriptOutput window and move on with processing at a specific rate ... say every 10000 rows or so. is this even possible? ... I mean... the main begin [pl/sql code] end structure is essentially a code block itself.
... the db I'm working on is a production environment, and i can only use a limited, read-only set of commands. ... in other words ... SYS stuff is beyond my reach.
As Dave Costa explains in the answer you link to, if you want the client to display the information before all your processing is done, you'd need to break your code up into separate blocks that you could send separately to the database. The client has no way to read the data out of the dbms_output buffer until the database returns control to the client.
You wouldn't realistically keep a single cursor open across these separate calls. You'd open a new cursor each time. For example, if you had a block
begin
for cursor in (select * from ... order by ... offset x fetch next y rows only)
loop
<<do something>>
end loop;
end;
you could have that run for 10,000 rows (or whatever value of y you want), get the output in SQL Developer, then run another block to process the next y rows (or have x and y be variables defined in SQL Developer that you update in each iteration or create a package to save state in package variables but I assume that is not an option here). But you'd need to submit a new PL/SQL block to the database for each set of y rows that you wanted to process. And this query will get less efficient as you try to get later and later pages since it has to sort more and more data to get the results.
Practically, it is pretty unlikely that you'd really want to break your logic up like this. You'd almost always be better off submitting a single PL/SQL block and letting the client fetch the data once at the end. But using dbms_output to generate a file is pretty weird in the first place. You'd generally be better served letting SQL Developer generate the file. For example, if you return a sys_refcursor for
select *
from table(someArrayData)
or just run whatever query that was used to generate someArrayData, SQL Developer can happily save that data into a CSV file, and XLS(X) file, etc.
You can clear the buffer by consuming the messages manually. This snippet puts something in the buffer and then clears it. My client shows no output.
DECLARE
var_status integer := 0;
var_dummy varchar2(32767);
BEGIN
dbms_output.put_line('this is a test');
WHILE var_status = 0
LOOP
DBMS_OUTPUT.GET_LINE (line => var_dummy,
status => var_status);
END LOOP;
END;
Related
I've searched across the whole internet for some examples, but I still can't get my head around why I can not use DML statement inside this cursor. I'm kind of missing the theory behind it, but I won't deny an example of how to do write this correctly would make my life lots easier as well. Here is the query I'm working on (Note: I removed exits when not found results, close if cursor already open and things like that just to focus on the main point here):
DECLARE
// lots of vars
// the cursor below gets all datasources connected to Node XXYZ123
CURSOR DataSourceCheck
IS
SELECT NODENAAM, NAAM, URL, DBNODE1, DBNODE2, DBUSERNAAM, DBNAAM
FROM SCHEMA.TABLENAME
WHERE NODENAAM = 'XXYZ123';
// this cursor will execute row-by-row based on the result set of above cursor
CURSOR CheckIfOnlyDataSource
IS
SELECT NODENAAM, NAAM, URL, DBNODE1, DBNODE2, DBUSERNAAM, DBNAAM
FROM SCHEMA.TABLENAME
WHERE DBUSERNAAM = var_dbusernaam AND (DBNode1 = var_dbnode1 OR DBNode2 = var_dbnode2);
BEGIN
OPEN DataSourceCheck;
LOOP
FETCH DataSourceCheck into var_nodenaam, var_naam, var_URL, var_dbnode1, var_dbnode2, var_dbusernaam, var_dbnaam;
var_rowcount:= 0;
OPEN CheckIfOnlyDataSource;
LOOP
FETCH CheckIfOnlyDataSource into var_nodenaam2, var_naam2, var_URL2, var_dbnode12, var_dbnode22, var_dbusernaam2, var_dbnaam2;
var_rowcount:= var_rowcount + 1;
END LOOP;
// only save result in a temp table when var_rowcount is 1 and not higher.
IF var_rowcount = 1
THEN
INSERT INTO global_temp_table
(t_dbusernaam, t_nodenaam, t_dbnode1, t_dbnode2, t_distinctcount)
VALUES
(var_dbusernaam2, var_nodenaam2, var_dbnode12, var_dbnode22, var_rowcount)
END IF;
CLOSE CheckIfOnlyDataSource;
END LOOP;
END;
The point of failure is this part, with the message that DML should be reconfigured into FORALL or BULK INTO statements:
IF var_rowcount = 1
THEN
INSERT INTO global_temp_table
(t_dbusernaam, t_nodenaam, t_dbnode1, t_dbnode2, t_distinctcount)
VALUES
(var_dbusernaam2, var_nodenaam2, var_dbnode12, var_dbnode22, var_rowcount)
END IF;
I don't understand why DML is not working in a row-by-row approach? The output is clearly stored inside the variables var_dbusernaam2, var_nodenaam2, var_dbnode12 and var_dbnode22, hence I can do a dbms_output.put_line to show them. But if it is stored into the variable already, then why can't I just store it simply into a table (this isn't billions of bulk data, not even 1000 records!).
Is there no simple workaround? I gave BULK COLLECT and FORALL a try, but I need a lot more time to invest to understand it and get the query right - the cursor in a cursor definately won't make it any easier.
In addition to the suggestion in Mottor's answer, the reason why Toad is flagging up your code is because row-by-row processing is slow. You've got a lot of context switching going on between the PL/SQL and SQL engines.
Think of it like building new wall near your house - if the bricks are delivered to the bottom of the drive, do you:
Go to the pile of bricks
Pick up a single brick
Go back to your wall
Add the brick onto the wall
Go back to step 1 and repeat until the wall is complete
(This is the equivalent of row-by-row processing)
Or:
Take your wheelbarrow down to the pile of bricks
Load your wheelbarrow with as many bricks as will fit and/or you can carry
Take the wheelbarrow back over to the wall
Add each brick into the wall
Go back to step 1 and repeat until the wall is complete.
(This is the equivalent of bulk processing.)
Of course, if you're canny, you could avoid all the walking and carrying required in the above scenarios by getting the bricks delivered right next to the wall in the first place. (This is the equivalent of set-based processing).
Turning your procedure into a set-based approach (incorporating Mottor's answer) would make it simply:
declare
-- lots of vars
begin
insert into global_temp_table (t_dbusernaam,
t_nodenaam,
t_dbnode1,
t_dbnode2,
t_distinctcount)
select dbusernaam,
nodenaam,
dbnode1,
dbnode2,
cnt
from (select nodenaam,
naam,
url,
dbnode1,
dbnode2,
dbusernaam,
dbnaam,
count(*) over (partition by dbnode1, dbnode2, dbusernaam) cnt
from schema.tablename
where nodenaam = 'XXYZ123')
where cnt = 1;
end;
/
This has the advantage of being more compact than your original code, making it easier to read, understand and therefore debug. Plus you can run the select statement on its own outside of the procedure - much easier to see what it's doing that way.
It will also be faster than your original approach of looping through two cursors (which, by the way, was reinventing the nested loop join - something that the database is optimised to do in pure SQL... and may not be the fastest way of doing the join anyway, if you had been stuck with keeping the join!).
I'd also be interested to know why you need to insert the rows into the GLOBAL_TEMP_TABLE (which I suspect is a GTT - global temporary table - rather than a normal heap table) - can you not do the subsequent processing in a single SQL statement, using the above select statement rather than inserting the data into the GTT?
This is not an error, but the TOAD suggestion with number Rule 4809.
P.S. If the table is the same in the both query, you can use
..., COUNT(*) OVER (PARTITION BY DBNODE1, DBNODE2, DBUSERNAAM) c
in the first query to get the number of rows per DBNODE1, DBNODE2, DBUSERNAAM and not to need the second one.
I have a PL/SQL package where i want to declare a select-statment which are used by different other Packages. So i see to ways. First way i define a cursor which can be called from other packages and store the select. Second way would be a procedure which stored the select.
Can someone tell me the advantages and disadvantages of each way? My Prof. say Cursor are old and statefull and noone use this today. My Chef tell me Cursor is faster to iterate and you can make Types of it.
Can someone tell me what's the best practice here?
For example:
CURSOR crs_active_customer IS
SELECT * FROM customer where status = 'active'
OR
PROCEDURE prc_getActiveCustomer IS
BEGIN
SELECT * FROM customer where status = 'active';
END prc_getActiveCustomer;
What is better way to store select-statements.
I would write a function that returns a new cursor instance every time you call it. A cursor variable in a package is actually a GLOBAL variable: you can have only one procedure at a time using it. This is probably the problem your professor is referring to.
Having a global cursor means that you will run into "cursor already open" errors if you write a a procedure that, while scanning the results of such cursor calls another function that internally needs to use the same cursor.
PL/SQL 101 to the rescue ! From Working with Cursors:
The central purpose of the Oracle PL/SQL language is to make it as easy and efficient as possible to query and change the contents of tables in a database. You must, of course, use the SQL language to access tables, and each time you do so, you use a cursor to get the job done.
So every time you have SQL in PL/SQL there will be a cursor. The next question is what kinds of cursors there is and when to use them. The above mentioned article touches also this topic.
You can also read the fine manual: Cursors
A cursor is a pointer to a private SQL area that stores information about processing a specific SELECT or DML statement.
And then carry on reading about implicit and explicit cursors.
Next find a better professor.
I want to write pl/sql code which utilizes a Cursor and Bulk Collect to retrieve my data. My database has rows in the order of millions, and sometimes I have to query it to fetch nearly all records on client's request. I do the querying and subsequent processing in batches, so as to not congest the server and show incremental progress to the client. I have seen that digging down for later batches takes considerably more time, which is why I am trying to do it by way of cursor.
Here is what should be simple pl/sql around my main sql query:
declare
cursor device_row_cur
is
select /my_query_details/;
type l_device_rows is table of device_row_cur%rowtype;
out_entries l_device_rows := l_device_rows();
begin
open device_row_cur;
fetch device_row_cur
bulk collect into out_entries
limit 100;
close device_row_cur;
end;
I am doing batches of 100, and fetching them into out_entries. The problem is that this block compiles and executes just fine, but doesn't return the data rows it fetched. I would like it to return those rows just the way a select would. How can this be achieved? Any ideas?
An anonymous block can't return anything. You can assign values to a bind variable, including a collection type or ref cursor, inside the block. But the collection would have to be defined, as well as declared, outside the block. That is, it would have to be a type you can use in plain SQL, not something defined in PL/SQL. At the moment you're using a PL/SQL type that is defined within the block, and a variable that is declared within the block too - so it's out of scope to the client, and wouldn't be a valid type outside it either. (It also doesn't need to be initialised, but that's a minor issue).
Dpending on how it will really be consumed, one option is to use a ref cursor, and you can declare and display that through SQL*Plus or SQL Developer with the variable and print commands. For example:
variable rc sys_refcursor
begin
open :rc for ( select ... /* your cursor statement */ );
end;
/
print rc
You can do something similar from a client application, e.g. have a function returning a ref cursor or a procedure with an out parameter that is a ref cursor, and bind that from the application. Then iterate over the ref cursor as a result set. But the details depend on the language your application is using.
Another option is to have a pipelined function that returns a table type - again defined at SQL level (with create type) not in PL/SQL - which might consume fewer resources than a collection that's returned in one go.
But I'd have to question why you're doing this. You said "digging down for later batches takes considerably more time", which sounds like you're using a paging mechanism in your query, generating a row number and then picking out a range of 100 within that. If your client/application wants to get all the rows then it would be simpler to have a single query execution but fetch the result set in batches.
Unfortunately without any information about the application this is just speculation...
I studied this excellent paper on optimizing pagination:
http://www.inf.unideb.hu/~gabora/pagination/article/Gabor_Andras_pagination_article.pdf
I used technique 6 mainly. It describes how to limit query to fetch page x and onward. For added improvement, you can limit it further to fetch page x alone. If used right, it can bring a performance improvement by a factor of 1000.
Instead of returning custom table rows (which is very hard, if not impossible to interface with Java), I eneded up opening a sys_refcursor in my pl/sql which can be interfaced such as:
OracleCallableStatement stmt = (OracleCallableStatement) connection.prepareCall(sql);
stmt.registerOutParameter(someIndex, OracleTypes.CURSOR);
stmt.execute();
resultSet = stmt.getCursor(idx);
It may sound like a silly question, but I hope I'll make myself clear enough.
When talking about Spaghetti Code, the basis of it is the
usage of GOTOs. I had a peer that was used to say if I put a breakpoint at the end of the code and this breakpoint isn't reached everytime, something is wrong.
Nevertheless, is a common (and I'd say, a rule) to use EXIT WHEN
structures within Oracle packages (usually followed by a %NOTFOUND
test).
Taking for granted that using EXIT breaks the programming flow, isn't something that doesn't match between 1 and 2?
Is everyone programming in PL/SQL following a bad practice? Does PL/SQL don't follow this specific pattern for conditionals?
Is there any performance reason under Oracle's hood to use such statements?
Apologies if this question has been already asked, I couldn't find anything similar around.
Yes, many people are following a bad practice.
Bad Style
I agree with #Osy that OPEN/FETCH/CLOSE adds completely unnecessary code. I would go even further, and say that you should almost never use CURSOR.
First of all, you normally want to do as much as possible in plain SQL. If you need to use PL/SQL, use an implicit cursor. It will save you a line of code and will help you keep related logic closer together.
I'm a strong believer in keeping individual units of code as small as possible. At first glance, it seems like a CURSOR can help you do this. You can define your SQL up top in one place, and then do the PL/SQL looping later.
But in reality, that extra layer of indirection is almost never worth it. Sometimes a lot of logic is in SQL, and sometimes a lot of logic is in PL/SQL. But in practice, it rarely makes sense to put a lot of complex logic in both. Your code usually ends up looking like
one of these:
for records in (<simple SQL>) loop
<complex PL/SQL>
end loop;
or:
for records in
(
<complex SQL>
) loop
<simple PL/SQL>;
end loop;
Either way, one of your code sections will be very small. The complexity of separating those two sections of code is greater than the complexity of a larger, single section of code. (But that is obviously my opinion.)
Bad Performance
There are significant performance implications with using OPEN/FETCH/CLOSE. That method is much slower than using a cursor for loop or an implicit cursor.
The compiler can automatically use bulk collect in some for loops. But, to quote from the Oracle presentation "PL/SQL Performance—Debunking the Myths", page 122:
Don’t throw this chance away by using the open, fetch loop, close form
Here's a quick example:
--Sample data
create table t(a number, b number);
insert into t select level, level from dual connect by level <= 100000;
commit;
--OPEN/FETCH/CLOSE
--1.5 seconds
declare
cursor test_cur is
select a, b from t;
test_rec test_cur%rowtype;
counter number;
begin
open test_cur;
loop
fetch test_cur into test_rec;
exit when test_cur%notfound;
counter := counter + 1;
end loop;
close test_cur;
end;
/
--Implicit cursor
--0.2 seconds
declare
counter number;
begin
for test_rec in (select a, b from t) loop
counter := counter + 1;
end loop;
end;
/
It´is very recommendable to keep it simple your code, so I can tell you what PL/SQL guru says about it:
NOTE : In some cases is not recommendable use of CURSOR-FOR-LOOP. You can consider one more intelligent manner of choose single SELECT-INTO or BULK COLLECT according of your needs.
Source : On Cursor FOR Loops, Oracle Magazine By Steven Feuerstein
(Reference: Feuerstein, TOP TWENTY PL/SQL TIPS AND TECHNIQUES):
Loops
12. Take advantage of the cursor FOR loop.
The cursor FOR loop is
one of my favorite PL/SQL constructs. It leverages fully the tight and
effective integration of the procedural aspects of the language with
the power of the SQL database language. It reduces the volume of code
you need to write to fetch data from a cursor. It greatly lessens the
chance of introducing loop errors in your programming - and loops are
one of the more error-prone parts of a program. Does this loop sound
too good to be true? Well, it isn’t - it’s all true!
Suppose I need to update the bills for all pets staying in my pet
hotel, the Share-a-Din-Din Inn. The example below contains an
anonymous block that uses a cursor, occupancy_cur, to select the room
number and pet ID number for all occupants at the Inn. The procedure
update_bill adds any new changes to that pet’s room charges.
DECLARE
CURSOR occupancy_cur IS
SELECT pet_id, room_number
FROM occupancy
WHERE occupied_dt = SYSDATE;
occupancy_rec occupancy_cur%ROWTYPE;
BEGIN
OPEN occupancy_cur;
LOOP
FETCH occupancy_cur
INTO occupancy_rec;
EXIT WHEN occupancy_cur%NOTFOUND;
update_bill
(occupancy_rec.pet_id,
occupancy_rec.room_number);
END LOOP;
CLOSE occupancy_cur;
END;
This code leaves nothing to the imagination. In addition to defining
the cursor (line 2), you must explicitly declare the record for the
cursor (line 5), open the cursor (line 7), start up an infinite loop,
fetch a row from the cursor set into the record (line 9), check for an
end-ofdata condition with the cursor attribute (line 10), and finally
perform the update. When you are all done, you have to remember to
close the cursor (line 14). If I convert this PL/SQL block to use a
cursor FOR loop, then I all I have is:
DECLARE
CURSOR occupancy_cur IS
SELECT pet_id, room_number
FROM occupancy WHERE occupied_dt =
SYSDATE;
BEGIN
FOR occupancy_rec IN occupancy_cur
LOOP
update_bill (occupancy_rec.pet_id,
occupancy_rec.room_number);
END LOOP;
END;
Here you see the beautiful simplicity of the cursor FOR loop! Gone is
the declaration of the record. Gone are the OPEN, FETCH, and CLOSE
statements. Gone is need to check the %FOUND attribute. Gone are the
worries of getting everything right. Instead, you say to PL/SQL, in
effect:: ÒYou and I both know that I want each row and I want to dump
that row into a record that matches the cursor. Take care of that for
me, will you?" And PL/SQL does take care of it, just the way any
modern programming language integrated with SQL should.
I have a statement such as below but its padded out to do 1000 calls at a time. Anything over that throws a PLS-123 error Program Too Large Diana Nodes
begin
sp_myprocedure(....)
sp_myprocedure(....)
sp_myprocedure(....)
sp_myprocedure(....)
end
We are moving to 11g and I was wondering if this limitation could be increased to 2000 for example.
Thanks
"I have a statement such as below but its padded out to do 1000 calls
at a time"
This is a very bad programming strategy. Writing the same thing multiple times is a code smell. Anytime we find ourselves programming with cut'n'paste and then a bit of editing is a time when we should stop and ask ourselves, 'hmmm is there a better way to do this?'
"The parameters are different for each stored procedure call"
Yes, but the parameters have to come from somewhere. Presumably at the moment you are hard-coding them one thousand times. Yuck.
A better solution would be to store them in a table. Then you could write a simple loop. Like this:
for prec in ( select p1, p2 from my_parameters
order by id -- if ordering is important
)
loop
sp_myprocedure(prec.p1, prec.p2);
end loop;
Because you are storing the parameters in a table you can have as many calls to that proc as you like, and you are not bound by the Diana node limit.
True you will have to move your parameter values to a table, but it is not harder to maintain data in a table than it is to maintain hardcoded values in source code.
If you're just moving from 10g then I don't believe the limit has changed. So, if you're having problems now then you'll have them again in 11g. Take a look at this Ask Tom article. A general suggestion is to put your procedure in a package. Or, break it down into smaller blocks. If you're only getting the error when running the block which calls the procedure 1000 times and in the procedure on its own then I suggest you try as APC says and loop through it instead as this should reduce the number of nodes.