Platform: Oracle
Language: PL/SQL
Issue: Want to output a procedure OUT cursor into the SQLDeveloper SQLWosksheet.
Anyone know how to use the Oracle "Select * from Table( PipelinedFunction( Param ) ) " to check procedure code output cursors?
I am using Crsytal Reports off of an Oracle stored procedure. Crystal requires that a procedure return a cursor, which it fetchs and reads.
The procedure code I have is currently working, but I want to find the easiest way to view the effects of changes to the procedure code. I have SQLDeveloper available, and I'm doing my creation and sql testing in that. I would like to get a quick result visible in the SQL Developer Query Result window ("SQL Worksheet").
Is there a (simple) way to use a Function to read the cursor from the procedure? (and pipe that out to the Table function?)
Convoluted, I know, but I deal best when I can just see the results of code changes. If I can view the record results directly, it will speed up development of the report.
I know of the Table function and a little about pipelining in Oracle. I know a little about cursors in general and sys_refcursor. I know diddly about types and why I need them. (Isn't sys_regCursor supposed to get us away from that?)
The current procedure does an adequate but ungraceful series of queries, inserts to global temp tables (GTT), joins from GTT and original tables, more inserts, and more self-joins and then SELECTS the results into the OUT cursor. I might be able to do better relying on just cursors and such, but the current method is good enough to get results to the report.
I think I can handle SQL pretty well (for our purposes), but I am not an Oracle-specific developer... but I need help.
Anybody run across this? The whole idea was to speed my development for the procedure code, but I've spent a couple of days looking for a way to just get at the output... not what I had in mind.
Update:
I have tried some hare-brained schemes based on slivers that I've seen on the web... such as
Create or replace FUNCTION GET_BACKPLANE (
Node VARCHAR2 ) RETURN SYS_REFCURSOR
AS
RESULTS SYS_REFCURSOR;
BEGIN
Open Results for
Select Backplane(Results, Node) from Dual ;
... etc.
and
Create or replace Function GET_BACKPLANE (
NODE VARCHAR2 ) RETURN My_Table_Stru%ROWTYPE PIPELINED
AS
BEGIN ...
I don't think that Oracle is even considering letting me re-reference the output cursor from the procedure ("Results" is a sys_refcursor that holds the results of the last SELECT in the procedure). I don't know how to define it, open it, and reference it from the procedure.
I never got to the place where I could try
SELECT * FROM TABLE(GET_BACKPLANE( ... etc )
Sorry for any typos and bad Oracle Grammar... it's been a long several days.
SQL Developer allows us to use SQL*Plus commands in the Worksheet. So all you need to do is define a variable to hold the output of the ref cursor.
I may have misinterpreted the actual code you want to run but I'm assuming your actual program is a procedure Backplane(Results, Node) where results is an OUT parameter of datatype sys_refcursor and node is some input parameter.
var rc refcursor
exec Backplane(results=>:rc, Node=>42)
print rc
The output of the print statement is written to the Script Output pane.
Note that the use of SQL*Plus commands means we have to use the Run Script option F5 rather than execute statement.
Thanks for the help. In the end, I wound up brute-force-ing it...
Step by step:
Make a query, test a query,
create a global temp table from the structure,
add code to make another query off of that GTT, test the query,
create a global temp table from the structure,
etc.
In the end, I wound up running (anonymous block) scripts and checking the GTT contents at every stage.
The last part was to use the same last query from the original procedure, stuffing everything into the Cursor that crystal likes...
tomorrow, I test that.
But, I'll just force it through for the next procedure, and get it done in a day and a half instead of 2+ weeks (embarrassed).
Thanks,
Marc
Related
I've seen lots of posts regarding the use of cursors in PL/SQL to return data to a calling application, but none of them touch on the issue I believe I'm having with this technique. I am fairly new to Oracle, but have extensive experience with MSSQL Server. In SQL Server, when building queries to be called by an application for returning data, I usually put the SELECT statement inside a stored proc with/without parameters, and let the stored proc execute the statement(s) and return the data automatically. I've learned that with PL/SQL, you must store the resulting dataset in a cursor and then consume the cursor.
We have a query that doesn't necessarily return huge amounts of rows (~5K - 10K rows), however the dataset is very wide as it's composed of 1400+ columns. Running the SQL query itself in SQL Developer returns results instantaneously. However, calling a procedure that opens a cursor for the same query takes 5+ minutes to finish.
CREATE OR REPLACE PROCEDURE PROCNAME(RESULTS OUT SYS_REFCURSOR)
AS
BEGIN
OPEN RESULTS FOR
<SELECT_query_with_1400+_columns>
...
END;
After doing some debugging to try to get to the root cause of the slowness, I'm leaning towards the cursor returning one row at a time very slowly. I can actually see this real-time by converting the proc code into a PL/SQL block and using DBMS_SQL.return_result(RESULTS) after the SELECT query. When running this, I can see each row show up in the Script output window in SQL Developer one at a time. If this is exactly how the cursor returns the data to the calling application, then I can definitely see how this is the bottleneck as it could take 5-10 minutes to finish returning all 5K-10K rows. If I remove columns from the SELECT query, the cursor displays all the rows much faster, so it does seem like the large amount of columns is an issue using a cursor.
Knowing that running the SQL query by itself returns instant results, how could I get this same performance out of a cursor? It doesn't seem like it's possible. Is the answer putting the embedded SQL in the application code and not using a procedure/cursor to return data in this scenario? We are using Oracle 12c in our environment.
Edit: Just want to address how I am testing performance using the regular SELECT query vs the PL/SQL block with cursor method:
SELECT (takes ~27 seconds to return ~6K rows):
SELECT <1400+_columns>
FROM <table_name>;
PL/SQL with cursor (takes ~5-10 minutes to return ~6K rows):
DECLARE RESULTS SYS_REFCURSOR;
BEGIN
OPEN RESULTS FOR
SELECT <1400+_columns>
FROM <table_name>;
DBMS_SQL.return_result(RESULTS);
END;
Some of the comments are referencing what happens in the console application once all the data is returned, but I am only speaking regarding the performance of the two methods described above within Oracle\SQL Developer. Hope this helps clarify the point I'm trying to convey.
You can run a SQL Monitor report for the two executions of the SQL; that will show you exactly where the time is being spent. I would also consider running the two approaches in separate snapshot intervals and checking into the output from an AWR Differences report and ADDM Compare Report; you'd probably be surprised at the amazing detail these comparison reports provide.
Also, even though > 255 columns in a table is a "no-no" according to Oracle as it will fragment your record across > 1 database blocks, thus increasing the IO time needed to retrieve the results, I suspect the differences in the two approaches that you are seeing is not an IO problem since in straight SQL you report fast result fetching all. Therefore, I suspect more of a memory problem. As you probably know, PL/SQL code will use the Program Global Area (PGA), so I would check the parameter pga_aggregate_target and bump it up to say 5 GB (just guessing). An ADDM report run for the interval when the code ran will tell you if the advisor recommends a change to that parameter.
I have a PL/SQL package where i want to declare a select-statment which are used by different other Packages. So i see to ways. First way i define a cursor which can be called from other packages and store the select. Second way would be a procedure which stored the select.
Can someone tell me the advantages and disadvantages of each way? My Prof. say Cursor are old and statefull and noone use this today. My Chef tell me Cursor is faster to iterate and you can make Types of it.
Can someone tell me what's the best practice here?
For example:
CURSOR crs_active_customer IS
SELECT * FROM customer where status = 'active'
OR
PROCEDURE prc_getActiveCustomer IS
BEGIN
SELECT * FROM customer where status = 'active';
END prc_getActiveCustomer;
What is better way to store select-statements.
I would write a function that returns a new cursor instance every time you call it. A cursor variable in a package is actually a GLOBAL variable: you can have only one procedure at a time using it. This is probably the problem your professor is referring to.
Having a global cursor means that you will run into "cursor already open" errors if you write a a procedure that, while scanning the results of such cursor calls another function that internally needs to use the same cursor.
PL/SQL 101 to the rescue ! From Working with Cursors:
The central purpose of the Oracle PL/SQL language is to make it as easy and efficient as possible to query and change the contents of tables in a database. You must, of course, use the SQL language to access tables, and each time you do so, you use a cursor to get the job done.
So every time you have SQL in PL/SQL there will be a cursor. The next question is what kinds of cursors there is and when to use them. The above mentioned article touches also this topic.
You can also read the fine manual: Cursors
A cursor is a pointer to a private SQL area that stores information about processing a specific SELECT or DML statement.
And then carry on reading about implicit and explicit cursors.
Next find a better professor.
I am writing a function similar to Tom Kyte's print_table, but with more output options and better formatting, etc., etc. It struck me that writing a function where you could enter arbitrary SQL with authid currentuser -- rather than passing a ref cursor -- was pretty dangerous! However, I need to know the column attributes and, AFAIK, there's no other way of getting this.
Is there a way, therefore, of restricting the dbms_sql.execute function so it only operates on select queries? Or, put another way, is there a way to check the type of DML being parsed through the cursor and then, for example, raise an exception if it's anything other than a select?
Textual analysis of the query (e.g., allowing select or with, but nothing else) wouldn't work, because you could just do print_table('select * from dual; drop someTable'); etc...
(P.S., Oracle 10gR2, if it makes a difference.)
I have a stored procedure that I have defined in Oracle. In that procedure I need to return a recordset. To do this, I am using the SYS_REFCURSOR, which works great inside of Oracle (and with cx_Oracle, for that matter). In my application I am using SqlAlchemy scoped sessions, to support multi-threading.
How can I use the scoped session to return the REF CURSOR? The only way I have been able to get this to work is by declaring an out cursor with the cursor that is active in the session and then executing the stored procedure, like below:
sql = """
BEGIN
example('%s', '%s', '%s', :cur);
END;
""" % (cid, show, type)
conn = sa_tool.session.connection()
in_cur = conn._Connection__connection.cursor()
out_cur = conn._Connection__connection.cursor()
in_cur.execute(sql, cur=out_cur)
results = out_cur.fetchall()
Ideally, I would like to avoid using the connection object in this way, and execute the procedure while letting SqlAlchemy manage the cursors. If that is not possible, is there a reason that the fetch would take so long?
Thanks
I saw this question which was not answered and I decided to jump in. First of all, let me just say that for your scenario there is no better option than SYS_REFCURSOR. Of course, you have alternatives.
An Oracle Cursor is a memory area location where an instruction to execute a SQL statement is stored. A Ref cursor is just a pointer to the cursor location. SYS_REFCURSOR is an specific oracle defined typed of ref cursor. So when you return a SYS_REFCURSOR variable to a client, you are returning a pointer towards the memory location where the instruction to execute the SQL resides. Your client can now execute the instruction using the FETCH operation and get the rows. So this is the best possible way to return a result set to the client.
As an alternative, you might use a PIPELINED FUNCTION, but I can assure you that you won't get any better performance. AskTom has a great explanation about this comparison in this article
https://asktom.oracle.com/pls/apex/f?p=100:11:0::::P11_QUESTION_ID:9532870400346178516
Another scenario is whether you want to analyse where the time is consumed, either in the EXECUTE phase or in the FETCH one. If you have a huge time in FETCH, perhaps you might consider transfer the data in another way to the client. If you have a problem in EXECUTION, then you need to make a tuning exercise over your procedure.
There have been a couple of hints that seem to have gotten me close here, but with some unique issues, I'm hoping this question is distinguishing enough to merit its own posting.
For starters here's what I have. I have an Oracle procedure that returns a standard REF CURSOR, and this REF CURSOR is passed back to my application. The REF CURSOR is a list of lookup IDs.
I then want to take this list and bring it to another data store and use it in a select statement. It will absolutely be possible to accomplish this by looping through the REF CURSOR, but I'm hoping to avoid that. I would much rather be able to write a SELECT...WHERE lookup_id IN result_ref_cursor OR SELECT...WHERE EXISTS...
First is this possible or should I just try a less than elegant solution? If it is possible, any hints as to where I should get started looking?
I'm relatively new to Oracle, but fairly experienced in RDBMs in general, so feel free to just through some links at me and I can study up. Much appreciated
Why kurosch didn't put his response as an "answer" I'll have no idea.
So, what you do is define a SQL type which describes one row of the output of the ref cursor, and also a SQL type which is a table of the previous. Then, you'll create a pipelined function which returns the rows returned by the ref cursor. This function can then be used in a standard SQL. I'm borrowing from Ask Tom on this one.
create or replace type myLookupId as object ( id int)
/
create or replace type myLookupIdTable as table of myLookupId
/
create or replace function f return myLookupIdTable PIPELINED is
l_data myLookupId;
l_id number;
p_cursor SYS_REFCURSOR;
begin
p_cursor := function_returning_ref_cursor();
loop
fetch p_cursor into l_id;
exit when p_cursor%notfound;
l_data := myLookupId( l_id );
pipe row (l_data);
end loop;
return;
end;
/
And now a sample query...
SELECT *
FROM SOME_TABLE
WHERE lookup_id in (SELECT ID FROM table(f));
Sorry if the code isn't exactly right, I don't have the DB to test right now.
There are several directions you could go with this, but I did a search on the specific solution you want and it seems like no-one has done it often enough to show up there. What you can do is search the oracle metalink - that is usually really good at finding obscure answers. (Though you do need a service agreement - just found out that mine expired :( )
Other possible solutions:
Create a link between the data stores so that you can do the select in the plsql directly
Create a function in Java that loops through it for you to create the string for the query. This will look a little more pretty at least.
Otherwise, REF CURSOR's need to go back and forth - I don't know how you can pipe the results of the REF CURSOR in one connection to the query in another without looping through it.