SQL Server is able to return the results of multiple queries in a single round-trip, e.g:
select a, b, c from y;
select d, e, f from z;
Oracle doesn't like this syntax. It is possible to use reference cursors, like this:
begin
open :1 for select count(*) from a;
open :2 for select count(*) from b;
end;
However, you incur a penalty in opening/closing cursors and you can hold database locks for an extended period. What I'd like to do is retrieve the results for these two queries in one shot, using Odp.net. Is it possible?
In Oracle, reference cursor is a pointer to data, rather than data itself.
So if a procedure returns two reference cursors, the the client still has to go and fetch the rows from those cursors (and incur the network hits).
As such, if the data volumes are small, you probably want to call a procedure that just returns the values.
If the data volumes are large (thousands of rows) then it won't be a single network trip anyway, so an extra one or two as you switch between cursors isn't going to make much difference.
Another choice is have a single select return all the rows. That might be a simple UNION ALL
select a, b, c from y union all select d, e, f from z;
It could be a pipelined table function
create or replace package test_pkg is
type rec_two_cols is record
(col_a varchar2(100),
col_b varchar2(100));
type tab_two_cols is table of rec_two_cols;
function ret_two_cols return tab_two_cols pipelined;
end;
/
create or replace package body test_pkg is
function ret_two_cols return tab_two_cols pipelined
is
cursor c_1 is select 'type 1' col_a, object_name col_b from user_objects;
cursor c_2 is select 'type 2' col_a, object_name col_b from user_objects;
r_two_cols rec_two_cols;
begin
for c_rec in c_1 loop
r_two_cols.col_a := c_rec.col_a;
r_two_cols.col_b := c_rec.col_b;
pipe row (r_two_cols);
end loop;
for c_rec in c_2 loop
r_two_cols.col_a := c_rec.col_a;
r_two_cols.col_b := c_rec.col_b;
pipe row (r_two_cols);
end loop;
return;
end;
end;
/
select * from table(test_pkg.ret_two_cols);
I believe the most recent versions of ODP for 11g allow user-defined types which may help.
Related
I have a cursor which fetches records from a table.
open p_cursor for select a1, a2 from my_table;
Thereafter I use fetch to get the columns and put all of them into a single CLOB column as follows : ( add_to_clob is a procedure which concatenates a text into existing CLOB - my_clob )
fetch p_cursor into l_a1, l_a2;
add_to_clob ( my_clob, l_a1 );
add_to_clob ( my_clob, l_a2 );
Essentially - the output of fetch are being written into a large CLOB.
But the operation is running slower than expected; and we have millions of records to process.
Is there any way such that use of cursor can be avoided to that the process runs faster ?
You can "cheat" a bit and use some XML aggregate functions like this:
DECLARE
l_clob1 CLOB;
l_clob2 CLOB;
BEGIN
SELECT XMLSERIALIZE(CONTENT EXTRACT(XMLAGG(XMLELEMENT(COL1, ao.object_name||', ')), '/COL1/text()') AS CLOB),
XMLSERIALIZE(CONTENT EXTRACT(XMLAGG(XMLELEMENT(COL2, ao.edition_name||', ')), '/COL2/text()') AS CLOB)
INTO l_clob1, l_clob2
FROM all_objects ao
WHERE ROWNUM <= 10;
END;
/
Just replace the simple query with your own. Also, you'll need to
I need to migrate all the records from a tableA to a tableB. At the moment I'am simply using the following statement:
INSERT INTO table1 (id, name, description) SELECT id, name, descriptionOld
FROM table2;
COMMIT;
The problem is that if there is a high number of records the temporary tablespace might not have enough space to handle this statement. For this reason I would like to know if there is any way to still have this statement over a loop that commits, for example, 1000 records at the time.
Thank you!
For huge data processing one must have a look on context switching between SQL and PLSQL engines. An approach can be let the insert from tableA to tableB and handle the error records after the insertion is completed. You create a error tableC same as your destination table to handle the error records. So once the copying of data from tableA is completed you can have a look at the error records and directly do and insert into to tableB after making correction. See below how you can do it.
declare
cursor C is
select *
from table_a;
type array is table of c%rowtype;
l_data array;
dml_errors EXCEPTION;
PRAGMA exception_init(dml_errors, -24381);
l_errors number;
l_idx number;
begin
open c;
loop
--Limit 100 will give optimal number of context switching and best perfomance
fetch c bulk collect into l_data limit 100;
begin
forall i in 1 .. l_data.count SAVE EXCEPTIONS
insert into table_b
values l_data(i);
exception
when DML_ERRORS then
l_errors := sql%bulk_exceptions.count;
for i in 1 .. l_errors
loop
l_idx := sql%bulk_exceptions(i).error_index;
--Insertnig error records to a error table_c so that later on these records can be handled.
insert into table_c
values l_data(l_idx);
end loop;
commit;
end;
exit when c%notfound;
end loop;
close c;
commit;
end;
/
Say you have these tables:
create table sourceTab( a number, b number, c number);
create table targetTab( a number, b number, c number, d number);
and you want to copy records from sourceTab to targetTab filling both the coumns c and d of the tagret table with the value of the column C in the source.
This is a way to copy the records not in a single statement, but in blocks of a given number of rows.
DECLARE
CURSOR sourceCursor IS SELECT a, b, c, c as d FROM sourceTab;
TYPE tSourceTabType IS TABLE OF sourceCursor%ROWTYPE;
vSourceTab tSourceTabType;
vLimit number := 10; /* here you decide how many rows you insert in one shot */
BEGIN
OPEN sourceCursor;
LOOP
FETCH sourceCursor
BULK COLLECT INTO vSourceTab LIMIT vLimit;
forall i in vSourceTab.first .. vSourceTab.last
insert into targetTab values vSourceTab(i);
commit;
EXIT WHEN vSourceTab.COUNT < vLimit;
END LOOP;
CLOSE sourceCursor;
END;
If you follow this approach, you may get an error when some records, but not all, have already been copied (and committed), so you have to consider the best way to handle this case, depending on your needs.
I want to create a procedure in PL/SQL that has 5 steps. Step 1 and 2 execute first and return an ID. In step 3, we have a SELECT statement that has a condition with that returned ID. I want then to take all of the results of that SELECT statement and use them in a JOIN in another SELECT statement and use THOSE results in a 3rd SELECT statement again using JOIN. From what I've seen, I can't use CURSOR in JOIN statements. Some of my co-workers have suggested that I save the results in a CURSOR and then use a loop to iterate through each row and use that data for the next SELECT. However since I'm going to do 2 selects this will create a huge fork of inside loops and that's exactly what I'm trying to avoid.
Another suggestion was to use Temprary Tables to store the data. However this procedure could be executed at the same time by many users and the table's data would conflict with each other. Right now I'm looking at LOCAL Temporary tables that supposedly filter the data according the the session but I'm not really sure I want to create dummy tables for my procedures since I want to avoid leaving trash in the schema (this procedure is for a custom part of the application). Is there a standard way of doing this? Any ideas?
Sample:
DECLARE
USERID INT := 1000000;
TEXT1 VARCHAR(100);
TEXT_INDEX INT;
CURSOR NODES IS SELECT * FROM NODE_TABLE WHERE DESCRIPTION LIKE TEXT || '%';
CURSOR USERS IS SELECT * FROM USERGROUPS JOIN NODES ON NODES.ID = USERGROUPS.ID;
BEGIN
SELECT TEXT INTO TEXT1 FROM TABLE_1 WHERE ID = USERID;
TEXT_INDEX = INSTR(TEXT, '-');
TEXT = SUBSTR(TEXT, 0, TEXT_INDEX);
OPEN NODES;
OPEN USERS;
END;
NOTE: This does NOT work. Oracle doesn't support joins between cursors.
NOTE2: This CAN be done in a single query but for the sake of argument (and in my real use case) I want to break those steps down in a procedure. The sample code is a depiction of what I'm trying to achieve IF joins between cursors worked. But they don't and I'm looking for an alternative.
I ended up using a function (although a procedure could be used as well) along with tables. Things I've learned and one should pay attention to:
PL/SQL functions can only return types that have been declared in the schema in advance and are clear. You can't create a function that returns something like MY_TABLE%ROWTYPE, even though it seems the type information is available it is not acceptable. You have to instead create a custom type of MY_TABLE%ROWTYPE is you want to return it.
Oracle treats tables of declared types differently from tables of %ROWTYPE. This confused the hell out of me at first but from what I've gathered this is how it works.
DECLARE TYPE MY_CUSTOM_TABLE IS TABLE OF MY_TABLE%ROWTYPE;
Declares a collection of types of MY_TABLE row. In order to add to this we must use BULK COLLECT INTO from an SQL statement that queries MY_TABLE. The resulting collection CANNOT be used in JOIN statements is not queryable and CANNOT be returned by a function.
DECLARE
CREATE TYPE MY_CUSTOM_TYPE AS OBJECT (COL_A NUMBER, COL_B NUMBER);
CREATE TYPE MY_CUSTOM_TABLE AS TABLE OF MY_CUSTOM_TYPE;
my_custom_tab MY_CUSTOM_TABLE;
This create my_custom_tab which is a table (not a collection) and if populated can be queried at using TABLE(my_custmo_tab) in the FROM statement. As a table which is declared in advance in the schema this CAN be returned from a function. However it CANNOT be populated using BULK COLLECT INTO since it is not a collection. We must instead use the normal SELECT INTO statement. However, if we want to populate it with data from an existing table that has 2 number columns we cannot simply do SELECT * INTO my_custom_tab FROM DOUBLE_NUMBER_TABLE since my_custom_tab hasn't been initialized and doesn't contain enough rows to receive the data. And if we don't know how many rows a query returns we can't initialize it. The trick into populating the table is to use the CAST command and cast our select result set as a MY_CUSTOM_TABLE and THEN add it.
SELECT CAST(MULTISET(SELECT COL_A, COL_B FROM DOUBLE_NUMBER_TABLE) AS MY_CUSTOM_TABLE) INTO my_custom_tab FROM DUAL
Now we can easily use my_custom_tab in queries etc through the use of the TABLE() function.
SELECT * FROM TABLE(my_custom_tab)
is valid.
You can do such decomposition in many ways, but all of them have a significant performance penalty in comaration with single SQL statement.
Maintainability improvement are also questionable and depends on specific situation.
To review all possibilities please look through documentation.
Below is some possible variants based on simple logic:
calculate Oracle user name prefix based on given Id;
get all users whose name starts with this prefix;
find all tables owned by users from step 2;
count a total number of found tables.
1. pipelined
Prepare types to be used by functions:
create or replace type TUserRow as object (
username varchar2(30),
user_id number,
created date
)
/
create or replace type TTableRow as object (
owner varchar2(30),
table_name varchar2(30),
status varchar2(8),
logging varchar2(3)
-- some other useful fields here
)
/
create or replace type TUserList as table of TUserRow
/
create or replace type TTableList as table of TTableRow
/
Simple function to find prefix by user id:
create or replace function GetUserPrefix(piUserId in number) return varchar2
is
vUserPrefix varchar2(30);
begin
select substr(username,1,3) into vUserPrefix
from all_users
where user_id = piUserId;
return vUserPrefix;
end;
/
Function searching for users:
create or replace function GetUsersPipe(
piNameStart in varchar2
)
return TUserList pipelined
as
vUserList TUserList;
begin
for cUsers in (
select *
from
all_users
where
username like piNameStart||'%'
)
loop
pipe row( TUserRow(cUsers.username, cUsers.user_id, cUsers.created) ) ;
end loop;
return;
end;
Function searching for tables:
create or replace function GetUserTablesPipe(
piUserNameStart in varchar2
)
return TTableList pipelined
as
vTableList TTableList;
begin
for cTables in (
select *
from
all_tables tab_list,
table(GetUsersPipe(piUserNameStart)) user_list
where
tab_list.owner = user_list.username
)
loop
pipe row ( TTableRow(cTables.owner, cTables.table_name, cTables.status, cTables.logging) );
end loop;
return;
end;
Usage in code:
declare
vUserId number := 5;
vTableCount number;
begin
select count(1) into vTableCount
from table(GetUserTablesPipe(GetUserPrefix(vUserId)));
dbms_output.put_line('Users with name started with "'||GetUserPrefix(vUserId)||'" owns '||vTableCount||' tables');
end;
2. Simple table functions
This solution use same types as a variant with pipelined functions above.
Function searching for users:
create or replace function GetUsers(piNameStart in varchar2) return TUserList
as
vUserList TUserList;
begin
select TUserRow(username, user_id, created)
bulk collect into vUserList
from
all_users
where
username like piNameStart||'%'
;
return vUserList;
end;
/
Function searching for tables:
create or replace function GetUserTables(piUserNameStart in varchar2) return TTableList
as
vTableList TTableList;
begin
select TTableRow(owner, table_name, status, logging)
bulk collect into vTableList
from
all_tables tab_list,
table(GetUsers(piUserNameStart)) user_list
where
tab_list.owner = user_list.username
;
return vTableList;
end;
/
Usage in code:
declare
vUserId number := 5;
vTableCount number;
begin
select count(1) into vTableCount
from table(GetUserTables(GetUserPrefix(vUserId)));
dbms_output.put_line('Users with name started with "'||GetUserPrefix(vUserId)||'" owns '||vTableCount||' tables');
end;
3. cursor - xml - cursor
It's is a specific case, which may be implemented without user-defined types but have a big performance penalty, involves unneeded type conversion and have a low maintainability.
Function searching for users:
create or replace function GetUsersRef(
piNameStart in varchar2
)
return sys_refcursor
as
cUserList sys_refcursor;
begin
open cUserList for
select * from all_users
where username like piNameStart||'%'
;
return cUserList;
end;
Function searching for tables:
create or replace function GetUserTablesRef(
piUserNameStart in varchar2
)
return sys_refcursor
as
cTableList sys_refcursor;
begin
open cTableList for
select
tab_list.*
from
(
XMLTable('/ROWSET/ROW'
passing xmltype(GetUsersRef(piUserNameStart))
columns
username varchar2(30) path '/ROW/USERNAME'
)
) user_list,
all_tables tab_list
where
tab_list.owner = user_list.username
;
return cTableList;
end;
Usage in code:
declare
vUserId number := 5;
vTableCount number;
begin
select count(1) into vTableCount
from
XMLTable('/ROWSET/ROW'
passing xmltype(GetUserTablesRef(GetUserPrefix(vUserId)))
columns
table_name varchar2(30) path '/ROW/TABLE_NAME'
)
;
dbms_output.put_line('Users with name started with "'||GetUserPrefix(vUserId)||'" owns '||vTableCount||' tables');
end;
Of course, all variants may be mixed, but SQL looks better at least for simple cases:
declare
vUserId number := 5;
vUserPrefix varchar2(100);
vTableCount number;
begin
-- Construct prefix from Id
select max(substr(user_list.username,1,3))
into vUserPrefix
from
all_users user_list
where
user_list.user_id = vUserId
;
-- Count number of tables owned by users with name started with vUserPrefix string
select
count(1) into vTableCount
from
all_users user_list,
all_tables table_list
where
user_list.username like vUserPrefix||'%'
and
table_list.owner = user_list.username
;
dbms_output.put_line('Users with name started with "'||vUserPrefix||'" owns '||vTableCount||' tables');
end;
P.S. All code only for demonstration purposes: no optimizations and so on.
Let's create two test procedures:
CREATE OR REPLACE PROCEDURE Aaaa_Test1(
pDog SYS_REFCURSOR
) IS
TYPE tDogRec is record (objid varchar2(7), lim number, debt number);
TYPE tDog IS TABLE OF tDogRec;
vDog tDog;
BEGIN
IF pDog%ISOPEN THEN
FETCH pDog BULK COLLECT INTO vDog;
IF vDog.count >= 1 THEN
FOR i IN vDog.First..vDog.Last LOOP
Dbms_Output.Put_Line('Aaaa_Test1 = '||vDog(i).Objid);
END LOOP;
END IF;
END IF;
END; -- Aaaa_Test1 Procedure
/
CREATE OR REPLACE PROCEDURE Aaaa_Test2(
pDog SYS_REFCURSOR
) IS
TYPE tDogRec is record (objid varchar2(7), lim number, debt number);
TYPE tDog IS TABLE OF tDogRec;
vDog tDog;
BEGIN
IF pDog%ISOPEN THEN
FETCH pDog BULK COLLECT INTO vDog;
IF vDog.count >= 1 THEN
FOR i IN vDog.First..vDog.Last LOOP
Dbms_Output.Put_Line('Aaaa_Test2 = '||vDog(i).Objid);
END LOOP;
END IF;
END IF;
END; -- Aaaa_Test2 Procedure
Then let's try to open cursor and pass it to these procedures in order:
DECLARE
Vcdogcur SYS_REFCURSOR;
BEGIN
OPEN Vcdogcur FOR
select '6518535' objid, 10000 lim,0 debt
from dual
union all
select '6518536', 0,500
from dual
union all
select '5656058', 0,899
from dual
union all
select '2180965', 5000,0
from dual
union all
select '2462902', 0,100
from dual;
Aaaa_Test1(Vcdogcur);
Aaaa_Test2(Vcdogcur);
CLOSE Vcdogcur;
END;
As you can see, I can't use already fetched cursor in second procedure, because ORACLE cursors are forward-and-read-only. What ways can help to solve this task?
I can't simply bring these procedures into one. Need to keep their logic separate from each other.
You need to open the cursor twice. Using a string variable to hold the query will prevent you from writing the query twice.
DECLARE
Vcdogcur SYS_REFCURSOR;
dyn_query varchar2(500);
BEGIN
dyn_query := 'select ''6518535'' objid, 10000 lim,0 debt
from dual
union all
select ''6518536'', 0,500
from dual
union all
select ''5656058'', 0,899
from dual
union all
select ''2180965'', 5000,0
from dual
union all
select ''2462902'', 0,100
from dual' ;
open Vcdogcur for dyn_query ;
Aaaa_Test1(Vcdogcur);
CLOSE Vcdogcur;
open Vcdogcur for dyn_query ;
Aaaa_Test2(Vcdogcur);
close Vcdogcur;
END;
/
Cursors are NOT designed to be re-used: you read them once, keep moving forward and as you're doing so you're discarding any previously scanned rows. Think of a Java stream... This is a feature, not a bug - cursors are intended to be very much memory/disk efficient.
So the options are:
1) As Nicolas mentioned, close and re-open the same cursor. You'll pay the performance penalty of running the same query twice
2) Store the query results in a temp table (good for very large sets as it would use disk)
3) Store the query results in a collection (nested table - good for small-medium sized tables)
4) If you really can't do any of the easy solutions above you can try to mess around with your code so that you have one "dispatch" procedure that reads the cursor and then passes each row to your 2 "worker" procedures. You'd have to modify your stored procs to be able to process row at a time
My situation:
I have a table named Table1. It has lots of columns, one of them is Column1. I don't know the other columns, they may even change sometimes.
There is a strongly typed ref cursor type which returns Table1%rowtype, named cur_Table1.
I have a stored procedure named SP1 which has an out parameter of type cur_Table1. I'm calling this SP1 stored procedure from another database that only sees this stored procedure, but not the table or the type itself.
How do I select only Column1 from the returned cursor?
I know I can fetch into a record or as many variables as the cursor has columns, but I only know of one column's existence so I can't declare the complete record or correct number of variables.
You can do this with DBMS_SQL, but it ain't pretty.
Table and sample data (COLUMN1 has the numbers 1 - 10):
create table table1(column1 number, column2 date, column3 varchar2(1000), column4 clob);
insert into table1
select level, sysdate, level, level from dual connect by level <= 10;
commit;
Package with a procedure that opens a ref cursor and selects everything:
create or replace package test_pkg is
type cur_Table1 is ref cursor return table1%rowtype;
procedure sp1(p_cursor in out cur_table1);
end;
/
create or replace package body test_pkg is
procedure sp1(p_cursor in out cur_table1) is
begin
open p_cursor for select column1, column2, column3, column4 from table1;
end;
end;
/
PL/SQL block that reads COLUMN1 data from the ref cursor:
--Basic steps are: call procedure, convert cursor, describe and find columns,
--then fetch rows and retrieve column values.
--
--Each possible data type for COLUMN1 needs to be added here.
--Currently only NUMBER is supported.
declare
v_cursor sys_refcursor;
v_cursor_number number;
v_columns number;
v_desc_tab dbms_sql.desc_tab;
v_position number;
v_typecode number;
v_number_value number;
begin
--Call procedure to open cursor
test_pkg.sp1(v_cursor);
--Convert cursor to DBMS_SQL cursor
v_cursor_number := dbms_sql.to_cursor_number(rc => v_cursor);
--Get information on the columns
dbms_sql.describe_columns(v_cursor_number, v_columns, v_desc_tab);
--Loop through all the columns, find COLUMN1 position and type
for i in 1 .. v_desc_tab.count loop
if v_desc_tab(i).col_name = 'COLUMN1' then
v_position := i;
v_typecode := v_desc_tab(i).col_type;
--Pick COLUMN1 to be selected.
if v_typecode = dbms_types.typecode_number then
dbms_sql.define_column(v_cursor_number, i, v_number_value);
--...repeat for every possible type.
end if;
end if;
end loop;
--Fetch all the rows, then get the relevant column value and print it
while dbms_sql.fetch_rows(v_cursor_number) > 0 loop
if v_typecode = dbms_types.typecode_number then
dbms_sql.column_value(v_cursor_number, v_position, v_number_value);
dbms_output.put_line('Value: '||v_number_value);
--...repeat for every possible type
end if;
end loop;
end;
/
Given the original question, jonearles's answer is still correct, so I'll leave it marked as such, but I ended up doing something completely different and much better.
The problem was/is that I have no control over SP1's database, I just have to call it from somewhere else as a 3rd party client. Now I managed to get permission to see not only SP, but also the type of the cursor. I still don't see the table but now there is a much cleaner solution:
In the other database I have been granted access to see this type now:
type cur_Table1 is ref cursor return Table1%rowtype;
So in my database I can do this now:
mycursor OtherDB.cur_Table1;
myrecord mycursor%rowtype;
...
OtherDB.SP1(mycursor);
fetch mycursor into myrecord;
dbms_output.put_line(myrecord.Column1);
See, I still don't need any access to the table, I see the cursor only. The key is that the magical %rowtype works for cursors as well, not just tables. It doesn't work on a sys_refcursor, but it does on a strongly typed one. Given this code, I don't have to care if anything changes on the other side, I don't have to define all the columns or records at all, I just specify the one column I'm interested in.
I really love this OOP attitude about Oracle.
Don't know if it's an option or not, but wouldn't a better solution be to create a function that returns the specific value you're looking for? That avoids the overhead of sending the extra data. Alternatively, you could define a cursor with a set of known fields in it that both parties know about.