I have this simple oracle plsql procedure:
declare
cursor A is
select column_A
from A_TAB; -- no order by
begin
for rec_ in A loop
procedure_A(rec_.column_A);
end loop;
end;
And this is running now for ages.
When I look into sys.v_$sql_bind_capture, value_string column, I can see the current value of bound column_A, and thankfully, that value keeps changing every few minutes.
As the cursor was not sorted by anything, is there a way to see how many more records to go (until this is finished)?
In other words I would need to see the currently fetched values of the query from that cursor. Where to look for it?
This is Oracle 12 database.
You can use dbms_application_info.set_session_longops to do this. The results are visible in V$SESSION_LONGOPS.
In your example, that could do something like:
DECLARE
rindex BINARY_INTEGER;
slno BINARY_INTEGER;
totalwork number;
sofar number;
obj BINARY_INTEGER;
cursor A is
select column_A,
COUNT(*) OVER () cnt
from A_TAB; -- no order by
begin
rindex := dbms_application_info.set_session_longops_nohint;
sofar := 0;
for rec_ in A loop
totalwork := rec_.cnt;
sofar := sofar + 1;
dbms_application_info.set_session_longops(rindex,
slno,
'Process a_tab',
'A_TAB',
0,
sofar,
totalwork,
'table',
'rows');
procedure_A(rec_.column_A);
end loop;
end;
Note that in order to get the totalwork value, I've used the analytic COUNT() function to get the total number of rows within the resultset. You could run a separate query to get the count before looping through your original cursor, if that is faster. You'd have to test both methods to work out which would be fastest for your data etc.
Of course, depending on what procedure_a does, you might be able to avoid the need to monitor the progress if you can refactor things so that all the work is being done in a single SQL statement. My answer above assumes that it's not possible to do that. If it is, I highly recommend you refactor your code instead!
Related
For some reason I'm trying to figure out why the following query executes by full table scan which takes ages because the table has ~31M rows
PROCEDURE d1(k_uni_in IN data_par.k_uni%TYPE) AS
CURSOR d1_cur IS
SELECT d.*
FROM data_par d
WHERE d.k_uni = k_uni_in
ORDER BY d.k_date;
BEGIN
FOR i IN d1_cur LOOP
...
END LOOP;
END;
However seemingly similar query runs index range scan and is pretty much instant
PROCEDURE d1(k_uni_in IN data_par.k_uni%TYPE) AS
CURSOR d1_cur(k_cv IN data_par.k_uni%TYPE) IS
SELECT d.*
FROM data_par d
WHERE d.k_uni = k_cv
ORDER BY d.k_date;
BEGIN
FOR i IN d1_cur(k_uni_in) LOOP
...
END LOOP;
END;
Why does that happen? Should I always use cursor parameters instead of using suprogram parameters in cursors?
If your table DATA_PAR has a column named K_UNI_IN, then Oracle is interpreting this line:
WHERE d.k_uni = k_uni_in
As meaning
WHERE d.k_uni = d.k_uni_in
And, since that's obviously not a condition that an index can help with, you're getting a full table scan.
See also: http://docs.oracle.com/cd/E11882_01/appdev.112/e25519/nameresolution.htm#LNPLS2038
Is there a quick way to flush REF CURSOR into a temporary table in oracle
CURSOR size will vary between 5K to 100K rows
currently I am using a for loop, but its very slow
There is no way to change the OUT parameter for SOME_FUNCTION_CALL function.
DECLARE
v_return SYS_REFCURSOR;
VAR_A varchar2(100);
VAR_B varchar2(100);
BEGIN
v_return := SOME_FUNCTION_CALL();
LOOP
FETCH v_return into VAR_A,VAR_B
EXIT WHEN v_return%NOTFOUND
INSERT INTO temp_table(a,b) values (VAR_A,VAR_B);
END LOOP;
CLOSE v_return;
END LOOP;
Probably not.
You could potentially make the code more efficient by doing a BULK COLLECT into local collections rather than doing row-by-row fetches. Something like
DECLARE
TYPE vc100_tbl IS TABLE OF VARCHAR2(100);
l_return SYS_REFCURSOR;
l_var_a_tbl vc100_tbl;
l_var_b_tbl vc100_tbl;
BEGIN
l_return := some_function_call();
LOOP
FETCH l_return
BULK COLLECT INTO l_var_a_tbl, l_var_b_tbl
LIMIT 100;
EXIT WHEN l_var_a_tbl.count = 0;
FORALL i IN 1 .. l_var_a_tbl.count
INSERT INTO temp_table( a, b )
VALUES( l_var_a_tbl(i), l_var_b_tbl(i) );
END LOOP;
close l_return;
END;
That will reduce the number of context shifts taking place between the SQL and PL/SQL engines. But depending on your definition of "very slow", it seems unlikely that context shifts are your most pressing problem. Were I to guess, I'd wager that if your code is "very slow" for most definitions of that term, the problem is likely that the query that is being used to open the cursor in some_function is slow and that most of your time is being spent executing that query in order to generate the rows that you want to fetch. If that's the case, you'd want to optimize the query.
Architecturally, this approach also seems rather problematic. Even if you can't change the definition of the function, can't you do something to factor out the query the function is executing so that you can use the same code without calling the function? For example, could you not factor out the query the function calls into a view and then call that view from your code?
I have a routine written in T-SQL for SQL Server. We are migrating to Oracle so I am trying to port it to PL/SQL. Here is the T-SQL routine (simplified); note the use of the table-valued variable which, in Oracle, will become a "nested table" type PL/SQL variable. The main thrust of my question is on the best ways of working with such "collection" objects within PL/SQL. Several operations in the ported code (second code sample, below) are quite awkward, where they seemed a lot easier in the SQL Server original:
DECLARE #MyValueCollection TABLE( value VARCHAR(4000) );
DECLARE #valueForThisRow VARCHAR(4000);
DECLARE #dataItem1Val INT, #dataItem2Val INT, #dataItem3Val INT, #dataItem4Val INT;
DECLARE theCursor CURSOR FAST_FORWARD FOR
SELECT DataItem1, DataItem2, DataItem3, DataItem4 FROM DataTable;
OPEN theCursor;
FETCH NEXT FROM theCursor INTO #dataItem1Val, #dataItem2Val, #dataItem3Val, #dataItem4Val;
WHILE ##FETCH_STATUS = 0
BEGIN
-- About 50 lines of logic that evaluates #dataItem1Val, #dataItem2Val, #dataItem3Val, #dataItem4Val and constructs #valueForThisRow
SET #valueForThisRow = 'whatever';
-- !!! This is the row that seems to have no natural Oracle equivalent
INSERT INTO #MyValueCollection VALUES(#valueForThisRow);
FETCH NEXT FROM theCursor INTO #dataItem1Val, #dataItem2Val, #dataItem3Val, #dataItem4Val;
END;
CLOSE theCursor;
DEALLOCATE theCursor;
-- !!! output all the results; this also seems harder than it needs to be in Oracle
SELECT * FROM #MyValueCollection;
I have been able to port pretty much everything, but in two places (see comments in the code), the logic is a lot more complex than the old SQL Server way, and I wonder if there might be, in Oracle, some more graceful way that is eluding me:
set serveroutput on; -- needed for DBMS_OUTPUT; see below
DECLARE
TYPE StringList IS TABLE OF VARCHAR2(4000);
myValueCollection StringList;
dummyTempCollection StringList; -- needed for my kludge; see below
valueForThisRow VARCHAR2(4000);
BEGIN
-- build all the sql statements
FOR c IN (
SELECT DataItem1, DataItem2, DataItem3, DataItem4 FROM DataTable;
)
LOOP
-- About 50 lines of logic that evaluates c.DataItem1, c.DataItem2, c.DataItem3, c.DataItem4 and constructs valueForThisRow
valueForThisRow := 'whatever';
-- This seems way harder than it should be; I would rather not need an extra dummy collection
SELECT valueForThisRow BULK COLLECT INTO dummyTempCollection FROM dual; -- overwrites content of dummy temp
myValueCollection := myValueCollection MULTISET UNION dummyTempCollection; -- merges into main collection
END LOOP;
-- output all the results... again, there's no shorter/easier/more-compact/single-line equivalent?
IF myValueCollection.COUNT > 0
THEN
FOR indx IN myValueCollection.FIRST .. myValueCollection.LAST
LOOP
DBMS_OUTPUT.PUT_LINE(myValueCollection(indx));
END LOOP;
END IF;
END;
/
Thanks in advance for any help!
Personally, I'd take the "50 lines of logic", move it into a function that you call in your SQL statement, and then do a simple BULK COLLECT to load the data into your local collection.
Assuming that you really want to load data element-by-element into the collection, you can simplify the code that loads the collection
DECLARE
TYPE StringList IS TABLE OF VARCHAR2(4000);
myValueCollection StringList := StringList();
valueForThisRow VARCHAR2(4000);
BEGIN
-- build all the sql statements
FOR c IN (
SELECT DataItem1, DataItem2, DataItem3, DataItem4 FROM DataTable;
)
LOOP
-- About 50 lines of logic that evaluates c.DataItem1, c.DataItem2, c.DataItem3, c.DataItem4 and constructs valueForThisRow
valueForThisRow := 'whatever';
myValueCollection.extend();
myValueCollection( myValueCollection.count ) := valueForThisRow;
END LOOP;
-- output all the results... again, there's no shorter/easier/more-compact/single-line equivalent?
IF myValueCollection.COUNT > 0
THEN
FOR indx IN myValueCollection.FIRST .. myValueCollection.LAST
LOOP
DBMS_OUTPUT.PUT_LINE(myValueCollection(indx));
END LOOP;
END IF;
END;
/
If you declare the collection as an associative array, you could avoid calling extend to increase the size of the collection. If you know the number of elements that you are going to load into the collection, you could pass that to a single extend call outside the loop. Potentially, you can also eliminate the valueForThisRow local variable and just operate on elements in the collection.
As for the code that processes the collection, what is it that you are really trying to do? It would be highly unusual for production code to write to dbms_output and expect that anyone will see the output during normal processing. That will influence the way that you would write that code. Assuming that your intention is really to just call dbms_output, knowing that will generally send the data into the ether
FOR indx IN 1 .. myValueCollection.count
LOOP
dbms_output.put_line( myValueCollection(indx) );
END LOOP;
This works when you have a dense collection (all indexes between 1 and the count of the collection exist and have values). If you might have a sparse collection, you would want to use FIRST, NEXT, and LAST in a loop but that's a bit more code.
I have to write a dynamic sql cursor where there are several possibilities in which the select query will be generated. Hence I am chosing dynamic and I am Using DBMS_SQL package to dynamically create a cursor and dynamically fetch the data.
However , Result set is going to be huge . around 11GB (there are 2.4 million records and the select statement will be approx 80 cols long assumning about 50Byte varchar per column)
Hence I cannot open the cursor at once . I want to know if there is a feature wherein i can fetch the data from the curosr keeping the curosr open for Blocks of say 1000 records at time(I will have to do this dynamically)
Please find the code attached which only fetches and prints the value of the columns (one sample case ) I want to use bul collect here \
Thanks
---------------code sample--------------------------------------
--create or replace type TY_DIMDEAL AS TABLE OF VARCHAR2(50) ;
create or replace procedure TEST_PROC (po_recordset out sys_refcursor)
as
v_col_cnt INTEGER;
v_ind NUMBER;
rec_tab DBMS_SQL.desc_tab;
v_cursor NUMBER;
lvar_output number:=0;
lvar_output1 varchar2(100);
lvar_output3 varchar2(100);
lvar_output2 varchar2(100);
LVAR_TY_DIMDEAL TY_DIMDEAL;
lvarcol varchar2(100);
begin
--
LVAR_TY_DIMDEAL := TY_DIMDEAL();
lvar_output1 := '';
v_cursor := dbms_sql.open_cursor;
dbms_sql.parse(v_cursor, 'select to_char(Field1) , to_char(fiel2) , to_char(field3) from table,table2 ', dbms_sql.native);
dbms_sql.describe_columns(v_cursor, v_col_cnt, rec_tab);
FOR v_pos in 1..rec_tab.LAST LOOP
LVAR_TY_DIMDEAL.EXTEND();
DBMS_SQL.define_column( v_cursor, v_pos ,LVAR_TY_DIMDEAL(v_pos),20);
END LOOP;
-- DBMS_SQL.define_column( v_cursor, 1 ,lvar_output1,20);
--DBMS_SQL.define_column( v_cursor, 2 ,lvar_output2,20);
--DBMS_SQL.define_column( v_cursor, 3 ,lvar_output3,20);
v_ind := dbms_sql.execute( v_cursor );
LOOP
v_ind := DBMS_SQL.FETCH_ROWS( v_cursor );
EXIT WHEN v_ind = 0;
lvar_output := lvar_output+1;
dbms_output.put_line ('row number '||lvar_output) ;
FOR v_col_seq IN 1 .. rec_tab.COUNT LOOP
LVAR_TY_DIMDEAL(v_col_seq):= '';
DBMS_SQL.COLUMN_VALUE( v_cursor, v_col_seq,LVAR_TY_DIMDEAL(v_col_seq));
dbms_output.put_line (LVAR_TY_DIMDEAL(v_col_seq));
END LOOP;
END LOOP;
end TEST_PROC;
Fetching data from a cursor in blocks of reasonable size, while keeping the cursor open, is one of PL/SQL Best Practices.
The above document (see Code 38 item) sketches an approach for when the select list is not known until runtime. Basically:
Define an appropriate type to fetch results into. Let's assume that all the returned columns will by of of type VARCHAR2:
-- inside DECLARE
Ty_FetchResults IS TABLE OF DBMS_SQL.VARCHAR2_TABLE;
lvar_results Ty_FetchResults;
Before each call to DBMS_SQL.FETCH_ROWS, call DBMS_SQL.DEFINE_ARRAY to enable batch fetching.
Call DBMS_SQL.FETCH_ROWS to fetch 1000 rows from the cursor.
Call DBMS_SQL.COLUMN_VALUE to copy the fetched data into your result array.
Process the results, record by record, in a FOR loop. Don't worry about the number of fetched records: if there are records to process, the FOR loop will run correctly; if the result array is empty, the FOR loop will not run.
Exit from the loop when the number of fetched records is less than the expected size.
Remember to DBMS_SQL.CLOSE the cursor.
Your loop body should look like this:
LOOP
FOR j IN 1..v_col_cnt LOOP
DBMS_SQL.DEFINE_ARRAY(v_cursor, j, lvar_results(j), 1000, 1);
END LOOP;
v_ind := DBMS_SQL.FETCH_ROWS(v_cursor);
FOR j IN 1..v_col_cnt LOOP
lvar_results(j).DELETE;
DBMS_SQL.COLUMN_VALUE(v_cursor, j, lvar_results(j));
END LOOP;
-- process the results, record by record
FOR i IN 1..lvar_results(1).COUNT LOOP
-- process a single record...
-- your logic goes here
END LOOP;
EXIT WHEN lvar_results(1).COUNT < 1000;
END LOOP;
-- don't forget: DBMS_CLOSE(v_cursor);
See also Doing SQL from PL/SQL: Best and Worst Practices.
LIMIT CLAUSE CAN COME TO RESCUE!
PL/SQL collections are essentially arrays in memory, so massive
collections can have a detrimental effect on system performance due to
the amount of memory they require. In some situations, it may be
necessary to split the data being processed into chunks to make the
code more memory-friendly. This “chunking” can be achieved using the
LIMIT clause of the BULK COLLECT syntax.
YOU CAN USE LIMIT CLAUSE AFTER BULK COLLECT INTO CLAUSE TO LIMIT YOUR RS.
AFTER YOU EXCEED TO LIMIT YOU CAN FETCH REMAINING ROWS.
SEE THIS ARTICLE
http://www.dba-oracle.com/plsql/t_plsql_limit_clause.htm
I have a select statement that I am trying to loop over and increment a variable based on the condition of the select statement, then return the variable as an out so I can do something with it in some front end code. I am using oracle 11g and I am seeing a few ways I can do this... but I am not sure which is the best way. I have some of what I am trying to do below, but again stopped because of confusion.
First I am setting my proc and 'in variable'
PROCEDURE SEEKER (pMonkeyID IN Number, vMarkCounter OUT Number)
AS
BEGIN
CURSOR seeker_cur IS
Select Mokney_approved, Monkey_vaulted
from MonkeyBookApps
where MonkeyID = pMonkeyID
and Monkey_doc_type = 'BANANA'
order by docu_approv_timestamp,monkey_doc_type,monkey_doc_approved desc
OPEN seeker_cur;
begin
OPEN Seeker_cur;
vMarkCounter := 0;
Here is the part I am not sure about. Should I loop and then exit if the condition is not met or should I do an if statement and somehow determine if there is a record that could be greater than one? If so how would that work? Is there a benefit to doing one way over the other? So... I am going to sudo-code what I am trying to do (below):
FOR (however many records) in Seeker_cur
IF seeker_cur (not found) or (returns no records)
EXIT or (break for loop);
ELSE
LOOP
vMarkCounter := vMarkCounter + 1;
EXIT WHEN seeker_cur is out of records (somehow)
END IF;
END LOOP;
END;
END SEEKER;
I am sure there are a few ways to do this. What ways would you suggest?
why dont you use implicit cursor , it will open and close itself:
DECLARE
CURSOR seeker_cur IS
Select Mokney_approved, Monkey_vaulted
from MonkeyBookApps
where MonkeyID = pMonkeyID
and Monkey_doc_type = 'BANANA'
order by docu_approv_timestamp,monkey_doc_type,monkey_doc_approved desc;
vMarkCounter number:=0;
BEGIN
FOR i IN seeker_cur
LOOP
vMarkCounter := vMarkCounter+1;
END LOOP;
dbms_output.put_line(vMarkCounter);
END;
It seems to me that the solution your problem might be as simple as this:
SELECT COUNT(*)
INTO l_some_local_variable
FROM monkey_book_apps
WHERE monkey_id = p_monkey_id
AND monkey_doc_type = 'BANANA';
RETURN l_some_local_variable;
Avoiding PL/SQL loops and using the simplest SQL possible is (almost always) the most efficient way. Tom Kyte calls the row-by-row execution of LOOPs "slow-by-slow".