Fetching the record one by one dynamically Oracle - oracle

I am creating a dynamic procedure which could accept 2 table names.Fetch the records from one table and after certain record (let's say 100 records) i have to issue the commit command.
Both tabName and temp_tabName are always be identical.Since I have billions of records in first table i am doing the commit after every 10000 records in order to get rid of undo table space problem.
Till now what i did is :
CREATE OR REPLACE PROCEDURE MyProdecure (
tabName IN USER_TABLES.table_name%TYPE,
temp_tabName IN USER_TABLES.table_name%TYPE
)
IS
v_sql VARCHAR2 (100) := 'select * from ' || tabName;
TEMP_CURSOR SYS_REFCURSOR;
COUNT NUMBER (6) := 0;
BEGIN
OPEN TEMP_CURSOR FOR v_sql;
LOOP
FETCH TEMP_CURSOR INTO V_ROW;
--=================================================================================
/*
* I need the code here to fetch the 100 record from TEMP_CURSOR into a Variable
* and insert into the second table. or one record increment the count and if
* count>= 100 commit
*What would be the data type of V_ROW. How to fetch the data from V_ROW and complete the insert into command.
*/
--================================================================================
EXIT WHEN TEMP_CURSOR%NOTFOUND;
END LOOP;
CLOSE TEMP_CURSOR;
END MyProdecure;

There is no way to define V_ROW in such a way as to make your PL/SQL block work correctly for an input table whose name and structure is not known until runtime.
To make your approach work, you would need to use DBMS_SQL.
Have you considered a variation of the following, to bypass the vast majority of the UNDO generation?
CREATE OR REPLACE PROCEDURE MyProcedure (
tabName IN USER_TABLES.table_name%TYPE,
temp_tabName IN USER_TABLES.table_name%TYPE
)
IS
l_log_io NUMBER;
C_BLOCK_SIZE NUMBER := 8192; -- assuming 8192 byte block size
l_undo_bytes NUMBER;
BEGIN
EXECUTE IMMEDIATE 'INSERT /*+ APPEND */ INTO ' || temp_tabName ||
' SELECT * FROM ' || tabName;
select t.log_io, t.used_ublk*C_BLOCK_SIZE undo_bytes
into l_log_io, l_undo_bytes
from v$transaction t
where t.addr = ( SELECT s.taddr FROM v$session s WHERE s.sid = USERENV('SID'));
dbms_output.put_line('Undo bytes used: ' || l_undo_bytes);
END;
INSERT /*+ APPEND */ comes with a number of caveats that you should look into before using it, but it could be a much simpler way of accomplishing your goal.

Related

How to access and query objects passed as parameter to a procedure while converting from Oracle to postgresql

I have a procedure in Oracle that I need to convert to Postgresql and need help on it. It paases a collection of objects in a procedure.The procedure then checks if each object is present in a database table or not and if present it gives a message that , that specific element is found/present. if some element that is paassed to the procedure is not present in the table, the procedure just doesnt do anything. I have to write equivalent of that in postgresql. I think the heart of the issue is this statement:
SELECT COUNT (*)
INTO v_cnt
FROM **TABLE (p_cust_tab_type_i)** pt
WHERE pt.ssn = cc.ssn;
In Oracle a collection can be treated as a table and one can query it but I dont know how to do that in postgresql. The code to create the table, add data, create the procedure, call the procedure by passing the collection (3 objects) and output of that is posted below. Can someone suggest how this can be done in postgresql?
Following the oracle related code and details:
--create table
create table temp_n_tab1
(ssn number,
fname varchar2(20),
lname varchar2(20),
items varchar2(100));
/
--add data
insert into temp_n_tab1 values (1,'f1','l1','i1');
--SKIP no. ssn no. 2 intentionally..
insert into temp_n_tab1 values (3,'f3','l3','i3');
insert into temp_n_tab1 values (4,'f4','l4','i4');
insert into temp_n_tab1 values (5,'f5','l5','i5');
insert into temp_n_tab1 values (6,'f6','l6','i6');
commit;
--create procedure
SET SERVEROUTPUT ON
CREATE OR REPLACE PROCEDURE temp_n_proc (
p_cust_tab_type_i IN temp_n_customer_tab_type)
IS
t_cust_tab_type_i temp_n_customer_tab_type;
v_cnt NUMBER;
v_ssn temp_n_tab1.ssn%TYPE;
CURSOR c
IS
SELECT ssn
FROM temp_n_tab1
ORDER BY 1;
BEGIN
--t_cust_tab_type_i := p_cust_tab_type_i();
FOR cc IN c
LOOP
SELECT COUNT (*)
INTO v_cnt
FROM TABLE (p_cust_tab_type_i) pt
WHERE pt.ssn = cc.ssn;
IF (v_cnt > 0)
THEN
DBMS_OUTPUT.put_line (
'The array element '
|| TO_CHAR (cc.ssn)
|| ' exists in the table.');
END IF;
END LOOP;
EXCEPTION
WHEN OTHERS
THEN
DBMS_OUTPUT.PUT_LINE (SQLERRM);
END;
/
--caller proc
SET SERVEROUTPUT ON
declare
array temp_n_customer_tab_type := temp_n_customer_tab_type();
begin
for i in 1 .. 3
loop
array.extend;
array(i) := temp_n_cust_header_type( i, 'name ' || i, 'lname ' || i,i*i*i*i );
end loop;
temp_n_proc( array );
end;
/
caller proc output:
The array element 1 exists in the table.
The array element 3 exists in the table.
When you create a table in Postgres, a type with the same name is also created. So you can simply pass an array of the table's type as a parameter to the function.
Inside the function you can then use unnest() to treat the array like a table.
The following is the closest match to your original Oracle code:
create function temp_n_proc(p_cust_tab_type_i temp_n_tab1[])
returns void
as
$$
declare
l_rec record;
l_msg text;
l_count integer;
BEGIN
for l_rec in select t1.ssn
from temp_n_tab1 t1
loop
select count(*)
into l_count
from unnest(p_cust_tab_type_i) as t
where t.ssn = l_rec.ssn;
if l_count > 0 then
raise notice 'The array element % exist in the table', l_rec.ssn;
end if;
end loop;
END;
$$
language plpgsql;
The row-by-row processing is not a good idea to begin with (neither in Postgres, nor in Oracle). It would be a lot more efficient to get the existing elements in a single query:
create function temp_n_proc(p_cust_tab_type_i temp_n_tab1[])
returns void
as
$$
declare
l_rec record;
l_msg text;
BEGIN
for l_rec in select t1.ssn
from temp_n_tab1 t1
where t1.ssn in (select t.ssn
from unnest(p_cust_tab_type_i) as t)
loop
raise notice 'The array element % exist in the table', l_rec.ssn;
end loop;
return;
END;
$$
language plpgsql;
You can call the function like this:
select temp_n_proc(array[row(1,'f1','l1','i1'),
row(2,'f2','l2','i2'),
row(3,'f3','l3','i3')
]::temp_n_tab1[]);
However a more "Postgres" like and much more efficient way would be to not use PL/pgSQL for this, but create a simple SQL function that returns the messages as a result:
create or replace function temp_n_proc(p_cust_tab_type_i temp_n_tab1[])
returns table(message text)
as
$$
select format('The array element %s exist in the table', t1.ssn)
from temp_n_tab1 t1
where t1.ssn in (select t.ssn
from unnest(p_cust_tab_type_i) as t)
$$
language sql;
This returns the output of the function as a result rather than using the clumsy raise notice.
You can use it like this:
select *
from temp_n_proc(array[row(1,'f1','l1','i1'),
row(2,'f2','l2','i2'),
row(3,'f3','l3','i3')
]::temp_n_tab1[]);

Compile error oracle procedure

Hej. I have a task that says to create a procedure that adds column "BRUTTO" to a table "TABELA_1, then fills that column with values based on values from column "NETTO" and output all records from TABLE_1, including newly created BRUTTO. It works without commented out code but doesn't otherwise. Apparently it doesn't see column BRUTTO yet so I can't reference it like that. Any help appreciated.
CREATE OR REPLACE PROCEDURE WSTAW_BRUTTO_I_WYSWIETL
AS
--CURSOR C IS
--SELECT NAZWISKO, NETTO, BRUTTO FROM TABELA_1;
V_VAT NUMBER(9,2) := 24;
BEGIN
EXECUTE IMMEDIATE 'ALTER TABLE TABELA_1 ADD BRUTTO NUMBER';
EXECUTE IMMEDIATE 'UPDATE TABELA_1 SET BRUTTO = NETTO * (1 + :1 /100)' USING V_VAT;
--FOR V_REC IN C
--LOOP
--DBMS_OUTPUT.PUT_LINE('| '||V_REC.NETTO||' | '||V_REC.BRUTTO);
--END LOOP;
END WSTAW_BRUTTO_I_WYSWIETL;
Your procedure won't compile because you can't access a column before it is added to the table. Not sure why you wrote a procedure with dynamic SQL for this. A plain SQL statement should work. Moreover, you can't use bind variable in a DDL, It would have raised
ORA-01027: bind variables not allowed for data definition operations
during run time.
You should also consider using BRUTTO as a VIRTUAL COLUMN, rather than a column itself.
ALTER TABLE TABELA_1 ADD BRUTTO NUMBER AS ( NETTO * (1 + 24 /100) );
Demo
If you still think you want a procedure and it has to compile, you should put the block inside EXECUTE IMMEDIATE, but it's not recommended.
CREATE OR REPLACE PROCEDURE wstaw_brutto_i_wyswietl AS
v_vat NUMBER(9,2) := 24;
BEGIN
EXECUTE IMMEDIATE 'ALTER TABLE TABELA_1 ADD BRUTTO NUMBER AS ( NETTO * (1 + '
|| v_vat
|| ' /100) )';
EXECUTE IMMEDIATE q'{BEGIN
FOR V_REC IN ( SELECT NETTO,BRUTTO FROM TABELA_1 )
LOOP
DBMS_OUTPUT.PUT_LINE(V_REC.NETTO||','||V_REC.BRUTTO);
END LOOP;
END;}'
;
END wstaw_brutto_i_wyswietl;
/
Demo2

Oracle: Pure PL/SQL data extraction and anonymization using temporary tables, read-only permissions

I am trying to create a PL/SQL script that extracts a root "object" together with all children and other relevant information from an oracle production database. The purpose is to create a set of test-data to recreate issues that are encountered in production. Due to data protection laws the data needs to be anonymized when extracted - object names, certain types of id's, and monetary amounts need to be replaced.
I was trying to create one or more temporary translation tables, which would contain both the original values and anonymized versions. Then I would join the real data with the translation tables and output the anonymized values wherever required.
DECLARE
rootId integer := 123456;
TYPE anonTableRow IS RECORD
(
id NUMBER,
fieldC NUMBER,
anonymizedFieldC NUMBER
);
TYPE anonTable IS TABLE OF anonTableRow;
anonObject anonTable;
BEGIN
FOR cursor_row IN
(
select
id,
fieldC,
1234 -- Here I would create anonymized values based on rowNum or something similar
from
prodTable
where id = rootId
)
LOOP
i := i + 1;
anonObject(i) := cursor_row;
END LOOP;
FOR cursor_row IN
(
select
prod_table.id,
prod_table.fieldB,
temp_table.anonymizedFieldC fieldC,
prod_table.fieldD
from
prod_table
inner join table(temp_table) on prod_table.id = temp_table.id
where prod_table.id = 123456789
)
LOOP
dbms_output.put_line('INSERT INTO prod_table VALUES (' || cursor_row.id || ', ' || cursor_row.fieldB || ', ' || cursor_row.fieldC || ', , ' || cursor_row.fieldD);
END LOOP;
END;
/
However I ran into several problems with this approach - it seems to be near impossible to join oracle PL/SQL tables with real database tables. My access to the production database is severely restricted, so I cannot create global temporary tables, declare types outside PL/SQL or anything of that sort.
My attempt to declare my own PL/SQL types failed with the problems mentioned in this question - the solution does not work for me because of the limited permissions.
Is there a pure PL/SQL way that does not require fancy permissions to achieve something like the above?
Please Note: The above code example is simplified a lot and would not really require a separate translation table - in reality I need access to the original and translated values in several different queries, so I would prefer not having to "recalculate" translations everywhere.
If your data is properly normalized, then I guess this should only be necessary for internal IDs (not sure why you need to translate them though).
The following code should work for you, keeping the mappings in Associative Arrays:
DECLARE
TYPE t_number_mapping IS TABLE OF PLS_INTEGER INDEX BY PLS_INTEGER;
mapping_field_c t_number_mapping;
BEGIN
-- Prepare mapping
FOR cur IN (
SELECT 101 AS field_c FROM dual UNION ALL SELECT 102 FROM dual -- test-data
) LOOP
mapping_field_c(cur.field_c) := mapping_field_c.COUNT; -- first entry mapped to 1
END LOOP;
-- Use mapping
FOR cur IN (
SELECT 101 AS field_c FROM dual UNION ALL SELECT 102 FROM dual -- test-data
) LOOP
-- You can use the mapping when generating the `INSERT` statement
dbms_output.put_line( cur.field_c || ' mapped to ' || mapping_field_c(cur.field_c) );
END LOOP;
END;
Output:
101 mapped to 1
102 mapped to 2
If this isn't a permanent piece of production code, how about "borrowing" an existing collection type - e.g. one define in SYS that you can access.
Using this script from your schema you can generate a SQL Plus script to describe all SYS-owned types:
select 'desc ' || type_name from all_types
where typecode = 'COLLECTION'
and owner = 'SYS';
Running the resulting script will show you the structure of all the ones you can access. This one looks potentially suitable for example:
SQL> desc KU$_PARAMVALUES1010
KU$_PARAMVALUES1010 TABLE OF SYS.KU$_PARAMVALUE1010
Name Null? Type
----------------------------------------- -------- ----------------------------
PARAM_NAME VARCHAR2(30)
PARAM_OP VARCHAR2(30)
PARAM_TYPE VARCHAR2(30)
PARAM_LENGTH NUMBER
PARAM_VALUE_N NUMBER
PARAM_VALUE_T VARCHAR2(4000)
Of course, you can't guarantee that type will still exist or be the same or be accessible to you after a database upgrade, hence my caveat at the start.
More generic way to achieve this goal.
In my example i'm using xquery flwor expressions and dbms_xmlstore. Knowledge about xquery is mandatory.
create table mask_user_objects as select * from user_objects where rownum <0;
declare
v_s_table varchar2(30) := 'USER_OBJECTS'; --uppercase!!!
v_d_table varchar2(30) := 'MASK_USER_OBJECTS'; --uppercase!!!
v_mask_columns xmltype := xmltype('<COLS><OBJECT_NAME>XXXX</OBJECT_NAME>
<DATA_OBJECT_ID>-1</DATA_OBJECT_ID>
<OBJECT_TYPE/>
</COLS>'); --uppercase!!!
insCtx DBMS_XMLSTORE.ctxType;
r NUMBER;
v_source_table xmltype;
v_cursor sys_refcursor;
begin
open v_cursor for 'select * from '||v_s_table||' where rownum <100 ';
v_source_table := xmltype(v_cursor);
close v_cursor;
-- Load source table into xmltype.
insCtx := DBMS_XMLSTORE.newContext(v_d_table); -- Get saved context
for rec in (
select tt.column_value from xmltable('
let $col := $anomyze/COLS
for $i in $doc/ROWSET/ROW
let $row := $i
return <ROWSET>
<ROW>
{
for $x in $row/*
return if(
exists($col/*[name() = $x/name()] )
) then element{$x/name()}{ $col/*[name() = $x/name()]/text() }
else element{$x/name()}{$x/text()}
}
</ROW>
</ROWSET>
'
passing v_source_table as "doc"
, v_mask_columns as "anomyze"
) tt) loop
null;
r := DBMS_XMLSTORE.insertXML(insCtx, rec.column_value);
end loop;
DBMS_XMLSTORE.closeContext(insCtx);
end;

How to use function returning Oracle REF_CURSOR in a procedure

I have to write an Oracle procedure which should invoke an Oracle function returning REF_CURSOR. The function is declared like that
FUNCTION "IMPACTNET"."TF_CONVERTPARA" (PARASTRING IN NVARCHAR2) RETURN SYS_REFCURSOR
AS
c SYS_REFCURSOR;
BEGIN
OPEN c FOR
SELECT SUBSTR(element, 1, INSTR(element, '|') - 1) as key,
SUBSTR(element, INSTR(element, '|') + 1, 99999) as val
FROM (
SELECT REGEXP_SUBSTR(PARASTRING, '[^;]+', 1, LEVEL) element
FROM dual
CONNECT BY LEVEL < LENGTH(REGEXP_REPLACE(PARASTRING, '[^;]+')) + 1
);
RETURN c;
END;
Can you tell me what I need to write in order to invoke the function from within my procedure? I'd like to insert all the returned values (shaped a table with two columns) into a rational table.
Thank you in advance!
Something along the lines of this should work (obviously, I'm guessing about table names and column names and the exact logic that you're trying to implement)
CREATE PROCEDURE some_procedure_name
AS
l_rc SYS_REFCURSOR := impactnet.tf_convertpara( <<some string>> );
l_key VARCHAR2(100);
l_val VARCHAR2(100);
BEGIN
LOOP
FETCH l_rc
INTO l_key, l_val;
EXIT WHEN l_rc%notfound;
INSERT INTO some_table( key_column, val_column )
VALUES( l_key, l_val );
END LOOP;
END;
As Ollie points out, it would be more efficient to do a BULK COLLECT and a FORALL. If you're just dealing with a few thousand rows (since your function is just parsing the data in a delimited string, I'm assuming you expect relatively few rows to be returned), the performance difference is probably minimal. But if you're processing more data, the difference can be quite noticeable. Depending on the Oracle version and your specific requirements, you may be able to simplify the INSERT statement in the FORALL to insert a record rather than listing each column from the record individually.
CREATE PROCEDURE some_procedure_name
AS
TYPE key_val_rec
IS RECORD(
key VARCHAR2(100),
val VARCHAR2(100)
);
TYPE key_val_coll
IS TABLE OF key_val_rec;
l_rc SYS_REFCURSOR := impactnet.tf_convertpara( <<some string>> );
l_coll key_val_coll;
BEGIN
LOOP
FETCH l_rc
BULK COLLECT INTO l_coll
LIMIT 100;
EXIT WHEN l_coll.count = 0;
FORALL i IN l_coll.FIRST .. l_coll.LAST
INSERT INTO some_table( key_column, val_column )
VALUES( l_coll(i).key, l_coll(i).val );
END LOOP;
END;

oracle stored procedure - select, update and return a random set of rows

oracle i wish to select few rows at random from a table, update a column in those rows and return them using stored procedure
PROCEDURE getrows(box IN VARCHAR2, row_no IN NUMBER, work_dtls_out OUT dtls_cursor) AS
v_id VARCHAR2(20);
v_workname VARCHAR2(20);
v_status VARCHAR2(20);
v_work_dtls_cursor dtls_cursor;
BEGIN
OPEN v_work_dtls_cursor FOR
SELECT id, workname, status
FROM item
WHERE status IS NULL
AND rownum <= row_no
FOR UPDATE;
LOOP
FETCH v_work_dtls_cursor
INTO v_id ,v_workname,v_status;
UPDATE item
SET status = 'started'
WHERE id=v_id;
EXIT
WHEN v_work_dtls_cursor % NOTFOUND;
END LOOP;
close v_work_dtls_cursor ;
/* I HAVE TO RETURN THE SAME ROWS WHICH I UPDATED NOW.
SINCE CURSOR IS LOOPED THRU, I CANT DO IT. */
END getrows;
PLEASE HELP
Following up on Sjuul Janssen's excellent recommendation:
create type get_rows_row_type as object
(id [item.id%type],
workname [item.workname%type],
status [item.status%type]
)
/
create type get_rows_tab_type as table of get_rows_row_type
/
create function get_rows (box in varchar2, row_no in number)
return get_rows_tab_type pipelined
as
v_work_dtls_cursor dtls_cursor;
l_out_rec get_rows_row_type;
BEGIN
OPEN v_work_dtls_cursor FOR
SELECT id, workname, status
FROM item sample ([ROW SAMPLE PERCENTAGE])
WHERE status IS NULL
AND rownum <= row_no
FOR UPDATE;
LOOP
FETCH v_work_dtls_cursor
INTO l_out_rec.id, l_out_rec.workname, l_outrec.status;
EXIT WHEN v_work_dtls_cursor%NOTFOUND;
UPDATE item
SET status = 'started'
WHERE id=l_out_rec.id;
l_out_rec.id.status := 'started';
PIPE ROW (l_out_rec);
END LOOP;
close v_work_dtls_cursor ;
END;
/
A few notes:
This is untested.
You'll need to replace the bracketed section in the type declarations with appropriate types for your schema.
You'll need to come up with an appropriate value in the SAMPLE clause of the SELECT statement; it might be possible to pass that in as an argument, but that may require using dynamic SQL. However, if your requirement is to get random rows from the table -- which just filtering by ROWNUM will not accomplish -- you'll want to do something like this.
Because you're SELECTing FOR UPDATE, one session can block another. If you're in 11g, you may wish to examine the SKIP LOCKED clause of the SELECT statement, which will enable multiple concurrent sessions to run code like this.
Not sure where you are doing your committing, but based on the code as it stands all you should need to do is SELECT ... FROM ITEM WHERE STATUS='started'
If it is small numbers, you could keep a collection of ROWIDs.
if it is larger, then I'd do an
INSERT into a global temporary table SELECT id FROM item .. AND ROWNUM < n;
UPDATE item SET status = .. WHERE id in (SELECT id FROM global_temp_table);
Then return a cursor of
SELECT ... FROM item WHERE id in (SELECT id FROM global_temp_table);
Maybe this can help you to do what you want?
http://it.toolbox.com/blogs/database-solutions/returning-rows-through-a-table-function-in-oracle-7802
A possible solution:
create type nt_number as table of number;
PROCEDURE getrows(box IN VARCHAR2,
row_no IN NUMBER,
work_dtls_out OUT dtls_cursor) AS
v_item_rows nt_number;
indx number;
cursor cur_work_dtls_cursor is
SELECT id
FROM item
WHERE status IS NULL
AND rownum <= row_no
FOR UPDATE;
BEGIN
open cur_work_dtls_cursor;
fetch cur_work_dtls_cursor bulk collect into nt_number;
for indx in 1 .. item_rows.count loop
UPDATE item
SET status = 'started'
WHERE id=v_item_rows(indx);
END LOOP;
close cur_work_dtls_cursor;
open work_dtls_out for select id, workname, status
from item i, table(v_item_rows) t
where i.id = t.column_value;
END getrows;
If the number of rows is particularly large, the global temporary solution may be better.

Resources