The following function generates the SELECT to produce INSERT STATEMENT of the records of an Oracle table:
CREATE OR REPLACE FUNCTION GEN_INSERT_STATEMENT (IN_TABLE_NAME VARCHAR2)
RETURN CLOB
IS
LC$COLS_SELECT CLOB;
LC$COLS_VALUES CLOB;
LC$COLOUMN CLOB;
CURSOR LCUR$TAB_COLUMNS (IN_TABLE_NAME VARCHAR2)
IS
SELECT COLUMN_NAME, DATA_TYPE, COLUMN_ID
FROM USER_TAB_COLS
WHERE TABLE_NAME = IN_TABLE_NAME
ORDER BY COLUMN_ID;
BEGIN
FOR LREC$TAB_COLUMNS IN LCUR$TAB_COLUMNS (UPPER (IN_TABLE_NAME))
LOOP
LC$COLS_SELECT :=
LC$COLS_SELECT
|| CASE LREC$TAB_COLUMNS.COLUMN_ID WHEN 1 THEN '' ELSE ',' END
|| LREC$TAB_COLUMNS.COLUMN_NAME;
IF INSTR (LREC$TAB_COLUMNS.DATA_TYPE, 'CHAR') > 0
THEN
LC$COLOUMN :=
'''''''''||REPLACE('
|| LREC$TAB_COLUMNS.COLUMN_NAME
|| ','''''''','''''''''''')||''''''''';
ELSIF INSTR (LREC$TAB_COLUMNS.DATA_TYPE, 'DATE') > 0
THEN
LC$COLOUMN :=
'''TO_DATE(''''''||TO_CHAR(' || LREC$TAB_COLUMNS.COLUMN_NAME
|| ',''mm/dd/yyyy hh24:mi:ss'')||'''''',''''mm/dd/yyyy hh24:mi:ss'''')''';
ELSE
LC$COLOUMN := LREC$TAB_COLUMNS.COLUMN_NAME;
END IF;
LC$COLS_VALUES :=
LC$COLS_VALUES
|| CASE LREC$TAB_COLUMNS.COLUMN_ID WHEN 1 THEN '' ELSE ',' END
|| '''||DECODE('
|| LREC$TAB_COLUMNS.COLUMN_NAME
|| ',NULL,''NULL'','
|| LC$COLOUMN
|| ')||''';
END LOOP;
RETURN 'SELECT ''INSERT INTO '
|| IN_TABLE_NAME
|| ' ('
|| LC$COLS_SELECT
|| ') VALUES ('
|| LC$COLS_VALUES
|| ');'' FROM '
|| IN_TABLE_NAME
|| ';';
END;
/
The problem is that this function DOES NOT HANDLE the case in which are existing some VARCHAR2 fields with strings ending with: CHR(0)
Usage:
SELECT GEN_INSERT_STATEMENT ('MY_ORACLE_TABLE_NAME') FROM DUAL;
... generates a SELECT to produce the INSERT statement.
If in a VARCHAR2 field the value ends with CHR(0), the INSERT statement will be truncated exactly where is CHR(0) located.
How can I fix this?
If I understand what you're seeing, you need to either strip the null character from the value:
IF INSTR (LREC$TAB_COLUMNS.DATA_TYPE, 'CHAR') > 0
THEN
LC$COLOUMN :=
'''''''''||REPLACE(REPLACE('
|| LREC$TAB_COLUMNS.COLUMN_NAME
|| ',CHR(0),NULL),'''''''','''''''''''')||''''''''';
which gives your function the output:
SELECT 'INSERT INTO T42 (STR) VALUES ('||DECODE(STR,NULL,'NULL',''''||REPLACE(REPLACE(STR,CHR(0),NULL),'''','''''')||'''')||');' FROM T42;
and with a dummy table called t42 with a single column str containing 'hello' followed by a null character, the output is:
INSERT INTO T42 (STR) VALUES ('Hello');
Or preserve it during the insert:
LC$COLOUMN :=
'''''''''||REPLACE(REPLACE('
|| LREC$TAB_COLUMNS.COLUMN_NAME
|| ','''''''',''''''''''''),CHR(0),''''''||CHR(0)||'''''')||''''''''';
which gives:
SELECT 'INSERT INTO T42 (STR) VALUES ('||DECODE(STR,NULL,'NULL',''''||REPLACE(REPLACE(STR,'''',''''''),CHR(0),'''||CHR(0)||''')||'''')||');' FROM T42;
and finally:
INSERT INTO T42 (STR) VALUES ('Hello'||CHR(0)||'');
So in the second version the null character is removed from the fixed string (anywhere, not just at the end), and is put back as part of the insert statement.
This seems like a horrible approach though, when you could export the data, or let SQL Developer or some other IDE generate the insert statements for you. There maybe other data types and values that give you headaches, and which someone else has already worked hard to overcome. Unless you really need to be able to see the actual statements, using expdp would be far simpler.
Use this:
https://github.com/teopost/oracle-scripts/blob/master/fn_gen_inserts.sql
Usage:
select fn_gen_inserts('select * from tablename', 'p_new_owner_name', 'p_new_table_name')
from dual;
where:
p_sql – dynamic query which will be used to export metadata rows
p_new_owner_name – owner name which will be used for generated INSERT
p_new_table_name – table name which will be used for generated INSERT
You can find original source code here:
http://dbaora.com/oracle-generate-rows-as-insert-statements-from-table-view-using-plsql/
Related
I have created a pl/sql procedure for a package that is performing a reconciliation between sets of tables which should match.
I am using listagg to concatenate the column names of the current table name in the loop into a string used in a dynamic SQL statement that compares two tables (34 sets, looped for each table name).
The procedure worked as expected, but the results were returned unexpectedly from the minus. After researching, I determined that some fields contained a HEX (00) character received in the flat file that populates the data on only the data from one side of the recon. In order to account for the special characters, I added a regexp_replace concatenated in line with the listagg in the column name select so it outputs the complete listagg results with each column name wrapped in a regexp_replace.
It works. However, some tables have over a hundred columns, and the listagg failes for the results being over 4000 characters.
Is there a better way to go about this entire thing?
Here is the code:
Collects column names into the comma-separated list (comma character is concatenated into the string itself for use as a separator in dynamic SQL select below)
execute immediate
'SELECT ' || q'{listagg('regexp_replace(' || column_name || ', ''[^A-Z0-9 ]'', '''')', '||'', '' || ')}' || ' within group (order by rownum) "COLUMN_NAME"
FROM user_tab_cols
where table_name =''' || csrpubtable.table_name || ''''
into v_column_names;
These two dynamic SQL statements perform the reconciliation in both directions. These aren't directly related to the error, but definitely to my question of a better overall way to accomplish the task.
--Insert data to RECON_PUB_TABLES where record exists in FILE but not PROD
execute immediate
'INSERT INTO RECON_PUB_TABLES
SELECT ''' || csrpubtable.table_name || ''', ''FILE'' , ' || v_column_names || ', trunc(sysdate) from ' || csrpubtable.table_name || '
minus
SELECT ''' || csrpubtable.table_name || ''', ''FILE'' , ' || v_column_names || ', trunc(sysdate) from ' || csrpubtable.table_name || '#pub_recon2prod where trunc(' || v_lastupdate_column || ') <= trunc(to_date(''' || v_compare_date || ''', ''dd-MON-yy''))';
--Insert data to RECON_PUB_TABLES where record exists in PROD but not FILE
execute immediate
'INSERT INTO RECON_PUB_TABLES
SELECT ''' || csrpubtable.table_name || ''', ''PROD'' , ' || v_column_names || ', trunc(sysdate) from ' || csrpubtable.table_name || '#pub_recon2prod where trunc(' || v_lastupdate_column || ') <= trunc(to_date(''' || v_compare_date || ''', ''dd-MON-yy''))
minus
SELECT ''' || csrpubtable.table_name || ''', ''PROD'' , ' || v_column_names || ', trunc(sysdate) from ' || csrpubtable.table_name ;
varchar2 is limited to 32k within plsql
if 32 is enough you can try something like this
create or replace procedure conc_col_names(tableName IN varchar2) as
collist varchar2(32767);
begin
for xx in (select * from user_tab_columns where table_name = tableName order by column_name asc) loop
if ( length(collist) > 0) then
collist := collist||',';
end if;
collist := collist||'regexp_replace('||xx.column_name||',''[^A-Z0-9 ]'')';
end loop;
/* add the rest code for comparing rows in the two table here */
end;
/
How do I get the data(i.e rows) from the column_name I retrieved from SQL statement? (Completely new to PL/SQL).
Here is my code:
create or replace procedure com_coll_cur
as
type comColcur is ref cursor;
com_col_cur comColcur;
sql_stmt varchar2(4000);
type newtab_field is table of comColcur%TYPE;
begin
sql_stmt :=
'select column_name from all_tab_cols where table_name in (''TAB1'', ''TAB2'') ' ||
'group by column_name having count(*)>1';
open com_col_cur for sql_stmt;
loop
fetch com_col_cur into newtab_field;
exit when com_col_cur%NOTFOUND;
end loop;
close com_col_cur;
end;
What I'm trying to do here is first find the common columns between the two tables. This part only grabs column_name but I also want the data in that common columns. So I used cursor to "point" that common column_name and used loop(fetch) to get the data inside that common column_name. Finally, I want to create new table with this common columns only with their data.
I am new to everything here and any help will be appreciate it.
You don't understand how works cursors and fetch.
Fetch get the data from the cursor, so in your procedure example you get only column names, not the data in the columns. To get data you need another cursor - select from the target table or use the dynamic sql.
This is a code that do what you describe. It is not clear to me how you want to store data from two tables - subsequently or in another manner. Let's assume that we store them subsequently. Also this code suggests than columns with the same names have the same datatypes. Part of this code (to make datatype) I get from another stackoverflow post to save time to write it:
How do I get column datatype in Oracle with PL-SQL with low privileges?
dbms_output.put_line - used to print sql statements that we create
declare
cSql varchar2(4000);
cCols varchar2(4000);
cNewTableName varchar2(30) := 'AABBCC';
cTb1 varchar2(30) := 'TAB1';
cTb2 varchar2(30) := 'TAB2';
begin
for hc in (
select T.column_name, T.typ
from
(
select column_name,
data_type||
case when data_precision is not null and nvl(data_scale,0)>0 then '('||data_precision||','||data_scale||')'
when data_precision is not null and nvl(data_scale,0)=0 then '('||data_precision||')'
when data_precision is null and data_scale is not null then '(*,'||data_scale||')'
when char_length>0 then '('||char_length|| case char_used
when 'B' then ' Byte'
when 'C' then ' Char'
else null
end||')'
end||decode(nullable, 'N', ' NOT NULL') typ
from all_tab_cols
where table_name in (cTb1, cTb2) ) T
group by T.column_name, T.typ having count(*) > 1)
loop
cSql := cSql || case when cSql is null then null else ',' end || hc.column_name || ' ' || hc.typ;
cCols := cCols || case when cCols is null then null else ',' end || hc.column_name;
end loop;
if (cSql is not null) then
-- First drop table if it exists
for hc in (select * from all_objects where object_type = 'TABLE' and object_name = cNewTableName)
loop
execute immediate 'drop table ' || hc.object_name;
end loop;
-- create table
cSql := 'create table ' || cNewTableName || '(' || cSql || ')';
dbms_output.put_line(cSql);
execute immediate cSql;
-- insert data
cSql := 'insert into ' || cNewTableName || '(' || cCols || ') select ' || cCols || ' from ' || cTb1;
dbms_output.put_line(cSql);
execute immediate cSql;
cSql := 'insert into ' || cNewTableName || '(' || cCols || ') select ' || cCols || ' from ' || cTb2;
dbms_output.put_line (cSql);
execute immediate cSql;
end if;
end;
CREATE OR REPLACE PROCEDURE SP_NEW_PROCEDURE1( )
RETURNS REFTABLE(employees)
LANGUAGE NZPLSQL AS
BEGIN_PROC
DECLARE
l_conditions varchar(1000);
p_rec RECORD;
BEGIN
FOR P_REC IN select empid, mgrid, empname, salary from employees where mgrid = 7
LOOP
l_conditions := 'insert into ' ||
REFTABLENAME ||
' VALUES (' ||
P_REC.EMPID ||
',' ||
P_REC.MGRID ||
',' ||
P_REC.EMPNAME ||
',' ||
P_REC.SALARY ||
' ) ; ' ;
execute immediate l_conditions;
l_conditions := ' ';
END LOOP;
RETURN REFTABLE;
END;
END_PROC;
When I run this:
select SP_NEW_PROCEDURE1()
I get the errors:
ERROR [01000] NOTICE: Error occurred while executing PL/pgSQL function SP_NEW_PROCEDURE1
ERROR [01000] NOTICE: line 24 at execute statement
ERROR [42S22] ERROR: Attribute 'DAN' not found
Can someone help whats wrong ...thanks
This has nothing do with the cursor itself, and everything to do with how you are building your dynamical SQL string.
When building dynamic SQL in a Netezza stored procedure, you can use the quote_ident and quote_literal helper functions to let the system know whether you are passing it a literal, or whether you are passing it an identifier. There is an example in the online documentation here. Essentially all they do is figure out the escaped quotation notation needed.
Since you are trying to put the values stored in the columns of your P_REC record into the VALUES part of an insert statement, you could use quote_literal like this:
CREATE OR REPLACE PROCEDURE SP_NEW_PROCEDURE1( )
RETURNS REFTABLE(employees)
LANGUAGE NZPLSQL AS
BEGIN_PROC
DECLARE
l_conditions varchar(1000);
p_rec RECORD;
BEGIN
FOR P_REC IN select empid, mgrid, empname, salary from employees where mgrid = 7
LOOP
l_conditions := 'insert into ' ||
REFTABLENAME ||
' VALUES (' ||
quote_literal(P_REC.EMPID) ||
',' ||
quote_literal(P_REC.MGRID) ||
',' ||
quote_literal(P_REC.EMPNAME) ||
',' ||
quote_literal(P_REC.SALARY ) ||
' ) ; ' ;
execute immediate l_conditions;
l_conditions := ' ';
END LOOP;
RETURN REFTABLE;
END;
END_PROC;
That being said, using a cursor to loop over records to insert a row one at a time is horribly inefficient in an MPP database like Netezza. Assuming this question is a follow-on from your question about an alternative to a recursive CTE to explore hierarchies, there's nothing wrong with looping in general, but try to avoid doing it record by record. Here is a version that will exploit the MPP nature of the system. For the record, if you are going to return your result set to a REFTABLE, then your only choice is Dynamic SQL.
CREATE OR REPLACE PROCEDURE SP_NEW_PROCEDURE1( )
RETURNS REFTABLE(employees)
LANGUAGE NZPLSQL AS
BEGIN_PROC
DECLARE
l_conditions varchar(1000);
p_rec RECORD;
BEGIN
-- FOR P_REC IN select empid, mgrid, empname, salary from employees where mgrid = 7
-- LOOP
l_conditions := 'insert into ' ||
REFTABLENAME ||
' select empid, mgrid, empname, salary from employees where mgrid = 7 ; ' ;
execute immediate l_conditions;
l_conditions := ' ';
-- END LOOP;
RETURN REFTABLE;
END;
END_PROC;
I suspect that you are building a query that is meant to insert a literal 'DAN' but which does not include the required quote marks, hence it is referencing DAN -- the optimiser is therefore trying to find an attribute of that name.
So the fix is to include the quotation marks when you build the SQL insert statement, or (preferably) to just use static SQL to insert the values instead of execute immediate.
When in doubt, always look at the data, as this would probably have been obvious to you if you inspected the value of l_conditions.
I have a table attribute_config with below columns:
table_name column_name key
Let us say is has below 2 rows
account accountphone accountnum
customer customernumber customerid
Key can be only accountnum or customerid.
I have to write code which will accept (i_accountnum,i_customerid) and;
fetch the respective values from columns mentioned in column_name in tables mentioned in table_name using the key in where condition.
For ex: select accountphone from account where accountnum = i_accountnum
select customernumber from customer where customerid = i_customerid
the complete query should be formed dynamically, whether to pass i_accountnum or i_customerid in the query also needs to be decided dynamically. if key - accountnum, i_accountnum will be passed to where condition.
I have been trying on these lines so far, this is not working, i know it is wrong.
declare
v_accountnum varchar2(20);
v_customerid varchar2(20);
v_attribute_value varchar2(20);
v_stmt varchar2(255);
begin
Account_Num := 'TestCustomer'; -- input to the function
v_customer_ref := 'TestAccount'; -- input to the function
for i in (Select * from attribute_config) loop
v_stmt := 'select ' || i.column_name || ' from ' || i.table_name ||' where ' || i.key|| ' = v_' || i.key;
execute immediate v_Stmt into v_attribute_value;
end loop;
end;
This will fix your code, but I do not see any advantage of using dynamic query when your code should accept 2 parameters(i_accountnum,i_customerid) - which is already static situation and fetch the relevant values, perhaps only in learning purposes.
declare
procedure fecth_values(i_accountnum account.accountnum%type,
i_customerid customer.customerid%type) return varchar2 is
v_attribute_value varchar2(20);
begin
for i in (select * from attribute_config) loop
execute immediate 'select ' || i.column_name || ' from ' ||
i.table_name || ' where ' || i.key || ' = ' || case when i.key = 'accountnum' then i_accountnum when i.key = 'customerid' then i_customerid end;
into v_attribute_value;
dbms_output.put_line(v_attribute_value);
end loop;
return null;
end;
begin
fecth_values(1, 1);
end;
Your where clause was wrong the i.key should be compared against the inputed values, not the 'v_' || i.key, which is undeclared when you execute your stmt.
There was a great answer from #ShannonSeverance on this question
Copying a row in the same table without having to type the 50+ column names (while changing 2 columns)
that showed how to dynamically copy a row within a table to the same table (changing the pk)
declare
r table_name%ROWTYPE;
begin
select *
into r
from table_name
where pk_id = "original_primary_key";
--
select pk_seq.nextval into r.pk_id from dual;
-- For 11g can use instead: r.pk_id := pk_seq.nextval;
r.fk_id := "new_foreign_key";
insert into table_name values r;
end;
I would like to apply this approach but within a function that is called each time from within an array of table names
So basically I can do the select using execute immediate - but how do I declare 'r'? Can I replace 'table_name' in the code with a variable that is passed to the function?
table(1)="Table1";
table(2)="Table2";
for t 1..table.count loop
CopyTableContacts(table(i));
end loop;
TIA
Mike
In the end, I have amended my function slightly
I now build 2 arrays
1 - the list of table names from the USER_TAB_COLUMNS table
2 - for each table in array1, I build a comma delimited list of the column names from the ALL_TAB_COLUMNS table
So I end up with 2 arrays (example...)
tableName(1) = 'MEMBER'
tableName(2) = 'SALARY'
tableColumns(1) = 'ID, SURNAME, SEX, DOB'
tableColumns(2) = 'ID, CURRENTSAL, BONUS, GRADE'
I then pass these 2 arrayvalues to my function and use some dynamic SQL during the looping of the tableName() array...
PROCEDURE CopyTableRow(inOrigMemNo NUMBER, inNewMemNo NUMBER, inTableName USER_TAB_COLUMNS.TABLE_NAME%TYPE, inTableString LONG) AS
selectString VARCHAR2(32000):=null;
newTableString LONG:=null;
insertTableString LONG:=null;
sqlResultCount NUMBER:=0;
BEGIN
/*CHECK IF THERE IS AT LEAST ONE ROW TO COPY*/
EXECUTE IMMEDIATE 'SELECT COUNT(*) FROM ' || inTableName || ' WHERE ID= ' || inOrigMemNo INTO sqlResultCount;
IF sqlResultCount > 0 THEN
/*BUILD INSERT STATEMENT FOR EACH ROW RETURNED*/
dbms_output.put_line('At least one row found on ' || inTableName || '(' || sqlResultCount || ')');
newTableString := REPLACE(inTableString, 'ID', 'REPLACE(ID, ID,' || inNewMemNo || ')');
selectString := 'SELECT ' || newTableString || ' FROM ' || inTableName || ' WHERE ID = ' || inOrigMemNo;
insertTableString := 'INSERT INTO ' || inTableName || '(' || inTableString || ') (' || selectString || ')';
END IF;
I am then left with an INSERT statement based on the table definition and value that I can execute
This seems to work fine and suits my current needs
NOTE - it only works if each table only has one row to copy. My next challenge is to cope with some of the tables returning multiple rows for the ID that require copying (which will be interesting seeing as I've painted myself into a non-cursor corner!)
Mike