I have a question regarding a unified insert query against tables with different data
structures (Oracle). Let me elaborate with an example:
tb_customers (
id NUMBER(3), name VARCHAR2(40), archive_id NUMBER(3)
)
tb_suppliers (
id NUMBER(3), name VARCHAR2(40), contact VARCHAR2(40), xxx, xxx,
archive_id NUMBER(3)
)
The only column that is present in all tables is [archive_id]. The plan is to create a new archive of the dataset by copying (duplicating) all records to a different database partition and incrementing the archive_id for those records accordingly. [archive_id] is always part of the primary key.
My problem is with select statements to do the actual duplication of the data. Because the columns are variable, I am struggling to come up with a unified select statement that will copy the data and update the archive_id.
One solution (that works), is to iterate over all the tables in a stored procedure and do a:
CREATE TABLE temp as (SELECT * from ORIGINAL_TABLE);
UPDATE temp SET archive_id=something;
INSERT INTO ORIGINAL_TABLE (select * from temp);
DROP TABLE temp;
I do not like this solution very much as the DDL commands muck up all restore points.
Does anyone else have any solution?
How about creating a global temporary table for each base table?
create global temporary table tb_customers$ as select * from tb_customers;
create global temporary table tb_suppliers$ as select * from tb_suppliers;
You don't need to create and drop these each time, just leave them as-is.
You're archive process is then a single transaction...
insert into tb_customers$ as select * from tb_customers;
update tb_customers$ set archive_id = :v_new_archive_id;
insert into tb_customers select * from tb_customers$;
insert into tb_suppliers$ as select * from tb_suppliers;
update tb_suppliers$ set archive_id = :v_new_archive_id;
insert into tb_suppliers select * from tb_suppliers$;
commit; -- this will clear the global temporary tables
Hope this helps.
I would suggest not having a single sql statement for all tables and just use and insert.
insert into tb_customers_2
select id, name, 'new_archive_id' from tb_customers;
insert into tb_suppliers_2
select id, name, contact, xxx, xxx, 'new_archive_id' from tb_suppliers;
Or if you really need a single sql statement for all of them at least precreate all the temp tables (as temp tables) and leave them in place for next time. Then just use dynamic sql to refer to the temp table.
insert into ORIGINAL_TABLE_TEMP (SELECT * from ORIGINAL_TABLE);
UPDATE ORIGINAL_TABLE_TEMP SET archive_id=something;
INSERT INTO NEW_TABLE (select * from ORIGINAL_TABLE_TEMP);
Related
I am using Toad for oracle 12c. I need to copy a table and data (40M) from one shcema to another (prod to test). However there is an unique key(not the PK for this table) called record_Id col which has something data like this 3.000*******19E15. About 2M rows has same numbers(I believe its because very large number) which are unique in prod. When I try to copy it violets the unique key of that col. I am using toad "export data to another schema" function to copy the data.
when I execute query in prod
select count(*) from table_name
OR
select count(distinct(record_id) from table_name
Both query gives the exact same numbers of data.
I don't have DBA permission. How do I copy all data without violating unique key of the table.
Thanks in advance!
You can use UPSERT for decisional INSERT or UPDATE or you may write small procedure for this.
you may consider to use NOT EXISTS, but your data is big and it might not be resource efficient.
insert into prod_tab
select * from other_tab t1 where NOT exists (
select 1 from prod_tab t2 where t1.id = t2.id
);
In Oracle you can use a MERGE query for that.
The following query proceeds as follows for each data row :
if the source record_id does not yet exist in the target table, a new record is inserted
else, the existing record is updated with source values
For the sake of the example, I assumed that there are two other columns in the table : column1 and column2.
MERGE INTO target_table t1
USING (SELECT * from source_table t2)
ON (t1.record_id = t2.record_id)
WHEN MATCHED THEN UPDATE SET
t1.column1 = t2.column1,
t1.column2 = t2.column2
WHEN NOT MATCHED THEN INSERT
(record_id, column1, column2) VALUES (t2.record_id, t2.column1, t2.column2)
I have to create a procedure which searches any recently added records and if there are then move them to ARCHIVE table.
This is my statement which filters recently added records
SELECT
CL_ID,
CL_NAME,
CL_SURNAME,
CL_PHONE,
VEH_ID,
VEH_REG_NO,
VEH_MODEL,
VEH_MAKE_YEAR,
WD_ID,
WORK_DESC,
INV_ID,
INV_SERIES,
INV_NUM,
INV_DATE,
INV_PRICE
FROM
CLIENT,
INVOICE,
VEHICLE,
WORKS,
WORKS_DONE
WHERE
Client.CL_ID=Invoice.INV_CL_ID and
Invoice.INV_CL_ID = Client.CL_ID and
Client.CL_ID = Vehicle.VEH_CL_ID and
Vehicle.VEH_ID = Works_Done.WD_VEH_ID and
Works_done.WD_INV_ID = Invoice.INV_ID and
WORKS_DONE.WD_WORK_ID = Works.WORK_ID and
Works_done. Timestamp >= sysdate -1;
You may need something like this (pseudo-code):
create or replace procedure moveRecords is
vLimitDate timestamp := systimestamp -1;
begin
insert into table2
select *
from table1
where your_date >= vLimitDate;
--
delete table1
where your_date >= vLimitDate;
end;
Here are the steps I've used for this sort of task in the past.
Create a global temporary table (GTT) to hold a set of ROWIDs
Perform a multitable direct path insert, which selects the rows to be archived from the source table and inserts their ROWIDs into the GTT and the rest of the data into the archive table.
Perform a delete from the source table, where the source table ROWID is in the GTT of rowids
Issue a commit.
The business with the GTT and the ROWIDs ensures that you have 100% guaranteed stability in the set of rows that you are selecting and then deleting from the source table, regardless of any changes that might occur between the start of your select and the start of your delete (other than someone causing a partitioned table row migration or shrinking the table).
You could alternatively achieve that through changing the transaction isolation level.
O.K. may be something like this...
The downside is - it can be slow for large tables.
The upside is that there is no dependence on date and time - so you can run it anytime and synchronize your archives with live data...
create or replace procedure archive is
begin
insert into archive_table
(
select * from main_table
minus
select * from archive_table
);
end;
I'm looking for the best way to change a data type of a column in a populated table. Oracle only allows changing of data type in colums with null values.
My solution, so far, is a PLSQL statement which stores the data of the column to be modified in a collection, alters the table and then iterates over the collection, restoring the original data with data type converted.
-- Before: my_table ( id NUMBER, my_value VARCHAR2(255))
-- After: my_table (id NUMBER, my_value NUMBER)
DECLARE
TYPE record_type IS RECORD ( id NUMBER, my_value VARCHAR2(255));
TYPE nested_type IS TABLE OF record_type;
foo nested_type;
BEGIN
SELECT id, my_value BULK COLLECT INTO foo FROM my_table;
UPDATE my_table SET my_value = NULL;
EXECUTE IMMEDIATE 'ALTER TABLE my_table MODIFY my_value NUMBER';
FOR i IN foo.FIRST .. foo.LAST
LOOP
UPDATE my_table
SET = TO_NUMBER(foo(i).my_value)
WHERE my_table.id = foo(i).id;
END LOOP;
END;
/
I'm looking for a more experienced way to do that.
The solution is wrong. The alter table statement does an implicit commit. So the solution has the following problems:
You cannot rollback after alter the alter table statement and if the database crashes after the alter table statement you will loose data
Between the select and the update users can make changes to the data
Instead you should have a look at oracle online redefinition.
Your solution looks a bit dangerous to me. Loading the values into a collection and subsequently deleting them fom the table means that these values are now only available in memory. If something goes wrong they are lost.
The proper procedure is:
Add a column of the correct type to the table.
Copy the values to the new column.
Drop the old column.
Rename the new column to the old columns name.
In PL SQL, I'm writing a stored procedure that uses a DB link:
CREATE OR REPLACE PROCEDURE Order_Migration(us_id IN NUMBER, date_id in DATE)
as
begin
INSERT INTO ORDERS(order_id, company_id)
SELECT ORDER_ID_SEQ.nextval, COMPANY_ID
FROM ORDERS#SOURCE
WHERE USER_ID = us_id AND DUE_DATE = date_ID;
end;
It takes all orders done on a certain day, by a certain user and inserts them in the new database. It calls a sequence to makes sure there are no repeat PKs on the orders, and it works well.
However, I want the same procedure to do a second INSERT into another table that has order_id as a foreign key. So I need to add all the order_id's just created, and the data from SOURCE that matches:
INSERT INTO ORDER_COMPLETION(order_id, completion_dt)
SELECT ????, completion_dt
FROM ORDER_COMPLETION#SOURCE
How can I keep track of which order_id that was just created matches up to the one whose data I need to pull from the source database?
I looked into making a temporary table, but you can't create those in a procedure.
Other info: I'll be calling this procedure from a C# app I'm writing
I'm not sure that I follow the question. If there is an ORDERS table and an ORDER_COMPLETION table in the remote database, wouldn't there be some key on the source system that related those two tables? If that key is the ORDER_ID, why would you want to re-assign that key in your procedure? Wouldn't you want to maintain the ORDER_ID from the source system?
If you do want to re-assign the ORDER_ID locally, I would tend to think that you'd want to do something like
CREATE OR REPLACE PROCEDURE order_migration( p_user_id IN orders.user_id%type,
p_due_date IN orders.due_date%type )
AS
TYPE order_rec IS RECORD( new_order_id NUMBER,
old_order_id NUMBER,
company_id NUMBER,
completion_dt DATE );
TYPE order_arr IS TABLE OF order_rec;
l_orders order_arr;
BEGIN
SELECT order_id_seq.nextval,
o.order_id,
o.company_id,
oc.completion_dt
BULK COLLECT INTO l_orders
FROM orders#source o,
order_completion#source oc
WHERE o.order_id = oc.order_id
AND o.user_id = p_user_id
AND o.due_date = p_due_date;
FORALL i IN l_orders.FIRST .. l_orders.LAST
INSERT INTO orders( order_id, company_id )
VALUES( l_orders(i).new_order_id, l_orders(i).company_id );
FORALL i IN l_orders.FIRST .. l_orders.LAST
INSERT INTO order_completion( order_id, completion_dt )
VALUES( l_orders(i).new_order_id, l_orders(i).completion_dt );
END;
You could also do a single FOR loop with two INSERT statements rather than two FORALL loops. And if you're pulling a lot of data each time, you probably want to pull the data in chunks from the remote system by adding a loop and a LIMIT to the BULK COLLECT
There must be some link between the rows in ORDERS#SOURCE and ORDERS, and between ORDERS#SOURCE and ORDER_COMPLETION#SOURCE, so can you not use a join?
Something like:
INSERT INTO ORDER_COMPLETION(order_id, completion_dt)
SELECT o.order_id, ocs.completion_dt
FROM ORDER_COMPLETION#SOURCE ocs
JOIN ORDERS o ON o.xxx = ocs.xxx
I create a table in Oracle 11g with the default value for one of the columns. Syntax is:
create table xyz(emp number,ename varchar2(100),salary number default 0);
This created successfully. For some reasons I need to create another table with same old table structure and data. So I created a new table with name abc as
create table abc as select * from xyz.
Here "abc" created successfully with same structure and data as old table xyz. But for the column "salary" in old table "xyz" default value was set to "0". But in the newly created table "abc" the default value is not set.
This is all in Oracle 11g. Please tell me the reason why the default value was not set and how we can set this using select statement.
You can specify the constraints and defaults in a CREATE TABLE AS SELECT, but the syntax is as follows
create table t1 (id number default 1 not null);
insert into t1 (id) values (2);
create table t2 (id default 1 not null)
as select * from t1;
That is, it won't inherit the constraints from the source table/select. Only the data type (length/precision/scale) is determined by the select.
The reason is that CTAS (Create table as select) does not copy any metadata from the source to the target table, namely
no primary key
no foreign keys
no grants
no indexes
...
To achieve what you want, I'd either
use dbms_metadata.get_ddl to get the complete table structure, replace the table name with the new name, execute this statement, and do an INSERT afterward to copy the data
or keep using CTAS, extract the not null constraints for the source table from user_constraints and add them to the target table afterwards
You will need to alter table abc modify (salary default 0);
new table inherits only "not null" constraint and no other constraint.
Thus you can alter the table after creating it with "create table as" command
or you can define all constraint that you need by following the
create table t1 (id number default 1 not null);
insert into t1 (id) values (2);
create table t2 as select * from t1;
This will create table t2 with not null constraint.
But for some other constraint except "not null" you should use the following syntax
create table t1 (id number default 1 unique);
insert into t1 (id) values (2);
create table t2 (id default 1 unique)
as select * from t1;