I need to do an Oracle data migration from 11g to 12c where schema changes are abundant. I have an excel sheet which describes all the schema changes. Excel sheet has the columns for 'old_table_name', 'old_column_name', 'old_value' and same for the new tables. Some values can be directly copied to the new table and some cannot be done that way.
For example I have to transform the old column value when it is moved to the new table. Some transformation are complex and they cannot be simply mapped. They should be transformed by joining with other tables in the old database. I was trying the Talend Open Studio Data Integration tool for this and found it is a bit complex to go ahead with that tool in my case. Does anyone have an idea of getting this done using Talend or any other tool? What is the ideal approach when doing a migration like this? I have included a sample of the excel sheet below which only has simple transformations.
The kind of converions shown in the spreadsheet can all be performed on the table itself using rename statements and/or basic ddl and dml statements. I would load the old table into the new database and perform these statement on the table.
alter table
old_table_one
rename to
new_table_one;
alter table
new_table_one
rename column
old_col_one
to
new_col_one;
update new_table_one
set new_col_one = 'A_NEW'
where new_col_one = 'A';
etc.
Related
Initial Note: Created Model by choosing to Import the Data Dictionary using one of my Connections and then choosing the Schema and lastly the Tables for which I want to model.
After making changes within Oracle SQL Developer Data Modeler how can I commit the changes made in the newly created relational model back to the database.
I can manually parse thru the generated DDL but that seems like unnecessary work. I attempted to use the 'Synchronize with Data Dictionary' option however when I went back to my tables within my schema they were not altered/updated in any way. No Primary Keys... Foreign Keys, Indexes or any other of the DDL actions I created in the model were seen in my database. What am I missing here?
I really thought the Synchronize options where what I should be using.
We will never commit changes to the database.
You'll do the compare, review the delta DDL, and then if you think it's good - load it up in SQLcl, SQL Developer, or SQLPlus to run.
It's not that we don't trust you to do the review part first, but also, it'd be just too easy to muck up a database if you hit the wrong button. Especially as some table structural changes could result in data loss.
i am working on a large database ,how do i Export some database tables without having dba privileges .do i have to copy the structures of the tables and using spool command to get the data in a text file then create the tables and inserting data from the text file?
One of the methods would be to install Oracle SQL Developer and export the required table structures and data using the wizard.
Here is the link to a tutorial which can guide you if you go with this option.
http://www.oracle.com/webfolder/technetwork/tutorials/obe/db/sqldev/r30/SQLdev3.0_Import_Export/sqldev3.0_import_export.htm
A second option would be to use SQL Loader to load data in your target tables. But for that you will have to first create the data structures on your target schema and spool the data from your source tables in CSV (comma separated values) or any other eligible format.
Here is a link for SQL Loader.
http://docs.oracle.com/cd/B28359_01/server.111/b28319/ldr_concepts.htm
A third option would be that you create the table structures on the target schema and generate the insert statements from the source schema using a script. Here is a link to such an example.
https://pandazen.wordpress.com/2008/08/18/generate-insert-statement-script-to-extract-data-from-oracle-table/
I would recommend going with the SQL Developer option since it is relatively simple.
I want to create a table (lets say table_copy) which has same columns as other table (lets call it table_original) in Oracle database, so the query will be like this :
create table table_copy as (select * from table_original where 1=0);
This will create a table, but the constraints of table_original are not copied to table_copy, so what should be done in this case?
Only NOT NULL constraints are copied using Create Table As Syntax (CTAS). Others should be created manually.
You might however query data dictionary view to see the definitions of constraints and implement them on your new table using PL/SQL.
The other tool that might be helpful is Oracle Data Pump. You could import the table using REMAP_TABLE option specifying the name for the new table.
Use a database tool to extract the DDL needed for the constraints (SQL Developer does the job). Edit the resulting script to match the name of the new class.
Execute the script.
If you need to do this programmatically you can use a statement like this:
DBMS_METADATA.GET_DDL('TABLE','PERSON') from DUAL;
Background: My team has an etl job that updates an aggregate table. Each row contains data for a particular date, but this row can and will get updated after the row date (which means any row can contain data from multiple jobs). This ETL job missed some data for one day last week and now I need to backfill it.
Problem: I have the missing data, and what I was planning on doing was dumping that data into a temporary table and then merging it with the agg table. That way I can deal with whether the ETL job already contains a row for that data (update) or whether a new row needs to be added (insert), but I don't have sufficient permissions to create a temp table, and I'd prefer not to involve the DBA.
Question: Can I do an insert/update sort of behavior without creating a temporary table (this is Oracle SQL by the way).
Edit: The data is coming from a tsv file.
Why do you want to avoid involving the DBA? The DBA should have full knowledge of what's going on in the database, as they are ultimately responsible for the condition of the data within it. So you shouldn't be playing sneaky commando with them.
As you have a file of missing data, the easiest way to present it to the database is with an external table. This requires the creation of the table and probably a directory object as well. You will need the DBA's help with this task.
The only way to avoid creating database objects is to convert your TSV file into a series of DML statements. An IDE which supports regex and/or records macros will prove invaluable here. I like TextPad; other editors are available.
The DML statement for doing upserts in Oracle is the MERGE statement. The one thing you need to watch for is recency. Your missing data comes from last week. If a row exists it may have have been added or amended in the intervening period. You must write your MERGE statement so it does not overwrite more recent data with the older stuff. Hopefully your table has useful metadata columns such as DATE_CREATED and LAST_UPDATED.
I've created a relational model in Oracle SQL Developer Data Modeler.
I want to add predefined\static data that should exist in the initial clean database: enum values, fixed lists( for example: contries ) using modeler. My goal is to receive script using "DDL File Editor" tool which contains not only "create table" commands and so on, but also "inserts" with initial data.
I there any way to do this?
What might be the easiest way would be to put the DML into the AFTER CREATE tab under Scripts for each table - and to make sure it's included in the DDL script.