Oracle Data Integrator
I've added a task to a copy of an IKM that will delete all of the rows in table A if they don't exist in table B. I can get the source table name using snpRef.getFrom(). This gives me the table name and its alias. Is there a way to get the alias name only so I can reference it in my WHERE clause.
Basically, in the snippet below I want to be able to replace SRC with the table alias. The way I'm doing it now, I had to go into each datastore that uses this and change the alias to "SRC". It works but there has to be a better way than making the alias all "SRC" in the datastore.
DELETE FROM <%=snpRef.getTable("L","TARG_NAME","A")%> TGT
WHERE NOT EXISTS (SELECT 'X'
FROM <%=odiRef.getFrom()%>
WHERE <%=odiRef.getColList("", "\tTGT.[COL_NAME]=SRC.[COL_NAME]", "\n\tand\t", "", "PK")%>)
Related
Suppose I have an existing schema (s1) with the following tables:
s1.table1
s1.table2
I would like to clone this schema to a new schema (s2) i.e.:
s2.table1
s2.table2
How can I achieve this?
The purpose is for testing without touching the original data.
I'm aware of the EXPORT command but it appears to be able to only export 1 table at a time.
You can copy a table's definition to a new table, without copying the data in the original table using CREATE TABLE ... LIKE:
CREATE TABLE s2.table1 (LIKE s1.table1 INCLUDING ALL);
CREATE TABLE s2.table2 (LIKE s1.table2 INCLUDING ALL);
You can then copy the rows from the original table if needed:
INSERT INTO s2.table1 SELECT * FROM s1.table1;
INSERT INTO s2.table2 SELECT * FROM s1.table2;
Assuming we have 2 hive tables created under the same HDFS file path.
I want to be able to drop a table WITH the HDFS files path, without corrupting the other table that's in the same shared path.
By doing the following:
drop table test;
Then:
hadoop fs -rm -r hdfs/file/path/folder/*
I delete both tables files, not just the one I've dropped.
In another post I found this solution:
--changing the tbl properties to to make the table as internal
ALTER TABLE <table-name> SET TBLPROPERTIES('EXTERNAL'='False');
--now the table is internal if you drop the table data will be dropped automatically
drop table <table-name>;
But I couldn't get passed the ALTER statement as I got a permission denied error (User does not have [ALTER] privilege on table)
Any other solution?
If you have two tables using the same location, then all files in this location belongs to both tables, does not matter how they were created.
Say if you have table1 with location hdfs/file/path/folder and table2 with the same location hdfs/file/path/folder and you inserted some data into table1, files are created and they are being read if you select from table2, and vice-versa: if you insert into table2, new files will be accessible from table1. This is because table data is being stored in the location, no matter how you put the files inside that location. You can insert data into table using SQL, put files into location manually, etc.
Each table or partition has it's location, you cannot specify files separately.
For better understanding, read also this answer with examples about multiple tables on top of the same location: https://stackoverflow.com/a/54038932/2700344
I wrote a UNIX batch job which updates a table with some "where" conditions. Before updating those records, i need to take the backup (insert statements) of the records that is returned with the "where conditions" and store it in ".dat" file. Could you please help on this???
The most straightforward way to create a backup of the table would be to use a create table statement using the where condition(s) of your update statement. For example, let's take a sample update statement:
UPDATE sometable
SET field1 = 'value'
WHERE company = 'Oracle'
This update would update the field1 column of every row where the company name is Oracle. You could create a backup of sometable by issuing the following command:
CREATE TABLE sometable_backup AS (SELECT * FROM sometable WHERE company = 'Oracle');
This will create a table called sometable_backup that will contain all of the rows that match the where clause of the update.
You can then use Data Pump or another utility to create an export .dat file of that specific table. You can use that .dat file to import into other databases.
Based on the Hive doc below:
Rename Table
ALTER TABLE table_name RENAME TO new_table_name;
This statement lets you change the name of a table to a different name.
As of version 0.6, a rename on a managed table moves its HDFS location as well. (Older Hive versions just renamed the table in the metastore without moving the HDFS location.)
Is there any way to rename a table without changing the location?
Yeah we can do that. You just need to follow below three commands in sequence.
Lets say you have a external table test_1 in hive. And you want to rename it test_2 which should point test_2 location not test_1. Then you need to convert this table into Managed table using below command.
test_1 -> pointing to test_1 location
ALTER TABLE db_name.test_1 SET TBLPROPERTIES('EXTERNAL'='FALSE');
Rename the table name.
ALTER TABLE db_name.test_1 RENAME TO db_name.test_2;
Again convert the managed table after renaming to external table.
ALTER TABLE db_name.test_2 SET TBLPROPERTIES('EXTERNAL'='TRUE');
db_name.test_2 table will point the test_2 location. If we do it without making the managed table it will point the test_1 location.
As of Hive 2.2.0 a managed table's HDFS location is moved only if the table is created without a LOCATION clause and under its database directory.Link
ALTER TABLE does not follow the databasename.tablename syntax in Hive like it does in CREATE or SELECT.
Mention the databasename first and then run alter table statement.
syntax as below
USE databasename;
ALTER TABLE old_tablename RENAME TO new_tablename;
Here is the command executed
ALTER TABLE old_ratings RENAME TO ratings;
I have to alter many columns in a Vertica table, and decided to drop the table altogether and create a new one. I also need 'undo' scripts ready to revert back the changes if needed (using mybatis migrations).
This is my plan:
rename mytable to mytable_backup
create mytable
create projection mytable_super (as select from mytable)
--undo
drop mytable if exists
rename mytable_backup to mytable
The original mytable was also created with a projection. The above script gives an error saying projection already exists.
DBCException: SQL Error [4482] [42710]: [Vertica][VJDBC](4482) ROLLBACK: Projection with base name "mytable_super" already exists
I believe when I rename the original table, the underlying projection is not being renamed.
What is the best way to rename a table with projections in vertica? Or what is the best way to back up a table and revert back?
You'll need to rename the projections as well.
alter projection mytable_super rename to mytable_super_backup;
For renaming a table, the above answer is the one.
One way to rename a projection, would be to get the projection inside a SQL file. For example:
select CONCAT(CONCAT(projection_schema,'.'),projection_name) from projections where projection_schema like '%one_table%'
Then modify it into the following SQL and execute it (and don't forget to run refresh):
https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/AdministratorsGuide/Projections/UpdatingProjectionsUsingRefresh.htm
Once you have the SQL, you can do \i /path/of/sql (inside vertica shell) or you can /opt/vertica/bin/vsql -f /path/to/sql/that.sql -U vertica_user -w vertica_passwd