I am trying to insert data from HDFS into a teradata table using TPT. I have read the documentation and tried to use DDL Operator to insert but I am unable to figure out a way to use INSERT OPERATOR along with DDL OPERATOR. Here is my attempt to do it.
DESCRIPTION 'Load a Teradata table from a file'
(
SET FileReaderHadoopHost = #HadoopHost;
STEP MAIN_STEP
(
APPLY
('INSERT INTO TABLE ' || #TargetTable || 'SELECT * FROM' || OPERATOR( $FILE_READER [#ReaderInstances] ) ' ; '),
TO OPERATOR ($DDL);
);
);
In order to use $FILE_READER I need to use $INSERT operator. How do I do it?
Related
Oracle - Alter all table column names with trim of white space in between names
For suppose column names before alter :
Home number
Mobile number
Local number
After alter column names shall be :
Homenumber
Mobilenumber
Localnumber
I've tried this way: but unable to crack:
UPDATE SA_VW_PHONENUMBER TN SET TN.Column_Name = TRIM (TN.Column_Name);
Fully automatic way
Use this cursor based DDL hacking - statement concat.
BEGIN
FOR alters IN
(
SELECT
'ALTER TABLE "'||table_name||'" RENAME COLUMN "'||column_name||
'" TO "'||replace(cols.column_name,' ','')||'"' sql_stmt
FROM all_tab_cols cols
WHERE REGEXP_LIKE(column_name,'[[:space:]]')
AND owner = user --Add real schema name here
ORDER BY 1
) LOOP
DBMS_OUTPUT.PUT_LINE ( alters.sql_stmt ||';') ;
EXECUTE IMMEDIATE alters.sql_stmt;
END LOOP;
END;
/
If you want to use the safe way
As I know you cannot perform a DDL as a dynamic SQL, so you cannot pass variables to the ALTER TABLE command, but here is what you can do instead of that.
Selecting the occurences:
SELECT table_name,column_name,replace(cols.column_name,' ','') as replace_name
FROM all_tab_cols
WHERE REGEXP_LIKE(column_name,'[[:space:]]');
Use the ALTER TABLE DDL command:
alter table T_TABLE rename column "COLUMN SPACE" TO "COLUMNNOSPACE";
Try the REPLACE function
UPDATE SA_VW_PHONENUMBER TN SET TN.Column_Name = REPLACE(TN.Column_Name,' ','')
I am trying to add partition to a hive table (partitioned by date)
My problem is that the date needs to be fetched from another table.
My query looks like :
ALTER TABLE my_table ADD IF NOT EXISTS PARTITION(server_date = (SELECT max(server_date) FROM processed_table));
When i run the query hive throws the following error:
Error: Error while compiling statement: FAILED: ParseException line 1:84 cannot recognize input near '(' 'SELECT' 'max' in constant (state=42000,code=40000)
Hive does not allow to use functions/UDF's for the partition column.
Approach 1:
To achieve this you can run the first query and store the result in one variable and then execute the query.
server_date=$(hive -e "set hive.cli.print.header=false; select max(server_date) from processed_table;")
hive -hiveconf "server_date"="$server_date" -f your_hive_script.hql
Inside your script you can use the following statement:
ALTER TABLE my_table ADD IF NOT EXISTS PARTITION(server_date =${hiveconf:server_date});
For more information on the hive variable substitution, you can refer link
Approach 2:
In this approach, you will need to create a temporary table if the partition data you are expecting is already not loaded in any other partitioned table.
Considering your data doesn't have the server_date column.
Load the data into temporary table
set hive.exec.dynamic.partition=true;
Execute the below query:
INSERT OVERWRITE TABLE my_table PARTITION (server_date)
SELECT b.column1, b.column2,........,a.server_date as server_date FROM (select max(server_date) as server_date from ) a, my_table b;
I am passing 100 table_names in the IN operator as strings, but I am getting numeric overflow error due to too many operands.
Is there a way where I can use something else besides IN ?
set serveroutput on
DECLARE
...
BEGIN
FOR r IN
(
SELECT table_name, column_name
FROM all_tab_columns
WHERE table_name IN (...100 strings)
)
AND data_type = 'NUMBER'
ORDER BY table_name, column_id
)
LOOP
execute immediate 'SELECT COUNT("' || r.column_name || '")
,COUNT(nvl2("' || r.column_name || '", NULL, 1))
FROM "' || r.table_name || '"'
INTO not_null_count, null_count;
DBMS_OUTPUT.PUT_LINE(..)
Note: For variables I am using PLS_Integer.
The suggested action for ORA-01426 is "reduce the operands". This doesn't mean reduce the number of operands. It means you're trying to put too large a number into a variable. So shrink the number, or enlarge the variable.
You say:
"for variables I am using PLS_Integer"
So, if you have a large table, and by large I mean more than 2,147,483,647 rows, you will get a numeric overflow. Because PLS_INTEGER is a 32-bit data type.
If this is your scenario then you need to declare your variables of data type INTEGER instead (or NUMBER(38,0)).
As #BobJarvis points out, PLS_INTEGER is optimized for hardware arithmetic. So the general advice would be to use it for counting type operations. However, your case simply requires a variable to hold the output of a SQL count() operation, so I don't think there will be any difference in efficiency.
I believe the limit on 'IN' clause is 1000 strings and not 100 strings. To debug:
a.) Try running your implicit cursor query in SQL.
b.) If it works fine then run the query in execute immediate after substituting the column name.
Also , try increasing the size of your not_null_count and null_count variables.
Hope it Helps
Vishad
some other possible solutions
use a temp table - populate it with the table names to filter join to it.
create a global array type
create type table_of_varchar2 is table of varchar2(30)
populate the array and filter using table_name member of arr_tables_list
Is there a way where I can use something else besides IN ?
Consider using a cursor instead.
I want to write a SQL that resets all colums in a table back to default, except a couple of colums like the primary key.
I just want to name the colums NOT to update, and reset everything else. There are quite many colums in the table, and I dont whant to write:
update my_table set column1 = DEFAULT, column2 = DEFAULT, ... where ...
for all colums, since there are quite many.
Any ideas? I am using Oracle
I don't think there is a procedure to do what you want, but if your only problem is the heavy burden of writing the SQL, you can automate that with ALL_TAB_COLUMNS view. You can improve the idea for your needs:
select
'update ' || TABLE_NAME ||
'set ' ||
COLUMN_NAME || ' = DEFAULT'
from ALL_TAB_COLUMNS
where
table_name = <YOUR_TABLE>
I would like to do an INSERT / SELECT, this means INSERT in the TARGET_TABLE the records of the SOURCE_TABLE, with this assumption:
The SOURCE and the TARGET table have only a SUBSET of common columns, this means in example:
==> The SOURCE TABLE has ALPHA, BETA and GAMMA columns;
==> The TARGET TABLE has BETA, GAMMA and DELTA columns.
What is the most efficient way to produce INSERT / SELECT statements, respecting the assumption that not all the target columns are present in the source table?
The idea is that the PL/SQL script CHECKS the columns in the source table and in the target table, makes the INTERSECTION, and then produces a dynamic SQL with the correct list of columns.
Please assume that the columns present in the target table, but not present in the source table, have to be left NULL.
I wish to extract the data from SOURCE into a set of INSERT statements for later insertion into the TARGET table.
You can assume that the TARGET table has more columns than the SOURCE table, and that all the columns in the SOURCE table are present in the TARGET table in the same order.
Thank you in advance for your useful suggestions!
In Oracle, You can get common columns with this SQL query:
select column_name
from user_tab_columns
where table_name = 'TABLE_1'
intersect
select column_name
from user_tab_columns
where table_name = 'TABLE_2'
Then you iterate a cursor with the mentioned query to generate a comma separated list of all values returned. Put that comma separated string into a varchar2 variable named common_fields. Then, you can:
sql_sentence := 'insert into TABLE_1 (' ||
common_fields ||
') select ' ||
common_fields ||
' from TABLE_2';
execute immediate sql_sentence;