I use WITH table_name AS (select...;) command in SQL Developer to create temporary table and use that temp table in following queries. What is the similar command in Hadoop Hive?
Using SQL assistant User interface on Hadoop Hive.
I tried the following example, which gives error-
Create table Failed,80:
CREATE TEMPORARY TABLE temp1(col1 string);
CREATE TEMPORARY TABLE temp2 AS Select * from table_name;
Maybe you must write case sensitive like this:
CREATE TEMPORARY TABLE temp1(col1 STRING);
The same CTE as in MySQL:
with your_table as (
select 'some value' --from etc etc
)
select * from your_table;
Another example: https://stackoverflow.com/a/54960324/2700344
Hive CTE Official docs
Related
I have few databases and all the databases have the same tables (i.e. table names). Now i want to get the DDL of the table with different schema.
Use the dbms_metadata package to get the DDL of any object of the DB.
SELECT
DBMS_METADATA.GET_DDL('<Object type>', '<Object name>', '<object schema>')
FROM
DUAL; -- How to
SELECT
DBMS_METADATA.GET_DDL('TABLE', 'MY_TABLE', 'MY_SCHEMA')
FROM
DUAL; -- In your case use something like this
Also, You can format the output using dbms_metadata.set_transform_param.
See Oracle documentation for more information on it.
Cheers!!
I have a table with a lots of columns with BLOB type and I need to change it to nvarchar2.
So, to change type I can use following script:
alter table AUDIT_LOG
modify
(
column_name type_name,
column_name2 type_name2
-- etc
);
And to get all columns with given datatype I can use the following:
select column_name, 'NVARCHAR2(4000)'
from all_tab_columns
where table_name = 'TAB_NAME' and data_type = 'BLOB';
But how to join this two scripts into one?
You cannot do DML and DDL operation together in same query. You have to use dynamic SQL in a PL/SQL block
Create a variable and generate the whole alter table query in it.
Execute Immidiate
Refer this and I am sure you will be able to add rest of the logic as per your requirement.
http://www.java2s.com/Tutorial/Oracle/0440__PL-SQL-Statements/EXECUTEIMMEDIATEdynamicsqltoaltersession.htm
I am interested in loading specific columns into a table created in Hive.
Is it possible to load the specific columns directly or I should load all the data and create a second table to SELECT the specific columns?
Thanks
Yes you have to load all the data like this :
LOAD DATA [LOCAL] INPATH /Your/Path [OVERWRITE] INTO TABLE yourTable;
LOCAL means that your file is on your local system and not in HDFS, OVERWRITE means that the current data in the table will be deleted.
So you create a second table with only the fields you need and you execute this query :
INSERT OVERWRITE TABLE yourNewTable
yourSelectStatement
FROM yourOldTable;
It is suggested to create an External Table in Hive and map the data you have and then create a new table with specific columns and use the create table as command
create table table_name as select statement from table_name;
For example the statement looks like this
create table employee as select id as id,emp_name as name from emp;
Try this:
Insert into table_name
(
#columns you want to insert value into in lowercase
)
select columns_you_need from source_table;
In postgresql, one can have all tables names by running the following query
SELECT table_name FROM information_schema.tables WHERE table_schema='public';
Is there something similar in monetdb to have the list of tables ?
I finally find this query
select tables.name from tables where tables.system=false ;
If you are using mclient, in the command line you could simply use the command "\d" to list all the tables in the current database.
sql> \d
TABLE sys.table1
TABLE sys.table2
TABLE sys.table3
Additionally, use the command "\d tablename" to view the metadata of a table - equivalent to "show create table" in mysql.
I'm using Amazon's Elastic MapReduce and I have a hive table created based on a series of log files stored in Amazon S3 and split in folders by day like so:
data/day=2011-09-01/log_file.tsv
data/day=2011-09-02/log_file.tsv
I am currently trying to create an additional table which filters out some unwanted activity in these log files but I can't figure out how to do this and keep getting errors such as:
FAILED: Error in semantic analysis: need to specify partition columns because the destination table is partitioned.
If my initial table create statement looks something like this:
CREATE EXTERNAL TABLE IF NOT EXISTS table1 (
... fields ...
)
PARTITIONED BY ( DAY STRING )
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
LOCATION 's3://bucketname/data/';
That initial table works fine and I've been able to query it with no problems.
How then should I create a new table that shares the structure of the previous one but simply filters out data? This doesn't seem to work.
CREATE EXTERNAL TABLE IF NOT EXISTS table2 LIKE table1;
FROM table1
INSERT OVERWRITE TABLE table2
SELECT * WHERE
col1 = '%somecriteria%' AND
more criteria...
;
As I've stated above, this returns:
FAILED: Error in semantic analysis: need to specify partition columns because the destination table is partitioned.
Thanks!
This always works for me:
CREATE EXTERNAL TABLE IF NOT EXISTS table2 LIKE table1;
INSERT OVERWRITE TABLE table2 PARTITION (day) SELECT col1, col2, ..., day FROM table1;
ALTER TABLE table2 RECOVER PARTITIONS;
Notice that I've added 'day' as a column in the SELECT statement. Also notice that there is an ALTER TABLE line which is necessary for Hive to become aware of the partitions that were newly created in table2.
I have never used the like option.. so thanks for showing me that. Will that actually create all of the partitions that the first table has as well? If not, that could be the issue. You could try using dynamic partitions:
create external table if not exists table2 like table1;
insert overwrite table table2 partition(part) select col1, col2 from table1;
Might not be the best solution, as I think you have to specify your columns in the select clause (as well as the partition column in the partition clause).
And, you must turn on dynamic partitioning.
I hope this helps.