I am using latest AWS Hive version 0.13.0.
FAILED: ParseException: cannot recognize input near 'exchange' 'string' ',' in column specification
I am getting the above error when I run the below(create table) query.
CREATE EXTERNAL TABLE test (
foo string,
exchange string,
bar string) ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n'
STORED AS TEXTFILE
LOCATION '/home/hadoop/test/';
If I rename the exchange like 'xch' it creates table successfully. Any reason?
You are getting an error because exchange is a keyword used to move the data in a partition from a table to another table that has the same schema but does not already have that partition for details view Hive Language Manual and HIVE-4095.
Try it like this after create statement
LOAD DATA LOCAL INPATH '/home/cloudera/Amit/xyz.csv' OVERWRITE INTO TABLE tabele_name;
Related
im trying to create a simple external table in hive cli and I got an error while creating a partition. I have searched on google, but I was not able to get a proper result. Can you please help
hive (sampledb)> create external table externalhive(id int,name varchar(100),age tinyint,city varchar(100),state varchar(100)) partitioned by (year string)
> row format delimited fields terminated by '/t' stored as textfile location '/user/ah12x/external';
OK
Time taken: 0.169 seconds
hive (sampledb)> show tables;
OK
externalhive
hive (sampledb)> alter table externalhive add partition (year ='2014')
> location ('/2012');
FAILED: ParseException line 2:9 extraneous input '(' expecting StringLiteral near '<EOF>'
There is no need for braces after location, https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterTable/PartitionLocation
I am trying to load data into a Hive table using partition.
The code is as follow:
CREATE EXTERNAL TABLE URL(url STRING, clicks INT)
COMMENT 'Unique Clicks per URL'
PARTITIONED BY(dt STRING)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n'
STORED AS TEXTFILE
LOCATION '/mypath/URL';
LOAD DATA INPATH '/inputpath/' INTO TABLE URL
PARTITION (dt=date_format(CURRENT_TIMESTAMP, "yyyy.MM.dd HH:mm:ss"));
I am gettin the following error:
FAILED: ParseException line 4:14 cannot recognize input near
'date_format' '(' 'CURRENT_TIMESTAMP' in constant
I tried using
SET hive.exec.dynamic.partition.mode=nonstrict;
but nothing changed.
Why is it not working?
How to set the current date as partition column?
Thank you in advance.
Lorenzo
Why move the files when you can create the external table on top of them?
LOAD DATA INPATH just moves the files (HDFS metadata operation) "as is", to the table's location.
Why define the partition column as a string when it is clearly a date?
CREATE EXTERNAL TABLE URL ... PARTITIONED BY(dt DATE) ...
Why are you trying to use non-ISO formats (yyyy.MM.dd)?
ISO date format is yyyy-MM-dd
Since it seems the partition information is not part of the data you have 3 options:
1.
Use a constant (no expression are allowed, including functions), e.g.
LOAD DATA INPATH '/inputpath/' INTO TABLE URL PARTITION (dt=date '2017-03-04');
2.
Create an additional table,URL_STG, similar to URL but without partition and use it to insert the partitions dynamically.
set hive.exec.dynamic.partition.mode=nonstrict;
insert into URL select *,current_date from URL_STG;
3.
Supply the date as a variable from the CLI
hive --hivevar dt=$(date +"%Y-%m-%d") -e \
'LOAD DATA INPATH '\''/inputpath/'\'' INTO TABLE URL PARTITION (dt=date '\''${hivevar:dt}'\'')'
Is it possible to create external partitioned table without location? I want to add all the locations later, together with partitions.
i tried:
CREATE EXTERNAL TABLE IF NOT EXISTS a.b
(line STRING)
COMMENT 'abc'
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\n'
STORED AS TEXTFILE
PARTITIONED BY day;
but i got ParseException: missing EOF at 'PARTITIONED' near 'TEXTFILE'
I don't think so, as said in alter location.
But anyway, i think your query as some errors and the correct script would be :
CREATE EXTERNAL TABLE IF NOT EXISTS a.b
(line STRING)
COMMENT 'abc'
PARTITIONED BY (day String)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\n'
STORED AS TEXTFILE
;
I think the issue is that you have not specified data type for your partition column "day". And you can create a HIVE external table without location and can use ALTER table options later to change the location.
I am running Hive and Hue on Cloudera.
I have the following text file uploaded to hdfs. And I'm trying to create an external table in hive partitioned by id. For whatever reason, it's not working.
/user/test2/test.csv
id,name,age
1,sam,10
2,john,5
1,rick,4
Hive:
CREATE EXTERNAL TABLE IF NOT EXISTS testDB (
name STRING,
age INT
)
COMMENT 'This is the test database'
PARTITIONED BY (id INT)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
LOCATION '/user/test2/'
TBLPROPERTIES ("skip.header.line.count" = "1");
On Hue, hive editor, when I tried to look at sample data, it says list index out of range. Not sure what this is. The external table will work correctly if I remove the partitioned by.
Your data located on '/user/test2/test.csv' is structured on three columns, but your schema defined for the table 'testDB' contains two columns, it is normal that you have this error.
You have to update your script, by adding the id column:
CREATE EXTERNAL TABLE IF NOT EXISTS testDB (
id INT,
name STRING,
age INT
)
...
you do not have the data partitioned because you have not created it. You should follow 3 steps:
1- Mount the data pointing to the .csv file.
CREATE EXTERNAL TABLE TableName (id int, name string, age int)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n'
STORED AS TEXTFILE LOCATION '/user/test2/';
Create the partitioned data
CREATE EXTERNAL TABLE IF NOT EXISTS testDB (
name STRING,
age INT )
COMMENT 'This is the test database'
PARTITIONED BY (id INT)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
LOCATION '/user/other/location'
TBLPROPERTIES ("skip.header.line.count" = "1");
Insert the data from your previous table to the final table.
insert into table testDB
partition (archive) select name, age from
TableName;
Hope this help you.
So I finally solved my problem. Below is my solution.
I am using Cloudera vm - 5.4.2, when I started Hue, the Hive setting in Hue is pointing to HiveServer2; But I created the tables using Hive CLI. So basically the table only exists in HiveServer1.
Solution:
Instead of using Hive CLI, create table in beeline, everything in Hue should work then.
I am using sqlldr for uploading data to oracle table from a csv.
For one column i need to use SEQUENCE.
My control file looks like this.
LOAD DATA
INFILE "D:\WORKER\ADS4014_GeneralJobWorkers_1426054269727.csv"
BADFILE dataFile.bad
APPEND INTO TABLE GJobWorkers
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(JobId,UserId,EmpId,ChkInTme,ChkOutTme,tDate,TotalHrs,JobCode,Modified_Time,GJ_SEQ "jobs_GJ.nextval")
But i get error in the log file :
Error on table GJobWorkers, column GJ_SEQ.
Column not found before end of logical record (use TRAILING NULLCOLS)
Can anyone help me doing this?
Thanks in advance.