I am unable to create hive table.
Following is the code
CREATE TABLE NYSE(exchange STRING, stock_symbol STRING, stock_date DATE,
stock_price_open FLOAT, stock_price_high FLOAT, stock_price_low FLOAT,
stock_price_close FLOAT, stock_volume INT, stock_price_avg_close FLOAT)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
This returns me the following error message:
NoViableAltException(332#[2189:1: columnNameTypeOrPKOrFK : ( ( foreignKeyWithName ) | ( primaryKeyWithName ) | ( primaryKeyWithoutName ) | ( foreignKeyWithoutName ) | ( columnNameType ) );])
at org.antlr.runtime.DFA.noViableAlt(DFA.java:158)
at org.antlr.runtime.DFA.predict(DFA.java:116)
at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeOrPKOrFK(HiveParser.java:42212)
at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeOrPKOrFKList(HiveParser.java:37938)
at org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:5259)
at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:2763)
at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1756)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1178)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:204)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:404)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:329)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1158)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1253)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
FAILED: ParseException line 1:18 cannot recognize input near 'exchange' 'STRING' ,' in column name or primary key or foreign key
I am using Hive 2.1.0 and Hadoop 2.7.3
please help me out.
You can not use "exchange" Reserved keywords in your column name definition.
try giving like exchange_1 or escape it using ` to your first column name.
Try these steps:
USE your_DB_Name;
DROP TABLE IF EXISTS NYSE PURGE;
CREATE TABLE NYSE (`exchange` STRING, stock_symbol STRING, stock_date DATE, stock_price_open FLOAT, stock_price_high FLOAT, stock_price_low FLOAT, stock_price_close FLOAT, stock_volume INT, stock_price_avg_close FLOAT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ;
Exchange is a Reserved keyword in HIVE
You can escape it using a backtick(``) symbol
This should work:
CREATE TABLE NYSE(
`exchange` STRING
,stock_symbol STRING
,stock_date DATE
,stock_price_open FLOAT
,stock_price_high FLOAT
,stock_price_low FLOAT
,stock_price_close FLOAT
,stock_volume INT
,stock_price_avg_close FLOAT)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
exchange is a keyword in hive so you cannot use. instead of use exchangee or anything else.
try this..
create table nyse (exchangee String,stock_symbol String,stock_date
String,stock_price_open double,stock_price_high double,stock_price_low
double,stock_price_close double,stock_volume
double,stock_price_adj_close double) row format delimited fields
terminated by ",";
Related
I have set up hive on mac. While executing simple create external table query. I am getting below stack trace :
hive> CREATE EXTERNAL TABLE weatherext ( wban INT, date STRING)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ‘,’
> LOCATION ‘ /hive/data/weatherext’;
NoViableAltException(80#[]) at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeOrPKOrFK(HiveParser.java:33341)
at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeOrPKOrFKList(HiveParser.java:29513)
at org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:6175)
at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:3808)
at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2382)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1333)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:204)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1316)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1456)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1236)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1226)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
FAILED: ParseException line 1:45 cannot recognize input near 'date' 'STRING' ')' in column name or primary key or foreign key.
I am able to run same query successfully on ubuntu.
Is it mandatory to surround coulmn names with ` in hive2?
date is a reserved word and should be qualified.
You are using the wrong qualifiers. You should use ' and not ‘ or ’.
You have space in the beginning of your location /hive/data...
CREATE EXTERNAL TABLE weatherext (wban INT, `date` STRING)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
LOCATION '/hive/data/weatherext';
Query:
hive> CREATE TABLE GREENTAXI(VendorID INT, pick_up_date DATE,drop_date DATE,Flag CHAR(1),rate_code INT, pick_up_long STRING,pick_up_lat STRING,drop_off_long STRING,drop_off_lat STRING,passenger_count INT,trip_distance DECIMAL,fare_amount DECIMAL,Extra DECIMAL,Tax DECIMAL,Tip DECIMAL,Tolls INT,Fee INT,Surcharge DECIMAL,total_amount DECIMAL,payment_type INT,trip_type INT)COMMENT 'Data about Green NYC Taxi for the year 2016-Jan’ ROW FORMAT DELIMITED FIELDS TERMINATED BY ','STORED AS TEXTFILE;
I get this error. Please advise
Looks like some character encoding problem. Use a simple editor. Tried this and worked:
CREATE TABLE greentaxi
(
vendorid INT,
pick_up_date DATE,
drop_date DATE,
flag CHAR(1),
rate_code INT,
pick_up_long STRING,
pick_up_lat STRING,
drop_off_long STRING,
drop_off_lat STRING,
passenger_count INT,
trip_distance DECIMAL,
fare_amount DECIMAL,
extra DECIMAL,
tax DECIMAL,
tip DECIMAL,
tolls INT,
fee INT,
surcharge DECIMAL,
total_amount DECIMAL,
payment_type INT,
trip_type INT
)
comment 'Data about Green NYC Taxi for the year 2016-Jan'
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE;
Using below command to create table in hive but giving error.
CREATE TABLE TestData ( id1 int, id2 int, id3 int, id4 String) ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,’ stored as textfile;
Error
FAILED: ParseException line 1:106 mismatched input ',' expecting StringLiteral near 'BY' in table row format's field separator
I have tried '\054' instead of ',' but not working.
You Didn't Mentioned How Rows are Separated..
Edit:
i.e. Lines
Most of the time things which are not visible from open eyes , can be seen through vi editor ;) . There are control characters in command.
cat -v file.txt
CREATE TABLE TestData ( id1 int, id2 int, id3 int, id4 String) ROW FORMAT DELIMITED
FIELDS TERMINATED BY M-bM-^#M-^X,M-bM-^#M-^Y stored as textfile;
Solution - don't copy paste command, just type it.
String type contains two data types in hive: VARCHAR and CHAR.
Instead of String use either VARCHAR or CHARas data type as this will resolve your issue.
*also don't forget to mention the size while specifying varchar.
Example: CREATE TABLE TestData ( id1 int, id2 int, id3 int, id4 varchar (65355)) ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,’ stored as textfile;
Input File:
[Tom,123,0,jump]
[jerry,345,1,run]
I want to read the above input in hive,
my ddl is
CREATE EXTERNAL TABLE IF NOT EXISTS db1.tomjerrry
( name string, id
int, isGood int, activity string )
row format delimited fields terminated by ','
LOCATION '/user/myname/sample.txt'
When i try reading ,
Select name from db1.tomjerrry
I get,
[Tom
[jerry
How do I remove the square bracket in the hive output.?
Add ESCAPED BY '['
ie
CREATE EXTERNAL TABLE IF NOT EXISTS db1.tomjerrry ( name ARRAY<string>, id int, isGood int, activity string )
row format delimited fields terminated by ',' ESCAPED BY '[';
LOCATION '/user/myname/sample.txt'
Or update CSV file remove [.
I'm able to create table successfully but whenever I'm fetching external table I got error
create table xyz ( t1 varchar2(20),t2 varchar(10),t3 varchar(10),t4 date,t5 varchar(10),t6 varchar(10),t7 varchar(10),t8 varchar(20),t9 varchar(25),t10 varchar(60),t11 varchar(60))
ORGANIZATION EXTERNAL (
TYPE ORACLE_LOADER
DEFAULT DIRECTORY DE_DUBFILE
ACCESS PARAMETERS
(
RECORDS DELIMITED BY NEWLINE
BADFILE DE_DUBFILE: 'xyz.bad'
DISCARDFILE DE_DUBFILE: 'xyz.dis'
LOGFILE DE_DUBFILE: 'xyz.log'
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
MISSING FIELD VALUES ARE NULL
(
t1,t2,t3,t4 date "YYYYMMDD",t5,t6,t7,t8,t9,
t10 "ORA_HASH(:t1||:t2||:t3 )",
t11 "ORA_HASH(:t6||:t8||:t9 )"
)
)
LOCATION (DE_DUBFILE:'ZZZ'));
I'm getting below error
ORA-29400: data cartridge error
KUP-00554: error encountered while parsing access parameters
KUP-01005: syntax error: found "double-quoted-string": expecting one of: "binary_double, binary_float, comma, char, date, defaultif, decimal, double, float, integer, (, no, nullif, oracle_date, oracle_number, position, raw, recnum, ), unsigned, varrawc, varchar, varraw, varcharc, zoned"
KUP-01007: at line 11 column 57
Script Terminated on line 26.
It seems to me that "ORA_HASH(:t1||:t2||:t3 )" can't be processed.
Try to load without them (t10 and t11) and generate these fields after all.