I am trying to load sample.log file on HDP-sandbox
My initial efforts
LOAD DATA LOCAL INPATH 'sample.log' OVERWRITE INTO TABLE logs;
It seems that path is not matching
Error: Error while compiling statement: FAILED: SemanticException Line 1:23 Invalid path ''sample.log'': No files matching path file:/home/hive/sample.log (state=42000,code=40000)
I logged out,moved to /root,then entered hive
0: jdbc:hive2://sandbox-hdp.hortonworks.com:2> LOAD DATA LOCAL INPATH '/root/Hadoop_Spark_Fundamentals_Code_Notes-V3.0/Lesson-6/Lesson-6.2_Hive/sample.log' OVERWRITE INTO TABLE logs;
Full path does not work either.
Error: Error while compiling statement: FAILED: SemanticException Line 1:23 Invalid path ''/root/Hadoop_Spark_Fundamentals_Code_Notes-V3.0/Lesson-6/Lesson-6.2_Hive/sample.log'': No files matching path file:/root/Hadoop_Spark_Fundamentals_Code_Notes-V3.0/Lesson-6/Lesson-6.2_Hive/sample.log (state=42000,code=40000)
It looks to me that it confuses /root and /home/hive.
How to set the proper path?
Your statement is being executed by user 'hive'. Make sure local file has permissions that allow 'hive' read access to it.
Related
I created a table the following way
CREATE TABLE `default.tmptbl` (id int, name string) ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.OpenCSVSerde' WITH SERDEPROPERTIES (
'escapeChar'='\\','quoteChar'='\"','separatorChar'=',');
And I have data in HDFS that have been structured in the following way:
/app/tmptbl/
/DIR1
/file1.csv
/file2.csv
/DIR2
/file3.csv
/file4.csv
I tried to load the data using the following command:
SET mapred.input.dir.recursive=true;
SET hive.mapred.supports.subdirectories=true;
LOAD DATA INPATH '/app/tmptbl/' INTO TABLE `default.tmptbl`;
However I get the following error:
FAILED: SemanticException Line 1:17 Invalid path ''/app/tmptbl/'': source contains directory: /app/tmptbl/dir1
I don't know why even after setting mapred.input.dir.recursive=true; hive.mapred.supports.subdirectories=true didn't make it load data recursively from sub-directories. Am I missing anything?
I am trying to remove the hard coding from Hive script. For that I have created a hql file(src_sys_cd_param.hql).
Setting the source system value thru parameter, below the of param file
hive -f /data/data01/dev/edl/md/sptfr/landing/src_sys_cd_param.hql;
param file is having command set src_sys_cd = 'M09';
After running the below script:
INSERT INTO TABLE SPTFR_CORE.M09_PRTY SELECT C.EDW_SK,A.PRTY_TYPE_HIER_ID,
A.PRTY_NUM,A.PRTY_DESC,A.PRTY_DESC,'N',${hiveconf:src_sys_cd},
A.DAI_UPDT_DTTM,A.DAI_CRT_DTTM
FROM SPTFR_STG.M09_PRTY_VIEW_STG A JOIN SPTFR_STG.BKEY_PRTY_STG C
ON ( CONCAT(A.PRTY_TYPE_LVL_1_CD,'|^',A.PRTY_NUM ,'|^',A.SRC_SYS_CD)= C.SRC_CMBN);
Receing the error:
Error while compiling statement: FAILED: ParseException line 1:113 cannot recognize input near '$' '{' 'hiveconf' in selection target
I am trying to change the partition location of my external hive table.
Command that I try to run:
ALTER TALBE sl_uploads PARTITION (hivetimestamp='2016-07-26 15:00:00') SET LOCATION '/data/dev/event/uploads/hivetimestamp=2016-07-26 15:00:00'
Error I get :
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.net.URISyntaxException: Illegal character in path
My data for a particular partition exists at the path:
/data/dev/event/uploads/hivetimestamp=date time/actual_data
I think space is creating an issue. But any help on this would be great.
your hdfs in path is right?
add /actual_data/?
Hive is unable to read the full hdfs path due to space in "2016-07-26 15:00:00";
you can use below commands;
hive> set part=2016-07-26 15:00:00;
hive>ALTER TALBE sl_uploads PARTITION (hivetimestamp='2016-07-26 15:00:00') SET LOCATION '/data/dev/event/uploads/hivetimestamp=#part';
First thing i saw that, you wrote TALBE instead of TABLE.
I am trying to import a file text from my machine windows to Hive. I create a table in Hive and I used this command:
load data local in path C:\Users\me\Desktop\test.txt into test_t sample;
But it doesn't work, and the Error is:
Error while compiling statement: FAILED: ParseException line 1:69 missing TABLE at 'test_t' near '' line 1:76 extraneous input 'sample' expecting EOF near '' [ERROR_STATUS]
Do you have any advise?
you have not mention table in your query
assume- your hive table name is sample
load data local inpath "C:\Users\me\Desktop\test.txt" into table sample
I'm trying to create an external table in Hive, but keep getting the following error:
create external table foobar (a STRING, b STRING) row format delimited fields terminated by "\t" stored as textfile location "/tmp/hive_test_1375711405.45852.txt";
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1)
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1)
Aborting command set because "force" is false and command failed: "create external table foobar (a STRING, b STRING) row format delimited fields terminated by "\t" stored as textfile location "/tmp/hive_test_1375711405.45852.txt";"
The contents of /tmp/hive_test_1375711405.45852.txt are:
abc\tdef
I'm connecting via the beeline command line interface, which uses Thrift HiveServer2.
System:
Hadoop 2.0.0-cdh4.3.0
Hive 0.10.0-cdh4.3.0
Beeline 0.10.0-cdh4.3.0
Client OS - Red Hat Enterprise Linux Server release 6.4 (Santiago)
The issue was that I was pointing the external table at a file in HDFS instead of a directory. The cryptic Hive error message really threw me off.
The solution is to create a directory and put the data file in there. To fix this for the above example, you'd create a directory under /tmp/foobar and place hive_test_1375711405.45852.txt in it. Then create the table like so:
create external table foobar (a STRING, b STRING) row format delimited fields terminated by "\t" stored as textfile location "/tmp/foobar";
We faced similar problem in our company (Sentry, hive, and kerberos combination). We solved it by removing all privileges from non fully defined hdfs_url. For example, we changed GRANT ALL ON URI '/user/test' TO ROLE test; to GRANT ALL ON URI 'hdfs-ha-name:///user/test' TO ROLE test;.
You can find the privileges for a specific URI in the Hive database (mysql in our case).