Hive No files matching path file and file Exists - hadoop

Question

I'm having a lot of trouble getting hive to work. I'm running CDH4.5 with YARN, all installed from Cloudera's yum repo. I followed their instructions to set up hive but for some reason it does not recognize legitimate files on my local file system.
[msknapp#localhost data]$ pwd
/home/msknapp/data
[msknapp#localhost data]$ ll | grep county_insurance_pp.txt
-rw-rw-rw- 1 msknapp msknapp 162537 Jan 5 14:58 county_insurance_pp.txt
[msknapp#localhost data]$ sudo -u hive hive
Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
Hive history file=/tmp/hive/hive_job_log_9e8bf55b-7ec8-4b79-be9b-cc2200a33f91_1795256456.txt
hive> describe count_insurance;
2014-01-08 02:42:59.000 GMT Thread[main,5,main] java.io.FileNotFoundException: derby.log (Permission denied)
----------------------------------------------------------------
2014-01-08 02:42:59.443 GMT:
Booting Derby version The Apache Software Foundation - Apache Derby - 10.4.2.0 - (689064): instance a816c00e-0143-6fbb-3f3a-000007a1d270
on database directory /var/lib/hive/metastore/metastore_db
Database Class Loader started - derby.database.classpath=''
OK
fips int
st string
stfips int
name string
a int
b int
c int
d int
e int
f int
total int
Time taken: 5.195 seconds
hive> LOAD DATA LOCAL INPATH 'county_insurance_pp.txt' OVERWRITE INTO TABLE count_insurance;
FAILED: SemanticException Line 1:23 Invalid path ''county_insurance_pp.txt'': No files matching path file:/home/msknapp/data/county_insurance_pp.txt
The file I'm trying to load does exist. I get the same exception when I use an absolute path in my load statement.
On a side note, I still don't know why it keeps giving me a FileNotFoundException for the derby log with a permission warning. A long time ago I went to /var/lib/hive and did 'sudo chmod -R 777 ./*', so permissions should not be a problem.
BTW I am running hadoop in pseudo-distributed mode, and have all three hive daemons running locally. I used hive-server2 not 1.
Somebody please let me know what I'm doing wrong here, or how to debug this.

Answer 1

This is Koji. I had a same problem recently.
The hive script run Hadoop server. If the file county_insurance_pp.txt does not exist on Hadoop server, it can not find the file.
You have to send your target file to Hadoop server before running the script. There are 2 ways to handle this:
use scp
use webhdfs (http://hadoop.apache.org/docs/r1.0.4/webhdfs.html)