I am running windows 10, hadoop 2.8.5 and hive 2.3.5. I installed Hive while following this guide:
https://kontext.tech/docs/DataAndBusinessIntelligence/p/apache-hive-300-installation-on-windows-10-step-by-step-guide
Each command in hive console ends with this exception after a while:
hive> show tables;
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
Before anyone points out this is a duplicate question, none of the other solved my problem, namely:
question 1
question 2
and others that I do not have links for at the moment. This is what I did:
run hadoop by using start-all.cmd, this works fine.
in cygwin, I created the metastore schematool -dbType derby -initSchema. Again, this finishes fine.
run hive server: hive --service hiveserver2 start
start hive console: hive
All of these work, but no command in hive console finishes. My .bashrc for cygwin:
export JAVA_HOME="/cygdrive/c/Program Files/Java/jdk1.8.0_212"
export PATH="$PATH:$JAVA_HOME/bin"
export HADOOP_HOME='/cygdrive/c/hadoop-2.8.5'
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HIVE_HOME='/cygdrive/c/hive-2.3.5'
export PATH=$PATH:$HIVE_HOME/bin
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HIVE_HOME/lib/*.jar
I have tried generating schema anew many times, and even setting the path in hive-site.xml in javax.jdo.option.ConnectionURL setting, and starting hive from HIVE_HOME and other directories, to no avail. I have run out of ideas as to what to try next.
Related
I'm trying to initialize hive and I'm getting this error when I try to use the schematool utility to initialize the Metastore by running the following command:
user#systemname /cygdrive/c/hive-3.1.2/bin
$ ./schematool -initSchema -dbType derby
ERROR: C:\Java\jdk1.8.0_161\bin/bin/java is not executable.
tap on link to see the screenschot
I think the reason for this may be that the directory name of hadoop is different than mentioned in some cygwin64/derby/hive configuration files.
Can someone please help me solve this error???
There is output the following error information:
Error: FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68,code=30000)
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
Use --verbose for detailed stacktrace.
*** schemaTool failed ***
Hive and Hadoop daemons are running fine. I ran Hive services after the Metastore services via Cygwin. I deleted all the Metastore db directories. Then I am facing also the same issue mentioned above after running the commands below.
In Windows command prompt window:
C:\hadoop_new\db-derby-10.14.2.0\bin\startNetworkServer -h 0.0.0.0
In Cygwin terminal window:
$HIVE_HOME/bin/schematool -dbType derby -initSchema
Error:
hive > FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
Solution :
Go to below directory:
C:\hadoop_new\apache-hive-3.1.2\scripts\metastore\upgrade\derby\hive-schema-3.1.0.derby.sql
comment the 'NUCLEUS_ASCII' function and 'NUCLEUS_MATCHES' function
rerun schematool -dbType derby -initSchema, then everything goes well!
I had a similar issue. The error was because I attempted to start hive before establishing the schema. Try the following:
$mv metastore_db metastore_db.tmp
Now run your schema setup.
I found this solution here.
I am trying to set up hive on my local. I started all Hadoop processes and set up the {hive}/bin path. On command prompt I can run hive commands , create and read tables. My questions are -
1) is hive-site.xml is optional file ?
2) in absence of hive-site.xml file, how hive get information regrading metastore and other configuration?
If you're running Hive queries from your local machine which has Hadoop installed, hive-site.xml is not needed as you are talking directly to hive/bin in the Hive installation directory. You don't need to tell Hive where to find Hive.
If you wanted to run Hive commands from another machine, but interacting with Hive on your local machine, you'd need hive-site.xml.
i am learning sqoop from few days and successfully installed and configure with hadoop.
hadoop_usr#sawai-Lenovo-G580:/usr/local/sqoop/bin$ sqoop2-server start
Setting conf dir: /usr/local/sqoop/bin/conf
Sqoop home directory: /usr/local/sqoop
The Sqoop server is already started.
hadoop_usr#sawai-Lenovo-G580:/usr/local/sqoop/bin$ sqoop
sqoop: command not found
sqoop server is already running and when i try to fire sqoop command then i get error message. command not found. sqoop home is already in path
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
export HADOOP_INSTALL=$HADOOP_HOME
export SQOOP_HOME=/usr/local/sqoop
export SQOOP_CONF_DIR=$SQOOP_HOME/conf
export SQOOP_CLASSPATH=$SQOOP_HOME/server/lib
export PATH=$PATH:$SQOOP_HOME/bin:$SQOOP_CONF:$SQOOP_CLASSPATH
$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/hadoop/sbin:/usr/local/hadoop/bin:/usr/local/sqoop/bin::/usr/local/sqoop/server/lib
Please help me to resolve this issue.
Thanks in advance.
command not found error in most of the cases happens because of path is not set for same.
Kindly set the paths of sqoop, which you already have done.
export PATH=$PATH:$SQOOP_HOME/bin:$SQOOP_CONF:$SQOOP_CLASSPATH
Compile the file where you have set $PATH or restart your terminal.
put below command in .bashrc file
export SQOOP_HOME=/home/pj/sqoop
export PATH=$PATH:$SQOOP_HOME/bin
and restart .bashrc
source .bashrc
If still issue persist, restart terminal.
I've just installed hadoop on windows using cygwin which works fine, and now I am installing Hive. I am running it as:
bin/hive -hiveconf java.io.tmpdir=/cygdrive/c/cygwin/tmp
OR
bin/hive -hiveconf java.io.tmpdir=/tmp
(both give the same problem) as I have found out there is a bug with the windows naming convension (https://issues.apache.org/jira/browse/HIVE-2388...)
When I run the above command, Hive seems to load fine, but when I enter "show tables;" I get no response. This is the same for all queries. CREATE TABLE etc, there is no response
Its the same problem as this guy:
http://mail-archives.apache.org/mod_mbox...
Any ideas?
I resolved a similar issue and successfully ran HIVE after starting all Hadoop daemons
namenode
datanode
jobtracker
Task Tracker
Running queries from files using hive -f <filename>, instead of writing queries directly at the HIVE command prompt. Additionally, you may also use bin/hive -e 'SHOW TABLES'