I am using hive for querying and data processing purpose in my hadoop main node,but I am not able to start hive in terminal and way taking too much time and not starting as show below.
#hive
Logging initialized using configuration in file:/etc/hive/2.3.4.7-4/0/hive-log4j.properties
you can lookup for the actual problem in HIVE
Hive uses log4j for logging. By default logs are not emitted to the console by the CLI. The default logging level is WARN for Hive releases prior to 0.13.0. Starting with Hive 0.13.0, the default logging level is INFO.
The logs are stored in the directory /tmp/<user.name>:
/tmp/<user.name>/hive.log
Note: In local mode, prior to Hive 0.13.0 the log file name was ".log" instead of "hive.log".
Related
I am trying to set up hive on my local. I started all Hadoop processes and set up the {hive}/bin path. On command prompt I can run hive commands , create and read tables. My questions are -
1) is hive-site.xml is optional file ?
2) in absence of hive-site.xml file, how hive get information regrading metastore and other configuration?
If you're running Hive queries from your local machine which has Hadoop installed, hive-site.xml is not needed as you are talking directly to hive/bin in the Hive installation directory. You don't need to tell Hive where to find Hive.
If you wanted to run Hive commands from another machine, but interacting with Hive on your local machine, you'd need hive-site.xml.
I installed Apache Kylin, following the official installation guide http://kylin.apache.org/docs/install/index.html, in HDP sandbox 2.6
When I run the script, $KYLIN_HOME/bin/kylin.sh start, I got the error below:
What can I do to fix this error?
Thanks in advance
Check if Hive service is up in your ambari, when Hive service is down Kylin cannot find it and gives the error. Check for .bash_profile as well. When those two issues are addressed kylin should be able to find location of hive dependency.
Kylin uses the find-hive-dependency.sh script to setup the CLASSPATH. This script uses a Hive CLI command (I test it with beeline) to query Hive env vars and extract the CLASSPATH from them.
beeline connect to Hive using the properties at kylin_hive_conf.xml but for some reason (probably due to the Hive version included in HDP 2.6) some of the loaded Hive properties cannot be set when the connection is stablished.
The Hive properties that causes the issue can be discarded for connecting to Hive to query the CLASSPATH, so, to fix this issue:
Edit $KYLIN_HOME/conf/kylin.properties and set kylin.source.hive.client=beeline
Open the find-hive-dependency.sh script, go to line 34 aprox and modify the line
hive_env=${beeline_shell} ${hive_conf_properties} ${beeline_params} --outputformat=dsv -e "set;" 2>&1 | grep 'env:CLASSPATH'
Just remove ${hive_conf_properties}
Check Hive depedencies have been configured by running the command find-hive-dependency.sh.
Now $KYLIN_HOME/bin/kylin.sh start should works.
I tried restarting my system, checked whether there is enough space or not and also made sure my hive server2 is running. But I'm getting these errors when given '$hive' in Cloudera.
Logging initialized using configuration in
file:/etc/hive/conf.dist/hive-log4j.properties
WARN: The method class
org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked.
Exception in thread "main" java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
The process of starting Hive2 is changed, as Hive got deprecated. Usage of Beeline is recommended.
Beeline was developed specifically to interact with the new server. Unlike Hive CLI, which is an Apache Thrift-based client, Beeline is a JDBC client based on the SQLLine CLI — although the JDBC driver used communicates with HiveServer2 using HiveServer2’s Thrift APIs.
As Hive development has shifted from the original Hive server (HiveServer1) to the new server (HiveServer2), users and developers accordingly need to switch to the new client tool. However, there’s more to this process than simply switching the executable name from “hive” to “beeline”.
More information provided over here
Use the below command to enter into interactive mode. Beeline supports same commands that Hive server does. You can execute same script in Beeline without any modifications.
beeline -u jdbc:hive2://
To start the Hive metastore,
sudo service hive-metastore start
I am new in spark and i want to print logs on console using apache spark in yarn cluster mode.
You need to check the value in log4j.properties file. In my case i have this file in /etc/spark/conf.dist directory
log4j.rootCategory=INFO,console
INFO - prints the all the logs on the console. You can change the value to ERROR, WARN to limit the information you would like to see on the console as sparks logs can be overwhelming
My Hive shell hangs at logging initialization at configuration
[cloudera#quickstart hive]$ hive
2017-03-01 08:23:50,909 WARN [main] mapreduce.TableMapReduceUtil: The hbase-prefix-tree module jar containing PrefixTreeCodec is not present. Continuing without it.
Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
This is the log file description.
2017-02-28 08:56:34,685 WARN [main]: hive.metastore
(HiveMetaStoreClient.java:open(448)) - set_ugi() not successful,
Likely cause: new client talking to old server. Continuing without it.
org.apache.thrift.transport.TTransportException:
java.net.SocketTimeoutException: Read timed out at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
I also had this problem when I started hive cli. I tried to find some ways to solve the problem, but it didn't work. When I changed the metastore's version (for example, mysql8 to mysql5.6, hive version is 1.10 and don't change) , the problem has been solved!!!!
the log file has already said new client talking to old server. So you should change your metastore's version to a new one. It should be kept in mind that your metastore's version match hive's version.