I'm trying to connect to hive using beeline !connect jdbc:hive2://localhost:10000 and I'm being asked for a username and password
Connecting to jdbc:hive2://localhost:10000'
Enter username for jdbc:hive2://localhost:10000:
Enter password for jdbc:hive2://localhost:10000:
As I don't know what username or password I'm supposed to type in I'm leaving it empty which causes the error: Error: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous (state=,code=0)
My setup is a single node hadoop cluster in ubuntu.
I can confirm that the services are up and running, both hadoop and hiveserver2
The question is , what are these username and password I'm being asked, where can I find them or set them?
Thanks in advance
You should provide a valid username and password that has privileges to access the HDFS and Hive Services (user running HiveServer2). For your setup, the user in which Hadoop and Hive are installed would be the superuser.
These credentials will be used by beeline to initiate a connection with HiveServer2.
And, add these properties in core-site.xml
<property>
<name>hadoop.proxyuser.username.groups</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.username.hosts</name>
<value>*</value>
</property>
Restart services after adding these properties.
Then run beeline with the specified user name username as below:
beeline -u jdbc:hive2://localhost:10000 -u username
ref: https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2#SettingUpHiveServer2-Impersonation
Alternatively, you can also set the parameter hive.server2.enable.doAs to false to disable user impersonation.
In hive-site.xml need to set the parameter hive.server2.enable.doAs to
false
<property>
<name>hive.server2.enable.doAs</name>
<value>FALSE</value>
<description>
Setting this property to true will have HiveServer2 execute
Hive operations as the user making the calls to it.
</description>
</property>
http://mail-archives.apache.org/mod_mbox/hive-user/201602.mbox/%3C54b7754ceb8370b7250bba929369763f#cloudtechnologypartners.co.uk%3E
Related
I am getting the below error when I try to start Hive using hiverserver2.
Connecting to jdbc:hive2://localhost:10000
18/10/25 09:45:38 [main]: WARN jdbc.HiveConnection: Failed to connect to localhost:10000
Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: deco is not allowed to impersonate anonymous (state=08S01,code=0)
The user name I am using is deco.
I have also added the below entry in core-site.xml file:
<property>
<name>hadoop.proxyuser.deco.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.deco.groups</name>
<value>*</value>
</property>
I am still unable to connect using beeline. I used the following commands:
$HIVE_HOME/bin/beeline -u jdbc:hive2://localhost:10000
and
$HIVE_HOME/bin/beeline -n $(whoami) -u jdbc:hive2://localhost:10000
I even took a backup of the metastore_db folder and reinitiated with the below command:
$HIVE_HOME/bin/schematool -dbType derby -initSchema
I even started hiveserver2 on 10001 port and connected beeline to 10001 and still got the same error
All the above prove futile.
Help I am dying
I ever got this error
User * is not allowed to impersonate anonymous
That's because by default hive tries to execute operations as the calling user, I add below lines to hive config file conf/hive-site.xml, to ask hive to execute operations as the hiveserver2 process user, then get rid of this error:
<property>
<name>hive.server2.enable.doAs</name>
<value>false</value>
<description>
Setting this property to true will have HiveServer2 execute
Hive operations as the user making the calls to it.
</description>
</property>
Here is the document:
Impersonation
By default HiveServer2 performs the query processing as
the user who submitted the query. But if the following parameter is
set to false, the query will run as the user that the hiveserver2
process runs as.
hive.server2.enable.doAs – Impersonate the connected user, default
true.
hive exception
I have installed Hadoop 3.0.0 and Hive 2.3.1 in my PC. Parallely i installed mysql and working with sql commands in sql shell mode and working fine. But While executing queries in Hive shell mode, i am receiving the following error,
hive> create table saurzcode(id int, name string);
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
Please let me know the reason for failure.
Also please clarify the following queries,
1) Difference between hive shell mode vs mysql shell mode.
2) Why to configure MySql Metastore for Hive?
Please find the hive-site.xml configuration,
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost/metastore?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hivelogin</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>apache</value>
</property>
</configuration>
Your Original exception is
Unable to load authentication plugin 'caching_sha2_password' as you can see in below error log.
org.apache.hadoop.hive.metastore.HiveMetaStore - Retrying creating default database after error: Unable to open a test connection to the given database. JDBC url = jdbc:mysql://localhost/metastore?createDatabaseIfNotExist=true
, username = hivelogin. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
java.sql.SQLException: Unable to load authentication plugin 'caching_sha2_password'.
Solution:
This error happens due to all new MySQL version come up with added password plugin called "caching_sha2_password", and it has to be configured properly at MySQL server or else you can simply use "mysql_native_password" parameter with "CREATE USER" in MySQL as below to get it resolved.
While creating the hive Meta Store user just follow the below command.
CREATE USER 'username'#'localhost' IDENTIFIED WITH mysql_native_password BY 'password';
GRANT ALL PRIVILEGES ON metastore_db.* TO 'hive'#'%';
I am hiding the metastore password in hive-site.xml using below property and I am using postgres as my metastore db.
<property>
<name>hadoop.security.credential.provider.path</name>
<value>jceks://file//...hive_new.jceks</value>
</property>
1)Iam getting error when I initiate db like "schematool -dbType postgres -initSchema"?
2)How can I use the these jceks file using beeline?.I have tried like below
beeline -u "jdbc:hive2://myhost:1000/default;principal=hive/principal#REALM?hadoop.security.credential.provider.path=jceks://hdfs#hostname/path/to/jceks"
but its halted at connecting.....
Do I need to set any other properties in hive-site.xml for beeline?
I am trying to implement Kerberos authentication. I am using Hadoop 2.3 version of hadoop on cdh5.0.1. I have done the following changes :
Added following properties to core-site.xml
<property>
<name>hadoop.security.authentication</name>
<value>kerberos</value>
</property>
<property>
<name>hadoop.security.authorization</name>
<value>true</value>
</property>
After restarting the daemon when i am issuing hadoop fs -ls / command, I am getting following error :
ls: Failed on local exception: java.io.IOException: Server asks us to fall back to SIMPLE auth, but this client is configured to only allow secure connections.; Host Details : local host is: "cldx-xxxx-xxxx/xxx.xx.xx.xx"; destination host is: "cldx-xxxx-xxxx":8020;
Please help me out.
Thanks in advance,
Ankita Singla
There is a lot more to configuring a secure HDFS cluster than just specifying hadoop.security.authentication as Kerberos. See Configuring Hadoop Security in CDH 5 about the required config settings. You'll need to create appropriate keytab files. Only after you configured everything and you confirmed that none of the Hadoop services report any error in their respective logs (namenode, datanode on all hosts, resourcemanager, nodemanager on all nodes etc) can you attempt to connect.
I was trying to run the hadoop fs -cp command but got the following error message:
-cp: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3 URL, or by setting the fs.s3.awsAccessKeyId or fs.s3.awsSecretAccessKey properties (respectively)
I'm new to hadoop and s3 so can anyone please give advice on what I should do?
Thanks!
Please refer .
go to
cd ${HADOOP_HOME}/conf/hadoop-site.xml and add/update
<property>
<name>fs.s3.awsAccessKeyId</name>
<value>Your AWS ACCESS KEY</value>
</property>
<property>
<name>fs.s3.awsSecretAccessKey</name>
<value>Your AWS Secret Access Key</value>
</property>