I created a 3 node Hadoop cluster with 1 namenode and 2 datanode.
I can perform a read/write query from Hive shell, but not beeline.
I found many suggestions and answers related to this issue.
In every suggestion it was mentioned to give the permission for the userX for each individual table.
But I don't know how to set the permission for an anonymous user once and for all.
Why I am getting the user anonymous while accessing the data from beeline or from a Java program?
I am able to read the data from the both beeline shell and using Java JDBC connection.
But I can't insert the data in the table.
This is my jdbc connection : jdbc:hive2://hadoop01:10000.
Below is the error i am getting while on insert request:
Permission denied: user=anonymous, access=WRITE, inode="/user/hive/warehouse/test_log/.hive-staging_hive_2017-10-07_06-54-36_347_6034469031019245441-1":hadoop:supergroup:drwxr-xr-x
Beeline syntax is
beeline -n username -u "url"
I assume you are missing the username. Also, no one but the hadoop user has WRITE access to that table anyway
If you don't have full control over the table permissions, you can try relocating the staging directory with the setting hive.exec.stagingdir
If no database is specified in the connection URL to connect, like
jdbc:hive2://hadoop01:10000/default
then beeline connects to the database DEFAULT , and while inserting the data into the table - first the data is loaded to a temporary table in default database and then loaded to the actual table.
So, you need to give the user access to the DEFAULT database also, or you can connect to the databases where you have access to.
jdbc:hive2://hadoop01:10000/your_db
Related
I have some tables in test database in HIVE that were created by another user.
I am trying to move these tables to another database called events
I have tried like below
alter table test.123 rename to events.123
But I am receiving permission denied error because the table 123 has been created by another.
1) How can I move this table?
2) How can we change the owner of the table to other user's?
3) Or How can we change ownership of all tables of a database to another user?
4) What is the best scenario where we don't need to worry about permission on Hive tables?
You can try following with superuser
ALTER (DATABASE|SCHEMA) database_name SET OWNER [USER|ROLE] user_or_role; -- (Note: Hive 0.13.0 and later; SCHEMA added in Hive 0.14.0)
If your Hadoop cluster is Kerberos enabled then there is no way you can change the ownership with non-super user account. If this would have been possible then there is no use of Permissions.
I need some information on Hive user impersonation. I did some research on it and found that by default HiveServer2 performs a query processing as the user who submitted query but if hive.server2.enable.doAs set it to false then query will be run as user who started hiveserver2 process.
I need to a create jdbc/thirft connection with hiveserver2 with service account (let’s say with user ‘ABC’ is logged in) but would like to run my hive statement with user that I pass , for example with user ‘XYZ’.
Let me know if anyone has done this before.
Is it possible to do this for Hive ?
With Hive impersonation enabled , you can run your queries that you will pass along with connection string .
For Example
jdbc:hive2://localhost:10000/default,username,password
In this case , your job will run with username that you are passing instead of hive user.
hope this helps.
I am unable to do an Update to my hive table via JDBC. I able to Select, but not Update.
Connecting to the hive database:
Connection connection =
DriverManager.getConnection("jdbc:hive2://localhost:10000/db", "", "");
My query:
ResultSet resultSet = statement.executeQuery("update db.test set name='yo yo' where id=1");
Stacktrace:
java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:275)
at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:355)
at com.spnotes.hive.App.main(App.java:63)
Again, I am able to Select but not Update via JDBC. I am however, able to Update my table via the hive shell only. I believe this is a user permissions issue. I have seen other problems where an HDFS directory needed to be granted permissions before it could be written to.
I had to invoke my hive shell with my HDFS user as so:
sudo -u hdfs hive
Can I somehow pass a "hfds" user via JDBC? It does not look like this is possible. This is how I'm thinking the exception will not happen anymore.
Here is the "secure way" of passing in a username and password as so:
Connection con = DriverManager.getConnection("jdbc:hive2:/hiveserver.domain.com:10000/default;user=username;password=password");
BUT this is NOT the same thing as passing the user hdfs. Perhaps it is possible to link the "username" with permissions to update the hive table?
Any help is welcome. Thanks!
You are trying to pass a update statement in a executeQuery()
For security reasons, any update statement will fail when using this method. Change it to executeUpdate()
Also, instead of using queries like this, I suggest using Prepared Statements, since by using parameters you make it less vulnerable to SQL Injections
I'm trying out Hive sql based authorization now, It works great.
But when I connect for example with beeline to hs2:
!connect jdbc:hive2://localhost:10000 alice#something.com password
and try to drop a table:
Permission denied: Principal [name=alice, type=USER] does not have following privileges for operation....
The hive see I'm "alice" instead of "alice#something.com". So I can not grant properly the users.
I appreacite any help about it, how I can prevent to cut the #something.com part.
thank you
I created a Cloudera cluster and imported some sample test files from oracle DB. But after a while I had to change the hostnames of the nodes. I followed the guide mentioned in cloudera site and everything worked fine. But when I try to access tables(using both hive and impala) I created earlier I get the following error:
Fetching results ran into the following error(s):
java.io.IOException: java.lang.IllegalArgumentException: java.net.UnknownHostException: [Old Host Name]
Then I created another table under the same DB (Using Hue>Metastore Tables) and I can access these new tables created under the new hostname with no issue.
Can someone explain how I can access my old tables without reverting back my hostnames. Can I access metastore db and change the table pointers to new hostname.
Never Mind I found the answer.
You can confirm that hive/impala is looking for the wrong location by executing
describe formatted [tablename];
O/P
14 Location: hdfs://[oldhostname]:8020/user/hive/warehouse/sample_07 NULL
Then you can change "Location" property using :
ALTER TABLE sample_07 SET LOCATION "hdfs://[newhostname]:8020/user/hive/warehouse/sample_07";
ps - sample_07 is my the table in concern
Some times this doesn't WORK !!
Above workaround works for sample table which is available by default but I had another table which I sqooped from external DB to a custome metastore DB, this gave me again an error similar to above.
Solution :
Go to host where you've installed hive.
Temporally add the old hostname of the hive server to /etc/hosts (if you don't have external DNS both new and old hostnames should exist in the same host file)
Execute the 'ALTER TABLE ....' at hive shell (or web interface)
Remove the oldhostname entry from /etc/hosts
Try this
hive --service metatool -updateLocation <newfsDefaultFSValue> <old_fsDefaultFSValue>
You can refer to https://www-01.ibm.com/support/knowledgecenter/SSPT3X_3.0.0/com.ibm.swg.im.infosphere.biginsights.trb.doc/doc/trb_inst_hive_hostnames.html