Have users on local machines that have HDFS /user dirs that do not show up as possible users when setting Ranger policies
I can see that Ranger already have a place where you can see and add users in the settings menu of the ranger UI, but not sure where this is getting populated from.
So my question then is what determines if Ranger can see cluster users for setting policies (and is there an easy way to manage this via ambari)?
The problem was that I had thought, looking at a answer on the Hortonworks community forums, that for a user to be recognized as "existing" on the HDP cluster, all that was required was for the user to 1) exist on a cluster node and 2) have a folder in hdfs:///user/<the username>. This apparantly is not correct (at least in the case of being recognized by Ranger as a valid user that can have policies set on them).
In order for a user to be recognized by Ranger (here, I do not have a cluster integrated with Kerberos or Active Directory), that user needs to exist on the usersync server machine which supports...
the ability [for Ranger] to get users and groups from the corporate AD to use in policy definitions.
Related
Is there an option to switch user in HUE?
In my organization, infrastructure team setup usecase id, which has all HDFS file system access and only usecase id can submit yarn jobs. Individual user can sudo
to usecase id sudo su - xyz. There are no password for usecase id.
I am able to login to HUE but can't submit any jobs as I don't have access to any queue so I want to switch to usecase id, after login to HUE. How to switch user ( sudo su - xyz) in hue?
Hue, by default, can only run under the first account that's logged in with.
You need to ask the infrastructure team to configure Hue with a PAM or LDAP login authentication, in which case the password will be required for any Hue login user
Once that's setup, you are also able to switch accounts.
There are other configurations, but for enterprise users, I think those are the best options other than some single sign on OAuth/OpenID tool.
There's also SPNEGO, and that'll require a completely kerberized cluster.
Realistically, your company sounds like their cluster is not using Kerberos, so it isn't even secure.
For example, don't even need to sudo... Just export a variable
export HADOOP_USER_NAME=usecase
Of course, this isn't possible in Hue, but if you already have SSH access, you really can do anything in the cluster you want to
Configure yarn core-site.xml to impersonate hue users to access yarn cluster for submitting jobs.
key: hadoop.proxyuser.default_user.hosts
value: *
key: hadoop.proxyuser.default_user.groups
value: *
Replace default_user with hue or any system user manages your application.
Hue supports different backend module for authentication and authorization.
django.contrib.auth.backends.ModelBackend
desktop.auth.backend.AllowAllBackend
desktop.auth.backend.AllowFirstUserDjangoBackend
desktop.auth.backend.LdapBackend
desktop.auth.backend.PamBackend
desktop.auth.backend.SpnegoDjangoBackend
desktop.auth.backend.RemoteUserDjangoBackend
libsaml.backend.SAML2Backend
libopenid.backend.OpenIDBackend
liboauth.backend.OAuthBackend (New oauth, support Twitter, Facebook, Google+ and Linkedin)
Each module has its own advantage and disadvantages. For a very large scale deployment where multiple user need to access HUE application, LDAP, SAML2, Oauth authentication is preferred over PAM and Django based login.
Every time i try to use the Hadoop Resource Manager web interface (http://resource-manger.host:8088/cluster/) i show up logged in as dr.who.
My question, how can I login as another user? In this case i want to login as myself and have a higher lever of privileges than dr.who.
The user infomation is got from HttpServletRequest#getRemoteUser().
1. If you deployed an insecure cluster, the simplest way to pass the username to server is by url parameter. For example, http://localhost:8088/cluster?user.name=babu
2. If you deployed a secure cluster, you probably use Kerberos authentication. You can use kinit to get a kerberos tgt, then configure the browser to negotiate. (network.negotiate-auth.trusted-uris for firefox, and --auth-server-whitelist for chromium. I'm sure there's lots of answers about this)
For more information, you can check hadoop official documentation.(https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/HttpAuthentication.html)
You should set the access control list by changing the default configuration of:
yarn.resourcemanager.zk-acl
from
world:anyone:rwcda
to something else,which is Cluster-specific
The ACLs the ResourceManager uses for the znode structure to store the internal state.
hadoop jar uses the name of the currently logged-in user. Is there a way to change this without adding a new system user?
There is, through a feature called Secure Impersonation, which lets one user submit on behalf of another (that user must exist though). If you're running as the hadoop superuser, it's as simple as setting the env variable $HADOOP_PROXY_USER.
If you want to impersonate a user which doesn't exist, you'll have to do the above and then implement your own AuthenticationHandler.
If you don't have to impersonate too many users, I find it easiest to just create those users on the namenode and use secure impersonation in my scripts.
In relation to building a Beowulf cluster, why is it necessary to create identical users on the slave nodes? If one were to create the users on the slave nodes in a different order to the order in which they were created on the master node, what problems would occur and how would one fix them?
I have been trying to find a concrete answer to this for a few hours but with no luck. Any help would be appreciate.
Probably because of SSH access/file permissions.
If one computer needs to access other it must have some sort of remote login technology, and SSH uses user names. Also if you have a file share between them, you may run into problems with file permissions when one pc writes them as one user and other tries to read them as other.
Regarding user creation, by default if you don't specify a user id your user gets the next available. In Ubuntu case, normal accounts start with UID 1000 so if you create 3 users you will get the following
USER NAME ID
user1 1000
user2 1001
user3 1002
If in a different machine you change the order, the users will have different user ids. Of course, you can avoid that providing the desired UID when you create the accounts.
I believe it is because they most likely share some sort of file system such as /home. Any shared software will need certain permissions and the permissions will correspond to a uid or groupid. If there is a user "user" on one machine with a different uid than "user" on another machine, some of the shared filesystem won't be accessible.
To fix it you would need to add the user on each machine with the specific matching uid.
When a MPI program is running in several nodes is necessary to login this nodes, write files etc. If the users is no sync between headnode and nodes you can't even to find the executable because the users permission in NFS share.
I'm looking for some guidance on how to automat applying a set of permissions withn the local security policy to a multiple users on multiple servers.
For example, via a script, I want to apply "act as part of the operating system" and "adjust memoroy quotas for a process" to user TEST1 and TEST2.
Any feedback on how to get started would be appreciated. thanks!
From a command line, the Microsoft-provided solution is secedit. AppDeploy is a great resource for packaging in general, and they have a good page on secedit here: http://www.osdeploy.com/tips/detail.asp?id=23
In short, change your policies using the Local Security Settings MMC snap-in, then export with secedit as in this page (http://www.webservertalk.com/message534715.html -- also assuming this computer isn't a member of a domain), then import as usual.
Is this machine domain joined? If so, you'll need to make sure no domain policies are applied. Otherwise the domain policies will be exported along with the local ones.
Simpler answer here:
Scripting Local Security Policy
Use ntrights.exe from the Windows 2003 Resource Kit.
However, this doesn't seem to help with the "adjust memory quotas for a process" right.