For now, in our HDP environment, we have been try set 1st and 2nd layer security, kerberos and ranger. Then we want to add the 3rd layer, Knox.
After we read some documentation in Knox reference for HDP 3.1.4, and the other page, we found 3 option for knox being implemented to a HDP cluster, first knox proxy (api), proxy (ui), and SSO.
In our needs, we want use the proxy (api) and we get the hint to implement it (combined with kerberos).
But we confused to implement the feature proxy ui and SSO
What is the different both of them?
when we use proxy ui and when we use SSO?
can we use three Knox options together?
Based on this link, there is a step that need to configure ambari authentication, "Configure Ambari Authentication for LDAP/AD". Does it mean we drop ambari authentication with kerberos?
But what about knox supported matrix that state Knox SSO can be configured in kerberized cluster?
We can't find how to use and configure Knox proxy (UI). Does it mean if we want launch atlas apps, the knox authentication form appear first or something different about that?
Regards
Related
According to the documentation, one prerequisite for using NiFi CLI against a secured NiFi instance is to configure proxy user request for the node's identity (e.g. CN=localhost, OU=NIFI).
https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#prerequisites-for-running-in-a-secure-environment
I understand how to configure it through the NiFi web user interface. However, is it possible to do the same through scripting?
The reason is that I am working on a NiFi installation script, and I would like to install NiFi and configure users/policies in one go if it is possible.
Thank you!
If you are trying to use NiFi CLI to setup NiFi itself, then you're only real option is for NiFi CLI to perform operations as the Initial Admin identity.
It then depends how NiFi is configured to perform authentication, meaning where is your initial admin identity coming from. Is it a DN from a client cert, a user in LDAP, a kerberos principal, etc?
If it is a client cert, then you can just configure NiFi CLI to use that cert and it should work.
If it is a LDAP user, then you need to have NiFi CLI use one of NiFi's server certs to proxy the LDAP user.
Both of these scenarios are shown in the docs:
https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#security-configuration
Is it possible to write DB connector using Knox Provider extension?
I want Knox to expose endpoint in response to which Knox would put to or get from database records.
Depends on how you want to connect and what functionality you want to add. Currently, Knox support HTTP and Websockets protocols so if you are using JDBC to connect then that will not work. If you are using JDBC over HTTP (e.g. beeline) then that might work. Here are few links to Knox documentation
User Guide
Dev Guide
I am new to OpenID connect & security domain. I have configured Nifi to use OpenID for authentication using online documentation. And to automate a few nifi related tasks we are using nipyapi.
I have already written python code which does automated flow deployment for basic nifi installation (unsecured & without user authentication)
Now, I have to move the code to secured Nifi installation. How to authenticate to OpenID connect using nipyapi/rest API ?
AS per discussion with Bryan, i am planning to use client certificate for authentication but it started giving authorization error. and have created another question with the details.
Nifi - Client Certificate Authorization Error
OpenID Connect generally requires that you follow a flow of re-directs, typically in the browser. NiFi re-directs you to the login page of the OIDC provider, upon completion, the OIDC provider redirects you back to NiFi. I'm not exactly sure how, or if you even can, perform this login process from scripts. An easy alternative would be to just generate a client certificate to represent an automation user for any NiPyApi scripts. Client certificate authentication is always enabled by default for NiFi.
I have a cluster secured by Kerberos, and have a REST API that needs to interact with the cluster on behalf of the user. I have used Spring Security with SPNEGO to authenticate the user, but when I try to use the Hadoop SDK, it fails for various reasons based on what I try.
When I try to use the SDK directly after the user logs in, it gives me SIMPLE authentication is not enabled.
I have noticed the session's Authenticator is UserNamePasswordAuthenticationToken which does not make sense, since I'm authenticating against the Kerberos realm with the credentials from the user.
I am trying to use this project out of the box with my own service account and keytab: https://github.com/spring-projects/spring-security-kerberos/tree/master/spring-security-kerberos-samples/sec-server-spnego-form-auth
For what it's worth, you can leverage Apache Knox (http://knox.apache.org) to consume the Hadoop REST APIs in a secured cluster. Knox will take care of the SPNEGO negotiation with the various components for you. You could use the HTTP header based pre-auth SSO provider to propagate the identity of your enduser to Knox.
Details: http://knox.apache.org/books/knox-0-8-0/user-guide.html#Preauthenticated+SSO+Provider
You will need to ensure that only trusted clients can call your service if you are using that provider however.
Alternatively, you can authenticate to Knox against LDAP with username/password with the default Shiro provider.
One of the great benefits of using Knox this way is that your service never needs to know anything about whether the cluster is kerberized. Knox abstracts that from you.
First of all, Spring Sec Kerberos Extension is a terrible piece of code. I have evaluated it once and abstained from using it. You need the credential of the client authenticating to your cluster. You have basically two options here:
If you are on Tomcat, you can try the JEE pre-auth wrapper from Spring Security along with my Tomcat SPNEGO AD Authenticator from trunk. If will receive the delegated credential from the client which will enable you to perform your task, assuming that your server account is trusted for delegation.
If the above is not an option, resort to S4U2Proxy/S4U2Self with Java 8 and obtain a Kerberos ticket on behalf of the user principal and perform then your REST API call.
As soon as you have the GSSCredential the flow is the same.
Disclaimer: I have no idea about Hadoop but the GSS-API process is always the same.
mine is a kerborised HDP 2.3 cluster with AD users support. we know that by using hadoop's web UI at http://namenode-ip/50070 anyone can access the hdfs details. Can we secure it only to certain authorized users and not publicly to all.. in production.
You can turn on kerberos/SPNEGO for UI access.
This will require SPNEGO to be enabled for your users' browsers as well.
See: https://hadoop.apache.org/docs/r1.2.1/HttpAuthentication.html for general instructions in securing the UIs.
At a high level, you can have the HTTP authentication set to simple, kerberos or the classname of a custom authentication handler.
You may also be interested in using Apache Knox to proxy the UIs in question. This will allow you to provide HTTP Basic Auth against LDAP or a number of other authentication options in order to access the ports. See: http://knox.apache.org/books/knox-0-7-0/user-guide.html#UI+Service+Details for UI proxying details.
This of course would require you to firewall off other direct access to the UIs. It is also worth noting that leaving the HTTP authentication type to simple in secure clusters also leaves the REST API open to anyone that can get to it.