I have a cluster secured by Kerberos, and have a REST API that needs to interact with the cluster on behalf of the user. I have used Spring Security with SPNEGO to authenticate the user, but when I try to use the Hadoop SDK, it fails for various reasons based on what I try.
When I try to use the SDK directly after the user logs in, it gives me SIMPLE authentication is not enabled.
I have noticed the session's Authenticator is UserNamePasswordAuthenticationToken which does not make sense, since I'm authenticating against the Kerberos realm with the credentials from the user.
I am trying to use this project out of the box with my own service account and keytab: https://github.com/spring-projects/spring-security-kerberos/tree/master/spring-security-kerberos-samples/sec-server-spnego-form-auth
For what it's worth, you can leverage Apache Knox (http://knox.apache.org) to consume the Hadoop REST APIs in a secured cluster. Knox will take care of the SPNEGO negotiation with the various components for you. You could use the HTTP header based pre-auth SSO provider to propagate the identity of your enduser to Knox.
Details: http://knox.apache.org/books/knox-0-8-0/user-guide.html#Preauthenticated+SSO+Provider
You will need to ensure that only trusted clients can call your service if you are using that provider however.
Alternatively, you can authenticate to Knox against LDAP with username/password with the default Shiro provider.
One of the great benefits of using Knox this way is that your service never needs to know anything about whether the cluster is kerberized. Knox abstracts that from you.
First of all, Spring Sec Kerberos Extension is a terrible piece of code. I have evaluated it once and abstained from using it. You need the credential of the client authenticating to your cluster. You have basically two options here:
If you are on Tomcat, you can try the JEE pre-auth wrapper from Spring Security along with my Tomcat SPNEGO AD Authenticator from trunk. If will receive the delegated credential from the client which will enable you to perform your task, assuming that your server account is trusted for delegation.
If the above is not an option, resort to S4U2Proxy/S4U2Self with Java 8 and obtain a Kerberos ticket on behalf of the user principal and perform then your REST API call.
As soon as you have the GSSCredential the flow is the same.
Disclaimer: I have no idea about Hadoop but the GSS-API process is always the same.
Related
We have an in-house OAuth2 server used by our applications. Now we want to use Windows SSO for our applications but without them to change anything: they'll still reach our OAuth2 server for an access token and the authentication part will be delegated to Kerberos (which Windows use, if I understood properly).
Is there a way to do that?
That is a standard setup and should just require configuration changes in the Authorization Server (AS) - with zero code changes in applications.
Most commonly:
The AS might be hosted in the cloud
It will redirect browsers to an on premise Identity Provider (IDP)
The IDP can connect to Active Directory
You may also need a fallback option for when users are not joined to the work domain. See this Curity guide for an example and some infrastructure factors to think about.
If the AS is in house it may even be able to make a direct Kerberos connection via an LDAP data source, though the preferred architecture is a separate IDP.
Of course you need an AS that supports the ability to make this type of connection, so would need to check the vendor docs.
REQUEST FLOW
Kerberos has always been the simplest protocol conceptually but the deepest to understand - here is a bit of a summary:
Your apps will make a standard OpenID Connect authorization redirect to the AS
The AS may then present an authentication selection screen to the user, unless there is only a single option
Alternatively an app can send the acr_values query parameter to say which authentication method to use
The AS will then redirect the browser to the next stage of processing, that uses a 'Windows SSO authenticator'
The redirect to the Windows SSO authenticator does not have to use OpenID Connect - it could be any vendor specific HTTP request
The browser will send an encrypted Kerberos ticket automatically by connecting to AD - a prerequisite for this to work might be that the domain in the URL is in the Local Intranet zone on end user computers
The Windows SSO authenticator will need to be able to decrypt this credential, which typically requires a Service Principal Name to be configured
Once the Kerberos ticket is decrypted, the authenticator will make an LDAP connection to an Active Directory data source via its standard LDAP endpoints, to verify the received ticket
I am new to OpenID connect & security domain. I have configured Nifi to use OpenID for authentication using online documentation. And to automate a few nifi related tasks we are using nipyapi.
I have already written python code which does automated flow deployment for basic nifi installation (unsecured & without user authentication)
Now, I have to move the code to secured Nifi installation. How to authenticate to OpenID connect using nipyapi/rest API ?
AS per discussion with Bryan, i am planning to use client certificate for authentication but it started giving authorization error. and have created another question with the details.
Nifi - Client Certificate Authorization Error
OpenID Connect generally requires that you follow a flow of re-directs, typically in the browser. NiFi re-directs you to the login page of the OIDC provider, upon completion, the OIDC provider redirects you back to NiFi. I'm not exactly sure how, or if you even can, perform this login process from scripts. An easy alternative would be to just generate a client certificate to represent an automation user for any NiPyApi scripts. Client certificate authentication is always enabled by default for NiFi.
We are using SCDF on PCF with File-based Authentication, it works fine on a single instance - however when we scale to 2 or more instances, it fails on login stating "Not Logged in" - there's no error message on the server..
Does SCDF store user info in session ? Not sure why login not working when scaled up
SCDF - 1.5.1.RELEASE
(Apparently it was working in 1.3.0.RELEASE)
The file-based authentication is not a recommended approach for cloud platforms like PCF.
In PCF in particular, you'd want to take advantage of the single-sign-on solution provided by the platform. With OAuth and SSO backed by UAA, it'd be a consistent security experience regardless of the number of instances. Please refer to the write-up on authentication options available for SCDF on PCF.
With this, you can centrally also renew an expired OAuth token or even revoke them as needed.
Also, as an FYI, when using the SCDF Tile, all this is automatically configured for you. You'd create an instance of SCDF service from the marketplace and the space-developer can gain access to the Dashboard, REST-APIs, and Shell - all of it works on an SSO model by default.
mine is a kerborised HDP 2.3 cluster with AD users support. we know that by using hadoop's web UI at http://namenode-ip/50070 anyone can access the hdfs details. Can we secure it only to certain authorized users and not publicly to all.. in production.
You can turn on kerberos/SPNEGO for UI access.
This will require SPNEGO to be enabled for your users' browsers as well.
See: https://hadoop.apache.org/docs/r1.2.1/HttpAuthentication.html for general instructions in securing the UIs.
At a high level, you can have the HTTP authentication set to simple, kerberos or the classname of a custom authentication handler.
You may also be interested in using Apache Knox to proxy the UIs in question. This will allow you to provide HTTP Basic Auth against LDAP or a number of other authentication options in order to access the ports. See: http://knox.apache.org/books/knox-0-7-0/user-guide.html#UI+Service+Details for UI proxying details.
This of course would require you to firewall off other direct access to the UIs. It is also worth noting that leaving the HTTP authentication type to simple in secure clusters also leaves the REST API open to anyone that can get to it.
I am designing a service to handle authentication across a number of hosted platforms. This service will need to be able to manage a number of different protocol for the users: LDAP, Shibboleth, no doubt others.
I was hoping to use the Shibboleth protocol internally and run the service as a Shibboleth IdP where depending on user category, nature of the protected resource, etc. the hard work of the authentication is passed on to the native LDAP, Shibboleth, or other server where the user already has an account.
It seems as though this should be possible, but I have not gotten sufficient clarity from the Shibboleth documentation to work out whether it is, let alone how to do it.
Is this possible? How do I do it? Useful documentation very much appreciated.
Shibboleth is not a protocol; it is the name of the software stak that uses the SAML protocol.
Shibboleth can authenticate users from LDAP, AD (and database via a JAAS plugin).
Documentation on hooking up Shibboleth to use LDAP for authentication and attribute retrieval is at: https://wiki.shibboleth.net/confluence/display/SHIB2/ResolverLDAPDataConnector