I am using HDF 3.0
I create keystore and truststore certificate with path and mention in Nifi ranger policy. While I test the connection I got an issue that I have shown below. I follow this link here
NiFi is secured using TLS certificates. When installed and secured via Ambari, by default Ambari installs a NiFi Certificate Authority (CA) and will generate certificates for each NiFi node and get those certificates signed by that CA.
The NiFI node private key is loaded in the keystore.jks file of each NiFi Node and the public key for the CA is loaded in the truststore.jks file on each NiFi node.
When a client (Ranger in the above case is the client) initiates a connection to NiFi, a two-way TLS connection is negotiated. This involves the server sending the node's public key (derived from the certificate in the keystore.jks) to the client. The client will check that key against a list of trustedCertEntries in its truststore.jks file. If it finds the servers public key or the public key of the CA who signed that server key in the truststore.jks, it will trusted the cert provided by the server. The client will then provide its client certificate (derived from ranger keystore.jks private key) to the target NiFi node. NiFi will follow the same steps above to determine if it should trusted the cert provided from that client using its truststore.jks file.
The error you are seeing indicates that this two-way TLS negotiating is failing because Ranger does not trust the cert being presented by NiFi. If you get past this, you will like have failure of trust the other way as well. You need to make sure the truststore.jks used by both NiFi's nodes and Ranger contains all the necessary "trustedCertEntries" for both sides of this connection. (This means having the public key of the NiFi CA loaded in Ranger's truststore.jks file and loading the public key for your Ranger certificate in the truststore.jks file used by your NiFi instances.
Related
I locally installed Elasticsearch with X-Pack and Kibana 8.2.2 and made it up and running. I also created a new project with Spring and Java that would connect with Elasticsearch. When running Elasticsearch for the first time it creates a certificate for the http and transport layers.
I want to use a certificate in Spring to login into Elasticsearch. Therefore I added elasticsearch-rest-client. My problem is, which certificate do I need from Elasticsearch to connect to it and be able to create e.g. role-mappings? So that I have superuser rights?
Quoting the documentation:
When Elasticsearch is configured to require client TLS authentication, for example when a PKI realm is configured, the client needs to provide a client certificate during the TLS handshake in order to authenticate. The following is an example of setting up the client for TLS authentication with a certificate and a private key that are stored in a PKCS#12 keystore.
If the client certificate and key are not available in a keystore but rather as PEM encoded files, you cannot use them directly to build an SSLContext. You must rely on external libraries to parse the PEM key into a PrivateKey instance. Alternatively, you can use external tools to build a keystore from your PEM files, as shown in the following example:
openssl pkcs12 -export -in client.crt -inkey private_key.pem -name "client" -out client.p12
How to set up PKI user authentication has a multi-page documentation and you'll be best served by using the one for your specific Elasticsearch version. The current version is available at https://www.elastic.co/guide/en/elasticsearch/reference/current/pki-realm.html but replace the current in the URL with whatever minor version you want to use (like 8.6).
PS: PKI authentication is a platinum licensed feature. But you can of course set it up with the free 30 day trial to see how it works.
NIFI is unable to connect to URL with https using invokeHTTP Processor, no certificate is required to access the site via browser(only user & pass).
The error observed is "Request Processing Failed: javax.net.SSLPeerUnverifiedException".
I have tried adding SSL Context with Java Truststore and nifi Keystore. But it is not working.Kindly suggest.
When using InvokeHTTP to connect to a HTTPS URL, you will need to add an SSLContextService which InvokeHTTP can use to verify the remote server. The SSLContextService will refer to a truststore which contains the public Certificate Authority. For example if connecting to stackoverflow with NiFi, you would need the CN = ISRG Root X1, O = Internet Security Research Group, C = US installed in a pkcs12 truststore, which is used by the SSLContextService. Another option is to use the truststore provided by Java, typically located at $JAVA_HOME/lib/security/cacerts, which will trust most publicly signed web domain certificates.
Please add more details of the error message if this still is not working.
Could you please help me setup the SSL on the Nifi Application.
To explain about the steps taken so far.
I have used the following link intructions to use the CA signed certs provided to us (This include root,intermediate and Server cert). I have sucessfully configured Nifi to run on SSL on server end but i am not getting the steps to create a client cert so that using the client cert we can login to Nifi.
Help in this regard will be highly appreciated.
You'll need to generate a Certificate Signing Request (CSR) or request from your security/IT team who provided the CA-signed server certs that they provide a client certificate (and private key) signed by the same intermediate or root CA. You could also generate your own client certificate signed by a self-signed CA and put the public certificate of that CA in the NiFi truststore. More documentation around this process can be found in the NiFi Toolkit Guide.
When an application optionally allows you to specify a certificate which represents a certificate authority, what is that doing?
Does that basically set that cert as a "trusted" certificate?
Specific example:
Kibana communicates with elasticsearch. When you configure kibana, you can set the following configuration value:
elasticsearch.ssl.ca
The documentation says this value is an "Optional setting that enables you to specify a path to the PEM file for the certificate authority for your Elasticsearch instance." (source)
Within that same config you're also specifying a certificate and key that can be used for communication with the elasticsearch instance.
If the setting is optional it means that the default behaviour is to use the system Root CAs to validate the SSL Server certificate used by your Elasticsearch instance. If you used a standard commercial SSL Server certificate this should be enough.
Yes, you are setting that certificate as a Root CA but only for this application.
Using the setting you can specify the Root CA used to generated the SSL Server certificate. This is useful if you:
use a selfsigned certificate
use a Root CA that is not available in the system Root CA repository
need to have stringent security settings limiting subset of Root CAs trusted by your application.
Is there any way to allow only https, instead of http, for the communication with the Neo4j server? Also, which channel does the Neo4j Shell's communication use, http or https?
This is from Neo4j Documentation:
HTTPS support
The Neo4j server includes built in support for SSL encrypted communication over HTTPS. The first time the server starts, it automatically generates a self-signed SSL certificate and a private key. Because the certificate is self signed, it is not safe to rely on for production use, instead, you should provide your own key and certificate for the server to use.
To provide your own key and certificate, replace the generated key and certificate, or change the neo4j-server.properties file to set the location of your certificate and key:
# Certificate location (auto generated if the file does not exist)
org.neo4j.server.webserver.https.cert.location=ssl/snakeoil.cert
# Private key location (auto generated if the file does not exist)
org.neo4j.server.webserver.https.key.location=ssl/snakeoil.key
Note that the key should be unencrypted. Make sure you set correct permissions on the private key, so that only the Neo4j server user can read/write it.
Neo4j also supports chained SSL certificates. This requires to have all certificates in PEM format combined in one file and the private key needs to be in DER format.
You can set what port the HTTPS connector should bind to in the same configuration file, as well as turn HTTPS off:
# Turn https-support on/off
org.neo4j.server.webserver.https.enabled=true
# https port (for all data, administrative, and UI access)
org.neo4j.server.webserver.https.port=443
From http://docs.neo4j.org/chunked/stable/security-server.html:
HTTPS support
The Neo4j server includes built in support for SSL encrypted communication over HTTPS. The first time the server starts, it automatically generates a self-signed SSL certificate and a private key. Because the certificate is self signed, it is not safe to rely on for production use, instead, you should provide your own key and certificate for the server to use.
To provide your own key and certificate, replace the generated key and certificate, or change the neo4j-server.properties file to set the location of your certificate and key:
# Certificate location (auto generated if the file does not exist)
org.neo4j.server.webserver.https.cert.location=ssl/snakeoil.cert
# Private key location (auto generated if the file does not exist)
org.neo4j.server.webserver.https.key.location=ssl/snakeoil.key
Note that the key should be unencrypted. Make sure you set correct permissions on the private key, so that only the Neo4j server user can read/write it.
Neo4j also supports chained SSL certificates. This requires to have all certificates in PEM format combined in one file and the private key needs to be in DER format.
You can set what port the HTTPS connector should bind to in the same configuration file, as well as turn HTTPS off:
# Turn https-support on/off
org.neo4j.server.webserver.https.enabled=true
# https port (for all data, administrative, and UI access)
org.neo4j.server.webserver.https.port=443
About your second question: which channel does the Neo4j Shell's communication use, http or https? The default channel is http.