Having my redis server with self_signed certificate (in certificate_base64 format) running on IBM cloud. wanted to connect to redis (server) through SSL from Spring Boot application (client).
Tried decrypting the base64 and saved it as .cer file and stored that certification in C:\Program Files\Java\jre1.8.0_341\lib\security\cacerts using the command line keytool -keystore "JRE_CA_CERT_PATH" -import -alias servername -file ".cer FILE"
but it was not as expected, still got unable to find valid certification path to requested target
I'm super new to this topic, any thoughts here would be helpful :-)
Related
community!
I'm trying to connect my Mulesoft application to a Kafka Server that is hosted at Heroku. Anypoint Studio 7 (Mule 4) have a Kafka connector that has a few options to connect (Basic, Kerberos, Kerberos SSL and SSL):
According to Heroku's docs it "support" SSL but does not mention that is required. Can anyone confirm?
Heroku documentation that I saw [https://devcenter.heroku.com/articles/kafka-on-heroku]
When I set up an app at Heroku, add the Kafka AddOn, create the topic, I get the bootstrap server(s) from running the command heroku config:get KAFKA_URL If I try to to abasic test I get the following error:
The error states org.mule.runtime.api.connection.ConnectionException: invalid connection!
org.mule.runtime.api.connection.ConnectionException: invalid connection!
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
what I'm trying to build is a PoC with 2 flows... one that produces a message to a Topic and the other flow to consume from it.
Any help on how to set up the connector and the Heroku environment will be most welcome
UPDATE:
I found that when you create the app at Heroku and include the KAFKA addon, you get the following variables:
KAFKA_URL: A comma-separated list of SSL URLs to the Kafka brokers making up the cluster. Example:
kafka+ssl://ec2-3-*****-100.compute-1.amazonaws.com:9096,kafka+ssl://ec2-3-******-127.compute-1.amazonaws.com:9096
KAFKA_TRUSTED_CERT: The brokers’ SSL certificate (in PEM format), to check that you’re connecting to the right servers. Example:
-----BEGIN CERTIFICATE----- MIIDfzCCAmegAwIBAgIBADANBgkqhkiG9w0BAQsFADAyMTAwLgYDVQQDDCdjYS1l
ZjQwOWUzNy00NjhhLTRiMGEtOGVkOC0wZWYxMmRhYjkyZWEwHhcNMTkxMjEyMTUx
NzU5WhcNMjkxMjEyMTUxNzU5WjAyMTAwLgYDVQQDDCdjYS1lZjQwOWUzNy00Njhh
KAFKA_CLIENT_CERT: The required client certificate (in PEM format) to authenticate clients against the broker. Example:
-----BEGIN CERTIFICATE----- MIIDQzCCAiugAwIBAgIBADANBgkqhkiG9w0BAQsFADAyMTAwLgYDVQQDDCdjYS1l
ZjQwOWUzNy00NjhhLTRiMGEtOGVkOC0wZWYxMmRhYjkyZWEwHhcNMjAwMTE1MTU1
MjU2WhcNMzAwMTE1MTU1MjU2WjAZMRcwFQYDVQQDDA51NnZtYWVzM2cwZnMyZjCC
KAFKA_CLIENT_CERT_KEY: The required client certificate key (in PEM format) to authenticate clients against the broker.
Kafka clusters require authenticating using the provided client certificate. Any requests not using the client certificate will be denied. Example:
-----BEGIN RSA PRIVATE KEY----- MIIEpAIBAAKCAQEAmmu+j9DulVnqwIOt02++6Ehw9Mg7KaocdIQjODVtVipC5AyK
iaHsdNVh9bgJQAJSfstIV/6O1mCLmjUS/YmyIEEgmBvATFxcldPGBGcpPVSV2R3Q
I assume that I need to "transform" those into the KeyStore and TrustStore for the Connector SSL config.. can anyone confirm? since for what I see from the docs, SSL is required
Another Update:
I downloaded the certificates I describe above and using the tool "Key explorer" I created a keystore.jks and imported the KAFKA_CLIENT_CERT and KAFKA_CLIENT_CERT_KEY into it and then I created a truststore.jsk and imported the file KAFKA_TRUSTED_CERT there. on both cases I set a basic password ... was looking good but I get an error:
org.mule.runtime.api.connection.ConnectionException: invalid connection!
org.mule.runtime.api.connection.ConnectionException: invalid connection!
Caused by: org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521)
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1197)
at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1165)
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
at org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:448)
at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:313)
at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:547)
at org.apache.kafka.common.network.Selector.poll(Selector.java:483)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:540)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1196)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
... 7 more
Caused by: java.security.cert.CertificateException: No name matching ec2-3-220-121-33.compute-1.amazonaws.com found
at sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:231)
at sun.security.util.HostnameChecker.match(HostnameChecker.java:96)
at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:455)
at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:252)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
... 16 more
Could be related to the line Caused by:
java.security.cert.CertificateException: No name matching
ec2-3-220-121-33.compute-1.amazonaws.com found
I got it to work.
create the JKS files using these commands (require HEROKU CLI)
client_key=`heroku config:get KAFKA_CLIENT_CERT_KEY --app <SET_HEROKU_APP_NAME_HERE>`
client_cert=`heroku config:get KAFKA_CLIENT_CERT --app <SET_HEROKU_APP_NAME_HERE>`
trusted_cert=`heroku config:get KAFKA_TRUSTED_CERT --app <SET_HEROKU_APP_NAME_HERE>`
# Write config vars to files.
echo "$client_key" >> keystore.pem
echo -n "$client_cert" >> keystore.pem
echo -n "$trusted_cert" > truststore.pem
# Set passwords
TRUSTSTORE_PASSWORD=<SET_PASSWORD_HERE>
KEYSTORE_PASSWORD=<SET_PASSWORD_HERE>
echo $TRUSTSTORE_PASSWORD
echo $KEYSTORE_PASSWORD
# Import cert.
keytool -importcert -file truststore.pem -keystore kafka.client.truststore.jks -deststorepass $TRUSTSTORE_PASSWORD -noprompt
# Create PKCS12 file.
openssl pkcs12 -export -in keystore.pem -out keystore.pkcs12 -password pass:$KEYSTORE_PASSWORD
# Create jks files.
keytool -importkeystore -srcstoretype PKCS12 \
-destkeystore kafka.client.keystore.jks -deststorepass $KEYSTORE_PASSWORD \
-srckeystore keystore.pkcs12 -srcstorepass $KEYSTORE_PASSWORD
Then config the connector with the SSL option and add the following INLINE:
I have configured HTTPS on a Keycloak server by placing a certificate A.crt in a Java keystore and configuring that keystore in standalone.xml. HTTPS access has worked well from a given client so far. The certificate is signed by an intermediate CA B, which is signed by a root CA C.
I now encounter problems when accessing Keycloak from another client. It says: "unable to get local issuer certificate" or "certificate signed by unknown authority" (depending on the client software). I'm quite sure the root cause is that CA certificate B.crt is not known to this client. A strong indication is that I can access Keycloak from that client with curl --cacert B.crt https://keycloak....
Since I cannot predict which clients will have what CA certificates installed I would like to configure a bundle consisting of A.crt, B.crt, and C.crt as root certificate. If this were a server that directly took a PEM certificate I would use the bundle resulting from cat A.crt B.crt C.crt > bundle.crt, but because of its use of a Java keystore this is not an option with Keycloak.
So how can I store a server certificate bundle (it that's the right terminology) into a Java keystore so that Keycloak can make use of it in such a way that "all" clients can access the service?
UPDATE I've tried adding the intermediate and root certificates B and C to the keystore already used by Keycloak, but this did not solve the current issue:
keytool -import -alias b -file B.crt -keystore $KEYSTORE
keytool -import -alias c -file C.crt -keystore $KEYSTORE
You said having a keystore configured in your standalone.xml where A.crt was imported, but do you have a truststore? You could try adding B.crt in your truststore.
I am new to SSL setup, please excuse me if my question is wrong.
I have deployed a Spring Boot application on AWS EC2 (Windows) instance with bunch of restful services, exposed through public IP address (AWS), i am able to access them publicly(http). I want to SSL(https) them now. I am in process of purchasing certificate, in one of the steps to setup, they have given these lines to validate a text file, is anyone aware of this ? Can you please suggest where i need to create ./well-known/pki-validation folder on my Spring Boot application(Tomcat) ?
The issuing vendor will provide you with a simple text-based file to place in sub-folders /.well-known/pki-validation/ in your site’s "home directory". If done properly, the vendor can view this file via HTTP:// and then issue the certificate upon confirmation.
1 Install certboot in the server.
git clone https://github.com/certbot/certbot
cd certbot
./certbot-auto --help
2 Obtain the certificate
In order to obtain the certificate you need to expose trough the server certain files. I do that using the target folder of spring boot tomcat.
./certbot-auto certonly --webroot -w {SpringBootProjectDir}/target/classes/static/ -d {yourDomain.com}
This command obtains the certificates and leaves them in:
/etc/letsencrypt/live/{yourDomain.com}/
Tomcat can't read the certificate provided since its not in p12 format. We have to generate the cert in this format. Use this command
sudo openssl pkcs12 -export -in /etc/letsencrypt/live/{yourDomain.com}/fullchain.pem -inkey /etc/letsencrypt/live/{yourDomain.com}/privkey.pem -out /etc/letsencrypt/live/{yourDomain.com}/keystore.p12 -name tomcat -CAfile /etc/letsencrypt/live/{yourDomain.com}/chain.pem -caname root
It will ask you a password. Keep the password.
3 Configure the server
server.port=443
server.ssl.enabled=true
server.ssl.key-store: /etc/letsencrypt/live/{yourDomain.com}/keystore.p12
server.ssl.key-store-password: {password}
server.ssl.keyStoreType: PKCS12
server.ssl.keyAlias: tomcat
Restart the server and Thats it!
By default setting up Solr on Https requires storing the certificate password as plain text in "bin\solr.in.cmd" (using windows to host solr)
And the certificate password is exposed in the portal.
How can I configure Solr with a Https certificate keeping the certificate password safe?
I want to use a private certifcate so the client trusts the certificate by default, rather than a self signed one.
I'm connecting via SolrNet from C#, currently without authentiation, but in the future can look at Basic Authentication which is supported in the provider which wraps SolrNet.
Seems like a basic thing which I would have thought should be supported? Can't find any article/documention on this.
Anyone solved this already?
Nearest I've found is:
Setting SOLR SSL properties
http://lucene.472066.n3.nabble.com/Prevent-the-SSL-Keystore-and-Truststore-password-from-showing-up-in-the-Solr-Admin-and-Linux-process-td4257422.html
Some suggestion of using jetty-https.xml and encrypt the configuration there.
Looking for a more complete answer.
update
I've come across the following Solr tickets:
https://issues.apache.org/jira/browse/SOLR-10307
https://issues.apache.org/jira/browse/SOLR-8897
Where support for loading the certificate passwords from environment variables has been added in Solr 7.0 and 6.7 (which hasn't been released yet).
I don't think using environment variables is much better, but some progress.
There is also a link to using Hadoop credential provider, but I'm not using Hadoop, so not applicable for me.
In the comments of the tickets, it mentions that someone tried the Jetty Password Utility, and Solr UI loaded over HTTPS. However the Collection API was erroring not being able to understand the password format.
So I'm still left without a way of configuring a certificate password for Solr, in a way I feel is secure.
For me the certificate password is not stored on the SOLR portal
see my setup
SSL
cd /path/to/solr/server/etc/
The following command will create a keystore file named solr-ssl.keystore.jks in the current directory
keytool -genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 -keypass secret -storepass secret -validity 9999 -keystore solr-ssl.keystore.jks -ext SAN=DNS:localhost,IP:#ipsolr,IP:127.0.0.1 -dname "CN=localhost, OU=xxx, O=xxx, L=xxx, ST=xxx, C=xxx"
Convert the certificate and key to PEM format for use with Curl
keytool -importkeystore -srckeystore solr-ssl.keystore.jks -destkeystore solr-ssl.keystore.p12 -srcstoretype jks -deststoretype pkcs12
Next convert the PKCS12 format keystore into PEM format using the openssl
openssl pkcs12 -in solr-ssl.keystore.p12 -out solr-ssl.pem
After go to your $JAVA_HOME (into jre/lib/security)
curl --cacert /pah/to/solr/server/etc/solr-ssl.pem https://localhost:8983/solr/
Set Common SSL-Related System Properties
vi /etc/default/solr.in.sh
SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.jks
SOLR_SSL_KEY_STORE_PASSWORD=secret
SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.jks
SOLR_SSL_TRUST_STORE_PASSWORD=secret
SOLR_SSL_NEED_CLIENT_AUTH=false
SOLR_SSL_WANT_CLIENT_AUTH=false
SOLR_SSL_KEY_STORE_TYPE=JKS
SOLR_SSL_TRUST_STORE_TYPE=JKS
Restart solr
service solr restart
For authentication
create security.json on SOLR_HOME (/var/solr/data/ for me)
Owner : root
See example of security.json to configure has you hope
After securing, add into solr.in.sh
SOLR_AUTH_TYPE="basic"
SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:xxx"
it allows to have an answer on solr status service because with authentication it is not accessible
For me it's work and i don't see the certificate password on solr UI
I have an existing spring webapp which uses SAML to let users login via a third party site. Users login on the third party site and use SAML to single-sign-on into my application. The third party site recently had their certificate expire and they sent me a new .crt file to import into my java keystore. I used the following commands to update my java keystore:
keytool -delete -alias thirdPartyCertificate -keystore myKeystore.jks
keytool -import -trustcacerts -alias thirdPartyCertificate -file 2017-third-party-certificate.crt -keystore myKeystore.jks
Both commands completed successfully (and I can run a -list on the keystore and see that the new certificate is present and not expired). However, after I restarted my java webserver, the single sign on did not work. It threw the following two warnings and then failed to login the user:
WARN org.opensaml.common.binding.security.BaseSAMLSimpleSignatureSecurityPolicyRule - Simple signature validation (with no request-derived credentials) failed
WARN org.opensaml.common.binding.security.BaseSAMLSimpleSignatureSecurityPolicyRule - Validation of request simple signature failed for context issuer: <certificate_url_omitted>
What would cause the certificate validation to fail? I don't think that their certificate is bad because several other webapp maintainers were able to successfully use the new certificate. Is there some step that I missed in installing the new certificate?
You will need to set the signingKey property to alias of the certificate you imported in the ExtendedMetadata of your IDP. See https://docs.spring.io/spring-security-saml/docs/1.0.x-SNAPSHOT/reference/htmlsingle/#configuration-key-management (8.2.1)