Kafka Failed SSL Handshake with Springboot - spring-boot

I have successfully setup SSL on Kafka broker, it's listening on port 9093. I'm using consumers built in Springboot, and when I attempt to bring up client consumer, I'm getting the following error:
....
Caused by: org.apache.kafka.common.errors.SslAuthenticationException:
SSL handshake failed Caused by: javax.net.ssl.SSLException:
Unrecognized SSL message, plaintext connection? ...
Below is the configuration on the broker: (server.properties)
listeners=PLAINTEXT://0.0.0.0:9092,SSL://0.0.0.0:9093
advertised.listeners=PLAINTEXT://192.168.xx.xx:9092,SSL://192.168.xx.xx:9093
ssl.keystore.location=/home/kafka/kafka/ssl/kafka.server.keystore.jks
ssl.keystore.password=password
ssl.truststore.location=/home/kafka/kafka/ssl/kafka.server.truststore.jks
ssl.truststore.password=password
ssl.key.password=password
ssl.client.auth=required
This is the configuration on the consumer (Springboot application.properties)
spring.kafka.bootstrap-servers=192.168.xx.xx:9093
spring.kafka.security.protocol=SSL
spring.kafka.ssl.trust-store-location=file:/var/ssl/kafka.client.truststore.jks
spring.kafka.ssl.trust-store-password=password
spring.kafka.ssl.key-store-location=file:/var/ssl/kafka.client.keystore.jks
spring.kafka.ssl.key-store-password=password
spring.kafka.ssl.key-password=password
This is how the keystore/truststore have been created:
CA certificate (root.crt) has signed server.crt and client.crt
kafka.server.keystore.jks contains root.crt, server.crt
kafka.server.truststore.jks contains root.crt
kafka.client.keystore.jks contains root.crt, client.crt
kafka.client.truststore.jks contains root.crt
What could I be missing? Anyone can help?

I have discovered 2 possible causes for this:
Server host name verification: this is likely to fail, so it's best to disabled it by setting ssl.endpoint.identification.algorithm to an empty string in application.properties
i.e.
ssl.endpoint.identification.algorithm=
Keystore generation: this is how I was initially doing it:
i. Generated self signed cert and key (output: ca.key, ca-cert.crt)
ii. Generated server key
and server cert, that was signed by root.crt (output: server.key, server.crt)
iii. Created
keystore by running this command:
keytool -keystore kafka.server.keystore.p12 -storetype PKCS12 -alias CARoot -import -file ca-cert.crt -storepass password -keypass password -noprompt
iv. Imported server.crt into kafka.server.keystore.p12
This will NOT work. This is the right way to do it:
i. Similar to step i above, generate self-signed ca-cert
ii. Generate keystore, that contains server.csr (command in the bash script below)
iii. 'Extract' server.csr, from the keystore generated in step ii, and sign it with ca-cert (output server.crt)
iv. Import ca-cert, and then server.crt into keystore
I decided to do all this in a bash file, so I can do it all in one command. I also preferred PKCS12 format - JKS is proprietory; however, in the configuration you may need to specify key-store-type as PKCS12 [key-store-type=PKCS12]
Here is it:
#!/bin/bash
##
#
# This script generates the following
# server.keystore.p12
# server.truststore.p12
# client.keystore.p12
# client.truststore.p12
# They are self signed, and so CA certificate is created first (ca.key, ca-cert.crt)
#
##
# Server Keystore and Truststore
#====================================
# 1. Generate CA key
openssl genrsa -out ca.key
# 2. Generate CA cert
openssl req -new -x509 -key ca.key -out ca-cert.crt -subj "/C=KE/ST=Kenya/L=Nairobi/O=John Doe Ltd/CN=*.mydomain.com"
# 3. Generate server keystore. A server CSR will be generated automatically
keytool -keystore kafka.server.keystore.p12 -alias serverkey -validity 3650 \
-genkey -keyalg RSA -ext SAN=DNS:*.mydomain.com -storetype PKCS12 -dname "CN=*.mydomain.com, OU=ICT, O=John Doe Ltd, L=Nairobi, ST=Kenya, C=KE" -storepass ****** -keypass ****** -noprompt
# 4. Export server CSR generated in step 3
keytool -keystore kafka.server.keystore.p12 -alias serverkey -certreq -file server.csr -storepass password -noprompt
# 5. Sign the server CSR generated in step 4 with the CA key - output is server.crt
openssl x509 -req -CA ca-root.crt -CAkey ca.key -in server.csr -out server.crt -days 3650 -CAcreateserial
# 6. Import CA cert into the server keystore
keytool -keystore kafka.server.keystore.p12 -alias CARoot -import -file ca-cert.crt -storepass ****** -noprompt
# 7. Import the server cert [generated in step 5] into the keystore
keytool -keystore kafka.server.keystore.p12 -alias serverkey -import -file server.crt -storepass ****** -noprompt
# 8. Generate server truststore and import CA cert
keytool -keystore kafka.server.truststore.p12 -alias CARoot -import -file ca-cert.crt -storepass ****** -noprompt
# Client Keystore and Truststore
#====================================
# 9. Generate client keystore. A client CSR will be generated automatically
keytool -keystore kafka.client.keystore.p12 -alias clientkey -validity 3650 \
-genkey -keyalg RSA -ext SAN=DNS:*.mydomain.com -storetype PKCS12 -dname "CN=*.mydomain.com, OU=ICT, O=ABC Bank Ltd, L=Nairobi, ST=Kenya, C=KE" -storepass ****** -keypass ****** -noprompt
# 10. Export client CSR generated in step 9
keytool -keystore kafka.client.keystore.p12 -alias clientkey -certreq -file client.csr -storepass ****** -noprompt
# 11. Sign the client CSR generated in step 4 with the CA key - output is client.crt
openssl x509 -req -CA ca-cert.crt -CAkey ca.key -in client.csr -out client.crt -days 3650 -CAcreateserial
# 12. Import CA cert into the client keystore
keytool -keystore kafka.client.keystore.p12 -alias CARoot -import -file ca-cert.crt -storepass ****** -noprompt
# 13. Import the client cert [generated in step 11] into the keystore
keytool -keystore kafka.client.keystore.p12 -alias clientkey -import -file client.crt -storepass ****** -noprompt
# 14. Generate client truststore and import CA cert
keytool -keystore kafka.client.truststore.p12 -alias CARoot -import -file ca-cert.crt -storepass ******a -noprompt
This worked perfectly for me. Any comments and corrections are welcome.

Related

Getting Invalid keystore format when running Springboot Application

Trying to use https for my springboot application and followed the below steps:
Created the keystore for the certificate and generated Private Key:
keytool -genkey -keysize 2048 -keyalg RSA -alias tomcat -keystore keystore.jks
Generated the CSR with above keytool and private key
keytool -certreq -alias tomcat -file your.csr -keystore keystore.jks
Recived and install certificate on server succesfully
keytool -import -trustcacerts -alias tomcat -file certificate.p7b -keystore keystore.jks -storepass ******
Configured the application with the below:
server.port=8443
server.ssl.key-store-type=JKS
server.ssl.key-store=/path/certificate.p7b
server.ssl.key-store-password=******
server.ssl.key-alias=tomcat
server.ssl.enabled=true
However I keep getting java.io.IOException: Invalid keystore format when I try to run the application.
Am I using the wrong store key type?
Do I have to use PKCS12 insted of JKS as keystore type?

cannot run thinkorswim behind corporate firewall: "unable to find valid certification path to requested target"

In windows, when I install and run thinkorswim inside company network (with its own self-signed ssl cert), it cannot connect via https to tdameritrade's server. How do I update tos's java runtime with the self-signed cert?
open command prompt in windows and run the following command to install company's cacert into thinkorswim's jre:
cd C:\<thinkorswim-install-dir>\jre\bin\
keytool.exe -import -trustcacerts -noprompt -storepass changeit -alias mycertificate -keystore ..\lib\security\cacerts -file c:\mycert.cer
For MacOS
cd /Applications/thinkorswim/.install4j/jre.bundle/Contents/Home/jre/bin
./keytool -import -trustcacerts -noprompt -storepass changeit -alias mycertificate -keystore ../lib/security/cacerts -file /path/to/mycert.cer

How do I generate X.509 certificate from key generated by openssl

I've a web server running on an ec2-instance which internally calls a REST server that is built using Spring Boot. Now, I am trying to get this REST server running under SSL. Here's what I've done so far:
1) Created a CSR & a key file using this command
openssl req -newkey rsa:2048 -nodes -keyout mydomain.key -out mydomain.csr
2) Copied 'csr' to get SSL certificate from GoDaddy.
3) Successfully installed the certificate under Nginx on my ec2-instance.
4) When I hit the home page under https, it works. I no longer get 'Not secure' message from the browser.
5) Login fails because it makes a REST call but REST server is not running under SSL so I am trying to get it running under SSL.
6) Ran following commands:
keytool -import -alias mydomain -keystore tomcat.keystore -trustcacerts -file mydomain.com.chained.crt
keytool -import -alias mydomain-key -keystore tomcat.keystore -trustcacerts -file mydomain.key
The previous command gives me an error message:
"keytool error: java.lang.Exception: Input not an X.509 certificate"
But this was the one created in step 1 above & the same file works under Nginx. What am I missing (other than the fact that I know very little about setting up SSLs!)? I need the second command to specify the value of 'server.ssl.keyAlias' in application.properties, I believe.
Not really an answer but overflowed comment.
You don't need to 'generate' an X.509 cert; you already got that from GoDaddy. If (and only if) the SpringBoot server is accessed by the same name(s) as (external) nginx -- which is unclear to me -- you need to convert the pair of private key AND certificate CHAIN from PEM format to a format Java uses. See:
How to import an existing x509 certificate and private key in Java keystore to use in SSL?
How can I set up a letsencrypt SSL certificate and use it in a Spring Boot application?
How to use .key and .crt file in java that generated by openssl?
Importing the private-key/public-certificate pair in the Java KeyStore
maybe Import key and SSL Certificate into java keystore
Thanks #Dave_thompson_085. Following 2 commands did the trick!
openssl pkcs12 -export -in mydomain.com.chained.crt -inkey mydomain.key -out keystore.p12 -name my-alias -caname root
keytool -importkeystore -deststorepass mypassword -destkeypass mypassword -destkeystore keystore.jks -srckeystore keystore.p12 -srcstoretype PKCS12 -srcstorepass mypassword -alias my-alias
and then in the application.properties I specified following properties:
server.port=8443
server.ssl.enabled=true
security.require-ssl=true
server.ssl.key-store=/etc/nginx/ssl/keystore.jks
server.ssl.key-store-password=mypassword
server.ssl.keyStoreType=JKS
server.ssl.keyAlias=my-alias

ssl setup in embedded tomcat Spring boot

I have a ssl certicate provided by globalsign. I want to import the certificate to use it with tomcat.
I am doing the following operation
Importing chain certificate
keytool -import -alias root11 -keystore server11.p12 -trustcacerts -file chain.pem -keyalg RSA -keysize 2048 -storetype PKCS12
Importing certificate
keytool -import -alias tomcatroot11 -keystore server11.p12 -trustcacerts -file file.cert -keyalg RSA -keysize 2048 -storetype PKCS12
In application.properties
server.ssl.key-store= /Users/Desktop/certificate/server11.p12
server.ssl.key-store-password= password
server.ssl.keyStoreType= PKCS12
server.ssl.keyAlias= tomcatroot11
I am getting the following error on start up of tomcat
java.lang.IllegalArgumentException: java.io.IOException: Alias name [hcmroot11] does not identify a key entry
at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:114) ~[tomcat-embed-core-8.5.16.jar:8.5.16]
What am i doing wrong?
You have imported only the certificates, not any privatekey. SSL/TLS server needs a privatekey AND certficate or usually cert chain. See dupe Java SSLHandshakeException: no cipher suites in common (which has additional links) or crossdupe https://serverfault.com/questions/858102/jboss-https-configuration-with-cer-p7b-certificate-fails (the webserver component of Jboss is a fork of Tomcat).
-keyalg and -keysize are used only for -genkeypair, if you choose the approach of generating the key and (then) CSR in Java with keytool. They are useless and ignored on -importcert because it is impossible to modify any attribute(s) of an already-certified key.

keytool asks for password despite -noprompt and -keypass is set

I'm trying to create an installation script, that reads data out of an .ini file and writes it into several configuration files. One step is to create an SSL key via keytool and to request it's certification (create a csr-file). I now have the problem that keytool asks for the password instead of using the provided one (in the variable $keyStorePassword), although i also have activated -noprompt. I would be glad if anyone could help.
keytool -genkey -noprompt -keyalg RSA -keysize 2048 \
-dname "CN=${HOSTNAME}-${microserviceName}-${environment}, O=${organizationName}, L=${localityName}, S=${stateName}, C=${country}" \
-validity ${validity} -keypass ${keyStorePassword} -keystore ${keyStorePath} -alias ${microserventerviceName}-ssl
keytool -certreq -keyalg RSA -file ${microserviceName}.csr -keystore ${keyStorePath} -alias ${microserviceName}-ssl
I've already searched the web but the problem doesn't seem to be common.
There are two parameters for passwords:
-keypass: The password for the key.
-storepass: The password for the keystore.
Each key entry in a Java keystore has its individual password, so you have to provide both the keystore password and the key password when generating a new key or accessing a key.

Resources