Kafka Failed SSL Handshake with Springboot - spring-boot

Question

I have successfully setup SSL on Kafka broker, it's listening on port 9093. I'm using consumers built in Springboot, and when I attempt to bring up client consumer, I'm getting the following error:
....
Caused by: org.apache.kafka.common.errors.SslAuthenticationException:
SSL handshake failed Caused by: javax.net.ssl.SSLException:
Unrecognized SSL message, plaintext connection? ...
Below is the configuration on the broker: (server.properties)
listeners=PLAINTEXT://0.0.0.0:9092,SSL://0.0.0.0:9093
advertised.listeners=PLAINTEXT://192.168.xx.xx:9092,SSL://192.168.xx.xx:9093
ssl.keystore.location=/home/kafka/kafka/ssl/kafka.server.keystore.jks
ssl.keystore.password=password
ssl.truststore.location=/home/kafka/kafka/ssl/kafka.server.truststore.jks
ssl.truststore.password=password
ssl.key.password=password
ssl.client.auth=required
This is the configuration on the consumer (Springboot application.properties)
spring.kafka.bootstrap-servers=192.168.xx.xx:9093
spring.kafka.security.protocol=SSL
spring.kafka.ssl.trust-store-location=file:/var/ssl/kafka.client.truststore.jks
spring.kafka.ssl.trust-store-password=password
spring.kafka.ssl.key-store-location=file:/var/ssl/kafka.client.keystore.jks
spring.kafka.ssl.key-store-password=password
spring.kafka.ssl.key-password=password
This is how the keystore/truststore have been created:
CA certificate (root.crt) has signed server.crt and client.crt
kafka.server.keystore.jks contains root.crt, server.crt
kafka.server.truststore.jks contains root.crt
kafka.client.keystore.jks contains root.crt, client.crt
kafka.client.truststore.jks contains root.crt
What could I be missing? Anyone can help?

Answer 1

I have discovered 2 possible causes for this:
Server host name verification: this is likely to fail, so it's best to disabled it by setting ssl.endpoint.identification.algorithm to an empty string in application.properties
i.e.
ssl.endpoint.identification.algorithm=
Keystore generation: this is how I was initially doing it:
i. Generated self signed cert and key (output: ca.key, ca-cert.crt)
ii. Generated server key
and server cert, that was signed by root.crt (output: server.key, server.crt)
iii. Created
keystore by running this command:
keytool -keystore kafka.server.keystore.p12 -storetype PKCS12 -alias CARoot -import -file ca-cert.crt -storepass password -keypass password -noprompt
iv. Imported server.crt into kafka.server.keystore.p12
This will NOT work. This is the right way to do it:
i. Similar to step i above, generate self-signed ca-cert
ii. Generate keystore, that contains server.csr (command in the bash script below)
iii. 'Extract' server.csr, from the keystore generated in step ii, and sign it with ca-cert (output server.crt)
iv. Import ca-cert, and then server.crt into keystore
I decided to do all this in a bash file, so I can do it all in one command. I also preferred PKCS12 format - JKS is proprietory; however, in the configuration you may need to specify key-store-type as PKCS12 [key-store-type=PKCS12]
Here is it:
#!/bin/bash
##
#
# This script generates the following
# server.keystore.p12
# server.truststore.p12
# client.keystore.p12
# client.truststore.p12
# They are self signed, and so CA certificate is created first (ca.key, ca-cert.crt)
#
##
# Server Keystore and Truststore
#====================================
# 1. Generate CA key
openssl genrsa -out ca.key
# 2. Generate CA cert
openssl req -new -x509 -key ca.key -out ca-cert.crt -subj "/C=KE/ST=Kenya/L=Nairobi/O=John Doe Ltd/CN=*.mydomain.com"
# 3. Generate server keystore. A server CSR will be generated automatically
keytool -keystore kafka.server.keystore.p12 -alias serverkey -validity 3650 \
-genkey -keyalg RSA -ext SAN=DNS:*.mydomain.com -storetype PKCS12 -dname "CN=*.mydomain.com, OU=ICT, O=John Doe Ltd, L=Nairobi, ST=Kenya, C=KE" -storepass ****** -keypass ****** -noprompt
# 4. Export server CSR generated in step 3
keytool -keystore kafka.server.keystore.p12 -alias serverkey -certreq -file server.csr -storepass password -noprompt
# 5. Sign the server CSR generated in step 4 with the CA key - output is server.crt
openssl x509 -req -CA ca-root.crt -CAkey ca.key -in server.csr -out server.crt -days 3650 -CAcreateserial
# 6. Import CA cert into the server keystore
keytool -keystore kafka.server.keystore.p12 -alias CARoot -import -file ca-cert.crt -storepass ****** -noprompt
# 7. Import the server cert [generated in step 5] into the keystore
keytool -keystore kafka.server.keystore.p12 -alias serverkey -import -file server.crt -storepass ****** -noprompt
# 8. Generate server truststore and import CA cert
keytool -keystore kafka.server.truststore.p12 -alias CARoot -import -file ca-cert.crt -storepass ****** -noprompt
# Client Keystore and Truststore
#====================================
# 9. Generate client keystore. A client CSR will be generated automatically
keytool -keystore kafka.client.keystore.p12 -alias clientkey -validity 3650 \
-genkey -keyalg RSA -ext SAN=DNS:*.mydomain.com -storetype PKCS12 -dname "CN=*.mydomain.com, OU=ICT, O=ABC Bank Ltd, L=Nairobi, ST=Kenya, C=KE" -storepass ****** -keypass ****** -noprompt
# 10. Export client CSR generated in step 9
keytool -keystore kafka.client.keystore.p12 -alias clientkey -certreq -file client.csr -storepass ****** -noprompt
# 11. Sign the client CSR generated in step 4 with the CA key - output is client.crt
openssl x509 -req -CA ca-cert.crt -CAkey ca.key -in client.csr -out client.crt -days 3650 -CAcreateserial
# 12. Import CA cert into the client keystore
keytool -keystore kafka.client.keystore.p12 -alias CARoot -import -file ca-cert.crt -storepass ****** -noprompt
# 13. Import the client cert [generated in step 11] into the keystore
keytool -keystore kafka.client.keystore.p12 -alias clientkey -import -file client.crt -storepass ****** -noprompt
# 14. Generate client truststore and import CA cert
keytool -keystore kafka.client.truststore.p12 -alias CARoot -import -file ca-cert.crt -storepass ******a -noprompt
This worked perfectly for me. Any comments and corrections are welcome.