How to use jaas configuration in spring-cloud-stream-binder-kafka-streams - spring

Question

I followed this example to create an authentication in my broker kafka, however there are two differences for my project
I use kafka streams binder, while in the example kafka binder is
used.
I only have one broker, while in the example two brokers are
used.
When I run my application it returns the following error:
https://gist.github.com/JacsonF/7363e8d7f4f07a5bc77c28fb1e882674
My function:
#Bean
public Function<KStream<Object, String>, KStream<?, String>[]> receive() {
Predicate<Object, String> isCompensado = (k, v) -> v.contains("XPTO");
Predicate<Object, String> isntCompensado = (k, v) -> !v.contains("XPTO");
return input -> input.branch(isCompensado, isntCompensado);
}
I would like to use only the properties of my application to authenticate, without an external file, just like in the example.
How can I do this?
application.yml

Answer 1

There was an issue with the way JAAS configuration was processed in Kafka Streams binder for Spring Cloud Stream. See this issue for more details. This is fixed now. Make sure that you are using version 3.1.4-SNAPSHOT. Here is a sample application that verifies this.
Update
The Kafka binders for Spring Cloud Stream have been released, version 3.1.4 has the fix.
If you're using spring-cloud-dependencies in your pom to manage dependencies, update it to (at least) version 2020.0.4
For example:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>2020.0.4</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>

Answer 2

This works fine for me:
spring.kafka.jaas.enabled=true
spring.kafka.properties.security.protocol=SASL_SSL
spring.kafka.properties.sasl.mechanism=PLAIN
spring.kafka.properties.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="username" password="password";
Output:
2021-05-18 18:50:56.505 INFO 75288 --- [ main] o.s.c.s.binder.DefaultBinderFactory : Creating binder: ktable
2021-05-18 18:50:56.595 INFO 75288 --- [ main] o.s.c.s.binder.DefaultBinderFactory : Caching the binder: ktable
2021-05-18 18:50:56.596 INFO 75288 --- [ main] o.s.c.s.binder.DefaultBinderFactory : Retrieving cached binder: ktable
2021-05-18 18:50:56.722 INFO 75288 --- [ main] o.a.k.clients.admin.AdminClientConfig : AdminClientConfig values:
............ (property)=(value) ............
sasl.mechanism = PLAIN
security.protocol = SASL_SSL
............ (property)=(value) ............
2021-05-18 18:50:56.960 INFO 75288 --- [ main] o.a.k.c.s.authenticator.AbstractLogin : Successfully logged in.
2021-05-18 18:50:57.131 WARN 75288 --- [ main] o.a.k.clients.admin.AdminClientConfig : The configuration 'sasl.jaas.config' was supplied but isn't a known config.
2021-05-18 18:50:57.134 INFO 75288 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka version: 2.6.0
2021-05-18 12:46:31.423 INFO 74020 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka commitId: 62abe01bee039651
2021-05-18 12:46:31.423 INFO 74020 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka startTimeMs: 1621367191408
2021-05-18 12:46:31.423 INFO 74020 --- [ main] o.s.c.s.b.k.p.KafkaTopicProvisioner : Auto creation of topics is disabled.
2021-05-18 12:46:31.456 INFO 74020 --- [ main] org.apache.kafka.streams.StreamsConfig : StreamsConfig values:
I tried the following but not working for me:
https://github.com/spring-cloud/spring-cloud-stream-binder-kafka#using-spring-boot-properties
Using the properties from the above link to set the same values as the first way will have the following error:
2021-05-18 18:59:44.140 INFO 73444 --- [ main] o.s.c.s.binder.DefaultBinderFactory : Creating binder: ktable
2021-05-18 18:59:44.231 INFO 73444 --- [ main] o.s.c.s.binder.DefaultBinderFactory : Caching the binder: ktable
2021-05-18 18:59:44.232 INFO 73444 --- [ main] o.s.c.s.binder.DefaultBinderFactory : Retrieving cached binder: ktable
2021-05-18 18:59:44.352 INFO 73444 --- [ main] o.a.k.clients.admin.AdminClientConfig : AdminClientConfig values:
............ (property)=(value) .............
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
............ (property)=(value) ............
2021-05-18 18:59:44.596 INFO 73444 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka version: 2.6.0
2021-05-18 18:59:44.597 INFO 73444 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka commitId: 62abe01bee039651
2021-05-18 18:59:44.597 INFO 73444 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka startTimeMs: 1621389584594
2021-05-18 18:59:44.599 INFO 73444 --- [ main] o.s.c.s.b.k.p.KafkaTopicProvisioner : Auto creation of topics is disabled.
2021-05-18 18:59:44.611 INFO 73444 --- [| adminclient-1] o.a.k.c.a.i.AdminMetadataManager : [AdminClient clientId=adminclient-1] Metadata update failed
org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1621389614599, tries=1, nextAllowedTryMs=-9223372036854775709) timed out at 9223372036854775807 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call.
From the logging, you can see that the second way is NOT using the values you provided and there is no "Successfully logged in".
By the way, I am using the following dependencies:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kafka-streams</artifactId>
<version>3.1.3-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream</artifactId>
<version>3.1.3-SNAPSHOT</version>
</dependency>
and the following repositories:
<repositories>
<repository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/libs-snapshot-local</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>false</enabled>
</releases>
</repository>
<repository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/libs-milestone-local</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>confluent</id>
<url>https://packages.confluent.io/maven/</url>
</repository>
</repositories>