how to access kafka with acl in my storm jar? - apache-storm

Question

Before I use kafka with no any SASL or ACL，it works fine, now i built SASL_PLAINTEXT for my kafka cluster，how to access my kafka cluster in my storm jar?
My storm version:1.1.0
Kafka version:0.10.1
I use HDP-2.6.3.0-235
any suggestions would be helpful.thanks

Answer 1

These are the steps I followed to get storm running with kafka in a secured environment
1.You need to use hdp specific dependencies for storm to enable the security protocol property
<repositories>
<repository>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
<id>hortonworks.extrepo</id>
<name>Hortonworks HDP</name>
<url>http://repo.hortonworks.com/content/repositories/releases</url>
</repository>
<repository>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
<id>hortonworks.other</id>
<name>Hortonworks Other Dependencies</name>
<url>http://repo.hortonworks.com/content/groups/public</url>
</repository>
</repositories>
and then include the hdp specific dependency for storm[change it as per your hdp version]
<storm-core.version>1.1.0.2.6.2.37-3</storm-core.version>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>${storm-core.version}</version>
<scope>provided</scope>
</dependency>
2.Now you need use the security protocol property while creating spouts(it will not be availaible unless using hdp specific repo)
spoutCfg.securityProtocol = "SASL_PLAINTEXT";
3.Use acl command on your topic by giving the appropriate storm user
bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:storm-user --allow-principal User:zriskcanvas --consumer --producer --group=* --topic testTopic
4.Make sure the storm jaas-conf is getting passed to each node with the user mentioned in ACL