How to subscribe to Kafka topic by key - spring-boot

Question

I'm super new to using Kafka, especially within a Kotlin service. I currently have a local broker which I'm able to send events to I'm trying to figure out how to subscribe to my topic and receive the message but I'm not able to actually consume the event by a key.
import io.confluent.kafka.serializers.KafkaAvroDeserializer
import io.confluent.kafka.serializers.KafkaAvroDeserializerConfig
import io.confluent.kafka.serializers.subject.TopicRecordNameStrategy
import org.apache.kafka.clients.consumer.ConsumerConfig
import org.apache.kafka.clients.consumer.KafkaConsumer
import org.apache.kafka.common.serialization.StringDeserializer
import org.apache.logging.log4j.LogManager
import org.springframework.beans.factory.annotation.Qualifier
import org.springframework.beans.factory.annotation.Value
import org.springframework.context.annotation.Bean
import org.springframework.kafka.annotation.KafkaListener
import org.springframework.stereotype.Service
#Service
class KafkaMessageConsumer {
#Bean("consumerConfig")
fun consumerConfigs(
#Value("\${KAFKA_SASL_USERNAME}")
kafkaServer: String,
#Value("\${KAFKA_SASL_USERNAME}")
userName: String,
#Value("\${KAFKA_SASL_PASSWORD}")
password: String,
#Value("\${KAFKA_SCHEMA_REGISTRY_URL}")
schemaRegistryUrl: String,
#Value("\${KAFKA_SCHEMA_REGISTRY_USER_INFO}")
schemaRegistryUserInfo: String
): Map<String, Any> {
val config: MutableMap<String, Any> = mutableMapOf()
config[ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG] = kafkaServer
config[ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG] = StringDeserializer::class.java
config[ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG] = KafkaAvroDeserializer::class.java
config[KafkaAvroDeserializerConfig.VALUE_SUBJECT_NAME_STRATEGY] = TopicRecordNameStrategy::class.java
config[KafkaAvroDeserializerConfig.SCHEMA_REGISTRY_URL_CONFIG] = schemaRegistryUrl
return config
}
#KafkaListener(topics = ["my.kafka.topic"], groupId = "group-id")
fun consume(
#Qualifier("consumerConfig")
config: Map<String, Any>,
message: String
) : Unit {
val consumer: KafkaConsumer<String, String> = KafkaConsumer(config)
println("message received from topic: $message")
}
}
This is pretty much all I have. There are a list of IDs I have in the db that I would like to use to consume these events via a key that matches the ID. I'm using this article as a reference

Answer 1

It is not possible to subscribe to a key.
You can change the function parameters to get a key in addition to the message record value (see docs on #Header(KafkaHeaders.RECEIVED_KEY)), then you will just use normal if-statement to compare and look for specific keys for all partitions in the topic.
Related - Message Filtering