SpringBoot Kafka: Bean method 'kafkaTemplate' in 'KafkaAutoConfiguration' is not loaded - spring

Question

I am using springboot and trying to write a KafkaProducer to push messages in Kafka queue.
I have created these methods in #Configuration class.
#Bean
public KafkaTemplate<String, String> kafkaTemplate(){
return new KafkaTemplate<>(producerFactory());
}
#Bean
public ProducerFactory<String, String> producerFactory() {
Map<String, Object> configProps = new HashMap<>();
configProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress); //bootstrapAddress holds address of kafka server
configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
configProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
return new DefaultKafkaProducerFactory<>(configProps);
}
And I have Autowired this KafkaTemplate bean in my KafkaMessageProducer class that takes care of the handling of send function of KafkaTemplate.
#Autowired
KafkaTemplate<String, String> kafkaTemplate;
But I am facing this error when I try to compile my code
Field kafkaTemplate in <pathoffile>.KafkaMessageProducer required a bean of type 'org.springframework.kafka.core.KafkaTemplate' that could not be found.
- Bean method 'kafkaTemplate' in 'KafkaAutoConfiguration' not loaded because #ConditionalOnMissingBean (types: org.springframework.kafka.core.KafkaTemplate; SearchStrategy: all) found bean 'avroKafkaTemplate'
Action:Consider revisiting the conditions above or defining a bean of type 'org.springframework.kafka.core.KafkaTemplate' in your configuration.
Also, if I try to exclude KafkaAutoConfiguration in my Spring project, I get error like "Bean cannot be loaded as KafkaAutoConfiguration is disabled'.
Any idea why I am getting this Bean error and what may be the solution?
EDIT:- I found following bean in a jar file used by my project
#Bean
#Conditional({EnableQueueCondition.class})
public KafkaTemplate<String, String> kafkaTemplate() {
KafkaTemplate<String, String> kafkaTemplate = new KafkaTemplate(this.producerFactory());
kafkaTemplate.setProducerListener(new ProducerListenerImpl());
return kafkaTemplate;
}
So, this is where the error is coming from, but I don't how to tell spring to not look to this bean, and use the bean I defined. I have tried using Primary and Qualifier annotations on the bean, it still gives the same error. Is there might be a possibility that my defined bean is not created or not found, and KafkaAutoConfiguration is then looking for the default bean that is override by avroKafkaTemplate bean? What may be the solution of this problem?

Answer 1

By default spring boot provide KafkaTemplate bean if you add kafka dependency in POM.
You just need to define the properties in your application.yml file
example:
server: port: 9000
spring:
kafka:
consumer:
bootstrap-servers: localhost:9092
group-id: group_id
auto-offset-reset: earliest
key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
producer:
bootstrap-servers: localhost:9092
key-serializer: org.apache.kafka.common.serialization.StringSerializer
value-serializer: org.apache.kafka.common.serialization.StringSerializer
enable auto configuration
#Configuration
#EnableKafka
and autowire the kafkaTemplate:
#Autowired
private KafkaTemplate<String, String> kafkaTemplate;
If your case the auto config factory is looking for ProducerFactory<String, Strng>which does not match with your configuration.
#Bean
#ConditionalOnMissingBean(ProducerFactory.class)
public ProducerFactory<String, Strng> kafkaProducerFactory()
so rename your producerFactory() to kafkaProducerFactory(), it will solve your issue.

Answer 2

From the stacktrace, there is another KafkaTemplate bean - avroKafkaTemplate. So I guess there is another configuration, duplicating KafkaTemplate definition.