Kafka producer is not recreated with #RefreshScope - spring

Question

I have a config server and several clients that work with Kafka. I would like my clients to work non-stop when I update config properties on a config server.
Here is my configuration setup:
#Component
#RefreshScope
#ConfigurationProperties(prefix = "params")
public class ConfigParams {
private KafkaParams kafkaParams = new KafkaParams(); // some custom params (doesn't matter)
// ...
}
#Configuration
public class KafkaConfig {
#Autowired
private ConfigParams configParams;
// ...
#Bean
#RefreshScope
MyKafkaBean myKafkaBean() {
return new MyKafkaBean(configParams.getKafkaParams()); // here I create my own kafka bean with topics, producers and consumers
}
// ...
}
Problem description:
If I change bootstrap.servers property to an incorrect and the kafka producer was invoked, there will be a WARN with a message that producer cannot connect to a kafka broker (that is correct). When I fix this property the connection is back and everything works fine, but I still have WARNs about no connection to the broker for "producer-1" ("producer-2" was created for a new connection).
P.S. Consumers work fine without any issues.
P.S.P.S. I've already tried to delete producer manually using DefaultKafkaConsumerFactory#reset or DefaultKafkaConsumerFactory#destroy, but it didn't help.
P.S.P.S.P.S. I assume the problem is that Kafka saves cache of producers. But I don't know how to solve it.
I'll be grateful for your help.

Answer 1

I had a similar issue. I used the RefreshRemoteApplicationEvent listener to manually close the producer before the refresh mechanism is processed.
For example:
#Slf4j
public class MyRefreshEventListener implements ApplicationListener<RefreshRemoteApplicationEvent> {
private final MyKafkaBean myKafkaBean;
public MyRefreshEventListener(#Autowired MyKafkaBean myKafkaBean) {
this.myKafkaBean = myKafkaBean;
}
#Override
public void onApplicationEvent(RefreshRemoteApplicationEvent event) {
log.info("Refresh myKafkaBean:");
if (myKafkaBean != null) {
closeProducer(myKafkaBean);
}
}
private void closeProducer(Producer<String, String> producer) {
if (producer != null) {
producer.flush();
producer.close();
}
}
}