Spring-Kafka: How to insert an application.yml topic in Producer Kafka - spring

Question

I have a spring-kafka microservice to which I recently added a dead letter to be able to send the various error messages
//some code..
#Component
public class KafkaProducer {
#Autowired
private KafkaTemplate<String, String> kafkaTemplate;
public void sendDeadLetter(String message) {
kafkaTemplate.send("myDeadLetter", message);
}
}
I would like to call the topic kafka of the dead letter as "messageTopic" + "_deadLetter", my main topic being "messageTopic". In my Consumer the topic name gives him the application.yml as follows:
#KafkaListener(topics = "${spring.kafka.topic.name}")
How can I set the same kafka topic by possibly inserting the "+ deadLetter" from the application.yml? I tried such a thing:
#Component
#KafkaListener(topics = "${spring.kafka.topic.name}"+"_deadLetter")
public class KafkaProducer {
#Autowired
private KafkaTemplate<String, String> kafkaTemplate;
public void sendDeadLetter(String message) {
kafkaTemplate.send("messageTopic_deadLetter", message);
}
}
but it creates me two different topics with the same name. I am waiting for some advice, thanks for the help!

Answer 1

Kafka Listener accepts constant for the Topic name, we can't modify the TOPIC name here.
Ideally good to go with separate methods (Kafka listeners) for actual topic and dead letter topic, define two different properties in YAML to hold two topic names.
#KafkaListener(topics = "${spring.kafka.topic.name}")
public void listen(......){
}
#KafkaListener(topics = "${spring.kafka.deadletter.topic.name}")
public void listenDlt(......){
}
To refer topic name inside send(...) from yml or property file
#Component
#KafkaListener(topics = "${spring.kafka.deadletter.topic.name}")
public class KafkaProducer {
#Value("${spring.kafka.deadletter.topic.name}")
private String DLT_TOPIC_NAME;
#Autowired
private KafkaTemplate<String, String> kafkaTemplate;
public void sendDeadLetter(String message) {
kafkaTemplate.send(DLT_TOPIC_NAME, message);
}
}

Answer 2

You can construct the topic name with SpEL:
#KafkaListener(topics = "#{'${spring.kafka.topic.name}' + '_deadLetter'"})
Note the single quotes around the property placeholder and literal.

Answer 3

This example may not be relevant to your use case, but sharing in case it's helpful to someone.
If you are building a Kafka Stream application, variable sink topic names can be achieved with the following:
When producing to the sink topic, pass a lambda that has the context as argument and the method that will handle the name definition.
... /* precedent stream operations */
// terminal operation 'to'.
.to(
(k, v, ctx) -> sinkTopicNameGenerator(ctx),
Produced.with(Serdes, Serdes)
);
Implement the method that generates the sink topic names:
protected static String sinkTopicNameGenerator(RecordContext ctx) {
return ctx.topic().concat("_deadLetter");
}
The above example is simple enough to be simplified to (k, v, ctx) -> ctx.topic().concat("_deadLetter"), but I wanted to keep the separate method approach for cases where further transformations are required, i.e. when part of the topic name will be replaced by some constant or regex defined in the config file.