How to load a compacted topic in memory before starting the context - spring

Question

I'm using a compacted topic in kafka which I load into a HashMap at the application startup.
Then I'm listening to a normal topic for messages, and processing them using the HashMap constructed from the compacted topic.
How can I make sure the compacted topic is fully read and the HashMap fully initialized before starting to listen to the other topics ?
(Same for RestControllers)

Answer 1

Implement SmartLifecycle and load the map in start(). Make sure the phase is earlier than any other object that needs the map.

Answer 2

This is an old question, I know, but I wanted to provide a more complete code sample of a solution that I ended up with when I struggled with this very problem myself.
The idea is that, like Gary has mentioned in the comments of his own answer, a listener isn't the correct thing to use during initialization - that comes afterwards. An alternative to Garry's SmartLifecycle idea, however, is InitializingBean, which I find less complicated to implement, since it's only one method: afterPropertiesSet():
#Slf4j
#Configuration
#RequiredArgsConstructor
public class MyCacheInitializer implements InitializingBean {
private final ApplicationProperties applicationProperties; // A custom ConfigurationProperties-class
private final KafkaProperties kafkaProperties;
private final ConsumerFactory<String, Bytes> consumerFactory;
private final MyKafkaMessageProcessor messageProcessor;
#Override
public void afterPropertiesSet() {
String topicName = applicationProperties.getKafka().getConsumer().get("my-consumer").getTopic();
Duration pollTimeout = kafkaProperties.getListener().getPollTimeout();
try (Consumer<String, Bytes> consumer = consumerFactory.createConsumer()) {
consumer.subscribe(List.of(topicName));
log.info("Starting to cache the contents of {}", topicName);
ConsumerRecords<String, Bytes> records;
do {
records = consumer.poll(pollTimeout);
records.forEach(messageProcessor::process);
} while (!records.isEmpty());
}
log.info("Completed caching {}", topicName);
}
}
For brevity's sake I'm using Lombok's #Slf4j and #RequiredArgsConstructor annotations, but those can be easily replaced. The ApplicationProperties class is just my way of getting the topic name I'm interested in. It can be replaced with something else, but my implementation uses Lombok's #Data annotation, and looks something like this:
#Data
#Configuration
#ConfigurationProperties(prefix = "app")
public class ApplicationProperties {
private Kafka kafka = new Kafka();
#Data
public static class Kafka {
private Map<String, KafkaConsumer> consumer = new HashMap<>();
}
#Data
public static class KafkaConsumer {
private String topic;
}
}