Report Metrics from Kafka to Actuator - spring

Question

I'm trying to get some metrics (client lag, ...) from kafka to provide it for consumption by prometheus.
My approach would be to write a simple springboot application which exposes the metrics for prometheus. I understand that kafka provides metrics to all its consumers via the interface MetricsReporter.
So I implemented a class which should do exactly that:
public class MonitoringIntegration implements MetricsReporter {
#Override
public void init(List<KafkaMetric> list) {
System.out.println("init");
for (KafkaMetric kafkaMetric : list) {
System.out.println(kafkaMetric.metricName());
System.out.println(kafkaMetric.metricValue());
}
}
#Override
public void metricChange(KafkaMetric kafkaMetric) {
System.out.println("Metric Change");
System.out.println(kafkaMetric.metricName());
System.out.println(kafkaMetric.metricValue());
}
#Override
public void metricRemoval(KafkaMetric kafkaMetric) {
System.out.println("Removal");
System.out.println(kafkaMetric.metricName());
System.out.println(kafkaMetric.metricValue());
}
#Override
public void close() {
System.out.println("close");
}
#Override
public void configure(Map<String, ?> map) {
System.out.println("Configuring");
System.out.println(map);
}
}
I registered this class with a bean:
#Configuration
public class MetricConfiguration {
#Bean
public ProducerFactory<?, ?> kafkaProducerFactory(KafkaProperties properties) {
Map<String, Object> producerProperties = properties.buildProducerProperties();
producerProperties.put(CommonClientConfigs.METRIC_REPORTER_CLASSES_CONFIG,
MonitoringIntegration.class.getName());
return new DefaultKafkaProducerFactory<>(producerProperties);
}
#Bean
public ConsumerFactory<?, ?> kafkaConsumerFactory(KafkaProperties properties) {
Map<String, Object> consumererProperties = properties.buildConsumerProperties();
consumererProperties.put(CommonClientConfigs.METRIC_REPORTER_CLASSES_CONFIG,
MonitoringIntegration.class.getName());
return new DefaultKafkaConsumerFactory<>(consumererProperties);
}
}
When I start the application some metrics will be printed out to cmd, but they have all default values (0.0, infinite, ..) and they will only be provided once after the application started.
Why am I not getting the metrics? What did I do wrong?
Cheers,
Fabian

Answer 1

Spring Kafka already exposes Kafka metrics as a JMX metrics. You dont need to update/send the metrics to Prometheus. Prometheus server will automatically read from your application's "/prometheus" endpoint. Enable Spring Actuator with Prometheus in your Spring project and configure the Prometheus server to read from it.
Here is a great example using Spring Boot - https://www.callicoder.com/spring-boot-actuator-metrics-monitoring-dashboard-prometheus-grafana/
MetricsReporter is not used to "report" metric values as they change. Check the docs. (For some reason I cant find the latest API).
https://archive.apache.org/dist/kafka/0.8.2-beta/java-doc/org/apache/kafka/common/metrics/MetricsReporter.html
A plugin interface to allow things to listen as new metrics are created so they can be reported.
metricChange() method will only be called when a metric is changed. This is the reason you see the first few outputs during application startup, because the metrics were created.

Answer 2

The consumer metrics support are only available on spring boot 2.1+ versions.
Auto-configuration Support For New Metrics
Metrics coverage has been improved to include:
Hibernate metrics
Spring Framework’s WebClient
Kafka consumer metrics
Log4j2 metrics
Jetty server thread pool metrics
Server-side Jersey HTTP request metrics
https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.1-Release-Notes#auto-configuration-support-for-new-metrics
I recommend you to upgrade to newer versions. But if you really need to use Spring Boot prior versions, you can check my kafka metrics micrometer implementation at:
https://github.com/luiztoscano/spring-boot-kmetrics