I'm using Spring Boot version 1.5.2.RELEASE along with Spring Kafka version 1.1.2.RELEASE.
Via the application.properties file I do see available options (spring.kafka.consumer.*) to configure Kafka Consumer.
What I'm not able to find though is a way to configure the acknowledgement mode.
spring.kafka.listener.ack-mode=
You can use Spring Cloud Stream Kafka Binder to streaming messages. In that case
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kafka</artifactId>
</dependency>
and configure consumer like this.
spring.cloud.stream.kafka.bindings.<channelName>.consumer..
and producer like this
spring.cloud.stream.kafka.bindings.<channelName>.producer..
to more detail follow this or this video
Related
I am trying to migrate to Spring Boot 3 with the new namespace jakarta.xx instead of javax.xx but the ActiveMQ "Classic" client has not been updated and was deprecated. Is there a way to continue using the old ActiveMQ client?
I tried the new ActiveMQ Artemis client but it seems like they are not interoperable with the ActiveMQ "Classic" server.
Including the old ActiveMQ client results in not being able to use JMSTemplate for configuration because JMSTemplate uses jakarta.xx and expects a ConnectionFactory from jakarta.xx not javax.xx
Edit: Didn't work so the only way is to upgrade to artemis. That way the codebase is also nearly unchanged.
As you noticed there is no ActiveMQ client that supports the Jakarta namespace JMS dependency or in fact none that supports JMS 2.0 so you really need to move to something else such as an ActiveMQ Artemis broker and the ActiveMQ Artemis client or Qpid JMS AMQP client v2.1.0 which both support JMS 2.0 and use the Jakarta APIs.
If you are dead set on sticking with ActiveMQ 5.x you can try using the Qpid JMS v2.1.0 client which does use the Jakarata JMS API but you will need to be somewhat careful as the 5.x broker doesn't support JMS 2.0 so some parts of the API can trigger unexpected behaviors. The AMQP support in the 5.x broker is not as fully integrated and JMS 2.0 aware as that of the Artemis broker so you can encounter issues you wouldn't see if you moved on to the Artemis broker.
Edit: Didn't work so the only way is to upgrade to artemis. That way the codebase is also nearly unchanged.
To continue to use ActiveMQ 5.x with Spring 3 and Jakarta EE 9, two updates are needed on the ActiveMQ 5.x side-- JMS 2.0 support, and then support for javax.jms -> jakarta.jms namespace change.
The first part is the biggest, and is underway. The ActiveMQ 5.x main branch has preview support for JMS 2.0, and there are plans to provide Jakarta namespace support shortly after.
This is a good page to track JMS 2.0 progress in ActiveMQ 5.x
ref: https://activemq.apache.org/jms2
UPDATE: A temporary, jakarta-based client suitable for transition activity is available: https://github.com/hyteio/activemq-client-jakarta
While waiting for ActiveMQ update to Jakarta EE 9 i removed
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-activemq</artifactId>
</dependency>
And included
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-client</artifactId>
<version>5.17.3</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jms</artifactId>
<version>5.3.24</version>
</dependency>
It worked like a charm with Spring Boot 3.0.0
In the Spring Cloud Stream application with Kafka binders, I am trying to PAUSE/RESUME the input binders. I searched and all of those sample solutions suggest using BindingsEndpoint.
References->
https://cloud.spring.io/spring-cloud-static/spring-cloud-stream-binder-kafka/2.2.0.M1/spring-cloud-stream-binder-kafka.html
Spring cloud stream kafka pause/resume binders
Using the BindingsEndpoint works in Spring Integration annotations based configuration Is there any way to use this feature in functional-style programming where Kafka listener is Consumer<T> bean?
Thanks in advance!
All,
I have a requirement to expose the Apache Kafka metrics to the spring boot (v 2.3.0.RELEASE) actuator endpoint.
Please note, I am NOT using spring-kafka library .
I am using the following libraries
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-streams</artifactId>
<version>2.5.0</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>2.5.0</version>
</dependency>
I have tried this spring.jmx.enabled=true but seems like this doesnt work. I assume this is happening because spring is not managing the kafka.
Is there a way I can bind these JMX metrics to the micrometer MeterRegistry?
I am able to make this work using the jmx-exporter provided by prometheus, but since that requires an agent running on a different port, I was hoping to make this work with default Micrometer and spring boot.
Have you tried KafkaClientMetrics, KafkaStreamsMetrics or KafkaConsumerMetrics?
KafkaConsumerMetrics collects metrics through JMX but it is deprecated in favor of KafkaClientMetrics.
You only need to create them and bind to your registry:
KafkaConsumer<String, String> consumer;
KafkaProducer<String, String> producer;
MeterRegistry registry;
...
new KafkaClientMetrics(consumer).bindTo(registry);
new KafkaClientMetrics(producer).bindTo(registry);
Similarly for KafkaStreams:
new KafkaStreamsMetrics(kafkaStreams).bindTo(registry);
If Spring does not manage your Kafka components, why do you expect spring.jmx.enabled=true to do anything?
I am using Spring boot 2.2.9.RELEASE and Spring Cloud Hoxton.SR7. I am using Spring Cloud Bus to signal all my containers in a docker swarm stack and when deployed in production with a running RabbitMQ cluster things work perfectly!
I am using the RabbitMQ implementation via the spring-cloud-starter-bus-amqp Spring Boot starter. We occasionally run tests without needing the bus. There is a spring boot flag for this:
spring.cloud.bus.enabled=false
this disables the bus, but rabbitMQ still starts, and spits out connection refused errors. I had to also add:
rabbitmq.autoStarting=false
I tried fussing around with disabling RabbitMQ's auto configuration, but it seems there is a RabbitAutoConfiguration class that implies it is a SB autoconfig class, but in actual fact it is a normal SB config class.
Is there a cleaner way to disable the Cloud Bus that also prevents RabbitMQ from starting?
You just need to include the spring-cloud-stream-test-support jar in your test scope. This jar includes binders that will override and replace the default binders. These test binders will not actually connect to the resources in the background.
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-test-support</artifactId>
<version>${spring.cloud.stream.version}</version>
<scope>test</scope>
</dependency>
i am designing realtime kafka consuming job, and considering between spring boot batch and spring boot.
kafka version : 2.11-1.1.1
jdk : 1.8
which is better?
spring boot batch + kafka
spring boot + kafka
and please tell me why :)
If you are thinking to create job and manage through spring cloud data flow further, go with the #1 and if you use case is to just keep consuming the kafka message when recieved to kafka topic, use spring boot and spring kafka. It totally depend on your use case.