We are using spring boot application for Kafka.
We want to do unit testing & integration testing for Kafka producer(kafkatemplate) & consumer (EventListner) application.
We explored couple of library for Kafka testing.
#1 Test container/ Kafka container - no access to docker images so we skipped this
#2 MockProducer & MockConsumer not worked with kafkatemplate & Kafkalietner. -- not found good working example for reference. Worked with core KafkaProducer & Kafka consumer interface.
#3 spring-kafka-test - EmbededKafka working as expected:)
#4 Kafka for junit
Anyone used Kafka for junit for Kafka testing?
What are the advantages of Kafka for junit over spring-kafka-test library ?
When to use Kafka for junit library vs spring-kafka-test ?
Advantages & Disadvantage of Kafka for junit ?
Thanks for advance.
Related
I want to do some integration testing in a spring boot project, where kafka is used.
For these tests I don't want any messages to be sent or received from kafka. Is there a way to configure this?
Kafka test containers help me to do integration tests with kafka
. May be it'll help you.
I am not using Spring-Kafka module to produce and consume messages. Instead, I am using Apache client library with producer and consumer implementations. As I am not using Spring-Kafka, the Spring Slueth auto configuration is not applied to generate the traces. I have referred https://docs.spring.io/spring-cloud-sleuth/docs/current-SNAPSHOT/reference/html/integrations.html I don't find any document around how to apply Spring Slueth for the code which uses 3rd party libraries?
If you're not using Spring, then don't add it to non-Spring code.
The traces in Sleuth are implemented using Brave Kafka Interceptor
I am looking to get the Spring Boot and Apache Kafka complex topics/scenarios examples. Whatever I found on web was very basic and similar demo.
Does anyone has Spring Boot and Apache Kafka example?
I was also looking for solid examples. And I could not find anything concrete but "hello world"... Then I just realized that I'm overlooking the Confluent's documents.
How to Work with Apache Kafka in Your Spring Boot Application
This is literally "hello world" but might be a good warm-up for beginners.
Spring for Apache Kafka Deep Dive – Part 1: Error Handling, Message Conversion and Transaction Support
Spring for Apache Kafka Deep Dive – Part 2: Apache Kafka and Spring Cloud Stream
Spring for Apache Kafka Deep Dive – Part 3: Apache Kafka and Spring Cloud Data Flow
Spring for Apache Kafka Deep Dive – Part 4: Continuous Delivery of Event Streaming Pipelines
I think the Confluent Platform's blog is a real hidden gem for the developers. There are lots of cool topics besides of example/tutorials. To name few of them:
Spring for Apache Kafka – Beyond the Basics: Can Your Kafka Consumers Handle a Poison Pill?
Advanced Testing Techniques for Spring for Apache Kafka
How to Use Schema Registry and Avro in Spring Boot Applications
please refer this tutorial. there are some steps.
https://howtodoinjava.com/kafka/spring-boot-with-kafka/
I'm thinking of developing a simulation of RabbitMQ that can be used in unit tests where it is not possible to start up an entire RabbitMQ server or not possible to connect to one. This RabbitMQ simulation would obviously have the same API as the RabbitMQ Java client. Question is now how to plug in this API of the RabbitMQ simulation into Spring Boot instead of the original one from RabbitMQ. Is there some hook in Spring Boot so that this could be done?
It's quite difficult to simulate RabbitMQ.
Some people have has some success using an embedded Apache QPID server running amqp 0.9.1.
However, it doesn't support any RabbitMQ extensions, if you are using those.
You'd be better off using something like TestContainers.
https://www.testcontainers.org/modules/rabbitmq/
i am designing realtime kafka consuming job, and considering between spring boot batch and spring boot.
kafka version : 2.11-1.1.1
jdk : 1.8
which is better?
spring boot batch + kafka
spring boot + kafka
and please tell me why :)
If you are thinking to create job and manage through spring cloud data flow further, go with the #1 and if you use case is to just keep consuming the kafka message when recieved to kafka topic, use spring boot and spring kafka. It totally depend on your use case.