How to listen to multiple topics by multiple StreamListener Spring Cloud Stream and Kafka stream - apache-kafka-streams

Question

I want to listen to two Kafka topics like in the code below and there are two source events that need to be handled and transform to another event.
So I want to listen these two event in one EnableBinding
#EnableBinding(PartnerOrderCancelledEventImporter.Targets.class)
public class PartnerOrderCancelledEventImporter {
#StreamListener(Targets.INPUT_ORDER_CANCELLED)
#SendTo(Targets.OUTPUT)
public KStream<?, TriggeringEvent> processOrderCancelled(KStream<?, OrderCancelledV1> input) {
LogInfo("PartnerOrderCancelled-OrderCancelledV1 stream started");
LogInfo(" KafkaBrokers: " + KafkaBrokers);
return input
.filter((key, value) -> IsFFFaultAndNoRoutes(value))
.peek((key, value) -> LogInfo("OrderCancelle", new LogObject(value)))
.map((key, value) -> KeyValue.pair(key, new TriggeringEvent(value)));
}
#StreamListener(Targets.INPUT_ORDER_ITEM_STOCK_CHECKED)
#SendTo(Targets.OUTPUT)
public KStream<?, TriggeringEvent> processOrderItemStockChecked(KStream<?, OrderItemStockCheckedV1> input) {
LogInfo("PartnerOrderCancelled-OrderItemStockCheckedV1 stream started");
LogInfo(" KafkaBrokers: " + KafkaBrokers);
return input
.filter((key, value) -> IsItemNoStock(value))
.peek((key, value) -> LogInfo("OrderItemStockChecke", new LogObject(value)))
.map((key, value) -> KeyValue.pair(key, new TriggeringEvent(value)));
}
public interface Targets {
String INPUT_ORDER_CANCELLED = "partnerOrderCancelledInputOrderCancelled";
String INPUT_ORDER_ITEM_STOCK_CHECKED = "partnerOrderCancelledInputOrderItemStockChecked";
String OUTPUT = "triggeringEventsOutputPartnerOrderCancelled";
#Input(INPUT_ORDER_CANCELLED)
KStream<?, ?> inputOrderCancelled();
#Input(INPUT_ORDER_ITEM_STOCK_CHECKED)
KStream<?, ?> inputOrderItemStockChecked();
#Output(OUTPUT)
KStream<?, ?> output();
}
spring.cloud.stream.bindings.triggeringEventsOutputPartnerOrderCancelled.destination=dev.comms.triggeringevents.TriggeringEvent-events-1.0
spring.cloud.stream.bindings.partnerOrderCancelledInputOrderItemStockChecked.destination=dev.ecom.order-management-service.order-item-stock-checked-events-v1
spring.cloud.stream.bindings.partnerOrderCancelledInputOrderItemStockChecked.group=TriggeringEvents-PartnerOrderCancelled
spring.cloud.stream.bindings.partnerOrderCancelledInputOrderCancelled.destination=dev.ecom.order-management-service.order-cancelled-events-v1
spring.cloud.stream.bindings.partnerOrderCancelledInputOrderCancelled.group=TriggeringEvents-PartnerOrderCancelled
thrown below exceptions
org.springframework.beans.factory.BeanInitializationException: Cannot setup StreamListener for public org.apache.kafka.streams.kstream.KStream com.farfetch.communication.triggeringeventsimporter.PartnerOrderCancelledEventImporter.processOrderItemStockChecked(org.apache.kafka.streams.kstream.KStream); nested exception is java.lang.reflect.UndeclaredThrowableException
at org.springframework.cloud.stream.binder.kafka.streams.KafkaStreamsStreamListenerSetupMethodOrchestrator.orchestrateStreamListenerSetupMethod(KafkaStreamsStreamListenerSetupMethodOrchestrator.java:195)
at org.springframework.cloud.stream.binding.StreamListenerAnnotationBeanPostProcessor.doPostProcess(StreamListenerAnnotationBeanPostProcessor.java:195)
at org.springframework.cloud.stream.binding.StreamListenerAnnotationBeanPostProcessor.lambda$postProcessAfterInitialization$0(StreamListenerAnnotationBeanPostProcessor.java:167)
at java.lang.Iterable.forEach(Iterable.java:75)
at org.springframework.cloud.stream.binding.StreamListenerAnnotationBeanPostProcessor.injectAndPostProcessDependencies(StreamListenerAnnotationBeanPostProcessor.java:285)
at org.springframework.cloud.stream.binding.StreamListenerAnnotationBeanPostProcessor.afterSingletonsInstantiated(StreamListenerAnnotationBeanPostProcessor.java:105)
。。。。。。。。。。。。。。。。。。。。。。。。。。。。。
at org.springframework.cloud.stream.binder.kafka.streams.KStreamStreamListenerResultAdapter.adapt(KStreamStreamListenerResultAdapter.java:41)
at org.springframework.cloud.stream.binder.kafka.streams.KStreamStreamListenerResultAdapter.adapt(KStreamStreamListenerResultAdapter.java:31)
at org.springframework.cloud.stream.binder.kafka.streams.KafkaStreamsStreamListenerSetupMethodOrchestrator.orchestrateStreamListenerSetupMethod(KafkaStreamsStreamListenerSetupMethodOrchestrator.java:187)
... 39 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.cloud.stream.binder.kafka.streams.KStreamBoundElementFactory$KStreamWrapperHandler.invoke(KStreamBoundElementFactory.java:99)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
... 43 more
Caused by: java.lang.IllegalArgumentException: delegate already set to org.apache.kafka.streams.kstream.internals.KStreamImpl#491cafec

Answer 1

You are getting this exception because you are trying to bind the same outbound KStream from both processors (output). Is there any chance that you can add another output binding for the second processor? That should address this particular error that you are getting. On the other hand, if this is a requirement that your use case demands, then currently the binder doesn't support that (It could be a good feature to add though). As a workaround, you could make the second processor returns nothing and then call the to method on KStream to send it manually.