Spring Integration : Publish Subscribe channel. Is it asynchronous? - spring

I am using a publish-subscribe channel after the inbound gateway receives a String message to parallely send it to the logger to log the message and to a transformer to transform the message. I want both these acivities to happen in parallel.
My question is very simple - Does the publish subscribe channel in spring integration sends messages to it's subscibers parallely?
Below is the code snippet from the source of spring-integration-context.xml.
<int:gateway id="gateway" service-interface="com.test.Gateway">
</int:gateway>
<int:publish-subscribe-channel id="publishsubscribechannel" />
<int:service-activator input-channel="publishsubscribechannel"
method="transformEvent" ref="transformer" output-channel="transformerreplychannel">
</int:service-activator>
<int:service-activator input-channel="publishsubscribechannel"
method="logMessage" ref="logger">
</int:service-activator>
Here the transformer and the logger are 2 subscribers to the publishsubscribechannel. In this setup will the message flow to logger and transformer from the gateway happen asynchronously by default??...OR I need to do some other configuration to achieve the same.

... or using JavaConfig
#Bean
public MessageChannel publishsubscribechannel() {
return new PublishSubscribeChannel(executor());
}
#Bean
public ThreadPoolTaskExecutor executor() {
ThreadPoolTaskExecutor pool = new ThreadPoolTaskExecutor();
pool.setCorePoolSize(10);
pool.setMaxPoolSize(10);
pool.setWaitForTasksToCompleteOnShutdown(true);
return pool;
}

By default, it runs in sequence. So in your case, it will be transformer, then logger. If you want to run it in parallel, you need to specify task-executor
<int:publish-subscribe-channel id="publishsubscribechannel" task-executor="executor" />
...
<task:executor id="executor" pool-size="10" />
And by using task-executor, the message handling is performed asynchronously.

As is from the Spring Integration documentation
PublishSubscribeChannel
The PublishSubscribeChannel implementation broadcasts any Message sent to it to all of its subscribed handlers. This is most often used for sending Event Messages whose primary role is notification as opposed to Document Messages which are generally intended to be processed by a single handler. Note that the PublishSubscribeChannel is intended for sending only. Since it broadcasts to its subscribers directly when its send(Message) method is invoked, consumers cannot poll for Messages (it does not implement PollableChannel and therefore has no receive() method). Instead, any subscriber must be a MessageHandler itself, and the subscriber's handleMessage(Message) method will be invoked in turn.
Prior to version 3.0, invoking the send method on a PublishSubscribeChannel that had no subscribers returned false. When used in conjunction with a MessagingTemplate, a MessageDeliveryException was thrown. Starting with version 3.0, the behavior has changed such that a send is always considered successful if at least the minimum subscribers are present (and successfully handle the message). This behavior can be modified by setting the minSubscribers property, which defaults to 0.
[Note] Note
If a TaskExecutor is used, only the presence of the correct number of subscribers is used for this determination, because the actual handling of the message is performed asynchronously.
Please note the Note.
It does mention if the TaskExecutor is used, the message handling would be asynchronous.
So, yes you have to add a TaskExecutor for this to be asynchronous.

Related

Spring integration messages queue

I have jms message endpoint like:
#Bean
public JmsMessageDrivenEndpoint fsJmsMessageDrivenEndpoint(ConnectionFactory fsConnectionFactory,
Destination fsInboundDestination,
MessageConverter fsMessageConverter) {
return Jms.messageDrivenChannelAdapter(fsConnectionFactory)
.destination(fsInboundDestination)
.jmsMessageConverter(fsMessageConverter)
.outputChannel("fsChannelRouter.input")
.errorChannel("fsErrorChannel.input")
.get();
}
So, my questions is did I get next message before current message will be processed? If it will...Did it will get all messages in mq queue until it fills up all the memory? How to avoid it?
The JmsMessageDrivenEndpoint is based on the JmsMessageListenerContainer, its threading model and MessageListener callback for pulled messages. As long as your MessageListener blocks, it doesn't go to the next message in the queue to pull. When we build an integration flow starting with JmsMessageDrivenEndpoint, it becomes as a MessageListener callback. As long as we process the message downstream in the same thread (DirectChannel by default in between endpoints), we don't pull the next message from JMS queue. If you place a QueueChannel or an ExecutorChannel in between, you shift a processing to a different thread. The current one (JMS listener) gets a control back and it is ready to pull the next message. And in this case your concern about the memory is correct. You can still use QueueChannel with limited size or your ExecutorChannel can be configured with limited thread pool.
In any way my recommendation do not do any thread shifting in the flow when you start from JMS listener container. It is better to block for the next message and let the current transaction to finish its job. So you won't lose a message when something crashes.

How to Ack/Nack with reactive RabbitListener in Spring AMQP?

I'm using Spring AMQP 2.1.6 for consuming messages with a RabbitListener returning a Mono<Void>. For example:
#RabbitListener
public Mono<Void> myListener(MyMessage myMessage) {
Mono<Void> mono = myService.doSomething(myMessage);
return mono;
}
Reading the documentation it says:
The listener container factory must be configured with AcknowledgeMode.MANUAL so that the consumer thread will not ack the message; instead, the asynchronous completion will ack or nack the message when the async operation completes.
I've thus configured the container factory with AcknowledgeMode.MANUAL, but it's not clear to me whether "the asynchronous completion will ack or nack the message when the async operation completes" means that this is handled by spring-amqp itself or if this is something that I have to do? I.e. do I have to the ack/nack the message after the call to myService.doSomething(myMessage) or does Spring AMQP automatically ack's it since I'm returning a Mono (even though AcknowledgeMode.MANUAL is set)?
If it's the case that I need to manually send ack's or reject's, what's the idiomatic way to do this in a non-blocking manner when using the RabbitListener?
The listener adapter takes care of the ack when the Mono is completed.
See AbstractAdaptableMessageListener.asyncSuccess() and asyncFailure().
EDIT
I am not a reactor person but, as far as I can tell, completing a Mono<Void> does nothing so the on...() methods are never called.
You can ack the delivery manually using channel.basicAck or basicReject...
#RabbitListener(queues = "foo")
public void listen(String in, Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG) long tag) throws IOException {
...
}

How to stop the Message driven adapter dynamically to stop receiving messages?

<control-bus input-channel ="inboundChannel/>
<channel id ="inboundChannel"/>
<message-driven-channel-adapter id="inAQueue" channel="inboundChannel"
auto-startup="false" container="DefaultContainer"/>
<service-activator input-channel="inboundChannel" ref="Something"
method="abc"/>
inboundChannel.send(MessageBuilder.withPayload("#inAQueue.stop()").build();
inboundChannel.send(MessageBuilder.withPayload("#inAQueue.start()").build());
But my service activator class receive this message and throws class cast exception that string can not be cast to jmstextMessage.
And I'm not sure if i am trying it in correct manner or not.
You have 2 subscribers on inboundChannel - the message-driven adapter and the control-bus. Messages will be distributed in round-robin fashion.
The control bus should subscribe to its own channel, to which the control message is sent.

Spring Integration poll aggregator programmatically

I've been using Spring boot and getting rid of all XML files in my project.
Unfortunately it also uses Spring integration which from my experience is very heavily XML based.
I have a scenario which requires me to have an aggregator, and have that aggregator polled every x amount of seconds.
This can be done using XML like so (example taken from a previous SO question):
<!--
the poller will process 100 messages every minute
if the size of the group is 100 (the poll reached the max messages) or 60 seconds time out (poll has less than 100 messages) then the payload with the list of messages is passed to defined output channel
-->
<int:aggregator input-channel="logEntryChannel" output-channel="logEntryAggrChannel"
send-partial-result-on-expiry="true"
group-timeout="60000"
correlation-strategy-expression="T(Thread).currentThread().id"
release-strategy-expression="size() == 100">
<int:poller max-messages-per-poll="100" fixed-rate="60000"/>
</int:aggregator>
I've managed to find a class that kinda sorta does the trick and it's bean definition is:
#Bean(name = "aggregatingMessageHandler")
public AggregatingMessageHandler aggregatingMessageHandler() {
AggregatingMessageHandler aggregatingMessageHandler =
new AggregatingMessageHandler(messageGroupProcessorBean(),
new SimpleMessageStore(10));
aggregatingMessageHandler.setCorrelationStrategy(customCorrelationStrategyBean());
aggregatingMessageHandler.setReleaseStrategy(
new TimeoutCountSequenceSizeReleaseStrategy(3,
TimeoutCountSequenceSizeReleaseStrategy.DEFAULT_TIMEOUT));
aggregatingMessageHandler.setExpireGroupsUponCompletion(true);
aggregatingMessageHandler.setOutputChannel(outputAggregatedChannelBean());
return aggregatingMessageHandler;
}
However this triggers the canRelease() method of the ReleaseStrategy only when a new message is received in the inboundChannel associated with this handler, and not at a fixed time interval which is not the desired result.
I want all groups older than one minute to be redirected to the output channel.
My question is - is there a way to programmatically attach a Poller such as the one in the XML definition?
For Java & Annotation configuration take a look here and here.
The Aggregator component has AggregatorFactoryBean for easier Java Configuration.
Anyway you have to pay attention that there is a #ServiceActivator annotation together with a #Bean on that handler definition. And exactly #ServiceActivator has poller attribute.
Also pay attention that there is a Java DSL for Spring Integration.
Another part of your question is a bit confusion. The poller fully isn't related to the release strategy. Its responsibility in this case to receive messages from the PollableChannel which is that logEntryChannel. And only after that already polled messages are placed to the aggregator for it correlation and release logic.
What is done in that sample is fully different story and we can discuss it in the separate SO thread.

spring integration bridge direct channel to queue Channel

At unit test time, I try to bridge the Spring Integration default channel to a queued channel, since I want to check the correctness of the amount of message flow into this channel.
<int:filter input-channel="prevChannel" output-channel="myChannel">
<int:bridge input-channel="myChannel" output-channel="aggregateChannel">
// the reason I have above bridge is I want to check the amount of message after filter.
// I cannot check prevChannel since it is before filtering, and I cannot check aggregateChannel
// because it has other processing branch
// in test xml I import above normal workflow xml and added below configuration to
// redirect message from myChannel to checkMyChannel to checking.
<int:bridge input-channel="myChannel"
output-channel="checkMyChannel"/>
<int:channel id="checkMyChannel">
<int:queue/>
</int:channel>
I autowired checkMyChannel in my unit test
but checkMyChannel.getqueuesize() always return 0.
Is there sth I did wrong?
You missed to share test-case with us. We don't have the entire picture. And looks like you have a race condition there. Someone polls all your messages from the checkMyChannel before you start assert that getqueuesize().
In our tests we don't use <poller> for such a cases. We use PollableChannel.receive(timeout) manually.
got this fixed, I have to declare myChannel to be a publish-subscribe-channel
How to test Spring Integration
This one helps, for my case there is a race condition since.
"A regular channel with multiple subscribers will round-robin ..."

Resources