I am getting the string messages in a Queue. I am consuming it and doing further processing using the below channel processMessage.
Now I have a requirement where I need to send this messages to another queue based on the selector expression mentioned below.
At the same time I need to make sure that the I am not missing any records in processMessage channel. Assuming that the below configuration is good to go?
or is there any better way to handle this situtaion:
<int:recipient-list-router id="router" input-channel="incomingMsg">
<int:recipient channel="publishToQueue" selector-expression="payload.contains('test trade') "/>
<int:recipient channel="processMessage"/>
</int:recipient-list-router>
<task:executor id="executor" pool-size="10" />
<int:publish-subscribe-channel id="publishToQueue" task-executor="executor"/>
The logical explanation is :
IF payload.contains(test trade)
THEN
PublishToQueue
END IF
ProcessMessage
Your configuration is correct and it is really a classical sample for the Recipient List Router. You send message to the processMessage unconditionally and the same message is sent to the publishToQueue if it fits to the selector expression.
Another way to reach the same is possible via Publish-Subscribe Channel and a Filter in the beginning of the second flow. But I would say your way with the Recipient List Router is really good.
Otherwise it isn't clear what is your question and why you are in doubt to go ahead.
Related
I've a spring-integration implementation with following:
Multiple publishing channels publishing on one common channel.
All the channels returns same response object.
Aggregator trying to aggregate response from all the above channels
Issue: Aggregator not able to combine all the responses together and provided method gets invoked on the first reponse from the channels
Here are the details. What is that I've to do to aggregate the responses?
<int:publish-subscribe-channel id="aggregate-channel" apply-sequence="true"/>
<int:publish-subscribe-channel id="input-channel" apply-sequence="true"/>
<int:service-activator input-channel="input-channel" output-channel="aggregate-channel" ref="...A" method="...A">
<int:service-activator input-channel="input-channel" output-channel="aggregate-channel" ref="...B" method="...B">
<int:service-activator input-channel="input-channel" output-channel="aggregate-channel" ref="...C" method="...C">
<int:service-activator input-channel="input-channel" output-channel="aggregate-channel" ref="...D" method="...D">
<!--This is the aggregator.
**Expecting a list of size 4 but then it gets list of size 1 for each response channel
-->
<int:aggregator input-channel="aggregate-channel" output-channel="gateway-response-channel" ref="Service" method="responseListProcessor"/>
Solution
Replace
<int:publish-subscribe-channel id="aggregate-channel" apply-sequence="true"/>
with
<int:publish-subscribe-channel id="aggregate-channel"/>
Issue
Initially it starts with sequence size 4 because input-channel has 4 subscribers
But when you add the attribute apply-sequence="true" to aggregate-channel, it reset the sequence size to 1 because aggregate-channel has only one subscriber which is the aggregator.
Reference
https://docs.spring.io/spring-integration/docs/5.1.7.RELEASE/reference/html/#channel-configuration-pubsubchannel
If you provide a aggregator downstream from a PublishSubscribeChannel, you can set the 'apply-sequence' property on the channel to true.
Doing so indicates that the channel should set the sequence-size and sequence-number message headers as well as the correlation ID prior to passing along the messages.
For example, if there are five subscribers, the sequence-size would be set to 5, and the messages would have sequence-number header values ranging from 1 to 5.
I would say your requirement is fully covered by specific for this kinda of tasks EIP - Scatter-Gather: https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#scatter-gather
I have a custom sink module and I would like to consume the messages from the input only if the the number of messages reaches a count or if they are in the channel since some time. In nutshell, I want to do a bulk push.
I have tried aggregating the number of messages after consuming and storing them in an aggregated channel backed by SimpleMessageStore and have MessageGroupStoreReaper checking for messages in the channel.
I am not satisfied with this approach as I am consuming the messages and storing them in an in-memory store, I am aware of the JDBC store as well but I don't want to follow this approach as the message channels in spring XD are backed by redis/mq I would like to consume from the input channel based on my conditions.
My current bean configuration is as shown below:
<int:aggregator id="messageAggregator" ref="messageAggregatorBean"
method="aggregate" input-channel="input" output-channel="aggregatorOutputChannel"
release-strategy="messageReleaseStrategyBean" release-strategy-method="canRelease"
send-partial-result-on-expiry="true" message-store="resultMessageStore">
</int:aggregator>
<int:service-activator id="contributionIndexerService"
ref="contributionIndexerBean" method="bulkIndex" input-channel="aggregatorOutChannel" />
<bean id="resultMessageStore"
class="org.springframework.integration.store.SimpleMessageStore" />
<bean id="resultMessageStoreReaper"
class="org.springframework.integration.store.MessageGroupStoreReaper">
<property name="messageGroupStore" ref="resultMessageStore" />
<property name="timeout" value="60000" />
</bean>
<task:scheduled-tasks>
<task:scheduled ref="resultMessageStoreReaper" method="run"
fixed-rate="10000" />
</task:scheduled-tasks>
Any thoughts or comments?
Thanks in advance.
I'm not sure that you will be able to determine the count of messages in the Broker's queue (Redis/RabbitMQ or even just normal JMS), more over how much they have been there.
You definitely should consume them to do such a logic.
And yes, I think Aggregator can help you. But right: that must be a Persistent Message Store.
for the case
if they are in the channel since some time
The Aggregator suggest an option like group-timeout to release those groups which haven't reached the releaseStrategy condition, but you would like to emit them anyway over some time: http://docs.spring.io/spring-integration/reference/html/messaging-routing-chapter.html#agg-and-group-to
Hey so i'm using spring integration's jms:outbound-channel-adapterand need to set a priority on a message before i push it through to the messaging system.
Now in plain JMS i had two ways of doing it.
Either set the priority on the MessageProducer:
this.producer.setPriority(i);
Or on the send method itself:
channel.send(message, DeliveryMode.PERSISTENT, 5, 1000);
Neither of these options are available for me anymore since the channel adapter abstracts me away from these details.
Setting the priority on the message itself works only in the in memory channels of spring integration and loses effect soon as i put it into an actual queue. And turns out setting the priority on the message isn't an option at all: JMS message priority not working on Message
There's an attribute on the channel adapter where i can set the priority, but this is static.
<jms:outbound-channel-adapter id="101Out"
channel="101MessageChannel"
connection-factory="101Factory"
destination="QUEUE_NAME"
priority="1" />
The max i can do i read it from a property file. (Or so i think. I'm not sure). I can use the destination-expression attribute to inspect the incoming message and dynamically route it to different destinations, but there's no priority-expression counter part for me to do the same with the priority.
I have a work around of sorts, but it's not a very good one:
<jms:outbound-channel-adapter id="101HighPriorityOut"
channel="101HighPriorityChannel"
connection-factory="101Factory"
destination-expression="headers.QUEUE_NAME"
priority="1"
explicit-qos-enabled="true" />
<jms:outbound-channel-adapter id="101LowPriorityOut"
channel="101LowPriorityChannel"
connection-factory="101Factory"
destination-expression="headers.QUEUE_NAME"
priority="0"
explicit-qos-enabled="true" />
I just route the messages to the appropriate outbound adapter once i determine what the priority needs to be. But if the number of priorities increases i'll be in trouble. Even if it doesn't, having two outbound adapters instead of one just coz i couldn't dynamically assign a priority is kinda clumsy i thought.
Appreciate the help :-)
Oh and i'm using Websphere MQ as my message broker. I don't know if this has anything to do with the message broker though.
Simply set the priority header in the message...
<int:header-enricher ...>
<int:priority value="2" />
</int:heaer-enricher>
The priority in the adapter configuration is a default which is used when there's no priority header (you can use a property placeholder to set it from a properties file).
Or, use an expression...
<int:header-enricher ...>
<int:priority expression="payload.foo == 'bar' ? 1 : 2" />
</int:heaer-enricher>
<int:header-enricher ...>
<int:priority expression="payload.priority" />
</int:heaer-enricher>
<int:header-enricher ...>
<int:priority expression="#someBean.calculatePriority(payload)" />
</int:heaer-enricher>
I would like to process incoming messages differently basing on the message header. How to implement it efficiently in RabbitMQ?
My listener is listenining to messages coming from presence.queue channels. The message header KEY has different keys: key1, key2. Messages with key1 should be processed by messageService.method1, messages with key2 should be processed with messageService.method2.
Here is the code I am playing with but obviously it is wrong. I probably should route messages from presence.queue to first.queue and second.queue ?
<!-- RECEIVER -->
<rabbit:queue id="presence.queue" name="presence" durable="true"/>
<rabbit:queue id="first.queue" name="first"/>
<rabbit:queue id="second.queue" name="second"/>
<rabbit:direct-exchange name="presence.direct" durable="true" >
<rabbit:bindings>
<rabbit:binding queue="presence.queue" key="key1"/>
<rabbit:binding queue="presence.queue" key="key2"/>
</rabbit:bindings>
</rabbit:direct-exchange>
<!-- RabbitMQ Asynchronous Receiver from PRESENCE_ENGINE queue -->
<rabbit:listener-container connection-factory="connectionFactory" acknowledge="auto" >
<rabbit:listener queues="first.queue" ref="messageService" method="method1" />
<rabbit:listener queues="second.queue" ref="messageService" method="method2" />
</rabbit:listener-container>
Why do you need presence.queue? Just bind your consumer queues with the appropriate routing key. If you want to use message headers for routing, use a Headers Exchange.
See Getting Started and Headers Exchange.
Correction. Producers send messages to the exchange, not the queue. Come back, please, to the AMQP theory.
They may send Messages with the same routing key as well, but as Gary suggests, you can use Headers Exchange and build binding to your queues by some value from MessageProperties.
Especially you already have that value.
There is nohing to change on producer side. But from other side: you are the server (consumer) and everything in your hadns: you can configure AMQP routing as it would be comfortably for you.
That's why AMQP is better than JMS!
I have a situation I don't know how to manage it.
The flow I neeed is the following one:
The first service makes its job and creates a message that needs to be retrieved in the chain's end.
When the first service finishes, I need to invoke a push notification server via a new service with a particular message but with some info related to the one created in step 1.
Finally, I the push notification has been sent successfully, I have to retrieve the message created in step 1.
The question is, how can I keep message created in step 1 when the outbound-gateway calling was produced and retrieved me the message from notification push server?
<int:chain input-channel="v1.inputChannel.input" output-channel="v1.inputChannel.output" send-timeout="50000">
<int:header-enricher>
<int:error-channel ref="v1.inputChannel.error" />
</int:header-enricher>
<int:service-activator ref="v1.input.service" method="methodName"/>
<int:service-activator ref="v1.notificationPusher.service" method="pushNotification"/>
<int-http:outbound-gateway url="http://example.com/api/elements/:element_id/objects" http-method="POST">
<int-http:uri-variable name="element_id" expression="#pathVariables.elementId"/>
</int-http:outbound-gateway>
<!-- here the transformer needs to get the messsage from v1.input.service -->
<int:object-to-json-transformer/>
</int:chain>
After some researching in 'Spring Integration in Action' I think the best option for resolving this situation is to use the wire tap pattern, having the push notification service as secondary flow.
Here you can see the book's example where, If I have understood it in a correct way, the auditChannel acts as a secondary flow separated from the main one.
<channel id="debitChannel">
<interceptors>
<wire-tap channel="auditChannel"/>
</interceptors>
</channel>
<service-activator input-channel="debitChannel" method="process">
<beans:bean class="siia.monitoring.wiretap.DebitService"/>
</service-activator>
<filter input-channel="auditChannel" expression="payload.amount > 10000" output-channel="logger"/>
<logging-channel-adapter id="logger" expression="'auditing debit: ' + payload"/>
I think you could achieve this with something like that:
Dupĺicate your message before send it to the outbound
Send one to the outbound and the another one to some channel
Use an Aggregator with the same timeout of your outbound to "join" them, but you could, in fact, only pass the message from the first step through (I haven't tested it)
Send it to the json transformer
To use this approach, I believe, you have to put the outbound and the json-transformer outside the chain as well as your logic to duplicate the message.
I can think of one way, which is to squirrel away the payload into a header after step 1, then retrieve and repopulate the payload right before you invoke step 3:
<int:service-activator ref="v1.input.service" method="methodName"/>
<int:header-enricher>
<int:header name="originalpayload" expression="payload"/>
</int:header-enricher>
<int:service-activator ref="v1.notificationPusher.service" method="pushNotification"/>
<int:enricher expression="headers.originalpayload"/>
<int-http:outbound-gateway url="http://xxx.com/api/elements/:element_id/objects" http-method="POST">
<int-http:uri-variable name="element_id" expression="#pathVariables.elementId"/>
</int-http:outbound-gateway>