require join thread like feature in spring integration. If 3 threads are complete, then execute 4th thread. As shown in the picture, 3 outbound gateways running in parallel and the 4th outbound will run only after execution of first 3.
However, I am not doing splitter/aggregater so not required. Nor required any chain operation I guess because it will execute things in sequence.
parallel execution and then wait for the execution to complete
<int:publish-subscribe-channel id="publishsubscribechannel"
task-executor="executor" />
<task:executor id="executor" pool-size="10" />
<int-http:inbound-channel-adapter
id="httpOrderCreateInboundAdapter"
channel="orderCreateInboundChannel" path="/input"
supported-methods="GET, POST"
</int-http:inbound-channel-adapter>
<int-http:outbound-gateway id="micros1" request-
channel="publishsubscribechannel"
reply-channel="channel1"
</int-http:outbound-gateway>
<int-http:outbound-gateway id="micros2" request-
channel="publishsubscribechannel"
reply-channel="channel2"
</int-http:outbound-gateway>
<int-http:outbound-gateway id="micros3" request-
channel="publishsubscribechannel"
</int-http:outbound-gateway>
<int-http:outbound-gateway id="micros4" request-channel="channel2"
reply-channel="channel4"
</int-http:outbound-gateway>
Consider to use Scatter-Gather component for that publish-subscribe channel. Yes, all your gateways must return to their reply channel header to let gatherer to collect replies from them . And really after that you will be able to proceed to the Outbound4.
Related
I am using Spring Integration for processing file. Here's my configuration
<int:channel id="startChannel">
<int:splitter input-channel="startChannel" output-channel="parseChannel" ref="splitter" method="split" />
<int:channel id="parseChannel">
<int:queue size="50"/>
</int:channel>
<int:service-activator input-channel="parseChannel" output-channel="aggregatChannel" ref="process" method="process" />
<int:channel id="aggregateChannel">
<int:queue size="50"/>
</int:channel>
<int:aggregate input-channel="aggregateChannel" output-channel="postProcessingChannel" ...(other attributes) />
<int:channel id="postProcessingChannel">
<int:queue size="50"/>
</int:channel>
<int:channel id="outboundChannel">
<int:queue size="50"/>
</int:channel>
<int:service-activator input-channel="postProcessingChannel" output-channel="outbound-channel" ... />
<int:outbound-adapter channel="outputChannel" ... />
..global poller
..global taskexecutor of size 40
My configuration works fine till the aggregator. Aggregator is able to put messages at the postProcessingChannel but nobody is reading from the postProcessing Channel.
On a closer look, i observe -
1) post processing channel is filled up with messages to its capacity.
2) threads are blocked in aggregator implementation, because they are not able to put messages in the postPostProcessing Channel.
My question is why no thread is reading from PostProcessingchannel ? From JProfiler I can see, many threads are idle/free not doing anything.
Can someone please help me understand this behavior.
You have a lot of queue channels; it is generally not necessary to make every channel a queue channel.
If you have more configuration, you are probably experiencing thread starvation in the default taskScheduler bean - it only has 10 threads by default.
If you have close to 10 queue channels in your application you will run out of threads because, by default, each queue channel will block a thread for 1 second (receiveTimeout).
Consider reducing the number of queue channels (as I said, it is not generally needed to have so many; you don't need to hand off to another thread between every component).
Or, see Configuring the Task Scheduler to increase the available threads for scheduled tasks.
I have a custom sink module and I would like to consume the messages from the input only if the the number of messages reaches a count or if they are in the channel since some time. In nutshell, I want to do a bulk push.
I have tried aggregating the number of messages after consuming and storing them in an aggregated channel backed by SimpleMessageStore and have MessageGroupStoreReaper checking for messages in the channel.
I am not satisfied with this approach as I am consuming the messages and storing them in an in-memory store, I am aware of the JDBC store as well but I don't want to follow this approach as the message channels in spring XD are backed by redis/mq I would like to consume from the input channel based on my conditions.
My current bean configuration is as shown below:
<int:aggregator id="messageAggregator" ref="messageAggregatorBean"
method="aggregate" input-channel="input" output-channel="aggregatorOutputChannel"
release-strategy="messageReleaseStrategyBean" release-strategy-method="canRelease"
send-partial-result-on-expiry="true" message-store="resultMessageStore">
</int:aggregator>
<int:service-activator id="contributionIndexerService"
ref="contributionIndexerBean" method="bulkIndex" input-channel="aggregatorOutChannel" />
<bean id="resultMessageStore"
class="org.springframework.integration.store.SimpleMessageStore" />
<bean id="resultMessageStoreReaper"
class="org.springframework.integration.store.MessageGroupStoreReaper">
<property name="messageGroupStore" ref="resultMessageStore" />
<property name="timeout" value="60000" />
</bean>
<task:scheduled-tasks>
<task:scheduled ref="resultMessageStoreReaper" method="run"
fixed-rate="10000" />
</task:scheduled-tasks>
Any thoughts or comments?
Thanks in advance.
I'm not sure that you will be able to determine the count of messages in the Broker's queue (Redis/RabbitMQ or even just normal JMS), more over how much they have been there.
You definitely should consume them to do such a logic.
And yes, I think Aggregator can help you. But right: that must be a Persistent Message Store.
for the case
if they are in the channel since some time
The Aggregator suggest an option like group-timeout to release those groups which haven't reached the releaseStrategy condition, but you would like to emit them anyway over some time: http://docs.spring.io/spring-integration/reference/html/messaging-routing-chapter.html#agg-and-group-to
I got this error.
org.springframework.messaging.MessageDeliveryException: failed to send message to channel 'org.springframework.integration.handler.MessageHandlerChain$ReplyForwardingMessageChannel#261b0001' within timeout: -1
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:112)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:44)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:94)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.sendMessage(AbstractReplyProducingMessageHandler.java:260)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.sendReplyMessage(AbstractReplyProducingMessageHandler.java:241)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.produceReply(AbstractReplyProducingMessageHandler.java:205)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleResult(AbstractReplyProducingMessageHandler.java:199)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:177)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
at org.springframework.integration.handler.MessageHandlerChain.handleMessageInternal(MessageHandlerChain.java:131)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:101)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97)
And my spring integration xml is like this.
<int-ws:inbound-gateway id="asmTestWSGateway" request-channel="webserviceInboundChannel"
reply-channel="outboundStartChannel" marshaller="asmMarshaller"
unmarshaller="asmMarshaller" />
<bean id="asmMarshaller" class="org.springframework.oxm.castor.CastorMarshaller">
<property name="targetClasses" value="com.sds.redca.test.vo.DependencyVO"></property>
<property name="mappingLocation" value="classpath:mapping-asm.xml"></property>
</bean>
<int:chain input-channel="webserviceInboundChannel" output-channel="checkAuthentificationChannel">
<int:service-activator>
<bean class="com.sds.redca.core.endpoint.activator.BuildMsgActivator" />
</int:service-activator>
<int:filter ref="sampleFilter" />
<int:header-enricher>
<int:header name="protocolType" value="#{T(com.sds.redca.cmm.RedCAConstants).WS_PROTOCOL}"/>
</int:header-enricher>
</int:chain>
<int:chain input-channel="checkAuthentificationChannel" output-channel="outboundStartChannel">
<int:service-activator>
<bean class="com.sds.redca.core.endpoint.activator.AuthentificationServiceActivator" />
</int:service-activator>
</int:chain>
Webservice gateway received soap message with marshalled message translated to vo object.
and normalized and checked whether or not it is valid then go to back to the reply channel for returning the client.
I don't know how to solve this. Even if I make logging level debug, i can't find the reason.
Any help would be greatly appreciated!
Show, please, your outboundStartChannel config.
Tipically failed to send message to channel ... within timeout: -1 is happened, when we use QueueChannel and can't overcome its current capacity, becuase there is not enough resources to poll messages from that queue to process.
Actoually I don't see reason for your use-case to use reply-channel on <int-ws:inbound-gateway> at all. Just remove it and output-channel="outboundStartChannel" from the last <chain> as well. And rely on the TemporaryReplyChannel from the MessageHeaders in that case to achieve a short circuit for the entire flow.
I have a situation I don't know how to manage it.
The flow I neeed is the following one:
The first service makes its job and creates a message that needs to be retrieved in the chain's end.
When the first service finishes, I need to invoke a push notification server via a new service with a particular message but with some info related to the one created in step 1.
Finally, I the push notification has been sent successfully, I have to retrieve the message created in step 1.
The question is, how can I keep message created in step 1 when the outbound-gateway calling was produced and retrieved me the message from notification push server?
<int:chain input-channel="v1.inputChannel.input" output-channel="v1.inputChannel.output" send-timeout="50000">
<int:header-enricher>
<int:error-channel ref="v1.inputChannel.error" />
</int:header-enricher>
<int:service-activator ref="v1.input.service" method="methodName"/>
<int:service-activator ref="v1.notificationPusher.service" method="pushNotification"/>
<int-http:outbound-gateway url="http://example.com/api/elements/:element_id/objects" http-method="POST">
<int-http:uri-variable name="element_id" expression="#pathVariables.elementId"/>
</int-http:outbound-gateway>
<!-- here the transformer needs to get the messsage from v1.input.service -->
<int:object-to-json-transformer/>
</int:chain>
After some researching in 'Spring Integration in Action' I think the best option for resolving this situation is to use the wire tap pattern, having the push notification service as secondary flow.
Here you can see the book's example where, If I have understood it in a correct way, the auditChannel acts as a secondary flow separated from the main one.
<channel id="debitChannel">
<interceptors>
<wire-tap channel="auditChannel"/>
</interceptors>
</channel>
<service-activator input-channel="debitChannel" method="process">
<beans:bean class="siia.monitoring.wiretap.DebitService"/>
</service-activator>
<filter input-channel="auditChannel" expression="payload.amount > 10000" output-channel="logger"/>
<logging-channel-adapter id="logger" expression="'auditing debit: ' + payload"/>
I think you could achieve this with something like that:
Dupĺicate your message before send it to the outbound
Send one to the outbound and the another one to some channel
Use an Aggregator with the same timeout of your outbound to "join" them, but you could, in fact, only pass the message from the first step through (I haven't tested it)
Send it to the json transformer
To use this approach, I believe, you have to put the outbound and the json-transformer outside the chain as well as your logic to duplicate the message.
I can think of one way, which is to squirrel away the payload into a header after step 1, then retrieve and repopulate the payload right before you invoke step 3:
<int:service-activator ref="v1.input.service" method="methodName"/>
<int:header-enricher>
<int:header name="originalpayload" expression="payload"/>
</int:header-enricher>
<int:service-activator ref="v1.notificationPusher.service" method="pushNotification"/>
<int:enricher expression="headers.originalpayload"/>
<int-http:outbound-gateway url="http://xxx.com/api/elements/:element_id/objects" http-method="POST">
<int-http:uri-variable name="element_id" expression="#pathVariables.elementId"/>
</int-http:outbound-gateway>
I'm currently using a spring integration channel through a gateway and see a very strange (and annoying) behaviour. messages are processed extremely late (more than a minute) after they were send.
Since the channel is handled asynchronously by its own thread pool i guess these threads simply aren't active. since sending of these messages is pretty important i would like to force the handling of the message synchronously in the current thread.
is this possible?
My current config:
<int:annotation-config />
<task:executor id="integrationPool" pool-size="0-100" />
<int:poller default="true" task-executor="integrationPool" fixed-rate="50" />
<int:channel id="loggableevents">
<int:queue />
</int:channel>
<int:channel id="outgoingevents">
<int:queue />
<int:interceptors>
<int:wire-tap channel="loggableevents" />
</int:interceptors>
</int:channel>
<int:outbound-channel-adapter channel="outgoingevents" method="handleMessage" ref="momadapter" />
<bean id="momadapter" class="my.project.mom.EventSendingMessageHandler" />
<!-- Logging -->
<int:outbound-channel-adapter channel="loggableevents" ref="loggingadapter" method="handleMessage" />
<bean id="loggingadapter" class="my.project.logging.EventLoggingHandler" />
<int:gateway id="eventgateway" service-interface="my.project.mom.EventGateway" default-request-channel="outgoingevents" />
I use the gateway for convenience, would it be better to access the channel directly?
Just remove the <queue/> element from the outgoingEvents channel and it will all run on the calling thread.
http://static.springsource.org/spring-integration/docs/2.1.2.RELEASE/reference/html/messaging-channels-section.html
If you get this answer and are using the interface based gateway. Using the message channel is mandatory. The solution in that case is, use the direct channel. The following code sample shows how to do this using the java DSL.
#Bean(name = "spa.import.zipcodes")
public MessageChannel queueRecordsChannel() {
return MessageChannels.direct().get();
}
Check this page for more information on the DirectChannel:
http://docs.spring.io/spring-integration/reference/html/messaging-channels-section.html