I'm currently using a spring integration channel through a gateway and see a very strange (and annoying) behaviour. messages are processed extremely late (more than a minute) after they were send.
Since the channel is handled asynchronously by its own thread pool i guess these threads simply aren't active. since sending of these messages is pretty important i would like to force the handling of the message synchronously in the current thread.
is this possible?
My current config:
<int:annotation-config />
<task:executor id="integrationPool" pool-size="0-100" />
<int:poller default="true" task-executor="integrationPool" fixed-rate="50" />
<int:channel id="loggableevents">
<int:queue />
</int:channel>
<int:channel id="outgoingevents">
<int:queue />
<int:interceptors>
<int:wire-tap channel="loggableevents" />
</int:interceptors>
</int:channel>
<int:outbound-channel-adapter channel="outgoingevents" method="handleMessage" ref="momadapter" />
<bean id="momadapter" class="my.project.mom.EventSendingMessageHandler" />
<!-- Logging -->
<int:outbound-channel-adapter channel="loggableevents" ref="loggingadapter" method="handleMessage" />
<bean id="loggingadapter" class="my.project.logging.EventLoggingHandler" />
<int:gateway id="eventgateway" service-interface="my.project.mom.EventGateway" default-request-channel="outgoingevents" />
I use the gateway for convenience, would it be better to access the channel directly?
Just remove the <queue/> element from the outgoingEvents channel and it will all run on the calling thread.
http://static.springsource.org/spring-integration/docs/2.1.2.RELEASE/reference/html/messaging-channels-section.html
If you get this answer and are using the interface based gateway. Using the message channel is mandatory. The solution in that case is, use the direct channel. The following code sample shows how to do this using the java DSL.
#Bean(name = "spa.import.zipcodes")
public MessageChannel queueRecordsChannel() {
return MessageChannels.direct().get();
}
Check this page for more information on the DirectChannel:
http://docs.spring.io/spring-integration/reference/html/messaging-channels-section.html
Related
require join thread like feature in spring integration. If 3 threads are complete, then execute 4th thread. As shown in the picture, 3 outbound gateways running in parallel and the 4th outbound will run only after execution of first 3.
However, I am not doing splitter/aggregater so not required. Nor required any chain operation I guess because it will execute things in sequence.
parallel execution and then wait for the execution to complete
<int:publish-subscribe-channel id="publishsubscribechannel"
task-executor="executor" />
<task:executor id="executor" pool-size="10" />
<int-http:inbound-channel-adapter
id="httpOrderCreateInboundAdapter"
channel="orderCreateInboundChannel" path="/input"
supported-methods="GET, POST"
</int-http:inbound-channel-adapter>
<int-http:outbound-gateway id="micros1" request-
channel="publishsubscribechannel"
reply-channel="channel1"
</int-http:outbound-gateway>
<int-http:outbound-gateway id="micros2" request-
channel="publishsubscribechannel"
reply-channel="channel2"
</int-http:outbound-gateway>
<int-http:outbound-gateway id="micros3" request-
channel="publishsubscribechannel"
</int-http:outbound-gateway>
<int-http:outbound-gateway id="micros4" request-channel="channel2"
reply-channel="channel4"
</int-http:outbound-gateway>
Consider to use Scatter-Gather component for that publish-subscribe channel. Yes, all your gateways must return to their reply channel header to let gatherer to collect replies from them . And really after that you will be able to proceed to the Outbound4.
I am using Spring Integration for processing file. Here's my configuration
<int:channel id="startChannel">
<int:splitter input-channel="startChannel" output-channel="parseChannel" ref="splitter" method="split" />
<int:channel id="parseChannel">
<int:queue size="50"/>
</int:channel>
<int:service-activator input-channel="parseChannel" output-channel="aggregatChannel" ref="process" method="process" />
<int:channel id="aggregateChannel">
<int:queue size="50"/>
</int:channel>
<int:aggregate input-channel="aggregateChannel" output-channel="postProcessingChannel" ...(other attributes) />
<int:channel id="postProcessingChannel">
<int:queue size="50"/>
</int:channel>
<int:channel id="outboundChannel">
<int:queue size="50"/>
</int:channel>
<int:service-activator input-channel="postProcessingChannel" output-channel="outbound-channel" ... />
<int:outbound-adapter channel="outputChannel" ... />
..global poller
..global taskexecutor of size 40
My configuration works fine till the aggregator. Aggregator is able to put messages at the postProcessingChannel but nobody is reading from the postProcessing Channel.
On a closer look, i observe -
1) post processing channel is filled up with messages to its capacity.
2) threads are blocked in aggregator implementation, because they are not able to put messages in the postPostProcessing Channel.
My question is why no thread is reading from PostProcessingchannel ? From JProfiler I can see, many threads are idle/free not doing anything.
Can someone please help me understand this behavior.
You have a lot of queue channels; it is generally not necessary to make every channel a queue channel.
If you have more configuration, you are probably experiencing thread starvation in the default taskScheduler bean - it only has 10 threads by default.
If you have close to 10 queue channels in your application you will run out of threads because, by default, each queue channel will block a thread for 1 second (receiveTimeout).
Consider reducing the number of queue channels (as I said, it is not generally needed to have so many; you don't need to hand off to another thread between every component).
Or, see Configuring the Task Scheduler to increase the available threads for scheduled tasks.
I got this error.
org.springframework.messaging.MessageDeliveryException: failed to send message to channel 'org.springframework.integration.handler.MessageHandlerChain$ReplyForwardingMessageChannel#261b0001' within timeout: -1
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:112)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:44)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:94)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.sendMessage(AbstractReplyProducingMessageHandler.java:260)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.sendReplyMessage(AbstractReplyProducingMessageHandler.java:241)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.produceReply(AbstractReplyProducingMessageHandler.java:205)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleResult(AbstractReplyProducingMessageHandler.java:199)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:177)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
at org.springframework.integration.handler.MessageHandlerChain.handleMessageInternal(MessageHandlerChain.java:131)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:101)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97)
And my spring integration xml is like this.
<int-ws:inbound-gateway id="asmTestWSGateway" request-channel="webserviceInboundChannel"
reply-channel="outboundStartChannel" marshaller="asmMarshaller"
unmarshaller="asmMarshaller" />
<bean id="asmMarshaller" class="org.springframework.oxm.castor.CastorMarshaller">
<property name="targetClasses" value="com.sds.redca.test.vo.DependencyVO"></property>
<property name="mappingLocation" value="classpath:mapping-asm.xml"></property>
</bean>
<int:chain input-channel="webserviceInboundChannel" output-channel="checkAuthentificationChannel">
<int:service-activator>
<bean class="com.sds.redca.core.endpoint.activator.BuildMsgActivator" />
</int:service-activator>
<int:filter ref="sampleFilter" />
<int:header-enricher>
<int:header name="protocolType" value="#{T(com.sds.redca.cmm.RedCAConstants).WS_PROTOCOL}"/>
</int:header-enricher>
</int:chain>
<int:chain input-channel="checkAuthentificationChannel" output-channel="outboundStartChannel">
<int:service-activator>
<bean class="com.sds.redca.core.endpoint.activator.AuthentificationServiceActivator" />
</int:service-activator>
</int:chain>
Webservice gateway received soap message with marshalled message translated to vo object.
and normalized and checked whether or not it is valid then go to back to the reply channel for returning the client.
I don't know how to solve this. Even if I make logging level debug, i can't find the reason.
Any help would be greatly appreciated!
Show, please, your outboundStartChannel config.
Tipically failed to send message to channel ... within timeout: -1 is happened, when we use QueueChannel and can't overcome its current capacity, becuase there is not enough resources to poll messages from that queue to process.
Actoually I don't see reason for your use-case to use reply-channel on <int-ws:inbound-gateway> at all. Just remove it and output-channel="outboundStartChannel" from the last <chain> as well. And rely on the TemporaryReplyChannel from the MessageHeaders in that case to achieve a short circuit for the entire flow.
I have a situation I don't know how to manage it.
The flow I neeed is the following one:
The first service makes its job and creates a message that needs to be retrieved in the chain's end.
When the first service finishes, I need to invoke a push notification server via a new service with a particular message but with some info related to the one created in step 1.
Finally, I the push notification has been sent successfully, I have to retrieve the message created in step 1.
The question is, how can I keep message created in step 1 when the outbound-gateway calling was produced and retrieved me the message from notification push server?
<int:chain input-channel="v1.inputChannel.input" output-channel="v1.inputChannel.output" send-timeout="50000">
<int:header-enricher>
<int:error-channel ref="v1.inputChannel.error" />
</int:header-enricher>
<int:service-activator ref="v1.input.service" method="methodName"/>
<int:service-activator ref="v1.notificationPusher.service" method="pushNotification"/>
<int-http:outbound-gateway url="http://example.com/api/elements/:element_id/objects" http-method="POST">
<int-http:uri-variable name="element_id" expression="#pathVariables.elementId"/>
</int-http:outbound-gateway>
<!-- here the transformer needs to get the messsage from v1.input.service -->
<int:object-to-json-transformer/>
</int:chain>
After some researching in 'Spring Integration in Action' I think the best option for resolving this situation is to use the wire tap pattern, having the push notification service as secondary flow.
Here you can see the book's example where, If I have understood it in a correct way, the auditChannel acts as a secondary flow separated from the main one.
<channel id="debitChannel">
<interceptors>
<wire-tap channel="auditChannel"/>
</interceptors>
</channel>
<service-activator input-channel="debitChannel" method="process">
<beans:bean class="siia.monitoring.wiretap.DebitService"/>
</service-activator>
<filter input-channel="auditChannel" expression="payload.amount > 10000" output-channel="logger"/>
<logging-channel-adapter id="logger" expression="'auditing debit: ' + payload"/>
I think you could achieve this with something like that:
Dupĺicate your message before send it to the outbound
Send one to the outbound and the another one to some channel
Use an Aggregator with the same timeout of your outbound to "join" them, but you could, in fact, only pass the message from the first step through (I haven't tested it)
Send it to the json transformer
To use this approach, I believe, you have to put the outbound and the json-transformer outside the chain as well as your logic to duplicate the message.
I can think of one way, which is to squirrel away the payload into a header after step 1, then retrieve and repopulate the payload right before you invoke step 3:
<int:service-activator ref="v1.input.service" method="methodName"/>
<int:header-enricher>
<int:header name="originalpayload" expression="payload"/>
</int:header-enricher>
<int:service-activator ref="v1.notificationPusher.service" method="pushNotification"/>
<int:enricher expression="headers.originalpayload"/>
<int-http:outbound-gateway url="http://xxx.com/api/elements/:element_id/objects" http-method="POST">
<int-http:uri-variable name="element_id" expression="#pathVariables.elementId"/>
</int-http:outbound-gateway>
I have 2 questions. Here they are:
Is there any way to determine the number of messages waiting to be
processed in a spring integration channel while the number of
customers keep increasing along the time?
In the app context, I want to be able to define x instances of
bean y which x and y, both consumes from channel p, programmatically
increase or decrease the consumers based on the load.
There is an example showed in spring 2gx but it used rabbitmq to determine load.
For 1), Spring Integration channels are just beans like any other bean. Assuming that you're using a standard pollable channel, you can autowire it in by name and then get the number of messages waiting:
http://docs.spring.io/spring-integration/api/org/springframework/integration/channel/QueueChannel.html#getQueueSize()
You may also be able to introspect this through JMX if you want.
For 2)... To do that, you'll want to use an AOP proxy on top of an object pool, then wire the proxy into (what I assume is a) Service Activator. Those properties can be externalized with a PropertyPlaceholderConfigurer. So, as an example:
<bean id="propertyPlaceholder"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="systemPropertiesMode" value="2" />
<property name="locations" value="classpath:my_config_params.properties" />
</bean>
<bean id="myBusinessObjectImpl" class="com.mycompany.whatever.impl.MyServiceImpl"
scope="prototype" autowire-candidate="false" />
<bean id="myBusinessObjPool" class="org.springframework.aop.target.CommonsPoolTargetSource">
<property name="targetBeanName" value="myBusinessObjectImpl" />
<!-- THIS IS THE KEY. myconfig.params.poolsize is the name of the property in the my_config_params.properties above. -->
<property name="maxSize" value="${myconfig.params.poolsize}" />
</bean>
<bean id="myBusinessObject" class="org.springframework.aop.framework.ProxyFactoryBean">
<property name="targetSource" ref="myBusinessObjPool" />
</bean>
<int:channel id="myInputChannel">
<int:queue size="500" />
</int:channel>
<int:service-activator inputChannel="myInputChannel" ref="myBusinessObject" method="processMessages">
<int:poller max-messages-per-poll="10" fixed-rate="5000"/>
</int:service>
This also allows you to make the Service Activator stateful.
Obligatory link to object pooling functionality:
http://docs.spring.io/spring/docs/3.0.x/spring-framework-reference/html/aop-api.html#aop-ts-pool