messagebased RPC with topic - spring

A simple message-based RPC is very easy to create. The server side exports the service, the client side uses a proxy.
What is the best way, to make the same thing with multiple repliers?
I want to send a request from a client. Then the client waits while all (maybe with timeout) replies are received.

You could use an aggregator with appropriate correlation and release strategies (and a group timeout).
EDIT:
Here's a version using a JMS Topic...
<bean id="connectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="vm://localhost"/>
</bean>
</property>
<property name="sessionCacheSize" value="10"/>
</bean>
<bean id="requestTopic" class="org.apache.activemq.command.ActiveMQTopic">
<constructor-arg value="topic.demo"/>
</bean>
<bean id="replyQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="queue.reply"/>
</bean>
<int-stream:stdin-channel-adapter id="stdin" channel="stdinToJmsoutChannel"/>
<int:channel id="stdinToJmsoutChannel"/>
<int:chain input-channel="stdinToJmsoutChannel">
<int:header-enricher>
<int:header name="jms_replyTo" ref="replyQueue" />
</int:header-enricher>
<int-jms:outbound-channel-adapter destination="requestTopic" />
</int:chain>
<int-jms:message-driven-channel-adapter channel="jmsReplyChannel"
destination="replyQueue"/>
<int:channel id="jmsReplyChannel" />
<int:chain input-channel="jmsReplyChannel">
<int:aggregator group-timeout="5000" expire-groups-upon-timeout="false"
send-partial-result-on-expiry="true"
discard-channel="logLateArrivers"
correlation-strategy-expression="headers['jms_correlationId']"
release-strategy-expression="size() == 2"/>
<int-stream:stdout-channel-adapter append-newline="true"/>
</int:chain>
<int:logging-channel-adapter id="logLateArrivers" />
<!-- Subscribers -->
<int-jms:inbound-gateway request-channel="upcase" request-destination="requestTopic" />
<int-jms:inbound-gateway request-channel="upcase" request-destination="requestTopic" />
<int:transformer input-channel="upcase" expression="payload.toUpperCase()" />
Type requests into the console:
Please type something and hit <enter>
foo
[FOO, FOO]
bar
[BAR, BAR]
baz
[BAZ, BAZ]

Related

How to use expression advice or error channel for exception handling?

I am new to spring integration. I have to handle few exceptions using spring integration.
we are using xml files for spring integration.
<int:chain id="sample-chain" input-channel="callTestChannel" output-channel="testResponseChannel">
<int:header-enricher>
<int:header name="Content-Type" value="application/json"/>
<int:header name="Accept" value="application/json"/>
</int:header-enricher>
<int-http:outbound-gateway id="testGateway"
url-expression="headers.externalUrl"
http-method="GET"
expected-response-type="com.example.test.TestProject"
charset="UTF-8"
request-factory="httpComponentClientRequestFactory"
mapped-request-headers="Content-Type,Accept"
reply-timeout="5000">
<int-http:uri-variable name="testId" expression="payload" />
<int-http:request-handler-advice-chain>
<bean class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice">
<property name="recoveryCallback">
<bean class="org.springframework.integration.handler.advice.ErrorMessageSendingRecoverer">
<constructor-arg ref="retryErrorChannel" />
</bean>
</property>
<property name="retryTemplate" ref="retryLoadTemplate" />
</bean>
</int-http:request-handler-advice-chain>
</int-http:outbound-gateway>
<int:transformer ref="fetchDetailsTransformer" method="processServiceDetails" />
</int:chain>
<int:channel id="retryErrorChannel"/>
<int:transformer input-channel="retryErrorChannel" output-channel="markErrorChannel"
expression="payload.getFailedMessage()"/>
<int:transformer input-channel="markErrorChannel" output-channel="tmsResponseChannel"
expression="'this Id :' + payload + ' could not find.'"/>
<bean id="retryLoadTemplate" class="org.springframework.retry.support.RetryTemplate">
<property name="retryPolicy">
<bean class="org.springframework.retry.policy.SimpleRetryPolicy">
<property name="maxAttempts" value="4" />
</bean>
</property>
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="initialInterval" value="1000" />
<property name="multiplier" value="5" />
</bean>
</property>
</bean>
Can someone please help how to create error channel for handling different exceptions? can I use some property on int-http:outbound-gateway?
I have read at some places to use expression advice but did not find any examples about using it in xml files or how to use them at all? or if there is any easy way to handle exceptions?
The documentation for that advice is here: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#message-handler-advice-chain
Over here you can find some samples how to configure those advices: https://github.com/spring-projects/spring-integration-samples/tree/main/intermediate/retry-and-more

How to use the RedeliveryPolicy in case of failed messages for outbound->inbound->httpgateway(messages failing here)->EXTERNAL SERVICE

How to use the RedeliveryPolicy in case of failed messages in outbound->inbound->httpoutboundgateway(messages failing here)->external service
the failed messages are not getting retried based on the RedeliveryPolicy mainly because the message is already dequeued before the exception occurs.
<int:channel id="jmsOutChannel" />
<jms:outbound-channel-adapter id="outboundJMSAdaptor" jms-template="jmsTemplate"
channel="jmsOutChannel"
destination-name="#{somebean.queueName}"/>
<int:channel id="jmsInChannel" />
<jms:message-driven-channel-adapter
channel="jmsInChannel" destination-name="#{somebean.queueName}"
connection-factory="jmsConnectionFactory" message-converter="jmsMessageConverter"/>
<int:header-enricher input-channel="jmsInChannel" output-channel="outbound_gateway_channel">
<int:header name="addressId" expression="payload.getId()"/>
<int:header name="Accept-Language" value="en_GB"/>
<int:header name="X-Source-CountryCode" value="GB"/>
<int:header name="X-Source-Operator" value="Enterprise"/>
<int:header name="X-Source-Division" value="CustomerManagement"/>
<int:header name="X-Source-System" value="${sapwebservices.http.header.source.system}"/>
<int:header name="X-Source-Timestamp" expression="new java.text.SimpleDateFormat('yyyy-MM-dd HH:mm:ss').format(new java.util.Date())"/>
<int:header name="Accept" value="application/json"/>
<int:header name="Content-Type" value="application/json;charset=UTF-8"/>
</int:header-enricher>
<bean id="httpRequestFactory" class="org.springframework.http.client.HttpComponentsClientHttpRequestFactory">
<property name="connectTimeout" value="${sapwebservices.http.rest.timeout}"/>
<property name="readTimeout" value="${sapwebservices.http.rest.timeout}"/>
<property name="httpClient" ref="httpClient"/>
</bean>
<int:object-to-json-transformer input-channel="outbound_gateway_channel"
output-channel="outbound_gateway_with_json"
object-mapper="nonNullObjectMapper"/>
<http:outbound-gateway mapped-request-headers="Accept*, Content-Type, X-*, HTTP_REQUEST_HEADERS"
request-channel="outbound_gateway_with_json"
reply-channel="print_payload"
url="${sapwebservices.ws.uri.updatecustomershippingaddress}"
http-method="PUT"
expected-response-type="java.lang.String"
charset="UTF-8"
request-factory="httpRequestFactory">
<http:uri-variable name="id" expression="headers['addressId']"/>
</http:outbound-gateway>
And for the redelivery policy I have,
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="messageConverter" ref="jmsMessageConverter" />
<property name="connectionFactory">
<bean class="org.springframework.jms.connection.SingleConnectionFactory">
<property name="targetConnectionFactory">
<ref local="jmsConnectionFactory" />
</property>
</bean>
</property>
</bean>
<bean id="jmsConnectionFactory" class="org.apache.activemq.spring.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616"/>
<property name="userName" value="admin"/>
<property name="password" value="admin"/>
<property name="redeliveryPolicy" ref="redeliveryPolicy"/>
<property name="nonBlockingRedelivery" value="true"/>
</bean>
<bean id="redeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<property name="maximumRedeliveries" value="${sapwebservice.redeliveryPolicy.maximumRedeliveries}"/>
<property name="initialRedeliveryDelay" value="${sapwebservice.redeliveryPolicy.initialRedeliveryDelay}"/>
<property name="backOffMultiplier" value="${sapwebservice.redeliveryPolicy.backOffMultiplier}"/>
<property name="useExponentialBackOff" value="${sapwebservice.redeliveryPolicy.useExponentialBackOff}"/>
<property name="redeliveryDelay" value="${sapwebservice.redeliveryPolicy.redeliveryDelay}"/>
</bean>
<bean id="demoQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="demo.queue"/>
</bean>
<bean id="jmsMessageConverter" class="sap.converter.JMSMessageConverter"/>
<amq:broker useJmx="true" persistent="false">
<amq:destinationPolicy>
<amq:policyMap>
<amq:defaultEntry>
<amq:policyEntry queue=">">
<amq:deadLetterStrategy>
<amq:individualDeadLetterStrategy queuePrefix="dlq." useQueueForQueueMessages="true"/>
</amq:deadLetterStrategy>
</amq:policyEntry>
</amq:defaultEntry>
</amq:policyMap>
</amq:destinationPolicy>
<amq:transportConnectors>
<amq:transportConnector uri="tcp://localhost:0" />
</amq:transportConnectors>
</amq:broker>
Expected result is to retry based in the RedeliveryPolicy and finally end up in dlq if all the retrials are failed.
Set acknowledge="transacted" on the message-driven channel adapter.
Spring Integration versions 4.2 and later (the current version is 5.1.7) set it to that by default; for earlier versions you have to set it in your configuration so the flow runs in a transaction and the dequeue is rolled back after an exception is thrown.

Spring Batch & Spring Integration (JMS) & Load Balance Slaves

I'm using
Spring Batch
Step 1
Step 2 Master (Partitioner)
Step 3
Spring Integration (JMS) to communicate Master and Slave
The issue we are seeing is, the first slave handles all JMS messages instead of even distribution between slaves.
See configuration as below
Master
<bean id="PreProcess" class="com.job.tasklet.PreProcessTasklet" scope="step">
<constructor-arg index="0" value="${run.slave}"/>
<property name="maxNumberOfSlaves" value="#{jobParameters['max-slave-count']}"/>
</bean>
<bean id="PostProcess" class="com.job.tasklet.PostProcessTasklet" scope="prototype">
<constructor-arg index="0" ref="chpsJobDataSource"/>
</bean>
<bean id="partitioner" class="com.job.partition.DatabasePartitioner" scope="step">
<constructor-arg index="3" value="${max.row.count}"/>
</bean>
<bean id="partitionHandler" class="com.job.handler.StepExecutionAggregatorHandler">
<property name="stepName" value="processAutoHoldSlaveStep"/>
<property name="gridSize" value="${grid.size}"/>
<property name="replyChannel" ref="aggregatedGroupRuleReplyChannel"/>
<property name="messagingOperations">
<bean class="org.springframework.integration.core.MessagingTemplate">
<property name="defaultChannel" ref="groupRuleRequestsChannel"/>
</bean>
</property>
</bean>
<!-- Request Start -->
<int:channel id="groupRuleRequestsChannel" />
<int-jms:outbound-channel-adapter channel="groupRuleRequestsChannel" jms-template="jmsTemplateToSlave"/>
<bean id="jmsTemplateToSlave" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="receiveTimeout" value="5000"/>
<property name="defaultDestinationName" value="defaultRequest"/>
</bean>
<bean id="jmsTemplateFromSlave" class="org.springframework.jms.core.JmsTemplate" parent="jmsTemplateToSlave">
<property name="defaultDestinationName" value="defaultRequest"/>
</bean>
<!-- Response Test Start -->
<int:channel id="groupRuleReplyChannel">
<!-- <int:queue/> -->
</int:channel>
<int-jms:inbound-channel-adapter channel="groupRuleReplyChannel" jms-template="jmsTemplateFromSlave">
<int:poller id="defaultPoller" default="true" max-messages-per-poll="1" fixed-rate="3000" />
</int-jms:inbound-channel-adapter>
<!-- define aggregatedReplyChannel -->
<int:channel id="aggregatedGroupRuleReplyChannel">
<int:queue/>
</int:channel>
<int:aggregator ref="partitionHandler"
input-channel="groupRuleReplyChannel"
output-channel="aggregatedGroupRuleReplyChannel"
send-timeout="3600000"/>
Slave
<int:channel id="requestsChannel" />
<bean id="connectionFactory" class="org.apache.activemq.spring.ActiveMQConnectionFactory">
<property name="brokerURL" value="${spring.activemq.broker-url}" />
<property name="trustAllPackages" value="true" />
</bean>
<int-jms:message-driven-channel-adapter id="jmsIn" destination-name="#{args[0]}" channel="requestsChannel" connection-factory="connectionFactory" max-messages-per-task="1"/>
<int:service-activator input-channel="requestsChannel" output-channel="replyChannel" ref="stepExecutionRequestHandler" />
<int:channel id="replyChannel" />
<int-jms:outbound-channel-adapter connection-factory="connectionFactory" destination-name="#{args[1]}" channel="replyChannel" />
Please advice if you have experience the issue.
Let me know if you need more information.
Note: I already search a lot at here and google but no luck for solution yet.
ActiveMQ uses a prefetch of 1000 by default see here.
In other words, the first (up to) 1000 partitions will go to the first consumer etc.
You can reduce the prefetch; 1 is probably fine for this application.

Spring Integration JMS Issue

We are using Spring Integration to consume messages from a queue and the requirement is to send the message to Error queue if there is any issue in the processing of the message consumed.
Flow works fine but one issue we see is when there is any exception thrown in processing of message, the message is redirected to the Error queue which we have configured but it is appended by the entire stack trace of the exception.
We are looking only for the original message to be delivered to the queue. Below is the configuration we have done,
<bean id="errorQ" class="com.ibm.mq.jms.MQQueue">
<constructor-arg value="${error.queue}" />
</bean>
<bean id="inQ" class="com.ibm.mq.jms.MQQueue">
<constructor-arg value="${inbound.queue}" />
</bean>
<int:channel id="error" />
<int:channel id="inbound" />
<int-jms:message-driven-channel-adapter
id="jmsIn" channel="inbound" container="messageListenerContainer"
acknowledge="transacted"></int-jms:message-driven-channel-adapter>
<int-jms:outbound-channel-adapter id="jmsError"
channel="error" connection-factory="mqConnectionFactory"
destination="errorQ" delivery-persistent="true"></int-jms:outbound-channel-adapter>
<int:service-activator id="service"
input-channel="inbound" ref="messageListener" method="someMethodInListerner">
<int:request-handler-advice-chain>
<ref bean="retryWithBackoffAdviceSession" />
</int:request-handler-advice-chain>
<bean id="retryWithBackoffAdviceSession"
class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice">
<property name="retryTemplate">
<bean class="org.springframework.retry.support.RetryTemplate">
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="initialInterval" value="${initialInterval}" />
<property name="multiplier" value="${multiplier}" />
<property name="maxInterval" value="${maxInterval}" />
</bean>
</property>
<property name="retryPolicy">
<bean class="org.springframework.retry.policy.SimpleRetryPolicy">
<property name="maxAttempts" value="${redelivery}" />
</bean>
</property>
</bean>
</property>
<property name="recoveryCallback">
<bean
class="org.springframework.integration.handler.advice.ErrorMessageSendingRecoverer">
<constructor-arg ref="error" />
</bean>
</property>
</bean>
The message sent to the error is an ErrorMessage with payload MessagingException.
The MessagingException has two properties cause and failedMessage.
So, if you want to send the original failed message to errorQ, you will need to add a transformer to the error flow...
<int:transformer ... expression="payload.failedMessage" />
EDIT
<int:chain input-channel="error">
<int:transformer expression="payload.failedMessage" />
<int-jms:outbound-channel-adapter ... />
</int:chain>
EDIT2
Generally, when using these techniques, it's useful to convey the reason for the failure. You can add the exception's message as a header...
<int:chain input-channel="error">
<int:header-enricher>
<int:header name="failureReason" expression="payload.cause.message" />
</int:header-enricher>
<int:transformer expression="payload.failedMessage" />
<int-jms:outbound-channel-adapter ... />
</int:chain>

Spring Integration queue is not getting message but channel is getting

Using Spring Integration for transferring message from one queue to other in Websphere MQ.
OutPut Channel shows message is going but I am not able to see in Queue from MQ Explorer.How the session commits ? and I am not able to stop the java application. How to gracefully stop with connection stop ?
where I am doing wrong ?
Here is the context config -
<int:logging-channel-adapter log-full-message="true" id="logger" level="INFO"/>
<bean id="jmsConnectionFactory" class="com.ibm.mq.jms.MQQueueConnectionFactory">
<property name="hostName" value="localhost" />
<property name="port" value="1414" />
<property name="queueManager" value="TEST" />
<property name="transportType" value="1" />
</bean>
<bean id="inQueue" class="com.ibm.mq.jms.MQQueue">
<constructor-arg value="TESTQ" />
</bean>
<bean id="outQueue" class="com.ibm.mq.jms.MQQueue">
<constructor-arg value="DEST_QUEUE" />
</bean>
<int:channel id="readFromChannel">
<int:interceptors>
<int:wire-tap channel="logger"/>
</int:interceptors>
</int:channel>
<int:channel id="sendToChannel" >
<int:queue/>
<int:interceptors>
<int:wire-tap channel="logger"/>
</int:interceptors>
</int:channel>
<int-jms:message-driven-channel-adapter id="jmsInAdapter"
connection-factory="cachingConnectionFactory"
destination="inQueue"
channel="readFromChannel" />
<int-jms:message-driven-channel-adapter id="jmsOutAdapter"
connection-factory="cachingConnectionFactory"
destination="outQueue"
channel="sendToChannel" />
<bean id="myMessenger" class="test.MyMessenger" />
<int:service-activator id="servAct" input-channel="readFromChannel" output-channel="sendToChannel" ref="myMessenger" method="transfer"/>
If I understand your use-case correctly, you want to shift messages from TESTQ to DEST_QUEUE. But you have <int-jms:message-driven-channel-adapter> for both to read messages from them, but no one to put messages to DEST_QUEUE.
Your <int:service-activator> get message from readFromChannel and it means 'get message from TESTQ', but it places message to the sendToChannel. But it doesn't say that you place a message to the DEST_QUEUE.
That is because <int-jms:message-driven-channel-adapter> is for reading messages from JMS (MQ in your case).
To send message to JMS (to the DEST_QUEUE in your case) you have to use <int-jms:outbound-channel-adapter> and there is no reason to mark sendToChannel as <queue>

Resources