<control-bus input-channel ="inboundChannel/>
<channel id ="inboundChannel"/>
<message-driven-channel-adapter id="inAQueue" channel="inboundChannel"
auto-startup="false" container="DefaultContainer"/>
<service-activator input-channel="inboundChannel" ref="Something"
method="abc"/>
inboundChannel.send(MessageBuilder.withPayload("#inAQueue.stop()").build();
inboundChannel.send(MessageBuilder.withPayload("#inAQueue.start()").build());
But my service activator class receive this message and throws class cast exception that string can not be cast to jmstextMessage.
And I'm not sure if i am trying it in correct manner or not.
You have 2 subscribers on inboundChannel - the message-driven adapter and the control-bus. Messages will be distributed in round-robin fashion.
The control bus should subscribe to its own channel, to which the control message is sent.
Related
I have jms message endpoint like:
#Bean
public JmsMessageDrivenEndpoint fsJmsMessageDrivenEndpoint(ConnectionFactory fsConnectionFactory,
Destination fsInboundDestination,
MessageConverter fsMessageConverter) {
return Jms.messageDrivenChannelAdapter(fsConnectionFactory)
.destination(fsInboundDestination)
.jmsMessageConverter(fsMessageConverter)
.outputChannel("fsChannelRouter.input")
.errorChannel("fsErrorChannel.input")
.get();
}
So, my questions is did I get next message before current message will be processed? If it will...Did it will get all messages in mq queue until it fills up all the memory? How to avoid it?
The JmsMessageDrivenEndpoint is based on the JmsMessageListenerContainer, its threading model and MessageListener callback for pulled messages. As long as your MessageListener blocks, it doesn't go to the next message in the queue to pull. When we build an integration flow starting with JmsMessageDrivenEndpoint, it becomes as a MessageListener callback. As long as we process the message downstream in the same thread (DirectChannel by default in between endpoints), we don't pull the next message from JMS queue. If you place a QueueChannel or an ExecutorChannel in between, you shift a processing to a different thread. The current one (JMS listener) gets a control back and it is ready to pull the next message. And in this case your concern about the memory is correct. You can still use QueueChannel with limited size or your ExecutorChannel can be configured with limited thread pool.
In any way my recommendation do not do any thread shifting in the flow when you start from JMS listener container. It is better to block for the next message and let the current transaction to finish its job. So you won't lose a message when something crashes.
I want to know the redelivery of JMS in a microservices.
For example, if I have a microservices system. And I have 2 instances of User service. And have a listener on a destination in user service. It means I have 2 listeners. The listener is like this:
#JmsListener(destination = "order:new", containerFactory = "orderFactory")
#Transactional
public void create(OrderDTO orderDTO) {
Order order = new Order(orderDTO);
orderRepository.save(order);
jmsTemplate.convertAndSend("order:need_to_pay", order);
}
So my question is, how many times a message will be delivered. And if there is some error in this function, and the message will be re-delivered. But I have 2 instances of the service. And on which this message will be delivered?
It's not part of the spec; it depends on the broker configuration how many times it will be delivered; many brokers can be configured to send the message to a dead-letter queue after some number of attempts.
There is no guarantee the redelivery will go to the same instance.
I am using redis as a queue (using the spring queue-in/outbound-channel-adapter) to distribute tasks (a message into the queue, etc)
As the throughput is quite high we observed that, although the messages were sent to the redis queue, a lot of them were lost and no messages arrived to the component after the inbound (a header router)
The channel config is attached below; the point is that we though that the problem was in this header router after the inbound addapter, that was unable to manage the rate of messages read from the queue, so they were lost.
We have use an intermediate element between the inbound adapter and this component (that is a header-router) and add a queue to fix this.
This works fine, but actually we don't fully understand the solution and if this is the proper one.
An expert view and opinion about this configuration will be wellcome!
Thanks
<!-- a Queue Inbound Channel Adapter is available to 'right pop' messages
from a Redis List. -->
<redis:queue-inbound-channel-adapter
id="fromRedis" channel="in" queue="${name}"
receive-timeout="1000" recovery-interval="3000" expect-message="true"
auto-startup="true"/>
<!-- a queue to avoid lost messages before the header router -->
<int:channel id="in">
<int:queue capacity="1000"/>
</int:channel>
<!-- a bridge to connect channels and have a poller -->
<int:bridge input-channel="in" output-channel="out">
<int:poller fixed-delay="500" />
</int:bridge>
<int:header-value-router id="router" timeout="15000"
input-channel="out" header-name="decision"
resolution-required="false" default-output-channel="defaultChannel" />
---added on 26/02
To insert messages into redis we have a web service, but actually is as you said, simply write messages into redis (
for... channel.send(msg)
Nothing more
About your answer I am now thinking in remove the in channel and its queue and use directly the header-value-router; but I have more questions:
I think the right solution is a low value for timeout in header-value-router, so I'll have the error notification faster if we don't have a consumer available. If I don't use a value as timeout, it will block indefinitely and this is a bad idea, isn't it?
I don't know how to manage the MesssageDeliveryException because the router don't have an error-channel configuration, ???
I think that if I can manage this error and get the message back I can re-send it to redis again. There are other servers that get the messages from redis and they luckily could attend it.
I add my proposed solution, but is not complete and we are not sure about the error management as I explained above
<!-- a Queue Inbound Channel Adapter is available to 'right pop' messages
from a Redis List. -->
<redis:queue-inbound-channel-adapter
id="fromRedis" channel="in" queue="${name}"
receive-timeout="1000" recovery-interval="3000" expect-message="true"
auto-startup="true"/>
<!-- a header-value-router with quite low timeout -->
<int:header-value-router id="router" timeout="150"
input-channel="in" header-name="decision"
resolution-required="false" default-output-channel="defaultChannel" />
<!-- ¿if MessageDeliveryException???? what to do??? -->
<int:channel id="someConsumerHeaderValue">
<int:dispatcher task-executor="ConsumerExecutor" />
</int:channel>
<!-- If 5 threads are busy we queue messages up to 5; if queue is full we can increase to 5 more working threads; if no more threads we have a... ¿¿MessageDeliveryException?? -->
<task:executor id="ConsumerExecutor" pool-size="5-5"
queue-capacity="5" />
Well, that's great to see such an observation. That might improve the Framework somehow.
So, I'd like to see:
Some test-case to reproduce from the Framework perspective.
Although I guess there is just enough to send a lot of messages to the Redis and use your config to consume. (Correct me if there is need anything else)
The downstream flow after the <int:header-value-router>. Look, you use there timeout="15000" which is synonym to the send-timeout :
Specify the maximum amount of time in milliseconds to wait
when sending Messages to the target MessageChannels if blocking
is possible (e.g. a bounded queue channel that is currently full).
By default the send will block indefinitely.
Synonym for 'timeout' - only one can be supplied.
From here I can say that if your downstream consumer if enough slow on some QueueChannel there you end up with the:
/**
* Inserts the specified element at the tail of this queue, waiting if
* necessary up to the specified wait time for space to become available.
*
* #return {#code true} if successful, or {#code false} if
* the specified waiting time elapses before space is available
* #throws InterruptedException {#inheritDoc}
* #throws NullPointerException {#inheritDoc}
*/
public boolean offer(E e, long timeout, TimeUnit unit)
....
while (count.get() == capacity) {
if (nanos <= 0)
return false;
nanos = notFull.awaitNanos(nanos);
}
Pay attention to that return false; indicating exactly the message lost.
That is also know like back-pressure drop strategy.
Let me know if you have different picture there.
You may consider to remove that timeout="15000" to meet the same in queue channel behavior.
UPDATE
Well, the error handling works a bit different way. The "guilty" component just throws Exception, like it is with raw Java and it is OK that this component isn't responsible for Exception catching that is up to the caller.
And caller in our case an upstream component - <redis:queue-inbound-channel-adapter>.
Any inbound channel adapter has an error-channel option. Through the <poller> if it is MessageSource or directly when it is MessageProducer.
I'm sure you will be able to handle:
if (!sent) {
throw new MessageDeliveryException(message,
"failed to send message to channel '" + channel + "' within timeout: " + timeout);
}
in that error-channel sub-flow and achieve your requirements for recovery.
We are using spring integration and daily in our logs we can see below stacktrace. Other JMS adapters are working fine, we think only below one is missing something:
Spring integration configuration:
<jms:message-driven-channel-adapter concurrent-consumers="1" id="jmsInLOAN" destination="queueLOAN" channel="LOANCommonDataChannel" acknowledge="transacted" />
Please find below MQ statistics of Put and Msgs read count, there should be exact count of Message read by adapter. I am worried about spring integration's message-driven-channel-adapter of reading extra messages from queue. Any help would be appreciated.
WARN 07/Jan/2016 09:04:15,438 [org.springframework.jms.listener.DefaultMessageListenerContainer#23-1] springframework.jms.listener.DefaultMessageListenerContainer - [SYSTEM_ID=HBUSLOANIQ] [MESSAGE_ID=null] Execution of JMS message listener failed, and no ErrorHandler has been set.
org.springframework.integration.MessagingException: unsupported payload type [com.ibm.jms.JMSMessage]
at org.springframework.integration.xml.DefaultXmlPayloadConverter.convertToDocument(DefaultXmlPayloadConverter.java:76)
at org.springframework.integration.xml.DefaultXmlPayloadConverter.convertToNode(DefaultXmlPayloadConverter.java:88)
at org.springframework.integration.xml.router.XPathRouter.getChannelIdentifiers(XPathRouter.java:119)
at org.springframework.integration.router.AbstractMessageRouter.determineTargetChannels(AbstractMessageRouter.java:247)
at org.springframework.integration.router.AbstractMessageRouter.handleMessageInternal(AbstractMessageRouter.java:211)
It looks like you are passing the unconverted JMS message (com.ibm.jms.JMSMessage) to the XML Payload converter...
org.springframework.integration.MessagingException: unsupported payload type [com.ibm.jms.JMSMessage]
at org.springframework.integration.xml.DefaultXmlPayloadConverter.convertToDocument(DefaultXmlPayloadConverter.java:76)
Perhaps you have set extract-payload to false ?
Although it's not in the configuration you show.
Turning on DEBUG logging will show the payload type of messages passing through the system.
The issue was because of valid and poisonous(which has a payload type [com.ibm.jms.JMSMessage]) messages we were getting onto queue. Valid messages processed well, but poisounous messages not able to digest by application and send to the BackoutQueue.
In our case, BOQ threshold is 3 that means 3 times my application will try to consume a perticular message and if the message backout 3 times then it will be moved to BOQ queue and (msgs read - msgs put)/3 on LOAIQ == the msgs put on to BOQ queue at that sampling interval. From msgs put on BOQ queue, we can see how many messages are backout from LOAIQ queue. That is why the message read count is more than that of msg received.
I am using a publish-subscribe channel after the inbound gateway receives a String message to parallely send it to the logger to log the message and to a transformer to transform the message. I want both these acivities to happen in parallel.
My question is very simple - Does the publish subscribe channel in spring integration sends messages to it's subscibers parallely?
Below is the code snippet from the source of spring-integration-context.xml.
<int:gateway id="gateway" service-interface="com.test.Gateway">
</int:gateway>
<int:publish-subscribe-channel id="publishsubscribechannel" />
<int:service-activator input-channel="publishsubscribechannel"
method="transformEvent" ref="transformer" output-channel="transformerreplychannel">
</int:service-activator>
<int:service-activator input-channel="publishsubscribechannel"
method="logMessage" ref="logger">
</int:service-activator>
Here the transformer and the logger are 2 subscribers to the publishsubscribechannel. In this setup will the message flow to logger and transformer from the gateway happen asynchronously by default??...OR I need to do some other configuration to achieve the same.
... or using JavaConfig
#Bean
public MessageChannel publishsubscribechannel() {
return new PublishSubscribeChannel(executor());
}
#Bean
public ThreadPoolTaskExecutor executor() {
ThreadPoolTaskExecutor pool = new ThreadPoolTaskExecutor();
pool.setCorePoolSize(10);
pool.setMaxPoolSize(10);
pool.setWaitForTasksToCompleteOnShutdown(true);
return pool;
}
By default, it runs in sequence. So in your case, it will be transformer, then logger. If you want to run it in parallel, you need to specify task-executor
<int:publish-subscribe-channel id="publishsubscribechannel" task-executor="executor" />
...
<task:executor id="executor" pool-size="10" />
And by using task-executor, the message handling is performed asynchronously.
As is from the Spring Integration documentation
PublishSubscribeChannel
The PublishSubscribeChannel implementation broadcasts any Message sent to it to all of its subscribed handlers. This is most often used for sending Event Messages whose primary role is notification as opposed to Document Messages which are generally intended to be processed by a single handler. Note that the PublishSubscribeChannel is intended for sending only. Since it broadcasts to its subscribers directly when its send(Message) method is invoked, consumers cannot poll for Messages (it does not implement PollableChannel and therefore has no receive() method). Instead, any subscriber must be a MessageHandler itself, and the subscriber's handleMessage(Message) method will be invoked in turn.
Prior to version 3.0, invoking the send method on a PublishSubscribeChannel that had no subscribers returned false. When used in conjunction with a MessagingTemplate, a MessageDeliveryException was thrown. Starting with version 3.0, the behavior has changed such that a send is always considered successful if at least the minimum subscribers are present (and successfully handle the message). This behavior can be modified by setting the minSubscribers property, which defaults to 0.
[Note] Note
If a TaskExecutor is used, only the presence of the correct number of subscribers is used for this determination, because the actual handling of the message is performed asynchronously.
Please note the Note.
It does mention if the TaskExecutor is used, the message handling would be asynchronous.
So, yes you have to add a TaskExecutor for this to be asynchronous.