In Spring Integration I have a DSL setup that looks something like this:
---> --->
---> --->
Messages --> Gateway ---> QueueChannel ---> MessageHandler(worker) -> QueueChannel ---> MH worker2 --> ...
---> --->
---> --->
I want to write an integration test for that, which will test if the message was in proper channels. I do not enrich headers right now anywhere, so I wanted to use:
this.someChannel.receive(TIMEOUT)
unfortunately, as I understand, it is blocking only the main thread of test, not all, so it cannot receive any message sent asynchronously.
I know, that there exists something like:
this.someChannel.getReceiveCount()
it gives me good results, but checking it is not enough for me.
How can I deal with it, to receive messages sent to queues in async way?
EDIT: It is important, that I need it, because the message is changed during going through workflow
You can add a channel interceptor to those queue channels in the test to track whatever comes through those channels. Of course you need to have in the test some barrier to wait for asynchronous messages or just blocking queue to gather them for verification in the end of test. Another way is to mock or spy message handlers in between, but again with similar CountDownLatch or BlockingQueue thread barrier to wait for asynchronous messages. See more info in docs: https://docs.spring.io/spring-integration/docs/5.2.3.RELEASE/reference/html/testing.html#test-context
Related
Using Mule 4.4 on premise along with Apache ActiveMQ. I am trying to get a better understanding of how Mule handles messaging.
I tried searching the internet but am not finding any details about the same.
I have a jms:listener:
<jms:listener doc:name="Listener" config-ref="JMS_Config" destination="Consumer.mine2.VirtualTopic.mine.test">
<jms:consumer-type >
<jms:queue-consumer />
</jms:consumer-type>
</jms:listener>
and I have a jms:consume:
<jms:consume doc:name="Consume" config-ref="JMS_Config" destination="Consumer.mine1.VirtualTopic.mine.test">
<jms:consumer-type >
<jms:queue-consumer />
</jms:consumer-type>
</jms:consume>
To me both seem to be doing the same job i.e. consume messages from a queue / topic. So why do we have these two components in Mule?
jms listener is a source, so using it you can trigger a flow whenever there is a new message in queue.
jms consume is an operation, so you can use it anywhere within a flow's execution, i.e. like a http request component that you put in the middle of a flow.
Both of them will consume a message from the queue/topic. However, when you are using a listener you are basically saying that "There is a queue, I do not know when a new message will come in, but whenever it comes I need to perform these actions"
When you use a consume operation, you are saying "I am expecting some message soon and with that I will to these actions".
Now in both cases a message may not come at all, and both have there own way to deal with it. A listener, since it is a source, will just not simply trigger the flow and keep on waiting. A consume will block your execution until a message is there, or you can configure a time out to not be blocked fore ever.
A common use case can be reprocessing messages from a DLQ. Generally when you use a queue, you also have a DLQ so that the messages that failed during processing, from the "main" queue, can be sent to the DLQ and reprocessed later.
Now, in this architecture, you will typically use the jms listener only with the main queue for the processing of the messages. And you will have a separate flow that can have a http listener so that you can trigger an HTTP Endpoint whenever you are ready to reprocess the messages from the DLQ. This flow with http listener with consume all the messages (probably in a loop) from the DLQ and publish them back to the main queue
I am using Spring AMQP v1.4.2 and trying to find a send method, possibly in RabbitTemplate, that blocks the thread until a confirm is received from the server to make sure we do not lose messages.
I have seen the sendAndReceive() method but it waits for a reply message in an RPC style, while I just want to wait for a confirm on the channel. I know that I can use the setConfirmCallback() method but then it means it is not synchronous and I may end up on a different thread when the confirmation arrives and I will lose my thread-locals, i.e. lots of Spring goodies.
Is there anyway to achieve that in Spring AMQP?
RabbitMQ gets huge performance benefits by using asynchronous publishing. As you say, you can configure publisher confirms to get an asynchronous confirmation that a message was delivered to a queue.
If you really want to block, you can use setChannelTransacted(true) on the RabbitTemplate; the commit will block until the message is secured in the queue(s).
We have an app that sends messages, and request specs that ensure that the messages are sent.
We are adding a queueing system for the messages. Each message is stored in the db, and then later processed and deleted. The records are de-queued asynchronously in another process. So the specs now fail.
What is a good way to automatically process the queue for the specs?
One approach would be to add an observer to the queue that automatically processes each message as it is queued. But I'm not sure if it makes sense to do it that way, especially since it is only for tests.
What is a good way to handle this?
If I'm understanding this correctly, you have spec that creates a message and sends it, then somehow verifies that it was sent. Now you are changing the app to queue messages and send them later. Where you had previously had one feature (send a message), now you have two features (1. queue a message; 2. send a message).
I'd say the specs should test those features separately, i.e., one spec that verifies that newly created messages are queued, and another spec that verifies that any queued message is sent. That will make the specs much easier to implement, and the specs will better reflect the behavior of the application.
I have a question regarding the behavior of a jdbc message store. I notice that if a failure occurs for instance if the following sequence is followed:
1)inbound-channel-adapter - Creates a message payload and sends it to a channel
2)The channel is polled for the payload by a jms:outbound-channel-adapter which adds an entry in the JMS queue. I am using ActiveMQ.
When I test for a situation like shutting down the MOM I would like to persist the message in the message-store, but this is not the default behavior. Looks like SI processes the payload in the channel and then pushes it out (deletes it from the store). I can see the message being inserted in the message-store in the database.
Am I wrong in understanding the function of a message-store? I thought that the message would persist until a successful run. Thanks for any feedback.
You need to set up durable subscriptions for that. I think most of the SI JMS components do support that.
I'm using a message listener to process some messages from MQ based on Spring's DefaultMessageListenerContainer. After I receive a message, I have to make a Web Service (WS) call. However, I don't want to do this in the onMessage method because it would block the onMessage method until the invocation of WS is successful and this introduces latency in dequeuing of messages from the queue. How can I decouple the invocation of the Web Service by calling it outside of the onMesage method or without impacting the dequeuing of messages?
Thanks,
I think you might actually want to invoke the web service from your onMessage. Why do you want to dequeue messages quickly, then delay further processing? If you do what you're saying, you'd probably have to introduce another level of queueing, or some sort of temporary "holding" collection, which is redundant. The point of the queue is to hold messages, and your message listener will pull them off and process them as quickly as possible.
If you are looking for a way to maximize throughput on the queue, you might think about making it multi-threaded, so that you have multiple threads pulling messages off the queue to invoke the web service. You can easily do this by setting the "concurrentConsumers" configuration on the DefaultMessageListenerContainer. If you set concurrentConsumers to 5, you'll have 5 threads pulling messages off the queue to process. It does get tricky if you have to maintain ordering on the messages, but there may be solutions to that problem if that's the case.
I agree with answer provided before me , however I can see a usecase similar to this very common in practice. I'm adding my two cents It might be valid in some cases that you don't want to do time consuming work in your onMessage Thread (which is pulling message from Q)
We have something similar in one workflow, where if user selects some XYZ option on GUI that means at server we need to connect to another external webservice to get ABCD in this case we do not make call to webservice in onMessage Thread and use ThreadPool to dispatch and handle that call.
If something wrong happens during webservice call we broadcast that to GUI as separate Message , there is concept of request id which is preserved across messages so that GUI can relate error messages. You can use ExecutorService implementation to submit task.
hope it helps.