For my master thesis I built a pub/sub with ActiveMQ and now need to evaluate the broker's performance with Jmeter. To do so, I need to simulate the use case I implemented. It contains multiple publishers and subscribers sending messages to several topics. By now, I do not see a possibility to define that in the JMS Sampler. I tried to simply separate the topics by "," but it does not function.
dynamicTopics/MyStaticTopic1, dynamicTopics/MyStaticTopic2
I read all information about JMS Test Plans, but none is discussion this aspect.
Is there a way to do it?
Best regards.
Related
I'm an IBM MQ novice, however have used other messaging systems in the past (Solace, RabbitMQ, BizTalk). I apologize if some of my MQ terminology is incorrect when it comes to local/remote/transmission queues etc.
I am integrating with a 3rd Party using MQ. I have no control over the use of MQ when it comes to this 3rd party. They define the set integration pattern.
I have my own local Queue Manager.
The 3rd party supports many 100s of different message types.
Typically each message sent to them will result in a response
Responses on overage will be delivered back to my QM within 0.5s (agreed SLA)
The basic model is as follows;
For all outbound messaging I publish to a single queue
All responses come to a single inbound queue on my QM
I do not believe they respect/use the ReplyToQ or ReplyToQmgr header properties
High level overview of the use case;
The MQ usage will be very active. Perhaps 500-1000 outbound messages per second
For the vast majority of these - there will be a user waiting for a response
The current design approach is to put a message onto the outbound queue and then create a subscription to the inbound queue with a JMS selector using the correlation ID.
Here is my challenge;
I am not sure if having so many concurrently subscriptions all with a unique filter on the single inbound queue will perform well. I would appreciate insights into this. I have a 2 node cluster running in a docker container. I don't have details on the spec yet.
My preference would be have messages arrive into the inbound queue and then be routed to many "function" specific local queues on the QM.
I would still use JMS selectors on the message ID in this model - however I would hope the load would be spread from a single deep queue to many shorter queues
It does not seem that I could route messages natively within MQ without an addon such as MQ Integration Broker or ESB (some other products within the WebSphere suite).
It is on this last point I could use some guidance. Can I route within my QM? If so - what options do I have. Alternatively - perhaps there are much better approaches that I have not considered?
Any guidance would be heartily appreciated!!
I have an environment where I have only one app server. I have some messages that take awhile to service (like 10 seconds or so) and I'd like to increase throughput by configuring multiple instances of my consumer application running code to process these messages. I've read about the "competing consumer" pattern and gather that this should be avoided when using MassTransit. According to the MassTransit docs here, each receive endpoint should have a unique queue name. I'm struggling to understand how to map this recommendation to my environment. Is it possible to have N instances of consumers running that each receive the same message, but only one of the instances will actually act on it? In other words, can we implement the "competing consumer" pattern but across multiple queues instead of one?
Or am I looking at this wrong? Do I really need to look into the "Send" method as opposed to "Publish"? The downside with "Send" is that it requires the sender to have direct knowledge of the existence of an endpoint, and I want to be dynamic with the number of consumers/endpoints I have. Is there anything built in to MassTransit that could help with the keeping track of how many consumer instances/queues/endpoints there are that can service a particular message type?
Thanks,
Andy
so the "avoid competing consumers" guidance was from when MSMQ was the primary transport. MSMQ would fall over if multiple threads where reading from the queue.
If you are using RabbitMQ, then competing consumers work brilliantly. Competing consumers is the right answer. Each competing consume will use the same receive from endpoint.
I am creating a hosted system where multiple customers can send messages. I am receiving thoses messages on a JMS queue.
Now, all processing is done in a similar way and I want my process to poll all incoming queues for messages and handle them. Is there a way in WSO2 ESB to subscribe to multiple queues?
If not possible, the workaround would be to create a seperate listener process for each queue and have this post the message to a central processing queue. But that seems to be a less clean solution (and I think it will scale worse than listening to multiple queues).
Any ideas on this?
If changes to activeMQ server is possible ie. if OP is able to influence the configuration to the server, something like ActiveMQ diverts could do the trick.
<divert name="prices-divert">
<address>jms.queue.ABC</address>
<forwarding-address>jms.queue.theone</forwarding-address>
<exclusive>true</exclusive>
</divert>
<divert name="prices-divert">
<address>jms.queue.xyz</address>
<forwarding-address>jms.queue.theone</forwarding-address>
<exclusive>true</exclusive>
</divert>
Basically, multiple diverts that converge the messages from multiple queues to the single queue. This method has advantage over the reading and writing to single queue-as mentioned by the OP and would in my view scale well as it is inbuilt feature.
You can define a sequence with all the required logic in it and then call it from multiple proxy services (each listening to a specific queue). Otherwise you can try something similar to this sample.
Here is my use case: I am developing a trading application and i want to send incremental stock updates (bidQty etc) to active consumers instead of the whole quote and a snapshot update to a new consumer (to start with).
Now, is it possible to override any ActiveMQ's class (implementors of Topic) to achieve this behavior? Any clues on this would be helpful .
If the same is possible in any other openSource provider, please let me know.
This is NOT a case where you simply can change the implementation of topic. You should actually avoid changing the implementation of core ActiveMQ features to solve specific business requirements. Fixing bugs and adding core messaging features is another thing.
There are multiple ways to solve your use case with regular ActiveMQ features.
Separate Sync and Update channel
I would probably divide the "sync/snapshot" channel from the "incremental update" channel.
One way is to implement the "snapshot-sync" as JMS request/reply where the consumer asks the provider for a sync, then continues to rely on incremental updates pushed via the topic.
Advisory messages and Selectors
You can also implement it all using a single topic using a mix of AdvisoryMessages and JMS Selectors.
An idea (you can do this in many ways):
Introduce two message properties: MsgType and Receiver
Mark each incremental update with MsgType=inc
Mark each snapshot with some client id of the consumer, Receiver=.
Have the producer listen to advisory messages from ActiveMQ and and fire a snapshot/sync message marked with Receiver= and MsgType=snapshot when there is a new client subscribing the stock topic.
The client subscribes with a selector of something like
MsgType='inc' OR (MsgType='snapshot' AND Receiver=<me>)
This way you can trigger snapshot syncs with specific clients as well as incremental updates for all clients.
If you start think about the dynamics you already have, you can probably think of another ten or so solutions.
Retroactive Consumers
You might have some use of a Retroactive Consumer - the example actually shows a scenario similar to yours.
I'm working on updating an existing Mule configuration and the task is to enhance it to route messages to different endpoints depending on some properties of the messages, therefore it would be nice to have some pros and cons on the two options I have at hand:
Add properties on the message, using the "message-properties-transformer" transformer which is later used by a "filtering-router" to single out the message and put it on the correct endpoint. This option allows me to use a single queue for all destinations.
Create one queue for each destination and thus instead of adding some property for later routing, I just put on on the right queue at once. I.e. this option would mean one queue per destination.
Any feedback would be welcome. Is there any "best practices" with regards to this?
I've had a great deal of success with using your first approach with a filtering-router. It reduces cohesion between your message producers and consumers. It forms a valuable abstraction, so any service can blindly drop messages within the generic "outbox".
We've come to depend on mule for filtering and routing messages so much so that we have a dedicated cluster of hardware to do only this. Using mule I was able to get far greater performance and not have to maintain connections to all queues.
The down side will be having to very carefully maintain your messaging object version globally, and having to keep a set of transformers on hands to accept and convert from different versions if you plan to upgrade only a portion of your infrastructure.
thanks, matt