I'm confused when it comes to JMS Queue/Topic. What I want is messages should go to every subscriber and I want subscribers to receive messages from inactive time when they become active. However, I don't have control over whether or not subscribers have durable subscription. Is there a way to set up a persistent Queue, and set it up so that every subscriber will receive same message? And how to set this up using spring config
Thanks much.
This is mostly a question where the design of your system affects the outcome.
You could use UI tooling to create durable subscriptions for the clients that need to but that is cumbersome and error prone. You could use something like camel or other configuration on the target broker to fanout messages from an incoming Queue to outgoing Queues that map to the consumer subscriptions.
It all depends mostly on the requirements and your overall design so a real answer is beyond the scope of a SO answer without you doing some more legwork to narrow the scope a bit. JMS itself does not define any answer for this so it will come down a bit to the broker you've chosen and possibly other third party tooling that you might pick to do what you need.
Related
We have a requirement to copy messages from one ActiveMQ broker to another. Here the message has to just copy and the message should exist in both broker.
I can think of a custom application that subscribes to a certain destination and read that message and re-post the messages to the destination in multiple brokers.
I do not have access to make changes in the Broker so I couldn't think of Network of Brokers option.
Is there any best practice or tools available to copy A-MQ messages from one broker to another?
Without having access to the target broker, as far as I know and I have read, I believe there is not shortcut to avoid the custom application that re-post those messages.
However, depending on the messages you want to re-post, there might be some functionalities offered by ActiveMQ that could facilitate your implementation (but they would not be for free, regarding the computational costs).
For example, in the case you want to copy ALL the messages sent through that broker to the other, then you might consider using Mirrored Queues, with a specific prefix (e.g. "copy"), that would allow you to just have a single consumer using a wildcard after that prefix (e.g. "copy.>"). That consumer would get ALL the messages sent to the broker, and it would simplify your implementation since you would just have to care about that single consumer and re-post from it. However this has costs, since as it is described in the documentation, enabling the mirrored queues will make a duplicate of each queue/topic in the system, and will post each message twice. You need to consider if this is an important inconvenient in your case, depending on the amount of messages and the available memory that your broker disposes.
In case you just wanted to copy SOME of the messages and not all, then I believe the most elegant way to handle it is by creating an abstraction of your Consumer class (or specific implementation), and use that special implementation for those queues you want to re-post. That class would be responsible of re-posting the messages to the other broker, in a way that would be transparent from the other Consumer class when using it.
I have talked above about consumers, but the same concept could apply to topics and subscribers. Hope these ideas help :)
Is there any way you can control the order delivery of messages in a topic in EMS or Tibco (using a JMS Topic Subscriber)? Something like the message selector, but instead of filtering to do ordering.
I would like to use a header like JMSXDeliveryCount, so that new messages will get a higher priority. I know that there's RedeliveryDelay, but that works only for queues, not topics.
Even JMSPriority could be an option if I can set it after getting the message with a topic subscriber. Can I do that? Maybe with a Java Code activity?
The broker will deliver messages in order as it receives them, but re-ordering messages is a distributed computing / EIP problem (see sequencer). The issue is the broker does't know how many messages are coming or how long to keep the window open for re-ordering messages. This activity is best done outside the broker in your application where that information is understood.
EIP frameworks and ESB's have done a nice job of standardizing a handler for this type of workflow.
ref: EIP Sequencer http://www.enterpriseintegrationpatterns.com/patterns/messaging/MessageSequence.html
JMSPriority is indeed an option, but it cannot be set on an already received message. The solution is to confirm the message and republished it with a different priority.
I am still learning about this activemq and jms stuff.
I already tried some example and now I can produce and consuming message from the queue/topic.
Now I have a problem, when my client/consumer lost the connection, the message in queue/topic still send out that message, that message become lost and not kept in the queue/topic. So my question is how I can keep that failed message and how to make the broker resend that message again?
thanks
You are mixing up terminology a bit.
Queues will hold messages until consumed or the broker is restarted, unless the message has been marked as persistent, in which case they will stick around even after a broker restart.
Topics only deliver the current message to any current subscriber. However there are several methods you can use to persist messages published to a topic:
Durable subscribers.
Virtual Destinations .
Virtual Topics tend to be popular for many reasons over durable subscribers, but it really depends on the use-case.
How you create a durable subscriber depends on what you are using to create the subscriber (Spring, POJO, some other API?). All methods will at some point call the Session.createDurableSubscriber method, but I suggest reading up on how they behave before choosing this over Virtual Topic or Composite Queues.
The thing which you are looking for might be Durable subscription
You can find documentation for same at http://activemq.apache.org/how-do-durable-queues-and-topics-work.html
I'm looking at swapping out ActiveMQ with RabbitMQ for a few reasons. I currently have multiple services which are each capable of publishing events (and they publish those events to a specific VirtualTopic in AMQ). Each of the services is also capable of consuming messages from the other services. Consumers are set up such that they subscribe as a consumer to a queue on the VirtualTopic.
This buys me the ability to fan messages out to multiple queues (topic-like functionality) while keeping the benefits of queues (load balancing and persistence).
It seems like this is roughly equivalent to RabbitMQ's fanout exchange. However, the part that I found very useful in ActiveMQ is that the producer doesn't need to have any knowledge of the consumers. It simply publishes to the virtual topic. It seems that in RabbitMQ, when the exchange is created, I need a definitive of queues to publish that message to.
tl;dr
Is there any routing scheme in RabbitMQ that is equivalent to ActiveMQ's Virtual Topic, such that I can produce messages to a topic that are distributed to any queue that has been created off of that Virtual Topic, without requiring a hard-coded routing scheme somewhere in RMQ?
I realized after posting this question that it is pretty trivial to do this (not sure why I never thought of it before).
I was looking at it from the wrong direction, wondering how I could automatically have the publisher configure queues for the recipients - which isn't the right way to approach this question.
Instead, I have the subscribers, when they start up, bind themselves to the exchange that the publisher users, which provides in the inversion of control I'm looking for (publishers need not know anything about their consumers).
I need some help with topics and selectors.
I have a scenario with a topic having multiple durable subscribers (each with a selector)
not all messages going into the topic aren't read by the consumers - because of unmatching selectors.
This is correct behavior.
However the problem occurs when the unmatched messages reach a certain quantity threshold, because at that point no other messages are being delivered to the consumers
activemq tries to dispatch those old unmatchable messages, but since there is no consumer for them everything is stuck
can anybody help with this?
My setup is ActiveMq 5.5
is there some configuration option, or is it just a flawed design?
I'd say this is a flawed design given there are better alternatives and perhaps a bug in ActiveMQ.
First question: is your producer publishing to this topic setting the JMSExpiration header on those messages?
If yes, the first thing I'd do is create a Jira issue detailing the scenario you described above, because it does seem incorrect that ActiveMQ will continue hold on to and continue send messages for which no selectors apply.
As for flawed design, the minute you hear yourself saying "I need durable subscribers" and you are using ActiveMQ, you should immediately turn to using Virtual Destinations instead. Virtual Destinations have the advantages of topics in that a producer can send a message to a destination and have that message propagated to N number of other destinations for consumption, but doesn't have the disadvantages that come with having durable subscribers on a topic. Read more about Virtual Destinations here.
This is related to the way that ActiveMQ handles sparse selectors. The current implementation doesn't page into the store to look for message matching sparse selectors so you need to make some configuration changes to try and work around this. You can set the maxBrowsePageSize in the configured destination policy, the default is 400. See this page.
Virtual destinations in ActiveMQ is probably the better choice, it almost always is when thinking about using durable subscribers. You could however add some message expiration to you messages and configure the policy to expire messages on inactive durable subscribers if you use a SNAPSHOT version of ActiveMQ 5.6.
It does seem like a bug, (or dashedly inconvenient at the least) but there is a work around using Virtual Destinations.
Quote:
Note that making a topic virtual does add a small CPU overhead when
sending messages to the topic but it is fairly small. From version
5.4, dispatch from virtual topics to subscription queues can be
selectorAware such that only messages that match one of the existing
subscribers are actually dispatched. Using this option prevents the
build up of unmatched messages when selectors are used by exclusive
consumers.