In direct exchange of RabbitMQ, Does it convey the message to all the consumers which are connected to that particular queue? or only for one consumer though there are many consumers?
I followed [https://www.baeldung.com/spring-amqp-reactive]. There it says " Multiple clients can subscribe to the same queue, but only one will receive a given message".But in another tutorial [https://medium.freecodecamp.org/rabbitmq-9e8f78194993] it says some other thing.
The type of exchange does not affect how messages are delivered to queue consumers.
If multiple consumers are subscribed to the same queue, each message will only be delivered to one of the consumers.
I suggest reading the RabbitMQ documentation, especially the tutorials, where these concepts are clarified.
The medium.com link is broken.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
Related
I am familiar with JMS and novice with Google Pub/Sub.
In JMS there are 2 options:
Queue: only one consumer can accept message.
Topic: each consumer accepts each message from the topic
I believe that Google Pub/Sub should support something like this, but a quick Googling didn't help me to answer that question.
Please point me out to the corresponding documentation part.
As the name "Pub/Sub" indicates, Google Pub/Sub supports publish/subscribes semantics which correspond to JMS topics. It doesn't support point-to-point semantics which correspond to JMS queues, at least not directly.
You can see an overview of the semantics in the documentation. The "Publisher-subscriber relationships" section may be helpful. To be clear, this documentation does use the word queue in two places:
In the "Pub/Sub message flow" section: "When a message is acknowledged by the subscriber, it is removed from the subscription's message queue."
In the "Common use cases" section: "For example, a large queue of tasks can be efficiently distributed among multiple workers, such as Google Compute Engine instances."
The term queue here is being used to refer to the actual subscription on the topic (i.e. where the messages are placed for subscribers to consume). Furthermore, the architectural overview includes this diagram:
This diagram demonstrates how multiple subscribers can receive messages from the same subscription (e.g. for balancing workloads). This would be akin to the "shared subscription" functionality added for topics in JMS 2.
I was looking for the answer to this question, and I've found this documentation that describe the behaviour of a queue rather than a topic :
https://cloud.google.com/pubsub/docs/subscriber
While a message is outstanding to a subscriber, however, Pub/Sub tries
not to deliver it to any other subscriber on the same subscription.
So my understanding is that if you want
a topic behaviour (one to many): you create many subscriptions, each will get a copy of the message
a queue behaviour : you create one subscription
Then on each subscription you can run one or more subscribers for load balancing, each message is delivered to only one subscriber
I didn't see what kind of distribution among subscribers of a subscription. likely to be round robin.
If a publisher publish some messages to a WMQ topic, but the subsciber didn't take it, then where the messages are saved? is there any way to know the message count?
As MQ is JMS compliant, the answer is mostly a JMS answer.
If the subscription is not durable and no subscription is registered, the messages for that subscriber are discarded.
If the subscription is durable, MQ creates a queue (or uses a predefined one if specified by the subscriber) to deliver the messages. The messages will collect there if the subscriber is not consuming them.
The 3rd case as Dave points out int he comments is that the non-durable subscriber is holding the subscription open but not consuming the messages. Since a queue is created to receive these that queue depth can be queried to determine if there's a back-up.
Based on there being a queue for every subscription (durable or otherwise) just look in the durable subscriber's queue to determine the number of messages outstanding.
Please also see Publish/subscribe lifecycles in the MQ Knowledge Center for more description of the behavior and specification of durable subscriber queues.
Of course, if that queue fills up the behavior changes. Depending on the settings either the publishers block or the publications continue but the messages are routed to an exception queue (if specified), the DLQ, or discarded.
Thanks Dave Ware for the comments about non-durable subscriptions.
I'm wondering from the question if you're asking if MQ keeps a store of all the messages published to a topic, independent of any registered subscriptions?
If that's the question, then no, it doesn't. When messages are published they are matched to each existing subscription and a copy is sent to each of their associated queues as T.Rob describes.
So the only queue depths to worry about are those of the subscriptions.
(There is a caveat in that MQ supports "retained publications", - it means MQ keeps just the most recent publication on that topic string for late subscriptions if you choose to do that).
I try to explain all this here (slides/video), which may help... http://www.slideshare.net/DavidWare1/ame-2271-mq-publish-subscribe-pdf
I am still learning about this activemq and jms stuff.
I already tried some example and now I can produce and consuming message from the queue/topic.
Now I have a problem, when my client/consumer lost the connection, the message in queue/topic still send out that message, that message become lost and not kept in the queue/topic. So my question is how I can keep that failed message and how to make the broker resend that message again?
thanks
You are mixing up terminology a bit.
Queues will hold messages until consumed or the broker is restarted, unless the message has been marked as persistent, in which case they will stick around even after a broker restart.
Topics only deliver the current message to any current subscriber. However there are several methods you can use to persist messages published to a topic:
Durable subscribers.
Virtual Destinations .
Virtual Topics tend to be popular for many reasons over durable subscribers, but it really depends on the use-case.
How you create a durable subscriber depends on what you are using to create the subscriber (Spring, POJO, some other API?). All methods will at some point call the Session.createDurableSubscriber method, but I suggest reading up on how they behave before choosing this over Virtual Topic or Composite Queues.
The thing which you are looking for might be Durable subscription
You can find documentation for same at http://activemq.apache.org/how-do-durable-queues-and-topics-work.html
I'm looking at swapping out ActiveMQ with RabbitMQ for a few reasons. I currently have multiple services which are each capable of publishing events (and they publish those events to a specific VirtualTopic in AMQ). Each of the services is also capable of consuming messages from the other services. Consumers are set up such that they subscribe as a consumer to a queue on the VirtualTopic.
This buys me the ability to fan messages out to multiple queues (topic-like functionality) while keeping the benefits of queues (load balancing and persistence).
It seems like this is roughly equivalent to RabbitMQ's fanout exchange. However, the part that I found very useful in ActiveMQ is that the producer doesn't need to have any knowledge of the consumers. It simply publishes to the virtual topic. It seems that in RabbitMQ, when the exchange is created, I need a definitive of queues to publish that message to.
tl;dr
Is there any routing scheme in RabbitMQ that is equivalent to ActiveMQ's Virtual Topic, such that I can produce messages to a topic that are distributed to any queue that has been created off of that Virtual Topic, without requiring a hard-coded routing scheme somewhere in RMQ?
I realized after posting this question that it is pretty trivial to do this (not sure why I never thought of it before).
I was looking at it from the wrong direction, wondering how I could automatically have the publisher configure queues for the recipients - which isn't the right way to approach this question.
Instead, I have the subscribers, when they start up, bind themselves to the exchange that the publisher users, which provides in the inversion of control I'm looking for (publishers need not know anything about their consumers).
Ref: Official GlassFish 4.0 docs/javaee-tutorial Java EE 7
Firstly, let us start with the destination-type of: topic.
As per GlassFish 4.0 tutorial, section “46.4 Writing High Performance and Scalable JMS Applications”:
This section describes how to use the JMS API to write applications
that can handle high volumes of messages robustly.
In the subsection “46.4.2 Using Shared Durable Subscriptions”:
The SharedDurableSubscriberExample.java client shows how to use shared
durable subscriptions. It shows how shared durable subscriptions
combine the advantages of durable subscriptions (the subscription
remains active when the client is not) with those of shared consumers
(the message load can be divided among multiple clients).
When we run this example as per “46.4.2.1 To Run the ShareDurableSubscriberExample and Producer Clients”, it gives us the same effect/functionality as previous example on destination-type of queue: if we follow “46.2.6.2 To Run the AsynchConsumer and Producer Clients”, points 5 onwards – and modify it slightly using 2 consumer terminal-windows and 1 producer terminal-window.
Yes, section “45.2.2.2 Publish/Subscribe Messaging Style” does mention:
The JMS API relaxes this requirement to some extent by allowing
applications to create durable subscriptions, which receive messages
sent while the consumers are not active. Durable subscriptions provide
the flexibility and reliability of queues but still allow clients to
send messages to many recipients.
.. and anyway section “46.4 Writing High Performance and Scalable ..” examples are queue style – one message per consumer:
Each message added to the topic subscription is received by only one
consumer, similarly to the way in which each message added to a queue
is received by only one consumer.
What is the precise technical answer for: why, in this example, the use of Shared-Durable-Consumer on Topic is supposed to be, and mentioned under, “High Performance and Scalable JMS Application” vs. use of Asynchronous-Consumer on Queue?
I was wonderign about the same issue, so I found out the following link. I understand that John Ament gave you the right reponse, maybe it was just too short to get a full understand.
Basically, when you create a topic you are assuming that only the subscribed consumers will receive its messages. However processing such a message may requires a heavy processing; in such a cases you can create a shared topic using as much threads as you want.
Why not use a queue? The answer is quite simple, if you use a queue only one consumer will be able to handle such a message.
In order to clarify I will give you an example. Let's say a federal court publishes thousand of sentences every day and you have three distinct applications that depends on it.
Application A just copy the sentences to a database.
Application B parse the sentence and try to find out all relation between people around all previously saved sentences.
Application C parse the sentence and try to find out all relation between companies around all previously saved sentences.
You could use a Topic for the sentences, where Application A, B and C would be subscribed. However it easy to see that Application A can process the message very quicly while Application B and C may take some time. An available solution would consist of create a shared subscription for application B and another one to application C, so multiple threads could act on each of them simultaneouly...
...Of course there are other solutions, you could for example use a unshared topic (i.e. a regular one) and post all received messages on a ArrayBlockingQueue that would be handled by a pool of threads some time later; howecer in such a decision the developer would be the one to worry about queue handling.
Hope this can help.
The idea is that you can have multiple readers on a subscription. This allows you to read more messages faster, assuming you have threads available.
JMS Queue :
queued messages are persisted
each message is guaranteed to be delivered once-and-only-once, even no consumer running when the messages are sent.
JMS Shared Subscription :
subscription could have zero to many consumers
if messages sent when there is no subscriber (durable or not), message will never be received.