I am familiar with JMS and novice with Google Pub/Sub.
In JMS there are 2 options:
Queue: only one consumer can accept message.
Topic: each consumer accepts each message from the topic
I believe that Google Pub/Sub should support something like this, but a quick Googling didn't help me to answer that question.
Please point me out to the corresponding documentation part.
As the name "Pub/Sub" indicates, Google Pub/Sub supports publish/subscribes semantics which correspond to JMS topics. It doesn't support point-to-point semantics which correspond to JMS queues, at least not directly.
You can see an overview of the semantics in the documentation. The "Publisher-subscriber relationships" section may be helpful. To be clear, this documentation does use the word queue in two places:
In the "Pub/Sub message flow" section: "When a message is acknowledged by the subscriber, it is removed from the subscription's message queue."
In the "Common use cases" section: "For example, a large queue of tasks can be efficiently distributed among multiple workers, such as Google Compute Engine instances."
The term queue here is being used to refer to the actual subscription on the topic (i.e. where the messages are placed for subscribers to consume). Furthermore, the architectural overview includes this diagram:
This diagram demonstrates how multiple subscribers can receive messages from the same subscription (e.g. for balancing workloads). This would be akin to the "shared subscription" functionality added for topics in JMS 2.
I was looking for the answer to this question, and I've found this documentation that describe the behaviour of a queue rather than a topic :
https://cloud.google.com/pubsub/docs/subscriber
While a message is outstanding to a subscriber, however, Pub/Sub tries
not to deliver it to any other subscriber on the same subscription.
So my understanding is that if you want
a topic behaviour (one to many): you create many subscriptions, each will get a copy of the message
a queue behaviour : you create one subscription
Then on each subscription you can run one or more subscribers for load balancing, each message is delivered to only one subscriber
I didn't see what kind of distribution among subscribers of a subscription. likely to be round robin.
Related
Here it states "the sender and the receiver do not have to be available at the same time in order to communicate.". And here it states that in pub/sub domain "A client that subscribes to a topic can consume only messages published after the client has created a subscription, and the subscriber must continue to be active in order for it to consume messages.". To me the italicized statement seems to contradict the first statement("the sender and the receiver do not have to be available at the same time").
If the subscriber must continue to be active to consume messages, it means the sender and the receiver must be available at the same time at least in the pub/sub domain. If they must be available, the pub/sub domain is only as good as RMI. Is this true?
...the sender and the receiver do not have to be available at the same time in order to communicate.
As far as I can tell this is a general statement about messaging and not a nuanced explanation of the semantics the JMS API provides. Notice it is under the "What Is Messaging?" heading before specific discussion of the JMS API begins.
For what it's worth, the JMS API does provide these semantics if you're using the point-to-point style of messaging (also discussed in the tutorial). It also provides a variation of these semantics using the pub-sub style of messaging, but I'll get to that later.
A client that subscribes to a topic can consume only messages published after the client has created a subscription, and the subscriber must continue to be active in order for it to consume messages.
If you read the next sentence after that you'll find some important additional details:
The JMS API relaxes this timing dependency to some extent by allowing subscribers to create durable subscriptions, which receive messages sent while the subscribers are not active.
So, as I mentioned previously, you can get a variation of the inactive sender/receiver semantics using the pub-sub style of messaging via durable subscriptions.
Keep in mind that the document you're referencing is just a tutorial. It's not the JMS specification. I doubt the wording of the tutorial was subjected to the same scrutiny as the specification so you are more likely to find ambiguous statements.
In direct exchange of RabbitMQ, Does it convey the message to all the consumers which are connected to that particular queue? or only for one consumer though there are many consumers?
I followed [https://www.baeldung.com/spring-amqp-reactive]. There it says " Multiple clients can subscribe to the same queue, but only one will receive a given message".But in another tutorial [https://medium.freecodecamp.org/rabbitmq-9e8f78194993] it says some other thing.
The type of exchange does not affect how messages are delivered to queue consumers.
If multiple consumers are subscribed to the same queue, each message will only be delivered to one of the consumers.
I suggest reading the RabbitMQ documentation, especially the tutorials, where these concepts are clarified.
The medium.com link is broken.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
If a publisher publish some messages to a WMQ topic, but the subsciber didn't take it, then where the messages are saved? is there any way to know the message count?
As MQ is JMS compliant, the answer is mostly a JMS answer.
If the subscription is not durable and no subscription is registered, the messages for that subscriber are discarded.
If the subscription is durable, MQ creates a queue (or uses a predefined one if specified by the subscriber) to deliver the messages. The messages will collect there if the subscriber is not consuming them.
The 3rd case as Dave points out int he comments is that the non-durable subscriber is holding the subscription open but not consuming the messages. Since a queue is created to receive these that queue depth can be queried to determine if there's a back-up.
Based on there being a queue for every subscription (durable or otherwise) just look in the durable subscriber's queue to determine the number of messages outstanding.
Please also see Publish/subscribe lifecycles in the MQ Knowledge Center for more description of the behavior and specification of durable subscriber queues.
Of course, if that queue fills up the behavior changes. Depending on the settings either the publishers block or the publications continue but the messages are routed to an exception queue (if specified), the DLQ, or discarded.
Thanks Dave Ware for the comments about non-durable subscriptions.
I'm wondering from the question if you're asking if MQ keeps a store of all the messages published to a topic, independent of any registered subscriptions?
If that's the question, then no, it doesn't. When messages are published they are matched to each existing subscription and a copy is sent to each of their associated queues as T.Rob describes.
So the only queue depths to worry about are those of the subscriptions.
(There is a caveat in that MQ supports "retained publications", - it means MQ keeps just the most recent publication on that topic string for late subscriptions if you choose to do that).
I try to explain all this here (slides/video), which may help... http://www.slideshare.net/DavidWare1/ame-2271-mq-publish-subscribe-pdf
Ref: Official GlassFish 4.0 docs/javaee-tutorial Java EE 7
Firstly, let us start with the destination-type of: topic.
As per GlassFish 4.0 tutorial, section “46.4 Writing High Performance and Scalable JMS Applications”:
This section describes how to use the JMS API to write applications
that can handle high volumes of messages robustly.
In the subsection “46.4.2 Using Shared Durable Subscriptions”:
The SharedDurableSubscriberExample.java client shows how to use shared
durable subscriptions. It shows how shared durable subscriptions
combine the advantages of durable subscriptions (the subscription
remains active when the client is not) with those of shared consumers
(the message load can be divided among multiple clients).
When we run this example as per “46.4.2.1 To Run the ShareDurableSubscriberExample and Producer Clients”, it gives us the same effect/functionality as previous example on destination-type of queue: if we follow “46.2.6.2 To Run the AsynchConsumer and Producer Clients”, points 5 onwards – and modify it slightly using 2 consumer terminal-windows and 1 producer terminal-window.
Yes, section “45.2.2.2 Publish/Subscribe Messaging Style” does mention:
The JMS API relaxes this requirement to some extent by allowing
applications to create durable subscriptions, which receive messages
sent while the consumers are not active. Durable subscriptions provide
the flexibility and reliability of queues but still allow clients to
send messages to many recipients.
.. and anyway section “46.4 Writing High Performance and Scalable ..” examples are queue style – one message per consumer:
Each message added to the topic subscription is received by only one
consumer, similarly to the way in which each message added to a queue
is received by only one consumer.
What is the precise technical answer for: why, in this example, the use of Shared-Durable-Consumer on Topic is supposed to be, and mentioned under, “High Performance and Scalable JMS Application” vs. use of Asynchronous-Consumer on Queue?
I was wonderign about the same issue, so I found out the following link. I understand that John Ament gave you the right reponse, maybe it was just too short to get a full understand.
Basically, when you create a topic you are assuming that only the subscribed consumers will receive its messages. However processing such a message may requires a heavy processing; in such a cases you can create a shared topic using as much threads as you want.
Why not use a queue? The answer is quite simple, if you use a queue only one consumer will be able to handle such a message.
In order to clarify I will give you an example. Let's say a federal court publishes thousand of sentences every day and you have three distinct applications that depends on it.
Application A just copy the sentences to a database.
Application B parse the sentence and try to find out all relation between people around all previously saved sentences.
Application C parse the sentence and try to find out all relation between companies around all previously saved sentences.
You could use a Topic for the sentences, where Application A, B and C would be subscribed. However it easy to see that Application A can process the message very quicly while Application B and C may take some time. An available solution would consist of create a shared subscription for application B and another one to application C, so multiple threads could act on each of them simultaneouly...
...Of course there are other solutions, you could for example use a unshared topic (i.e. a regular one) and post all received messages on a ArrayBlockingQueue that would be handled by a pool of threads some time later; howecer in such a decision the developer would be the one to worry about queue handling.
Hope this can help.
The idea is that you can have multiple readers on a subscription. This allows you to read more messages faster, assuming you have threads available.
JMS Queue :
queued messages are persisted
each message is guaranteed to be delivered once-and-only-once, even no consumer running when the messages are sent.
JMS Shared Subscription :
subscription could have zero to many consumers
if messages sent when there is no subscriber (durable or not), message will never be received.
I need some help with topics and selectors.
I have a scenario with a topic having multiple durable subscribers (each with a selector)
not all messages going into the topic aren't read by the consumers - because of unmatching selectors.
This is correct behavior.
However the problem occurs when the unmatched messages reach a certain quantity threshold, because at that point no other messages are being delivered to the consumers
activemq tries to dispatch those old unmatchable messages, but since there is no consumer for them everything is stuck
can anybody help with this?
My setup is ActiveMq 5.5
is there some configuration option, or is it just a flawed design?
I'd say this is a flawed design given there are better alternatives and perhaps a bug in ActiveMQ.
First question: is your producer publishing to this topic setting the JMSExpiration header on those messages?
If yes, the first thing I'd do is create a Jira issue detailing the scenario you described above, because it does seem incorrect that ActiveMQ will continue hold on to and continue send messages for which no selectors apply.
As for flawed design, the minute you hear yourself saying "I need durable subscribers" and you are using ActiveMQ, you should immediately turn to using Virtual Destinations instead. Virtual Destinations have the advantages of topics in that a producer can send a message to a destination and have that message propagated to N number of other destinations for consumption, but doesn't have the disadvantages that come with having durable subscribers on a topic. Read more about Virtual Destinations here.
This is related to the way that ActiveMQ handles sparse selectors. The current implementation doesn't page into the store to look for message matching sparse selectors so you need to make some configuration changes to try and work around this. You can set the maxBrowsePageSize in the configured destination policy, the default is 400. See this page.
Virtual destinations in ActiveMQ is probably the better choice, it almost always is when thinking about using durable subscribers. You could however add some message expiration to you messages and configure the policy to expire messages on inactive durable subscribers if you use a SNAPSHOT version of ActiveMQ 5.6.
It does seem like a bug, (or dashedly inconvenient at the least) but there is a work around using Virtual Destinations.
Quote:
Note that making a topic virtual does add a small CPU overhead when
sending messages to the topic but it is fairly small. From version
5.4, dispatch from virtual topics to subscription queues can be
selectorAware such that only messages that match one of the existing
subscribers are actually dispatched. Using this option prevents the
build up of unmatched messages when selectors are used by exclusive
consumers.