By default, it appears MassTransit is using transient exchanges. I see there is a durable switch in both the DeclareExchange and BindQueue methods here:
https://github.com/phatboyg/MassTransit/blob/217d51b9d6ea19d5b01904dcceb7bacd399ace67/src/Transports/MassTransit.Transports.RabbitMq/RabbitMqConnection.cs
I hoping this would be somewhat easy... Similar to EasyNetQ, where you can access the advanced bus properties, toggle the durable flag, and you are done...
I am not, however, finding where you can actually access the RabbitMqConnection object in the bus configurations, and call these methods.
Anyone happen to have an example of how to use the DeclareExchange method?
Update: The default behavior is in fact durable. The issue was due to an unsubscribe.
All exchanges and queues are durable by default, including any bindings between exchanges.
The only non-durable exchanges are those that are bound to publish events that are private classes, those are declared as temporary to avoid cluttering the virtual host with a bunch of internal/private class names and bindings once the connection is closed.
The only other way is if the address URI includes:
?temporary=true
In which case, it will declare it as temporary. This is often used when a temporary queue/exchange/binding is required for a transient consumer that goes away once the process is exited.
Related
I would like to use the same service classes in both the publisher (which will be a REST API) and consumer. Since sending messages can be a part of these service classes, they have an instance of IBus injected into them so they can publish/send messages. This is fine on the REST API side, but the MassTransit documentation states the following:
Once you have consumers you will ALWAYS use ConsumeContext to interact with the bus, and never the IBus.
What's the reason behind this? Is it just performance related or does using IBus have any other consequences? And what are the alternatives to doing this? Would injecting IPublishEndpoint and ISendEndpointProvider be the accepted solution here, or does that not really change anything?
The reason why I want to do this is because some actions can be done either synchronously by using the API, or happen automatically in the background by using a message, and having to duplicate the business logic would be very inconvenient and hard to maintain.
Bonus question: The documentation states the same thing for TransactionalBus:
Never use the TransactionalBus or TransactionalEnlistmentBus when writing consumers. These tools are very specific and should be used only in the scenarios described.
However, if I want to support transactions in the above mentioned services, I will probably have to use TransactionalBus, but is it safe to do so in consumers? I do know about the in-memory outbox, but I have 2 problems with it:
It can only be used on the consumer side, so the publisher would not support transactions
It does not support "partial transactions" - the codebase that I'm working on has certain places where transactions don't wrap the entire API call, but rather only parts of it, so cases where some entities are successfully written to the database before the transaction is even started can happen, and in these cases the corresponding messages would need to be sent/published as well. This could easily be done by calling Release on the TransactionalBus at the right time, but couldn't be done when using the outbox since it's all or nothing (if an exception happens, nothing will be sent).
This bonus question isn't that important since I could probably work around it, but is still something I'm curious about, as it could be resolved by using TransactionalBus (if that won't cause any issues in consumers).
You should be using IPublishEndpoint or ISendEndpointProvider to publish or send messages from your components and/or services. There is almost never a reason to use IBus.
IPublishEndpoint and ISendEndpointProvider are registered a scoped, so a valid scope is required. In a service that normally doesn't have a scope, one can easily be created using provider.CreateScope(). Scopes should also be disposed of when they are no longer used.
Note that current versions should use provider.CreateAsyncScope() instead, and to make it easy just assign it using:
await using var scope = provider.CreateAsyncScope()
var publishEndpoint = scope.ServiceProvider.GetService<IPublishEndpoint>();
For any components, consumers, etc. simply use constructor injection for either of those two types, and they will resolve the proper services depending upon the context.
Also, don't use ITransactionBus. The new outbox is a better solution, as it's actually in the transaction. I will eventually remove ITransactionBus from MassTransit.
We have a requirement to copy messages from one ActiveMQ broker to another. Here the message has to just copy and the message should exist in both broker.
I can think of a custom application that subscribes to a certain destination and read that message and re-post the messages to the destination in multiple brokers.
I do not have access to make changes in the Broker so I couldn't think of Network of Brokers option.
Is there any best practice or tools available to copy A-MQ messages from one broker to another?
Without having access to the target broker, as far as I know and I have read, I believe there is not shortcut to avoid the custom application that re-post those messages.
However, depending on the messages you want to re-post, there might be some functionalities offered by ActiveMQ that could facilitate your implementation (but they would not be for free, regarding the computational costs).
For example, in the case you want to copy ALL the messages sent through that broker to the other, then you might consider using Mirrored Queues, with a specific prefix (e.g. "copy"), that would allow you to just have a single consumer using a wildcard after that prefix (e.g. "copy.>"). That consumer would get ALL the messages sent to the broker, and it would simplify your implementation since you would just have to care about that single consumer and re-post from it. However this has costs, since as it is described in the documentation, enabling the mirrored queues will make a duplicate of each queue/topic in the system, and will post each message twice. You need to consider if this is an important inconvenient in your case, depending on the amount of messages and the available memory that your broker disposes.
In case you just wanted to copy SOME of the messages and not all, then I believe the most elegant way to handle it is by creating an abstraction of your Consumer class (or specific implementation), and use that special implementation for those queues you want to re-post. That class would be responsible of re-posting the messages to the other broker, in a way that would be transparent from the other Consumer class when using it.
I have talked above about consumers, but the same concept could apply to topics and subscribers. Hope these ideas help :)
I have an ActiveMQ broker (version 5.14.3) running, which is being used by producers and consumers of different origins.
For some reason recently a lot of empty queues have been automatically created labeled either communication_{SOME_HASH} or communication_{NAME_OF_ORIGIN_MACHINE}_{SOME_PROCESS_ID}. These queues are all empty queues without any enqueued or dequeued messages, which I assume are useless but are still generated for some reason.
Has somebody come across the same issue and can either explain why these queues are being generated or point out how I can avoid this behavior?
These are not prefixes for any type of temporary or advisory based destinations that ActiveMQ would ever create so the fault must lay in your own application. You would need to debug this from your own side to find out where they are coming from. Since ActiveMQ creates destinations whenever a Producer or Consumer is created on a Topic or Queue unless security policies prevent that your applications may be creating these resources accidentally an never using them.
Suppose you have a system on the other side of a network that sends events and data that needs to be cached to some intermediate broker.
Instead of giving every component of your application that needs to be informed of such events a new subscription to the broker, I decide for performance and simplicity (the third party library that handles broker subscriptions isnt pretty) I should have only one Event Processor that subscribes to the broker and programatically fires events as it receives them to subscribed listeners provided by the components. The cached data can also be shared from this singleton. This will greatly reduce network connections.
However according to most discussions about singletons, they are always evil PERIOD unless for concurrency reasons or hardware reasons you need only one access point. This is not my situation since every component could have their own subscription and their own personal cache of data since all the data can be requested over the broker. However this could easily add 200 more network connections.
Because singletons are evil does that mean 200 more connections to a broker with 200 copies of data is better than using singleton I don't need to use? After all this slows things down quite a bit but its not game breaking, the application is still usable.
There's nothing inherently wrong with your broker client object servicing multiple clients within your process.
All the talk about singletons being evil is really about global variables being evil. A singleton becomes evil because it provides a static access point to mutable state, not because there is only one instance of it.
In that light, you might want to use dependency injection to hook it up rather than calling Broker.getInstance(). This avoids client code making the assumption that it is in fact a singleton.
Messaging middleware solutions (JMS, Tibco, etc.) allow publish/subscribe with "topic" filtering using wildcards to subscribe to all messages of a certain "topic", e.g. SUBSCRIBE("ACCOUNT.*") topic allows you to subscribe to both "ACCOUNT.WITHDRAW" message and "ACCOUNT.CHECKBALANCE" message.
The problem is that such subscription also receives my own published messages.
I'm looking for a mechanism, similar to, say, UDP multicast loopback which can be turned ON or OFF by the transport layer without messing with the data being sent.
Is there a common, declarative (no custom code, configuration only) way to configure the middleware not to receive messages which that very same service instance has published? Ideally, this should also be able to filter out everything published by ALL servers (nodes) of the same "kind".
Thanks in advance.
The JMS API contains this option for TopicSubscribers, e.g. TIBCO EMS let's you create a consumer with the "noLocal" property. That means no messages published over the same connection, get consumed by clients on the same connection.
e.g. take a look here how to create a topic subscriber with the "noLocal" option:
https://docs.tibco.com/pub/enterprise_message_service/7.0.1-march-2013/doc/html/tib_ems_api_reference/api/javadoc/javax/jms/TopicSession.html
No one is answering, so I'll chime in (in a hand-wavey way).
I believe there's nothing in the JMS spec around controlling whether you get your own sent messages back on a topic receiver. So any capability like this would be a non-portable vendor feature. Especially for your second requirement (based on "kind" of JMS client versus some control based on the same connection doing the sending/receiving).
If you've got no flexibility to modify code or message content (properties), I think you've got no portable solutions. And likely no solution at all for that second "kind" requirement.
If you want to investigate vendor-specific options, you'll need to tell us which vendor you're interested in. You may get nothing, but there's no way to know without asking.