Can I share one instance of ISubscriber for publishing messages into the same or different channels from different threads?
This is not documented.
Thanks.
Yes. All the key APIs in SE.Redis (the IDatabase, IServer, ISubscriber implementations) are fully thread-safe, with the expectation that they will be potentially shared and hammered from multiple threads.
Related
I would like to use the same service classes in both the publisher (which will be a REST API) and consumer. Since sending messages can be a part of these service classes, they have an instance of IBus injected into them so they can publish/send messages. This is fine on the REST API side, but the MassTransit documentation states the following:
Once you have consumers you will ALWAYS use ConsumeContext to interact with the bus, and never the IBus.
What's the reason behind this? Is it just performance related or does using IBus have any other consequences? And what are the alternatives to doing this? Would injecting IPublishEndpoint and ISendEndpointProvider be the accepted solution here, or does that not really change anything?
The reason why I want to do this is because some actions can be done either synchronously by using the API, or happen automatically in the background by using a message, and having to duplicate the business logic would be very inconvenient and hard to maintain.
Bonus question: The documentation states the same thing for TransactionalBus:
Never use the TransactionalBus or TransactionalEnlistmentBus when writing consumers. These tools are very specific and should be used only in the scenarios described.
However, if I want to support transactions in the above mentioned services, I will probably have to use TransactionalBus, but is it safe to do so in consumers? I do know about the in-memory outbox, but I have 2 problems with it:
It can only be used on the consumer side, so the publisher would not support transactions
It does not support "partial transactions" - the codebase that I'm working on has certain places where transactions don't wrap the entire API call, but rather only parts of it, so cases where some entities are successfully written to the database before the transaction is even started can happen, and in these cases the corresponding messages would need to be sent/published as well. This could easily be done by calling Release on the TransactionalBus at the right time, but couldn't be done when using the outbox since it's all or nothing (if an exception happens, nothing will be sent).
This bonus question isn't that important since I could probably work around it, but is still something I'm curious about, as it could be resolved by using TransactionalBus (if that won't cause any issues in consumers).
You should be using IPublishEndpoint or ISendEndpointProvider to publish or send messages from your components and/or services. There is almost never a reason to use IBus.
IPublishEndpoint and ISendEndpointProvider are registered a scoped, so a valid scope is required. In a service that normally doesn't have a scope, one can easily be created using provider.CreateScope(). Scopes should also be disposed of when they are no longer used.
Note that current versions should use provider.CreateAsyncScope() instead, and to make it easy just assign it using:
await using var scope = provider.CreateAsyncScope()
var publishEndpoint = scope.ServiceProvider.GetService<IPublishEndpoint>();
For any components, consumers, etc. simply use constructor injection for either of those two types, and they will resolve the proper services depending upon the context.
Also, don't use ITransactionBus. The new outbox is a better solution, as it's actually in the transaction. I will eventually remove ITransactionBus from MassTransit.
We have multiple application environments (development, QA, UAT, etc) that need to connect to fewer provider environments through MQ. For example, the provider only has one test (we'll call it TEST1) environment to which all of the client application environments need to interact. It is imperative that each client environment only receives MQ responses to the messages sent by that respective environment. This is a high volume scenario so correlating message IDs has been ruled out.
Right now TEST1 has a queue set up and is functional, but if one of the client app's environments wants to use it the others have to be shut off so that messaging doesn't overlap.
Does MQ support a model having multiple clients connect to a single queue while preserving the client-specific messaging? If so, where is that controlled (i.e. the channel, queue manager, etc)? If not, is the only solution to set up additional queues for each corresponding client?
Over the many years I have worked with IBM MQ, I have gone back and forth on this issue. I've come to the conclusion that sharing a queue just makes life more difficult. Queues should be handed out like candy on Halloween. If an application team says that they have 10 components to their application then the MQAdmin should give them 10 queues. To the queue manager or server or CPU or hard disk, there is no difference in resource usage.
Also, use an MQ naming standard that makes sense and is easy to apply security to. i.e. for HR (Human Resource) department
HR.PAYROLL.SALARY
HR.PAYROLL.DEDUCTIONS
HR.PAYROLL.BENEFITS
HR.EMPLOYEE.DETAILS
HR.EMPLOYEE.REVIEWS
etc...
You could use a selector such as MQGET(where applname="myapp") or based on a specific user-defined property assuming the sender populates such a property but that's likely to be worse performance than any retrieval by msgid or correlid. Though you've not given any information to demonstrate that get-by-correlid is actually problematic.
And of course any difference between a test and production environment - whether it involves code or configuration - is going to be very risky.
You would not normally share a single destination queue between multiple different application types - multiple queues is far more standard.
I'm trying to use the InMemoryBus transport in unit tests with 2 bus instances (same process). One of the bus instances should publish an event and the other one consume it.
What I need, I think, is the Shared Transport feature documented here http://masstransit-project.com/MassTransit/usage/transports.html but it seems no longer available in version 5.2.1. Is there another way to achieve the same result?
Multiple bus instances with a shared in-memory transport are no longer supported. You can use a single bus instance with multiple receive endpoints if you need to test multiple consumers in the same process.
I'm guessing that you're upgrading from a much older version of MT, since that ability was removed quite some time ago.
I have an environment where I have only one app server. I have some messages that take awhile to service (like 10 seconds or so) and I'd like to increase throughput by configuring multiple instances of my consumer application running code to process these messages. I've read about the "competing consumer" pattern and gather that this should be avoided when using MassTransit. According to the MassTransit docs here, each receive endpoint should have a unique queue name. I'm struggling to understand how to map this recommendation to my environment. Is it possible to have N instances of consumers running that each receive the same message, but only one of the instances will actually act on it? In other words, can we implement the "competing consumer" pattern but across multiple queues instead of one?
Or am I looking at this wrong? Do I really need to look into the "Send" method as opposed to "Publish"? The downside with "Send" is that it requires the sender to have direct knowledge of the existence of an endpoint, and I want to be dynamic with the number of consumers/endpoints I have. Is there anything built in to MassTransit that could help with the keeping track of how many consumer instances/queues/endpoints there are that can service a particular message type?
Thanks,
Andy
so the "avoid competing consumers" guidance was from when MSMQ was the primary transport. MSMQ would fall over if multiple threads where reading from the queue.
If you are using RabbitMQ, then competing consumers work brilliantly. Competing consumers is the right answer. Each competing consume will use the same receive from endpoint.
Suppose you have a system on the other side of a network that sends events and data that needs to be cached to some intermediate broker.
Instead of giving every component of your application that needs to be informed of such events a new subscription to the broker, I decide for performance and simplicity (the third party library that handles broker subscriptions isnt pretty) I should have only one Event Processor that subscribes to the broker and programatically fires events as it receives them to subscribed listeners provided by the components. The cached data can also be shared from this singleton. This will greatly reduce network connections.
However according to most discussions about singletons, they are always evil PERIOD unless for concurrency reasons or hardware reasons you need only one access point. This is not my situation since every component could have their own subscription and their own personal cache of data since all the data can be requested over the broker. However this could easily add 200 more network connections.
Because singletons are evil does that mean 200 more connections to a broker with 200 copies of data is better than using singleton I don't need to use? After all this slows things down quite a bit but its not game breaking, the application is still usable.
There's nothing inherently wrong with your broker client object servicing multiple clients within your process.
All the talk about singletons being evil is really about global variables being evil. A singleton becomes evil because it provides a static access point to mutable state, not because there is only one instance of it.
In that light, you might want to use dependency injection to hook it up rather than calling Broker.getInstance(). This avoids client code making the assumption that it is in fact a singleton.