I'm trying to understand the principles of HornetQ as well as core/JMS messaging using this solution.
In my experimental app, I'd like my end-user application(client) to send messages to a HornetQ which will be read by a backend app. So far this is no problem and I love HornetQ.
But now, i'd like to send an "answer" message from the backend app back to the end-user. For this, I have the condition that no other client app should be able to read the answer message (let's say it contains the current bank balance). So user A should only fetch messages for himself and the same applies to any other user.
Is this possible using HornetQ? If so, how do I have to do it?
with hornetq (or any other message system) you always send to a queue, not to a specific consumer.
ON this case you have to create a queue matching your client.
This answer here will provide you some feedback on request-response where I won't need to repeat myself after this approach:
Synchronous request-reply pattern in a Java EE container
Related
Recently I am trying to use MassTransit in our microservice ecosystem.
According to MassTransit vocabulary and from documents my understanding is :
Publish: Sends a message to 1 or many subscribers (Pub/Sub Pattern) to propagate the message.
Send: Used to send messages in fire and forget fashion like publish, but instead It is just used for one receiver. The main difference with Publish is that in Send if your destination didn't receive a message, it would return an exception.
Requests: uses request/reply pattern to just send a message and get a response in a different channel to be able to get response value from the receiver.
Now, my question is according to the Microservice concept, to follow the event-driven design, we use Publish to propagate messages(Events) to the entire ecosystem. but what is exactly the usage (use case) of Send here? Just to get an exception if the receiver doesn't exist?
My next question is that is it a good approach to use Publish, Send and Requests in a Microservices ecosystem at the same time? like publish for propagation events, Send for command (fire and forget), and Requests for getting responses from the destination.
----- Update
I also found here which Chris Patterson clear lots of things. It also helps me a lot.
Your question is not related to MassTransit. MassTransit implements well-known messaging patterns thoughtfully described on popular resources such as Enterprise Integration Patterns
As Eben wrote in his answer, the decision of what pattern to use is driven by intent. There are also technical differences in the message delivery mechanics for each pattern.
Send is for commands, you tell some other service to do something. You do not wait for a reply (fire and forget), although you might get a confirmation of the action success or failure by other means (an event, for example).
It is an implementation of the point-to-point channel, where you also can implement competing consumers to scale the processing, but those will be instances of the same service.
With MassTransit using RabbitMQ it's done by publishing messages to the endpoint exchange rather than to the message type exchange, so no other endpoints will get the message even though they can consume it.
Publish is for events. It's a broadcast type of delivery or fan-out. You might be publishing events to which no one is listening, so you don't really know who will be consuming them. You also don't expect any response.
It is an implementation of the publish-subscribe channel.
MassTransit with RabbitMQ creates exchanges for each message type published and publishes messages to those exchanges. Consumers create bindings between their endpoint exchanges and message exchanges, so each consumer service (different apps) will get those in their independent queues.
Request-response can be used for both commands that need to be confirmed, or for queries.
It is an implementation of the request-reply message pattern.
MassTransit has nice diagrams in the docs explaining the mechanics for RabbitMQ.
Those messaging patterns are frequently used in a complex distributed system in different combinations and variations.
The difference between Send and Publish has to do with intent.
As you stated, Send is for commands and Publish is for events. I worked on a large enterprise system once running on webMethods as the integration engine/service bus and only events were used. I can tell you that it was less than ideal. If the distinction had been there between commands and events it would've made a lot more sense to more people. Anyway, technically one needs a message enqueued and on that level it doesn't matter, which is why a queueing mechanism typically would not care about such semantics.
To illustrate this with a silly example: Facebook places and Event on my timeline that one of my friends is having a birthday on a particular day. I can respond directly (send a message) or I could publish a message on my timeline and hope my friend sees it. Another silly example: You send an e-mail to PersonA and CC 4 others asking "Please produce report ABC". PersonA would be expected to produce the report or arrange for it to be done. If that same e-mail went to all five people as the recipient (no CC) then who gets to do it? I know, even for Publish one could have a 1-1 recipient/topic but what if another endpoint subscribed? What would that mean?
So the sender is responsible, still configurable as subscriptions are, to determine where to Send the message to. For my own service bus I use an implementation of an IMessageRouteProvider interface. A practical example in a system I once developed was where e-mails received had to have their body converted to an image for a content store (IBM FileNet P8 if memory serves). For reasons I will not go into the systems were stopped each night at 20h00 and restarted at 6h00 in the morning. This led to a backlog of usually around 8000 e-mails that had to be converted. The conversion endpoint would process a conversion in about 2 seconds but that still takes a while to work through. In the meantime the web front-end folks could request PDF files for conversion to paged TIFF files. Now, these ended up at the end of the queue and they would have to wait hours for that to come back. The solution was to implement another conversion endpoint, with its own queue, and have the web front-end configured to send the same message type, e.g. ConvertDocumentCommand to that "priority" queue for processing. Pretty easy to do. Now, if that had been a publish how would I do that split? The same event going to 2 different endpoints under different circumstances? Well, you could have another subscription store for your system but now you'd need to maintain both. There could be another answer such as coding this logic into the send bit but that is a design choice and would require coding changes.
In my own Shuttle.Esb service bus I only have Send and Publish. For request/response both the sender and receiver have an inbox and a request would be sent (Send) to the receiver and it in turn could reply (also a Send but uses the sender's URI).
I'm new to microservices architecture and want to create a centralised notification microservice to send emails/sms to users.
My first option was to create a notification Kafka queue where all other microservices can send notifications to. The notification microservice would then listen to this queue and send messages accordingly. If the notification service was restarted or taken down, we would not lose any messages as the messages will be stored on the queue.
My second option was to add a notification message API on the notifications microservice. This would make it easier for all other microservices as they just have to call an API as opposed to integrate with the queue. The API would then internally send the message to the notification Kafka queue and send the message. The only issue here is if the API is not available or there is an error, we will lose messages.
Any recommendations on the best way to handle this?
Either works. Some concepts that might help you decide:
A service that fronts "Kafka" would be helpful to:
Hide the implementation. This gives you the flexibility to change Kafka out later for something else. Your wrapper API would only respond with a 200 once it has put the notification request on the queue. I also see giving services direct access to "your" queue similar to allowing services to directly interact with a database they don't own. If you allow direct-access to Kafka and Kafka proves to be inadequate, a change to Kafka will require all of your clients to change their code.
Enforce the notification request contract (ensure the body of the request is well-formed). If you want to make sure that all of the items put on the queue are well-formed according to contract, an API can help enforce that. That will help prevent issues later when the "notifier" service picks notifications off the queue to send.
Adding a wrapper API would be less desirable if:
You don't want to/can't spend the time. Maybe deadlines are driving you to hurry and the days it would take to stand up a wrapper is just too much.
You are a small team and you don't have the resources/tools/time for service-explosion.
Your first design is simple and will work. If you're looking for the advantages I outlined, then consider your second design. And, to make sure I understand it, I would see it unfold like:
Client 1 needs to put out a notification and calls Service A POST /notifications
Service A that accepts POST /notifications
Service A checks the request, puts it on Kafka, responds to client with 200
Service B picks up notification request from Kafka queue.
Service A should be run as multiple instances for reliability.
I have list of emails (to many). I want to write scheduler, which sends emails periodically.
I read emails from database
I send messages to this emails.
As I see for good performance, it is good to use JMS (Topic) to make this.
In documentation I read that Topic sends messages to all clients. Could you enplane me what does "client" means at this case? In my opinion, wich my example, they are the owner of the emails and my system will send the message text to this owner of email (Clients). Is it right?
No, in this context, "all clients" means all java processes that have an open subscription to the topic.
You would need to write code to convert from JMS to Email (and send it). Frameworks like Spring Integration can be used for this, it does all the heavy lifting for you; you would simply wire a JMS message-driven-channel-adapter to receive the message from a queue (not a topic), do a JDBC query to get the emails, then send them via a mail outbound-channel-adapter.
Read the project documentation for more information (there's a link to it from the project page link above).
I am relatively new to JMS and I have been reading a lot on it lately.
I am planning to design a web app which would do the following:
User logs into the system and publishes a message/question to a topic.
All the users who have subscribed to the topic read the message/question and reply to it.
The originator reviews all the answers and picks the best answer.
The originator now replies to only the user whose answer he/she picked and asks for further clarification.
The responder gets the message and replies.
So, once the originator has picked the answer, the JMS now becomes a request/reply design.
My questions are:
Is it possible to publish to a topic with setJmsReplyTo(tempQueue)?
Can request/reply approach be async?
Is it a good idea to have per user queue?
These questions might some dumb to some of the experts here, but please bare in mind that I am still learning.
Thanks.
Is it possible to publish to a topic with setJmsReplyTo(tempQueue)?
You should be able but I'm not 100% sure about it. By the way, I searched in my bookmarks and found this link that should explain what you have to do to build up a Request/Response system using JMS
http://activemq.apache.org/how-should-i-implement-request-response-with-jms.html
Can request/reply approach be async?
A message listener is an object that acts as an asynchronous event handler for messages. So you approach about request/reply, if using JMS, is by default async.
http://docs.oracle.com/javaee/1.3/jms/tutorial/1_3_1-fcs/doc/prog_model.html#1023398
Is it a good idea to have per user queue?
I don't know how many user you expect to have but having one queue for each user is not a good way to handle the messages. I had a problem similar to yours but we used a single queue for each of the macro area and we structured the message to hold the information of the user that sent it in order to store the information later and use it to further analysis.
JMSReplyTo is just a message header, nothing else. So It is possible to publish a message withing a topic with specific value in this header.
Sure! If you would like to create a scalable system you should design event driven system using async instead of blocking aproach. MessageListener can help you.
It is specific to JMS broker implementation. If queue creation is quite cheap there is no problems with such a solution.
Messaging middleware solutions (JMS, Tibco, etc.) allow publish/subscribe with "topic" filtering using wildcards to subscribe to all messages of a certain "topic", e.g. SUBSCRIBE("ACCOUNT.*") topic allows you to subscribe to both "ACCOUNT.WITHDRAW" message and "ACCOUNT.CHECKBALANCE" message.
The problem is that such subscription also receives my own published messages.
I'm looking for a mechanism, similar to, say, UDP multicast loopback which can be turned ON or OFF by the transport layer without messing with the data being sent.
Is there a common, declarative (no custom code, configuration only) way to configure the middleware not to receive messages which that very same service instance has published? Ideally, this should also be able to filter out everything published by ALL servers (nodes) of the same "kind".
Thanks in advance.
The JMS API contains this option for TopicSubscribers, e.g. TIBCO EMS let's you create a consumer with the "noLocal" property. That means no messages published over the same connection, get consumed by clients on the same connection.
e.g. take a look here how to create a topic subscriber with the "noLocal" option:
https://docs.tibco.com/pub/enterprise_message_service/7.0.1-march-2013/doc/html/tib_ems_api_reference/api/javadoc/javax/jms/TopicSession.html
No one is answering, so I'll chime in (in a hand-wavey way).
I believe there's nothing in the JMS spec around controlling whether you get your own sent messages back on a topic receiver. So any capability like this would be a non-portable vendor feature. Especially for your second requirement (based on "kind" of JMS client versus some control based on the same connection doing the sending/receiving).
If you've got no flexibility to modify code or message content (properties), I think you've got no portable solutions. And likely no solution at all for that second "kind" requirement.
If you want to investigate vendor-specific options, you'll need to tell us which vendor you're interested in. You may get nothing, but there's no way to know without asking.