Just wondering what the best way of capturing "fanout" calls from RabbitMQ is in Laravel subscriber services?
Service 1 sends out the message, say UserUpdated with their UUID, and this goes into RabbitMQ now.
Service 2/3/4/n capture UserUpdated and perform their appropriate actions.
I just don't know the best way to have a long running service on the Laravel subscribers to catch these messages and perform their own actions. I've tried multiple packages on GitHub so far but none go into this detail of where to place a class to receive the messages.
All help is much appreciated.
You can achieve that with enqueue/laravel-queue package. It comes with Enqueue Simple Client support. The client supports, pub/sub, message bus and friendly for use in microservers oriented systems.
Related
I'm new to microservices architecture and want to create a centralised notification microservice to send emails/sms to users.
My first option was to create a notification Kafka queue where all other microservices can send notifications to. The notification microservice would then listen to this queue and send messages accordingly. If the notification service was restarted or taken down, we would not lose any messages as the messages will be stored on the queue.
My second option was to add a notification message API on the notifications microservice. This would make it easier for all other microservices as they just have to call an API as opposed to integrate with the queue. The API would then internally send the message to the notification Kafka queue and send the message. The only issue here is if the API is not available or there is an error, we will lose messages.
Any recommendations on the best way to handle this?
Either works. Some concepts that might help you decide:
A service that fronts "Kafka" would be helpful to:
Hide the implementation. This gives you the flexibility to change Kafka out later for something else. Your wrapper API would only respond with a 200 once it has put the notification request on the queue. I also see giving services direct access to "your" queue similar to allowing services to directly interact with a database they don't own. If you allow direct-access to Kafka and Kafka proves to be inadequate, a change to Kafka will require all of your clients to change their code.
Enforce the notification request contract (ensure the body of the request is well-formed). If you want to make sure that all of the items put on the queue are well-formed according to contract, an API can help enforce that. That will help prevent issues later when the "notifier" service picks notifications off the queue to send.
Adding a wrapper API would be less desirable if:
You don't want to/can't spend the time. Maybe deadlines are driving you to hurry and the days it would take to stand up a wrapper is just too much.
You are a small team and you don't have the resources/tools/time for service-explosion.
Your first design is simple and will work. If you're looking for the advantages I outlined, then consider your second design. And, to make sure I understand it, I would see it unfold like:
Client 1 needs to put out a notification and calls Service A POST /notifications
Service A that accepts POST /notifications
Service A checks the request, puts it on Kafka, responds to client with 200
Service B picks up notification request from Kafka queue.
Service A should be run as multiple instances for reliability.
I am new to Microservices and have a question with RabbitMQ / EasyNetQ.
I am sending messages from one microservice to another microservice.
Each Microservice are Web API's. I am using CQRS where my Command Handler would consume message off the Queue and do some business logic. In order to call the handler, it will need to make a request to the API method.
I would like to know without having to explicit call the API endpoint to hit the code for consuming messages. Is there an automated way of doing it without having to call the API endpoint ?
Suggestion could be creating a separate solution which would be a Console App that will execute the RabbitMQ in order to start listening. Create a while loop to read messages, then call the web api endpoint to handle business logic every time a new message is sent to the queue.
My aim is to create a listener or a startup task where once messages are in the queue it will automatically pick it up from the Queue and continue with command handler but not sure how to do the "Automatic" way as i describe it. I was thinking to utilise Azure Webjob that will continuously be running and it will act as the Consumer.
Looking for a good architectural way of doing it.
Programming language being used is C#
Much Appreciated
The recommended way of hosting RabbitMQ subscriber is by writing a windows service using something like topshelf library and subscribe to bus events inside that service on its start. We did that in multiple projects with no issues.
If you are using Azure, the best place to host RabbitMQ subscriber is in a "Worker Role".
I am using CQRS where my Command Handler would consume message off
the Queue and do some business logic. In order to call the handler, it
will need to make a request to the API method.
Are you sure this is real CQRS? CQRS occures when you handle queries and commands differently in your domain logic. Receiving a message via a calss, that's called CommandHandler and just reacting to it is not yet CQRS.
My aim is to create a listener or a startup task where once messages
are in the queue it will automatically pick it up from the Queue and
continue with command handler but not sure how to do the "Automatic"
way as i describe it. I was thinking to utilise Azure Webjob that will
continuously be running and it will act as the Consumer. Looking for
a good architectural way of doing it.
The easier you do that, the better. Don't go searching for complex solutions until you tried out all the simple ones. When I was implementing something similar, I was just running a pool of message handler scripts using Linux cron. A handler poped a message off the queue, processed it and terminated. Simple.
I think using the CQRS pattern, you will have events as well and corresponding event handlers. As you are using RabbitMQ for asynchronous communication between command and query then any message put on specific channel on RabbitMQ, can be listened by a callback method
Receiving messages from the queue is more complex. It works by subscribing a callback function to a queue. Whenever we receive a message, this callback function is called by the Pika library.
I have created Web API which allows messages to be sent to the Queue. My Web API is designed with CQRS and DDD in mind. I want my message consumer to always be waiting for any messages on the queue to receive. Currently the way its done, this will only read messages if I make a request to the API to hit the method.
Is there a way of either using console application or something that will always be running to consume messages at anytime given without having to make a request from the Web Api. So more of a automation task ?
If so, how do I go about with it i.e. if its console app how would I keep it always running (IIS ?) and is there way to use Dependency Injection as I need to consume the message then send to my repository which lives on separate solution. ?
or a way to make EasyNetQ run at start up ?
The best way to handle this situation in your case is to subscribe to bus events using AMPQ through EasyNetQ library. The recommended way of hosting it is by writing a windows service using topshelf library and subscribe to bus events inside that service on start.
IIS processes and threads are not reliable for such tasks as they are designed to be recycled on a regular basis which may cause some instabilities and inconsistencies in your application.
and is there way to use Dependency Injection as I need to consume the message then send to my repository which lives on separate solution.
It is better to create a separate question for this, as it is obviously off-topic. Also, it requires a further elaboration as it is not clear what specifically you are struggling with.
I'm trying to understand the principles of HornetQ as well as core/JMS messaging using this solution.
In my experimental app, I'd like my end-user application(client) to send messages to a HornetQ which will be read by a backend app. So far this is no problem and I love HornetQ.
But now, i'd like to send an "answer" message from the backend app back to the end-user. For this, I have the condition that no other client app should be able to read the answer message (let's say it contains the current bank balance). So user A should only fetch messages for himself and the same applies to any other user.
Is this possible using HornetQ? If so, how do I have to do it?
with hornetq (or any other message system) you always send to a queue, not to a specific consumer.
ON this case you have to create a queue matching your client.
This answer here will provide you some feedback on request-response where I won't need to repeat myself after this approach:
Synchronous request-reply pattern in a Java EE container
So our team has recently implemented torquebox into our jruby on rails applications. The purpose of this was to be able to receive queue/topic messages from an outside source which is streaming live data.
We have setup our queues/topics and they are receiving the messages without an issue. The next step we want to take is to get these messages on the browser.
So we started to look into leveraging the power of stomp. But we have come across some issues with this. It seems from the documentation that the purpose of using stomp + websockets is to receive messages from the client-side and push those messages to other clients. But we want to receive messages on our queues, and then push these messages to the client-side using websockets. Is this possible? Or would we have to implement a different technology such as Pusher or socket.io to get the queue/topic messages to the browser?
Thanks.
I think stomplets is good solution for this task. In rails application you should use ruby base stomp client, in browser javascript base stomp client. In rails just send data, and in browser just receive.
More detail how do it you can find in torquebox documentation
http://torquebox.org/documentation/2.0.0/stomp.html
It is indeed possible to push messages straight from the server to clients. It took me quite a bit of digging to find it as it is not listed in the documentation directly. Their blog lists it in their example of how to build a chat client using websockets.
http://torquebox.org/news/2011/08/23/stomp-chat-demo-part3/
Basically you use the inject method to choose which channel you're publishing to, and then use the publish method on the returned object to actually send the message. This code excerpt from the article should get you pointed in the right direction.
inject( '/topics/chat' ).publish( message,
:properties=>{
:recipient=>username,
:sender=>'system'
} )
It looks like :properties is the same thing as message headers. I'll be giving this a go over the next couple of days to see how well this works in Rails.