I tried to use SNS as platform to post http messages to clients, but it have 2 major problems.
i can't send the subscribers id's / endpoints dynamically. i must create a topic for every combination, but the combinations change every time according to specific message parameters which change very often.
trying to make a work around the 1 issue, i tried to create a service which will generate the topics run-time, but even when i create new topic i need confirmation from the client after adding him to the subscribers considering this happens pretty often i can't expect clients to confirm being added endlessly which creates an issue even so.
can anyone suggest alternative service which uses http to publish the messages?
Don't use an SNS subscription model and just create endpoints in the SNS application as the users register/login your app.
You will have to store on the back end a mapping of the users account to the endpoint ARN.
FYI, any one user can have many endpoints and some may be invalid.
Related
My first attempt to implement a microservice architecture using events with Kafka.
I have problems finding out how can I check for user credentials in a event.
My application is simple:
a service that controls users with email and passwords, able to create, edit and delete them.
a service that sends emails from those users.
My idea is to call create an event with a json like.
{
"status":"sendEmail",
"message":{
"sender":"abc#zxy.com",
"password":"123456",
"recipient":"jkl#asd.com",
"content":"this is my emails body"
}
}
Once I create this event at the second service, how can I validate with event that the user exist in the first service? I could easily do this wiht a REST communication but I would like to find out how to communicate responses between services with events messages.
Thanks.
You would need to either cache all user accounts in the second service (by consuming all user topic records), or perform an external lookup upon consuming email records. Messaging and RESTful services aren't necessarily exclusive.
FWIW, at least encrypt passwords before sending over unsecured/plaintext topics
I'm new to microservices architecture and want to create a centralised notification microservice to send emails/sms to users.
My first option was to create a notification Kafka queue where all other microservices can send notifications to. The notification microservice would then listen to this queue and send messages accordingly. If the notification service was restarted or taken down, we would not lose any messages as the messages will be stored on the queue.
My second option was to add a notification message API on the notifications microservice. This would make it easier for all other microservices as they just have to call an API as opposed to integrate with the queue. The API would then internally send the message to the notification Kafka queue and send the message. The only issue here is if the API is not available or there is an error, we will lose messages.
Any recommendations on the best way to handle this?
Either works. Some concepts that might help you decide:
A service that fronts "Kafka" would be helpful to:
Hide the implementation. This gives you the flexibility to change Kafka out later for something else. Your wrapper API would only respond with a 200 once it has put the notification request on the queue. I also see giving services direct access to "your" queue similar to allowing services to directly interact with a database they don't own. If you allow direct-access to Kafka and Kafka proves to be inadequate, a change to Kafka will require all of your clients to change their code.
Enforce the notification request contract (ensure the body of the request is well-formed). If you want to make sure that all of the items put on the queue are well-formed according to contract, an API can help enforce that. That will help prevent issues later when the "notifier" service picks notifications off the queue to send.
Adding a wrapper API would be less desirable if:
You don't want to/can't spend the time. Maybe deadlines are driving you to hurry and the days it would take to stand up a wrapper is just too much.
You are a small team and you don't have the resources/tools/time for service-explosion.
Your first design is simple and will work. If you're looking for the advantages I outlined, then consider your second design. And, to make sure I understand it, I would see it unfold like:
Client 1 needs to put out a notification and calls Service A POST /notifications
Service A that accepts POST /notifications
Service A checks the request, puts it on Kafka, responds to client with 200
Service B picks up notification request from Kafka queue.
Service A should be run as multiple instances for reliability.
Just wondering what the best way of capturing "fanout" calls from RabbitMQ is in Laravel subscriber services?
Service 1 sends out the message, say UserUpdated with their UUID, and this goes into RabbitMQ now.
Service 2/3/4/n capture UserUpdated and perform their appropriate actions.
I just don't know the best way to have a long running service on the Laravel subscribers to catch these messages and perform their own actions. I've tried multiple packages on GitHub so far but none go into this detail of where to place a class to receive the messages.
All help is much appreciated.
You can achieve that with enqueue/laravel-queue package. It comes with Enqueue Simple Client support. The client supports, pub/sub, message bus and friendly for use in microservers oriented systems.
I am implementing an event-driven microservice architecture. Imagine the following scenario:
Chat service: Ability to see conversations and send messages. Conversations can have multiple participants.
Registration-login service: Deals with the registration of new users, and login.
User service: Getting/updating user profiles.
The registration-login service emits the following event with the newly created user object:
registration-new
login-success
logout-success
The chat service then listens on registration-new and stores some fields of user in its own redis cache. It also listens on login-success and stores the token, and on logout-success to delete the token.
The user service has the following event: user-updated. When this is fired, a listener in the chat service updates the data corresponding to the user id in redis. Like the chat service, the user service also listens on login-success and logout-success and does the same thing as what the chat service does.
My question is the following: is this a good way to do this? It feels a bit counterintuitive to be sharing data everywhere. I need some advice on this. Thank you!
Seems that there's no other way. Microservices architecture puts lots of stress in avoiding data sharing so as to not create dependencies. That means that each microservice will have some data duplicated. That also means that there must exist a way of getting data from other contexts. The preferred methods strive for eventual consistency, such as sending messages to event sourcing or AMQP systems and subscribing to them. You can also use synchronous methods (RPC calls, distributed transactions). That creates additional technologic dependencies, but if you cannot accept eventual consistency it could be the only way.
We're use MassTransit with RabbitMQ. Is there a way to check that endpoints aren't available before we publish any messages? I want to setup our IoC to use another strategy if servicebus isn't available and I don't want to get to the point when I'll catch RabbitMQ.Client.Exceptions.BrockerUnreachableException on publishing messages.
If you're using a container, you could create a decorator that could monitor the outcome of the Publish method call, and if it starts throwing exceptions, you could switch the calls over to an alternative publisher.
Ideally such an implementation would include some type of progressive retry capability so that once the endpoint becomes available the calls resume back to the actual endpoint, as well as triggering some replay of the previously failed messages to the endpoint as well.
I figure you're already dealing with the need to have an alternative storage available, such as a local endpoint or some sort of local storage.
Not currently, you can submit an issue requesting that feature: https://github.com/MassTransit/MassTransit/issues. It's not trivial to implement, but maybe not impossible.
A couple of other options people have done include a remote cluster or having a local instance to forward/cluster across all machines included in the bus.