Is there any way to use MassTransit with AmazonMQ or SNS+SQS and AWS Lambda?
I am new to serverless, AWS Lambda and AWS in general. I am implementing an event-driven architecture that is going to need some event bus for communication between microservices/lambda.
I have an app that needs to publish events, and I want to have many AWS Lambda that react to those events, do their work and go to sleep. I have had a look at AWS EventBridge but the latency (300ms to 600ms) is way too high for real time needs and it's not really designed for a large amount of subscribers. SNS + SQS is another option because I know MassTransit also supports it. Notice that RabbitMQ broker does not seem yet compatible with AWS Lambda as Chris mentioned in the comment below pointing to https://docs.aws.amazon.com/lambda/latest/dg/with-mq.html.
I've had good results with MassTransit in the past and I like the way it abstracts the transport layer away, but I can't find any opinion/sample on how to use it with AWS Lambda. Is it even possible? Any sample/link or reasons why it's not possible would be appreciated.
UPDATE: RabbitMQ cannot be used as broker for AmazonMQ if I want to use AWS Lambda. Question edited and it remains but applied to MassTransit + AmazonMQ (ActiveMQ broker) or SQS/SNS + AWS Lambda
Related
I'm relatively new to AWS Lambdas, but what I want to do is the following:
When a message lands on SQS, I want some lambda process to hit an endpoint. Can anyone guide me how to accomplish this?
I use node.js if that matters.
Yes, you can hit lambda function through SQS. Below is the screenshot for your reference.
I found out that the task queue is being primarily used for App Engine standard environment. I am migrating our existing services from App Engine to Kubernetes. What would be a good alternative for task queue? Push queue is the one which is being currently used.
I read documentation online as well as gone through this link: When to use PubSub vs Task Queues
But there is no clear answer as to whether Pub/Sub is a good alternative on Kubernetes.
Edit:
My current use case is that a service performs similar tasks for a set of ID's and some task which takes some time to complete so the queue would take this task and process it while the service can perform other things in parallel. While Pub/Sub is mainly needed where we have publisher and subscriber here the service itself has some tasks which it needs to keep processing in parallel!
I would think Cloud Pub/Sub is a great tool for message queues. It's orthogonal to how you deploy/run your services, whether with Kubernetes or something else.
There's a lot of relevant documentation for using pubsub with Kubernetes on GCP, like this page.
Im new to aws.
I am going to develop a REST full app which is going host on aws.
I decided to use
Amazon S3 for static contents
Amazon Cognito User Pool for Authentication
Amazon DynamoDB as db
I am confused on where my app is going to be hosted. I have 2 ideas for that.
AWS Lambda Function + api gateway
Can I implement entire app on it ?
Elastic Beanstalk
Can i integrate all the above aws services with it ?
(Backend on .net core web api 2.0)
Please guid me
As the experience of working with cloud, after 1y 6m I can give a proper answer for my own question.
Yes.
There is a possibility to use API Gateway + Lambda for the entire app as the back end. But you have to manage your most of the app logic from the front end. On there you have to get a risk because the source code can be viewed by the public.
Keeping your all business logic in the client code is not a good practice. And keeping all the logic in the Lambda also not easy or cost effective. The reason is when you making a real world app, you will need thousands of functions. To do one task, you will have to call many functions (Then its a function run time). So it will be very expensive.
Best solution is hosting the backend on Elastic Beanstalk and front end on S3. If you have any heavy task ? then you can make Lambda functions for that.
Lambda is best for CPU bounded functions. But not to have all the application logic on it.
Since you might not be interested in managing the underlying system, you should opt for AWS Lambda + API Gateway.
I am looking forward to work in a Amazon Lambda with Node.js
They call it server less, So is it a better way to host our code then traditional hosting servers ?
I am open for the suggestions, thanks in advance!!
It is called serverless as you dont manage and maintain the underlying server and the runtime.
Basically you write your code in one of the supported languages, say node.js, and then configure events that will trigger your code.
Example in case of AWS, the events can be a API GW call, a SQS message, a SNS notification etc.
So it can be better depending on what you are planning on doing.
Do note that there are certain limits that AWS imposes by default on accounts for AWS Lambda.
Also there can be slight startup penalty for a Lambda.
A plus point of Lambda vs Hosting your code in EC2 is that with Lambda you dont get charged if your code is not used/triggered.
However, do note that for functions that have heavy usage it might be better to
host your own EC2.
Most important a Lambda has to be stateless.
Considering all the above factors you can take a call on whether AWS Lambda and Serverless Architecture fits your needs.
I'm a noob using Hubot, so please bear with me.
I was going through Hubot's documentation ( https://hubot.github.com/docs/deploying/ ) and saw that we can deploy to Heroku, Unix systems and others.
May I know how can Hubot be deployed on AWS Lambda ?
Best Regards.
I was wondering the same thing. I think the answer is probably "yes, if your chat system can send you http when traffic happens". I think Lambda is probably a great choice, since each message in chat is an event to process according to the rules that your bot has. So, the Lambda function invocation model is a good fit - lots of tiny invokes. You might, though, want to filter the traffic that gets to the function, though, if your chat system is high-traffic.
For example Slack's Events API lets you subscribe to all of the things you might need the bot to see, I think. You'd then need to route those requests to your lambda function, which you could do via AWS API Gateway.
The existing hubot-slack adapter uses the Real-Time Messaging API, though, so you'd need to write your own adaptor for the Events API http that the API Gateway receives.
The same approach (and requirement) applies to other chat systems.