I have a lambda that is triggered whenever an event is dropped in the eventbus to which my lambda is connected and is triggered automatically. How can I performance test it to test how it performs of 500 events are dropped at a time?
Also I know aws has some inbuilt metrics like lambda execution time, xray tracing etc. Can anyone let me know how to use them for my use case?
If by "eventbus" you mean AWS EventBus which is a part of Amazon EventBrigde my expectation is that the easiest would be using PutEvents API endpoint, you can come up with a JSON payload having 500 events or make 500 separate calls with 1 event or any combination you can think of.
Be aware that the AWS API requests need to be signed to the load testing tool you choose must have the possibility to calculate this signature. A guide for Apache JMeter: How to Handle Dynamic AWS SigV4 in JMeter
With regards to metrics - check out AWS CloudWatch
Related
I have created a alexa smart home function and want to run it asynchronously so plan to use amazon sqs (Simple que service) functionality. I connected amazon sqs trigger output to lambda function and successfully able to send message from sqs to lambda. Now need to connect the alexa to sqs input. When i try to use sqs arn in alexa developer console it does not support it. Is there any way to solve this or will alexa support only lambda function for invocation.
The alexa skill is for smart home service to control switches (Turn on/off), so when try to control the multiple switches because of synchronous nature execution of lambda it turns on switches one after the other. I need to control them at single shot so need asynchronous execution for lambda where requests need to execute without waiting for the response.
Thanks in advance for answers.
It will not work as SQS works asynchronus and just reply that message was put there. But Alexa needs a valid JSON response with speech tag and so on immediately and SQS is not able to fulfill this.
What you could do:
Alexa -> Lambda (new) -> SQS - Lambda
In your new created lambda you could give a valid reply to Alexa and put a message in SQS.
AWS Lambda can work asynchronously. You can have a bunch of back-end processes all working as they need to, triggering various Lambdas as needed.
But the exchange with Alexa opens a session to your backend, sends its request, and the full response is expected to end that session. That response may have directives to download other content to incorporate into the response, like a sound file or lazy loading a list in APL. But it is expecting a full response.
If you go through the basic Cake Time tutorial for building Alexa skills, they actually use async-await for some APIs because that response has to be complete before it's sent.
There are some async APIs like reminders and proactive events, but they're NOT conversational. They're unique one-way messages.
The real questions are why do you feel you need to do it this way and what are you optimizing for by queuing?
I am using #nest/microservices with the latest version of nextjs 6.x. I have inter-process communication working correctly using #EventPattern and ClientProxy.emit.
This works fine locally but when I deploy I use AWS Eventbridge for communication directly to a lambda. This could of course be any transport mechanism so my question is not related to Eventbridge per se.
What I need to do is receive a payload into my lambda and then I need to push this into the nestjs microservices lifecycle so that my event can be processed using the #EventHandler(...). So I am looking for something like a publishLocal or emitLocal so that the communication stays in-process.
I would be bootstrapping the nestjs application context with NestFactory.createApplicationContext(...), I am just not sure how I can process the subscription payload in-process in a decoupled fashion.
Any ideas would be greatly appreciated, thanks in advance.
Cheers,
Mark
Example use case
Send the user a notification 2 hours after signup.
Options considered
setTimeout(() => { /* send notification */ }, 2*60*60*1000); is not an option in serverless environments since the function terminates after execution (so it has to be stateless).
CloudWatch events can schedule lambda invocations using cron expressions - but this was designed for repetitive invocations (there's a limit of 100 rules/region).
I have not seen scheduling options in AWS SNS/SQS or GCP Pub/Sub. Are there alternatives with scheduling?
I want to avoid (if possible) setting up a dedicated message broker (overkill) or stateful/non-serverless instance - is there a serverless way to do this?
I can queue the events in a database and invoke a lambda function every minute to poll the database for events to execute in that minute... is there a more elegant solution?
Use AWS Step functions, they are like serverless functions that don't have the 15 minute limit like AWS Lambda does. You can design a workflow in AWS step that integrates with API Gateway, Lambda and SNS to send email and text notifications as follows:
Create a REST API via API gateway that will invoke a Lambda function passing in for example, the destination address (email, phone #) of the SNS notification, when it should be sent, notification method (e.g. email, text, etc.).
The Lambda function on invocation will invoke the Step function passing in the data (Lambda is needed because API Gateway currently can't invoke Step functions directly).
The Step function is basically a workflow, you can define states for waiting (like waiting for the specified time to send the notification e.g. 30 seconds), and states for invoking other Lambda functions that can use SNS to send out an email and/or text notifications.
A rudimentary example is provided by AWS w/ their Task Timer example.
Things are coming on GCP for doing this, but not very soon. Thereby, today, the solution is to poll a database.
You can to that with Datastore/firestore with the execution datetime indexed (to prevent to read all the documents each minute). But be careful of traffic spike, you could create hotspot.
You can use Cloud Scheduler on Google Cloud Platform. As is is stated in the official documentation :
Cloud Scheduler is a fully managed enterprise-grade cron job scheduler. It allows you to schedule virtually any job, including batch, big data jobs, cloud infrastructure operations, and more. You can automate everything, including retries in case of failure to reduce manual toil and intervention. Cloud Scheduler even acts as a single pane of glass, allowing you to manage all your automation tasks from one place.
Here you can check a quickstart for using it with Pub/Sub and Cloud Functions.
I am making a stress test in my system that is designed with AWS API Gateway + AWS Lambda.
I am setting 2K Virtual Users each of them making 1 transaction with a Ramp-up of 1 minute.
When making a dummy lambda the system can handle the load.
If I change my lambda to have a sleep(5), I started to see some errors on my dashboard. They're 5xx errors, but there's no logging information on Lambda function. It seems that the Lambda function was not called... The request was "blocked" on API Gateway.
There is a possiblility that you are hitting the lambda concurrency limit.aws lambda
Each AWS Account has an overall AccountLimit value that is fixed at any point in time, but can be easily increased as needed,As of May 2017, the default limit is 1000 of concurrency per AWS Region.
Also check the API gateway throttling limits if there are any set at method level for your project (default is 10,000/rps with burst of 5,000) aws apiGateway
Also make sure you inform aws that you are doing a stress test as there is a possibility that they might block you.
You can have a look at cloudwatch logs for both apigateway and lambdas which might give us some more insight.
I'm beginner of greengrass core application, and finished the demo setup following greengrass developper guide. but i'm still confusing about how lambda functio works.the bellow is the quesitons I want to ask for help.
I want to run a lambda function in my raspberry pi 3 as greengrass core, which can recieve multiple IoT devices' MQTT messages and do some process according to task tpye(i.e various signal filtering or house-hold machine learning algorithms). After proceesing, I need send the information using MQTT to my own server(not AWS IoT cloud) for higher level processing with some topics.
my questions are as follows( I want to use JAVA language):
1 To receive multiple aws iot devices connected to the GGC, should I need to set up a AWSIoTMQTTClient in aws-iot-device-sdk-java?
I also find in aws_greengrass_core_sdk_java, there is “IotDataClient” class,what's it for?and what's the different with AWSIoTMQTTClient. here is really very confusing, even with sdk document description.
2 In GGC, when I deployed my lambda function, will it has an internal MQTT broker to receive messages for AWSIoTMQTTClient ?
3 for lambda functions, after creation and deployment on GGC, will it start to work. I saw there is method to invoke another lambda funciton from a lambda funciton. I don't understand the mechanism how lambda works.
4 Can i have multiple lambda functions for different uage,for instance, one is only to receive MQTT messages, another is to process the received info, other one is to send the processed info out to my own MQTT server? if permitted, how to make the work together to perform all the tasks.
5 I saw there is event input to lambda interface, how can I call a lambda only when some specific topic arriverd to AWSIoTMQTTClient defined in the lambda function?
6 the below is JAVA lambda interface template:
outputType handler-name(inputType input, Context context) {
...
}
i think it should permit user to define input data type as he need. but the quesiton is if I define inputtype is string. how to the lambda handler to receive the string. the development guidence have no clear description.
7 finally, can you please share some demo codes for the above questions?
Thanks for you attention and kind help in advance.
your help is highly expected
AWSIoTMQTTClient from the device SDK is not for Greengrass Lambda functions. Instead use IotDataClient from the Greengrass Java SDK, create a publish request, and then invoke the publish method. There is an example of that here - https://github.com/aws-samples/aws-greengrass-lambda-functions/blob/master/foundation/CDDBaselineJava/src/main/java/com/timmattison/greengrass/cdd/communication/GreengrassCommunication.java
AWSIoTMQTTClient is for devices/applications that run outside of Greengrass.
If you'd like to see some example Greengrass Lambda function code in Java check out at least this skeleton example - https://github.com/aws-samples/aws-greengrass-lambda-functions/tree/master/functions/CDDSkeletonJava. Note this function and other other ones in the repo depend on a framework called CDD (Cloud Device Driver). It is shared in the same repo and does most of the heavy lifting (messaging, startup, etc). That combined with the Greengrass provisioner - https://github.com/awslabs/aws-greengrass-provisioner - gives you a quick way to develop Java functions on Greengrass. Let me know if you try it out.
If you want to see the internals of CDD the root of it is here - https://github.com/aws-samples/aws-greengrass-lambda-functions/tree/master/foundation/CDDBaselineJava
As far as Lambda functions and how they run briefly I'll say that they can run on-demand (when they receive a message) or they can run "pinned" (forever). Pinned functions can receive messages too. Pinned functions are good when you need to track some kind of state. On-demand functions are more efficient for stateless data processing.