how do i load test aws appsync? - performance

I am attempting to use Load runner to simulate 10k clients all subscribed to AWS Appsync.
My understanding is that it uses MQTT over Websockets.
I haven't been able to find documentation on how to achieve this with load runner.

Currently (12.63) LoadRunner supports only MQTT over HTTP.

See VUGEN, advanced topics, creating virtual users in Visual Studio. Use the published appsync C#, C++, VB interfaces.
An eclipse path exits as well if you are so inclined
Should AWS Appsync fail to meet your requirements, how will you post a defect to Amazon? Do you have a strict SLA from Amazon on response times?

Related

how to configure x-ray traces for amazon-neptune?

I've an api going thru Lambda (node.js) to Aws Neptune. X-ray shows the traces from api-gateway --> lambda and stops here. Has anyone enabled deeper tracing all the way into Neptune ?
Thanks !
you can use AWS X-Ray SDK for Node.js to instrument your lambda function so the calls to Neptune are traced: https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-nodejs-awssdkclients.html
Please let me know if you need further help.
As of now, the most you can do is to use XRay clients and explicitly trace [1] the requests that you make from your Lambda. Neptune's AWS SDK currently only tracks management API calls, and not queries to the database. So unlike the dynamo db example called out in XRay docs, you cannot get granular insights (eg: query that was executed, the latency breakdowns etc) via XRay from Neptune at the moment.
It does sound like a useful feature, so I would recommend making a feature request for the same, or build something custom for the client you are using. Just curious, what client are you using from within the lambda? (ie Gremlin GLV? Raw HTTP request? Jena? etc..) For example, if you're using Gremlin GLV, then maybe all you need is to build a custom netty handler that can do tracing on your behalf.
[1] https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-nodejs-httpclients.html

OpenWhisk support of websockets and static websites

I'm choosing a serverless platform for my projects. I have explored AWS and found it excessively complicated: they provide an enormous bunch of settings but some basic scenarios are been too hard to implement.
The other platform looking promising for me is IBM Cloud with its OpenWhisk. And I'd like to check if the necessary capabilities are either implemented or in close plans for implementation.
Questions
Can I use websocket for my functions as a trigger for connect, message and disconnect? I found only a half year old discussion and nothing more. But this feature is demanded for real time applications.
Can I have static websites in both my custom domain and in subpath? I saw recipes where a docker container and lambda functions were employed. But writing my own implementation of Nginx looks nonsense. But this feature is also strongly demanded for single page applications (SPA) and there can be multiple such SPAs on one domain.
This blog with an IBM Cloud Functions overview has links and answers to your second question. There are tutorials on how to use custom domains with IBM Cloud Functions as backend for applications (see this tutorial with static page / SPA custom domain, and recipes for Express and Flask).
IBM Cloud Functions also has a package to post to Websockets. AFAIK there is functionality to listen to Websockets. My understanding is that serverless is incompatible with the "always on" nature of Websockets and the serverless runtime would need an API gateway or similar to manage the communication. If something is received, the action would be invoked.
Support for websockets for the ActionLoop proxy (used by Go,Swift,Python,PHP,Rust and Java) is here: https://github.com/sciabarracom/incubator-openwhisk-runtime-go/tree/websocket-support.
It can be used to build runtimes that support websocket but you need to deploy the runtime by yourself using Kubernetes. The support had ben postponed as an integration of OpenWhisk with Knative is a better path to include it in OpenWhisk.

Unit Test GraphQL schemas/queries made in AWS AppSync?

I have a simple question: is there a way/program/method to create unit tests to test the API Url generated on AWS AppSync to verify the validity of created GraphQL schemas, queries, mutations, etc?
There is an open-source AppSync Serverless plugin which has offline emulator support. You may find it useful: https://github.com/sid88in/serverless-appsync-plugin#offline-support
Another good recommendation is to have two separate AppSync APIs. One API is hosting you production traffic. The other is to test changes before they go to production. This is significantly easier if you use Cloudformation (highly recommended) to manage your infrastructure.
If you want to validate your API is working periodically (every minute or so), you could create a canary like the following:
Create a Lambda function which runs on a schedule. This lambda function will make various GraphQL requests. It can emit success/failure metrics to CloudWatch.
Setup a CloudWatch alarm so you can be notified if your success/failure metric is out of the ordinary.
For the canary use-case see:
https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/RunLambdaSchedule.html
https://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html
There is also amplify amplify-appsync-simulator package that is supposed to help with testing appsync, but there is no documentation on how you should use it. It is used by serverless-appsync-simulator Michael wrote and Amplify itself.

How to setup Amazon Lambda with micro services in Node.js

I am looking forward to work in a Amazon Lambda with Node.js
They call it server less, So is it a better way to host our code then traditional hosting servers ?
I am open for the suggestions, thanks in advance!!
It is called serverless as you dont manage and maintain the underlying server and the runtime.
Basically you write your code in one of the supported languages, say node.js, and then configure events that will trigger your code.
Example in case of AWS, the events can be a API GW call, a SQS message, a SNS notification etc.
So it can be better depending on what you are planning on doing.
Do note that there are certain limits that AWS imposes by default on accounts for AWS Lambda.
Also there can be slight startup penalty for a Lambda.
A plus point of Lambda vs Hosting your code in EC2 is that with Lambda you dont get charged if your code is not used/triggered.
However, do note that for functions that have heavy usage it might be better to
host your own EC2.
Most important a Lambda has to be stateless.
Considering all the above factors you can take a call on whether AWS Lambda and Serverless Architecture fits your needs.

How to do automated functional testing of AWS components?

In my project we have implemented custom auto scaling module. This module takes advantage of AWS CloudWatch API and uses its custom logic to auto scale up/down the cluster. All this code is in written in Java + Shell scripts. We have written unit test cases using JUnit.
Now we want to automate the functional testing but I do not know how other people do automated functional/integration testing of AWS components and best practices for AWS component functional testing.
Consider the following scenario and expected output:
Scenario : HDFS utilization of EC2 based Hadoop cluster goes above given threshold.
Expected result : Attach new EBS volume to one of the EC2 instance in the cluster.
I would like to know which technology, language can be used to do functional testing of this scenarios.
Take a look at LocalStack. It provides an easy-to-use test/mocking framework for developing AWS-related applications by spinnin up the AWS-compatible APIs on your local machine or in Docker. It supports two dozen of AWS APIs and EC2 and CloudWatch are among them. It is really a great tool for functional testing without using a separate environment in AWS for that.
As for the language: you can use any language has AWS SDK. Even the AWS CLI works with LocalStack.
If you are a lucky user of Java/Kotlin and JUnit 5 I would like to recommend you to use aws-junit5, a set of JUnit 5 extensions for AWS. And, yes, I am it's author. These extensions can be used to inject clients for AWS services provided by tools like localstack or any other AWS-compatible API (including the real AWS, of course). Both AWS Java SDK v 2.x and v 1.x are supported. You can use aws-junit5 to inject clients for S3, DynamoDB, Kinesis, SES, SNS and SQS.

Resources