Unable to query dynamodb GSI in lambda locally - aws-lambda

So I added a lambda function category using the amplify CLI, in order to query data from the GSI(Global secondary Index) I created using the #key directive in the graphql schema. Whenever I try mocking the function locally using the amplify mock function <functionName> the callback function of the query keeps on returning null. The function can be seen below
const AWS = require("aws-sdk");
const db = new AWS.DynamoDB.DocumentClient({
region: process.env.REGION,
apiVersion: "2012-08-10",
});
const params = {
// ProjectionExpression: ["province", "gender", "updatedAt", "createdAt"],
ExpressionAttributeValues: {
":provinceVal": "Sichuan",
},
IndexName: "RegistreesByProvince",
KeyConditionExpression: "province = :provinceVal",
TableName: process.env.API_PORTAL_SUBMISSIONSTABLE_NAME,
};
const calculateStatistics = async () => {
try {
const data = await db.query(params).promise();
console.log(data);
} catch (err) {
console.log(err);
}
};
const resolvers = {
Query: {
getStatistics: () => {
return calculateStatistics();
},
},
};
exports.handler = async (event) => {
// TODO implement
const typeHandler = resolvers[event.typeName];
if (typeHandler) {
const resolver = typeHandler[event.fieldName];
if (resolver) {
var result = await resolver(event);
return result;
}
}
}; // };
I then tried to capture the whole event and logged it to the console as can be seen in the calculateStatistics function, which now showed me a bit more explicit error as follows.
{ UnknownEndpoint: Inaccessible host: `dynamodb.us-east-1-fake.amazonaws.com'. This service may not be available in the `us-east-1-fake' region.
at Request.ENOTFOUND_ERROR (/Users/apple/Documents/work/web/portal/amplify/backend/function/calcStatistics/src/node_modules/aws-sdk/lib/event_listeners.js:501:46)
at Request.callListeners (/Users/apple/Documents/work/web/portal/amplify/backend/function/calcStatistics/src/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at Request.emit (/Users/apple/Documents/work/web/portal/amplify/backend/function/calcStatistics/src/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
at Request.emit (/Users/apple/Documents/work/web/portal/amplify/backend/function/calcStatistics/src/node_modules/aws-sdk/lib/request.js:688:14)
at ClientRequest.error (/Users/apple/Documents/work/web/portal/amplify/backend/function/calcStatistics/src/node_modules/aws-sdk/lib/event_listeners.js:339:22)
at ClientRequest.<anonymous> (/Users/apple/Documents/work/web/portal/amplify/backend/function/calcStatistics/src/node_modules/aws-sdk/lib/http/node.js:96:19)
at ClientRequest.emit (events.js:198:13)
at ClientRequest.EventEmitter.emit (domain.js:448:20)
at TLSSocket.socketErrorListener (_http_client.js:401:9)
at TLSSocket.emit (events.js:198:13)
message:
'Inaccessible host: `dynamodb.us-east-1-fake.amazonaws.com\'. This service may not be available in the `us-east-1-fake\' region.',
code: 'UnknownEndpoint',
region: 'us-east-1-fake',
hostname: 'dynamodb.us-east-1-fake.amazonaws.com',
retryable: true,
originalError:
{ Error: getaddrinfo ENOTFOUND dynamodb.us-east-1-fake.amazonaws.com dynamodb.us-east-1-fake.amazonaws.com:443
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:56:26)
message:
'getaddrinfo ENOTFOUND dynamodb.us-east-1-fake.amazonaws.com dynamodb.us-east-1-fake.amazonaws.com:443',
errno: 'ENOTFOUND',
code: 'NetworkingError',
syscall: 'getaddrinfo',
hostname: 'dynamodb.us-east-1-fake.amazonaws.com',
host: 'dynamodb.us-east-1-fake.amazonaws.com',
port: 443,
region: 'us-east-1-fake',
retryable: true,
time: 2020-08-12T10:18:08.321Z },
time: 2020-08-12T10:18:08.321Z }
Result:
null
Finished execution.
I then did more research and came across this thread about inaccessible-dynamodb-host-when-running-amplify-mock which I followed and tried implementing to but to no avail. Any help on this would be very much appreciated.
PS: It is worth mentioning that I was able to successfully query for this data through the Appsync console, which led me to strongly believe the problem lies in the function itself.

After doing more research and asking around, I finally made sense of the answer that was provided to me on github that
When running mock on a function which has access to a dynamodb
table generated by API. It will populate the env with fake values. If
you would like to mock your lambda function against your deployed
dynamodb table you can edit the values in the sdk client so it can
make the call accurately.
In summary, if you are running things locally, then you wouldn't have access to your backend variables which you might try mocking. I hope this helps someone. Thanks!

Related

Graphql subscription in playground during local development throwing "Could not connect to websocket endpoint" in basic nestjs project

This is happening on a simple project during local development, so cloud infrastructure isn't an issue.
This is also happening in the application playground.
My module registration:
GraphQLModule.forRootAsync<ApolloDriverConfig>({
driver: ApolloDriver,
imports: [YeoConfigModule],
useFactory: (configService: YeoConfigService<AppConfig>) => {
const config: ApolloDriverConfig = {
debug: true,
subscriptions: {
'graphql-ws': true,
},
playground: true,
autoSchemaFile: './apps/event-service/schema.gql',
sortSchema: true,
context: ({ req, res }) => ({ req, res }),
};
const origins = configService.get('CORS_ORIGINS')();
config.cors = { origin: origins, credentials: true };
// config.path = '/apis/event-service/graphql';
return config;
},
inject: [YeoConfigService],
My app startup:
async function bootstrap(): Promise<void> {
const app = await getApp();
await app.listen(process.env.PORT ?? 3600);
}
bootstrap();
My versions:
"graphql-ws": "5.11.2",
"graphql-redis-subscriptions": "2.5.0"
"#apollo/gateway": "2.1.3",
"#nestjs/graphql": "10.1.3",
"graphql": "16.5.0",
Result:
{
"error": "Could not connect to websocket endpoint ws://localhost:3600/graphql. Please check if the endpoint url is correct."
}
Any ideas why this isn't working as expected? I've been reading the nestjs docs up at https://docs.nestjs.com/graphql/subscriptions but there's nothing that I can find about extra setup required other than adding
subscriptions: {
'graphql-ws': true,
},
when registering the graphql module.
For anyone else stumbling upon this, I have started using altair which allows me to specify the ws endpoint as well as the type of connection, among which there is a graphql-ws option.
So I went with it.
If anyone knows how to achieve this using the playground referred in the original answer, happy to mark that one as the right answer over my own.

AWS Lambda Timeout when connecting to Redis Elasticache in same VPC

Trying to publish from a Lambda function to a Redis Elasticache, but I just continue to get 502 Bad Gateway responses with the Lambda function timing out.
I have successfully connected to the Elasticache instance using an ECS in the same VPC which leads me to think that the VPC settings for my Lambda are not correct. I tried following this tutorial (https://docs.aws.amazon.com/lambda/latest/dg/services-elasticache-tutorial.html) and have looked at several StackOverflow threads to no avail.
The Lambda Function:
export const publisher = redis.createClient({
url: XXXXXX, // env var containing the URL which is also used in the ECS server to successfully connect
});
export const handler = async (
event: AWSLambda.APIGatewayProxyWithCognitoAuthorizerEvent
): Promise<AWSLambda.APIGatewayProxyResult> => {
try {
if (!event.body || !event.pathParameters || !event.pathParameters.channelId)
return ApiResponse.error(400, {}, new InvalidRequestError());
const { action, payload } = JSON.parse(event.body) as {
action: string;
payload?: { [key: string]: string };
};
const { channelId } = event.pathParameters;
const publishAsync = promisify(publisher.publish).bind(publisher);
await publishAsync(
channelId,
JSON.stringify({
action,
payload: payload || {},
})
);
return ApiResponse.success(204);
} catch (e) {
Logger.error(e);
return ApiResponse.error();
}
};
In my troubleshooting, I have verified the following in the Lambda functions console:
The correct role is showing in Configuration > Permissions
The lambda function has access to the VPC (Configuration > VPCs), Subnets, and the same SG as the Elasticache instance.
The SG is allowing all traffic from anywhere.
It is indeed the Redis connection. Using console.log the code stops at this line: await publishAsync()
I am sure it is something small, but it is racking my brain!
Update 1:
Tried adding an error handler to log any issues with the publish in addition to the main try/catch block, but it's not logging a thing.
publisher.on('error', (e) => {
Logger.error(e, 'evses-service', 'message-publisher');
});
Also have copied my Elasticache setup:
And my Elasticache Subnet Group:
And my Lambda VPC settings:
And that my Lambda has the right access:
Update 2:
Tried to follow the tutorial here (https://docs.aws.amazon.com/lambda/latest/dg/services-elasticache-tutorial.html) word for word, but getting the same issue. No logs, just a timeout after 30 seconds. Here is the test code:
const crypto = require('crypto');
const redis = require('redis');
const util = require('util');
const client = redis.createClient({
url: 'rediss://clusterforlambdatest.9nxhfd.0001.use1.cache.amazonaws.com',
});
client.on('error', (e) => {
console.log(e);
});
exports.handler = async (event) => {
try {
const len = 24;
const randomString = crypto
.randomBytes(Math.ceil(len / 2))
.toString('hex') // convert to hexadecimal format
.slice(0, len)
.toUpperCase();
const setAsync = util.promisify(client.set).bind(client);
const getAsync = util.promisify(client.get).bind(client);
await setAsync(randomString, 'We set this string bruh!');
const doc = await getAsync(randomString);
console.log(`Successfully receieved document ${randomString} with contents: ${doc}`);
return;
} catch (e) {
console.log(e);
return {
statusCode: 500,
};
}
};
If you have timeout, assuming the lambda network is well configured, you should check the following:
redis SSL configuration: check diffs between redisS connection url and cluster configuration (in-transit encryption and client configuration with tls: {})
configure the client with a specific retry strategy to avoid lambda timeout and catch connection issue
check VPC acl and security groups
I had same issue with my elasticache cluster, here are few findings -
Check the number client connections with elasticache and resources used
Check VPC subnet and CIDR for nodes security group
Try to increase the TTL for lambda and see which service is taking more time to respond Lambda or elasticache

NodeJS: AWS SDK V3: Not receiving any response data from lambda function

I'm trying to use the v3 javascript sdk to invoke a AWS Lambda function, and I'm having problems getting any meaningful response.
My code looks like so...
const { Lambda } = require("#aws-sdk/client-lambda");
const client = new Lambda();
const params = {
FunctionName: "MyLamdaFuncton",
Payload: JSON.stringify({ "action": "do_something" }),
InvocationType: "Event"
};
client.invoke(params)
.then((response) => {
console.log(JSON.stringify(response,null,4));
})
.catch((err) => {
console.error(err);
})
I can confirm from checking the CloudWatch logs that the lambda function works as exepcted. However this is the response I get in my NodeJS code...
{
"$metadata": {
"httpStatusCode": 202,
"requestId": "d6ba189d-9156-4f01-bd51-efe34a66fe34",
"attempts": 1,
"totalRetryDelay": 0
},
"Payload": {}
}
How do I get the actual response and status from the Lambda function?
If I change the payload above to intentionally throw an exception in my Lambda, the response in the console is still exactly the same.
update:
The Lambda function is written in Ruby. The response is returned like so...
{ statusCode: 200, body: JSON.generate(response.success?) }
where "response" is from another service it calls internally.
I've figured out what I was doing wrong. The issue was the "InvocationType". I got it working by changing to...
InvocationType: "RequestResponse"
Then I had to extract the response data like so...
const response_data = JSON.parse(new TextDecoder("utf-8").decode(response.Payload))

How to call one lambda from another in AWS SAM

I'm writing application with multiple functions inside SAM application. I can invoke lambda function that is already deployed to AWS with code similar to AWS Lambda call Lambda but it doens't work with local functions. I tried things from https://github.com/awslabs/aws-sam-cli/issues/510 but nothing seem to work yet.
This is closest I got yet (you need sam local start-lambda --host 172.17.0.1 where host is in docker network)
var AWS = require("aws-sdk");
exports.lambdaHandler = async (event, context) => {
let lambda = new AWS.Lambda({});
if (process.env.AWS_SAM_LOCAL) {
var ep = new AWS.Endpoint("http://172.17.0.1:3001");
lambda = new AWS.Lambda({ endpoint: ep });
}
const body = await new Promise(r => {
lambda.invokeAsync(
{
FunctionName: "myFunction",
InvokeArgs: JSON.stringify({ arguments: "for other function" })
},
function(err, data) {
if (err) console.log(err, err.stack);
else console.log(data);
r(data);
}
);
});
response = {
statusCode: 200,
body: JSON.stringify(body)
};
return response;
};
and at least I see some activity, but I get error in invokeAsync
PathNotFoundLocally: PathNotFoundException\n at Object.extractError (/var/task/node_modules/aws-sdk/lib/protocol/json.js:51:27)\n
and this error in start-lambda console
2019-12-20 16:07:02 172.17.0.7 - - [20/Dec/2019 16:07:02] "POST /2014-11-13/functions/myFunction/invoke-async/ HTTP/1.1" 404 -
UPDATE:
I was able to fix ssl error that I had in first version of this question, and it looks like it works in sync mode with lambda.invoke
It looks like this is not currently supported until https://github.com/awslabs/aws-sam-cli/pull/749 is merged

Apollo GraphQL Client 'Network error: Can't find field XXXX on object undefined' with disableOffline: false

I am having trouble using apollo-client version 2.4.6 to query my AWS AppSync endpoint.
I can successfully query the AWS AppSync endpoint using a curl command, but the exact same
GraphQL executed over the Apollo client is returning "Can't find field getTickets on object undefined."
I am a newby at GraphQL and Apollo. Am I doing something stupid to cause that error? Why does it say NetworkError? Why is the object undefined?
EDIT: I noticed that if I pass disableOffline: true in the constructor to AWSAppSyncClient then it starts working. Why? Why is the default client behavior with disableOffline: false not working?
Here is my super simple schema.graphql deployed at AWS:
schema {
query: Query
}
type Query {
getTickets: [EmmDDavidTickets]
#aws_api_key
}
type EmmDDavidTickets #aws_api_key {
ticketNumber: ID!
pnrNumber: String
}
Here is the curl command that works to query that endpoint at AWS. Note the valid response:
$ curl -X POST -H "x-api-key: --REDACTED--" https://wm3mz6anrjbrfpgbewnyyrio3u.appsync-api.us-east-1.amazonaws.com/graphql -d '{ "query": "query list {\ngetTickets { ticketNumber\n pnrNumber\n }\n}"}'
{"data":{"getTickets":[{"ticketNumber":"12345","pnrNumber":null},{"ticketNumber":"0001020202020","pnrNumber":"ABC123"}]}}
Here is my NodeJS code to execute the same query using Apollo:
const apiKey ='--REDACTED--';
const region = 'us-east-1';
const type = 'API_KEY';
const url = 'https://wm3mz6anrjbrfpgbewnyyrio3u.appsync-api.us-east-1.amazonaws.com/graphql';
const gql = require('graphql-tag');
const query = gql(`
query list {
getTickets {
ticketNumber
}
}`);
// Set up Apollo client
const client = new AWSAppSyncClient({
url: url,
region: region,
auth: {
type: type,
apiKey: apiKey,
},
disableOffline: false
});
client.hydrated().then(function (client) {
//Now run a query
client.query({ query: query })
.then(function logData(data) {
console.log('results of query: ', data);
})
.catch(console.error);
});
Here is the error response from Apollo:
ApolloError: Network error: Can't find field getTickets on object undefined.
at new ApolloError (/Users/dyoung/workspace//appsync_javascript_test/node_modules/apollo-client/bundle.umd.js:85:32)
at /Users/dyoung/workspace//appsync_javascript_test/node_modules/apollo-client/bundle.umd.js:1039:45
at /Users/dyoung/workspace//appsync_javascript_test/node_modules/apollo-client/bundle.umd.js:1411:21
at Array.forEach (<anonymous>)
at /Users/dyoung/workspace//appsync_javascript_test/node_modules/apollo-client/bundle.umd.js:1410:22
at Map.forEach (<anonymous>)
at QueryManager.broadcastQueries (/Users/dyoung/workspace//appsync_javascript_test/node_modules/apollo-client/bundle.umd.js:1405:26)
at /Users/dyoung/workspace//appsync_javascript_test/node_modules/apollo-client/bundle.umd.js:988:35 {
graphQLErrors: [],
networkError: Error: Can't find field getTickets on object undefined.
at /Users/dyoung/workspace//appsync_javascript_test/node_modules/apollo-cache-inmemory/lib/bundle.umd.js:429:27
at Array.forEach (<anonymous>)
at StoreReader.diffQueryAgainstStore (/Users/dyoung/workspace//appsync_javascript_test/node_modules/apollo-cache-inmemory/lib/bundle.umd.js:426:36)
at StoreReader.readQueryFromStore (/Users/dyoung/workspace//appsync_javascript_test/node_modules/apollo-cache-inmemory/lib/bundle.umd.js:401:25)
at processOfflineQuery (/Users/dyoung/workspace//appsync_javascript_test/node_modules/aws-appsync/lib/link/offline-link.js:154:34)
at /Users/dyoung/workspace//appsync_javascript_test/node_modules/aws-appsync/lib/link/offline-link.js:110:28
at new Subscription (/Users/dyoung/workspace//appsync_javascript_test/node_modules/zen-observable/lib/Observable.js:183:34)
at Observable.subscribe (/Users/dyoung/workspace//appsync_javascript_test/node_modules/zen-observable/lib/Observable.js:262:14)
at /Users/dyoung/workspace//appsync_javascript_test/node_modules/aws-appsync/lib/client.js:182:67,
message: "Network error: Can't find field getTickets on object undefined.",
extraInfo: undefined
}
According Apollo Docs on local state management:
We need to write an initial state to the cache before the query is run
to prevent it from erroring out.
I had the almost same error thrown out when I did not initialise the state in the Apollo's InMemoryCache.
To initialize the state for AWSAppSyncClient, you can refer to React-Native + Apollo-Link-State + AWS Appsync : Defaults are not stored in cache, and https://github.com/awslabs/aws-mobile-appsync-sdk-js/pull/96

Resources