In the process of updating serverless from V2 to V3 and have run into this issue where I am getting the following error
Runtime.ImportModuleError: Error: Cannot find module 'get-fixtures-today'
My serverless.yml looks like
functions:
get-fixtures-today:
handler: src/get-fixtures/today/get-fixtures-today.run
name: get-fixtures-today
description: Get fixtures from api for today's date
package:
patterns:
- '!src/**'
- src/get-fixtures/today/**'
- src/helpers/get-fixtures.js
- src/helpers/utilities.js
- src/helpers/upcoming-fixtures.js
- src/sql/queries.js
- src/sql/query_template.js
- src/test/**
- src/config/**
get-fixtures-today
module.exports.run = async (event, context) => {
try {
await....
} catch (error) {
console.log(`Error');
}
};
Am i potentially missing something obvious?
Related
I'm trying to find a way to run Next.js (v13.0.6) with OG image generation logic (using #vercel/og) in AWS Lambda
Everything works fine locally (in dev and prod mode) but when I try execute lambda handler getting "statusCode": 500,
It only fails for apis that involve ImageResponse (and runtime: 'experimental-edge' as a requirement for #vercel/og)
I'm pretty sure the problem is caused by Edge Runtime is not being configured correctly
There is my handler code
next build with next.config.js output: 'standalone' creates folder .next/standalone
insde standalone handler.js
const { parse } = require('url');
const NextServer = require('next/dist/server/next-server').default
const serverless = require('serverless-http');
const path = require('path');
process.env.NODE_ENV = 'production'
process.chdir(__dirname)
const currentPort = parseInt(process.env.PORT, 10) || 3000
const nextServer = new NextServer({
hostname: 'localhost',
port: currentPort,
dir: path.join(__dirname),
dev: false,
customServer: false,
conf: {...} // copied from `server.js` in there same folder
});
const requestHandler = nextServer.getRequestHandler();
// this is a AWS lambda handler that converts lambda event
// to http request that next server can process
const handler = serverless(async (req, res) => {
// const parsedUrl = parse(req.url, true);
try {
await requestHandler(req, res);
}catch(err){
console.error(err);
res.statusCode = 500
res.end('internal server error')
}
});
module.exports = {
handler
}
testing it locally with local-lambda, but getting similar results when test against AWS deployed lambda
what is confusing is that server.js (in .next/standalone) has a similar setup, it only involves http server on top of of it
update:
aws lambda logs show
ERROR [Error [CompileError]: WebAssembly.compile(): Compiling function #64 failed: invalid value type 'Simd128', enable with --experimental-wasm-simd #+3457 ]
update 2:
the first error was fixed by selecting Node 16 for AWS lambda, now getting this error
{
"errorType": "Error",
"errorMessage": "write after end",
"trace": [
"Error [ERR_STREAM_WRITE_AFTER_END]: write after end",
" at new NodeError (node:internal/errors:372:5)",
" at ServerlessResponse.end (node:_http_outgoing:846:15)",
" at ServerlessResponse.end (/var/task/node_modules/next/dist/compiled/compression/index.js:22:783)",
" at NodeNextResponse.send (/var/task/node_modules/next/dist/server/base-http/node.js:93:19)",
" at NextNodeServer.handleRequest (/var/task/node_modules/next/dist/server/base-server.js:332:47)",
" at processTicksAndRejections (node:internal/process/task_queues:96:5)",
" at async /var/task/index.js:34:5"
]
}
At the moment of writing Vercel's runtime: 'experimental-edge' seems to be unstable (run into multiple issues with it)
I ended up recreating #vercel/og lib without wasm and next.js dependencies, can be found here
and simply use it in AWS lambda. It depends on #resvg/resvg-js instead of wasm version, which uses binaries, so there should not be much perf loss comparing to wasm
I have the following code:
const func = new NodejsFunction(this, <function name>, {
memorySize: 2048,
timeout: Duration.seconds(60),
runtime: Runtime.NODEJS_14_X,
handler: 'handler',
role: <role>,
entry: path.join(__dirname, <filePath>),
currentVersionOptions: {
description: `Version created on ${new Date(Date.now())}`,
},
});
const version = func.currentVersion;
const alias = new Alias(this, 'VersionAlias', {
aliasName: 'current',
version,
});
I do this with a handful of Lambda functions all in the same stack. The first deployment works, however the lambda functions are created with random version numbers (some have v4, some with v5, some with v7).
Subsequent deployments then fail with a vague Internal Failure error message. So I check the CloudTrail logs and find a series of ResourceNotFoundException errors. The "Version" resources are unable to be updated because they have the incorrect version number stemming from the first deploy. How can I force CloudFormation to start at #1 for versioning my lambda functions?
For anyone visiting this later, the problem was with the following code:
currentVersionOptions: {
description: `Version created on ${new Date(Date.now())}`,
},
Apparently you can't have a dynamic description as it is an immutable field
Description of the problem
I have created a Lambda function with API Gateway in SAM, then deployed it and it was working as expected. In API Gateway I used HttpApi not REST API.
Then, I wanted to add a Lambda authorizer with Simple Response. So, I followed the SAM and API Gateway docs and I came up with the code below.
When I call the route items-list it now returns 401 Unauthorized, which is expected.
However, when I add the header myappauth with the value "test-token-abc", I get a 500 Internal Server Error.
I checked this page but it seems all of the steps listed there are OK https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-http-lambda-integrations/
I enabled logging for the API Gateway, following these instructions: https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-logging.html
But all I get is something like this (redacted my IP and request ID):
[MY-IP] - - [07/Jul/2021:08:24:06 +0000] "GET GET /items-list/{userNumber} HTTP/1.1" 500 35 [REQUEST-ID]
(Perhaps I can configure the logger in such a way that it prints a more meaningful error message? EDIT: I've tried adding $context.authorizer.error to the logs, but it doesn't print any specific error message, just prints a dash: -)
I also checked the logs for the Lambda functions, there is nothing there (all logs where from the time before I added the authorizer).
So, what am I doing wrong?
What I tried:
This is my Lambda Authorizer function which I have deployed using sam deploy, when I test it in isolation using an event with the myappauth header, it works:
exports.authorizer = async (event) => {
let response = {
"isAuthorized": false,
};
if (event.headers.myappauth === "test-token-abc") {
response = {
"isAuthorized": true,
};
}
return response;
};
and this is the SAM template.yml which I deployed using sam deploy:
AWSTemplateFormatVersion: 2010-09-09
Description: >-
myapp-v1
Transform:
- AWS::Serverless-2016-10-31
Globals:
Function:
Runtime: nodejs14.x
MemorySize: 128
Timeout: 100
Environment:
Variables:
MYAPP_TOKEN: "test-token-abc"
Resources:
MyAppAPi:
Type: AWS::Serverless::HttpApi
Properties:
FailOnWarnings: true
Auth:
Authorizers:
MyAppLambdaAuthorizer:
AuthorizerPayloadFormatVersion: "2.0"
EnableSimpleResponses: true
FunctionArn: !GetAtt authorizerFunction.Arn
FunctionInvokeRole: !GetAtt authorizerFunctionRole.Arn
Identity:
Headers:
- myappauth
DefaultAuthorizer: MyAppLambdaAuthorizer
itemsListFunction:
Type: AWS::Serverless::Function
Properties:
Handler: src/handlers/v1-handlers.itemsList
Description: A Lambda function that returns a list of items.
Policies:
- AWSLambdaBasicExecutionRole
Events:
Api:
Type: HttpApi
Properties:
Path: /items-list/{userNumber}
Method: get
ApiId: MyAppAPi
authorizerFunction:
Type: AWS::Serverless::Function
Properties:
Handler: src/handlers/v1-handlers.authorizer
Description: A Lambda function that authorizes requests.
Policies:
- AWSLambdaBasicExecutionRole
Edit:
User #petey suggested that I tried returning an IAM policy in my authorizer function, so I changed EnableSimpleResponses to false in the template.yml, then I changed my function as below, but got the same result:
exports.authorizer = async (event) => {
let response = {
"principalId": "my-user",
"policyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Action": "execute-api:Invoke",
"Effect": "Deny",
"Resource": event.routeArn
}]
}
};
if (event.headers.myappauth == "test-token-abc") {
response = {
"principalId": "my-user",
"policyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Action": "execute-api:Invoke",
"Effect": "Allow",
"Resource": event.routeArn
}]
}
};
}
return response;
};
I am going to answer my own question because I have resolved the issue, and I hope this will help people who are going to use the new "HTTP API" format in API Gateway, since there is not a lot of tutorials out there yet; most examples you will find online are for the older API Gateway standard, which Amazon calls "REST API". (If you want to know the difference between the two, see here).
The main problem lies in the example that is presented in the official documentation. They have:
MyLambdaRequestAuthorizer:
FunctionArn: !GetAtt MyAuthFunction.Arn
FunctionInvokeRole: !GetAtt MyAuthFunctionRole.Arn
The problem with this, is that this template will create a new Role called MyAuthFunctionRole but that role will not have all the necessary policies attached to it!
The crucial part that I missed in the official docs is this paragraph:
You must grant API Gateway permission to invoke the Lambda function by using either the function's resource policy or an IAM role. For this example, we update the resource policy for the function so that it grants API Gateway permission to invoke our Lambda function.
The following command grants API Gateway permission to invoke your Lambda function. If API Gateway doesn't have permission to invoke your function, clients receive a 500 Internal Server Error.
The best way to solve this, is to actually include the Role definition in the SAM template.yml, under Resources:
MyAuthFunctionRole
Type: AWS::IAM::Role
Properties:
# [... other properties...]
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- apigateway.amazonaws.com
Action:
- 'sts:AssumeRole'
Policies:
# here you will put the InvokeFunction policy, for example:
- PolicyName: MyPolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action: 'lambda:InvokeFunction'
Resource: !GetAtt MyAuthFunction.Arn
You can see here a description about the various Properties for a role: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-role.html
Another way to solve this, is to separately create a new policy in AWS Console, which has InvokeFunction permission, and then after deployment, attach that policy to the MyAuthFunctionRole that SAM created. Now the Authorizer will be working as expected.
Another strategy would be to create a new role beforehand, that has a policy with InvokeFunction permission, then copy and paste the arn of that role in the SAM template.yml:
MyLambdaRequestAuthorizer:
FunctionArn: !GetAtt MyAuthFunction.Arn
FunctionInvokeRole: arn:aws:iam::[...]
Your lambda authorizer is not returning what is expected to be an actual lambda authorizer (an IAM policy). This could explain that internal error 500.
To fix, replace is with something like this that returns an IAM policy (or rejects):
// A simple token-based authorizer example to demonstrate how to use an authorization token
// to allow or deny a request. In this example, the caller named 'user' is allowed to invoke
// a request if the client-supplied token value is 'allow'. The caller is not allowed to invoke
// the request if the token value is 'deny'. If the token value is 'unauthorized' or an empty
// string, the authorizer function returns an HTTP 401 status code. For any other token value,
// the authorizer returns an HTTP 500 status code.
// Note that token values are case-sensitive.
exports.handler = function(event, context, callback) {
var token = event.authorizationToken;
// modify switch statement here to your needs
switch (token) {
case 'allow':
callback(null, generatePolicy('user', 'Allow', event.methodArn));
break;
case 'deny':
callback(null, generatePolicy('user', 'Deny', event.methodArn));
break;
case 'unauthorized':
callback("Unauthorized"); // Return a 401 Unauthorized response
break;
default:
callback("Error: Invalid token"); // Return a 500 Invalid token response
}
};
// Help function to generate an IAM policy
var generatePolicy = function(principalId, effect, resource) {
var authResponse = {};
authResponse.principalId = principalId;
if (effect && resource) {
var policyDocument = {};
policyDocument.Version = '2012-10-17';
policyDocument.Statement = [];
var statementOne = {};
statementOne.Action = 'execute-api:Invoke';
statementOne.Effect = effect;
statementOne.Resource = resource;
policyDocument.Statement[0] = statementOne;
authResponse.policyDocument = policyDocument;
}
// Optional output with custom properties of the String, Number or Boolean type.
authResponse.context = {
"stringKey": "stringval",
"numberKey": 123,
"booleanKey": true
};
return authResponse;
}
Lots more information here : https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html#api-gateway-lambda-authorizer-lambda-function-create
Just to complete the answer. You have to add an AssumeRolePolicyDocument under Properties.
The role will then state
MyAuthFunctionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- apigateway.amazonaws.com
Action:
- 'sts:AssumeRole'
Policies:
# see answer above
I'm trying to update a cloud formation template with a few lambda functions in it. The last version of the template was deployed a few years ago, and all the lambda functions currently have a runtime of node6.10.
I have updated the runtime for all functions to node10.x, but when I deploy the template, I get the following message:
The runtime parameter of nodejs6.10 is no longer supported for creating or updating AWS Lambda functions
I've created a change set, and reviewed it, and it includes an update to the runtime property for each lambda function, however Cloud Formation seems to be ignoring it.
Is there something I'm missing?
Context:
I assume that you encountered this issue - (you got "nodejs version not supported error" message after you tried to amplify push followed by amplify add auth)
Go to amplify -> backend -> auth -> cognito -> click cognito cloudformation ->
search for "Runtime: node"
change it to "Runtime: nodejs8.10" - whatever latest recommended in error message
re-run
$ amplify push
Unfortunately, I found I had to update the runtime of all functions in a template outside of Cloud Formation, to get the stacks to deploy. I used this script:
const AWS = require('aws-sdk')
const lambda = new AWS.Lambda(...)
main().catch(err => {
console.error(err)
process.exit(1)
})
async function main() {
const functions = await getFunctions()
await Promise.all(
functions
// filter only functions you want to update
.filter(...)
.filter(x => x.Runtime !== 'nodejs10.x')
.map(updateFunction)
)
}
async function updateFunction(func) {
await lambda
.updateFunctionConfiguration({
FunctionName: func.FunctionName,
Runtime: 'nodejs10.x'
})
.promise()
console.log(`function updated: ${func.FunctionName}`)
}
async function getFunctions() {
let marker
let functions = []
do {
const result = await lambda
.listFunctions({
Marker: marker
})
.promise()
functions = [...functions, ...result.Functions]
marker = result.NextMarker
} while (marker)
return functions
}
I want to reuse my serverless.yml in different environments (dev, test, prod).
In the config I have:
provider:
name: aws
stage: ${opt:stage, 'dev'}
environment:
NODE_ENV: ${self:provider.stage}
Right now the value will be dev, test or prod (all in lower-case).
Is there a way to convert it toUpperCase() in a way that the input and self:provider:stage will stay as it is (i.e. lower-case) but the value of NODE_ENV will be UPPER-CASE?
Update (2022-10-13)
This answer was correct at the time of its writing (circa 2018). A better answer now is to use serverless-plugin-utils as stated in #ShashankRaj's comment below.
varName: ${upper(value)}
AFAIK, there is no such function in YAML.
You can achieve what you want though by using a map between the lowercase and uppercase names.
custom:
environments:
dev: DEV
test: TEST
prod: PROD
provider:
name: aws
stage: ${opt:stage, 'dev'}
environment:
NODE_ENV: ${self:custom.environments.${self:provider.stage}}
You can achieve something to this effect using the reference variables in javascript files functionality provided.
To take your example, this should work (assuming you're running in a node.js environment that supports modern syntax)
serverless.yml
...
provider:
name: aws
stage: ${opt:stage, 'dev'}
environment:
NODE_ENV: ${file(./yml-helpers.js):provider.stage.uppercase}
...
yml-helpers.js (adjacent to serverless.yml)
module.exports.provider = serverless => {
// The `serverless` argument containers all the information in the .yml file
const provider = serverless.service.provider;
return Object.entries(provider).reduce(
(accumulator, [key, value]) => ({
...accumulator,
[key]:
typeof value === 'string'
? {
lowercase: value.toLowerCase(),
uppercase: value.toUpperCase()
}
: value
}),
{}
)
};
I arrived at something that works, via reading some source code and console logging the entire serverless object. This example applies a helper function to title-case some input option values (apply str.toUpperCase() instead, as required). There is a result of parsing the input options already available in the serverless object.
// serverless-helpers.js
function toTitleCase(word) {
console.log("input word: " + word);
let lower = word.toLowerCase();
let title = lower.replace(lower[0], lower[0].toUpperCase());
console.log("output word: " + title);
return title;
}
module.exports.dynamic = function(serverless) {
// The `serverless` argument contains all the information in
// the serverless.yaml file
// serverless.cli.consoleLog('Use Serverless config and methods as well!');
// this is useful for discovery of what is available:
// serverless.cli.consoleLog(serverless);
const input_options = serverless.processedInput.options;
return {
part1Title: toTitleCase(input_options.part1),
part2Title: toTitleCase(input_options.part2)
};
};
# serverless.yaml snippet
custom:
part1: ${opt:part1}
part2: ${opt:part2}
dynamicOpts: ${file(./serverless-helpers.js):dynamic}
combined: prefix${self:custom.dynamicOpts.part1Title}${self:custom.dynamicOpts.part2Title}Suffix
This simple example assumes the input options are --part1={value} and --part2={value}, but the generalization is to traverse the properties of serverless.processedInput.options and apply any custom helpers to those values.
Using Serverless Plugin Utils:
plugins:
- serverless-plugin-utils
provider:
name: aws
stage: ${opt:stage, 'dev'}
environment:
NODE_ENV: ${upper(${self:provider.stage})}
Thanks to #ShashankRaj...