Lambda Layers not installing with Serverless - aws-lambda

Currently getting the following error with MongoDB:
no saslprep library specified. Passwords will not be sanitized
We are using Webpack so simply installing the module doesn't work (Webpack just ignores it). I found this thread which talks about how to exclude it from Webpack compilations, but then I have to manually load it into every Lambda function which led me to Lambda Layers.
Following the Serverless guide on using Lambda layers allowed me to get my layer published to AWS and included in all of my functions, but for some reason, it doesn't install the modules. If I download the layer using the AWS GUI, I get a folder with just the package.json and package-lock.json files.
My file structure is:
my-project
|_ layers
|_ saslprep
|_ package.json
and my serverless.yml is:
layers:
saslprep:
path: layers/saslprep
compatibleRuntimes:
- nodejs14.x

This is not my preferred solution as I'd like to use 256, but the way I got around this error/warning was by changing the authMechanism from SCRAM-SHA-256 to SCRAM-SHA-1 in the connection string. The serverless-bundle most likely needs to add this dependency into their package to enable support for Mongo 4.0 SHA256 (my best guess!).
You can specify this authentication mechanism by setting the authMechanism parameter to the value SCRAM-SHA-1 in the connection string as shown in the following sample code.
const { MongoClient } = require("mongodb");
// Replace the following with values for your environment.
const username = encodeURIComponent("<username>");
const password = encodeURIComponent("<password>");
const clusterUrl = "<MongoDB cluster url>";
const authMechanism = "SCRAM-SHA-1";
// Replace the following with your MongoDB deployment's connection string.
const uri =
`mongodb+srv://${username}:${password}#${clusterUrl}/?authMechanism=${authMechanism}`;
// Create a new MongoClient
const client = new MongoClient(uri);
// Function to connect to the server
async function run() {
try {
// Connect the client to the server
await client.connect();
// Establish and verify connection
await client.db("admin").command({ ping: 1 });
console.log("Connected successfully to server");
} finally {
// Ensures that the client will close when you finish/error
await client.close();
}
}
run().catch(console.dir);

Related

How to run nextjs in AWS lambda with `experimental-edge` runtime

I'm trying to find a way to run Next.js (v13.0.6) with OG image generation logic (using #vercel/og) in AWS Lambda
Everything works fine locally (in dev and prod mode) but when I try execute lambda handler getting "statusCode": 500,
It only fails for apis that involve ImageResponse (and runtime: 'experimental-edge' as a requirement for #vercel/og)
I'm pretty sure the problem is caused by Edge Runtime is not being configured correctly
There is my handler code
next build with next.config.js output: 'standalone' creates folder .next/standalone
insde standalone handler.js
const { parse } = require('url');
const NextServer = require('next/dist/server/next-server').default
const serverless = require('serverless-http');
const path = require('path');
process.env.NODE_ENV = 'production'
process.chdir(__dirname)
const currentPort = parseInt(process.env.PORT, 10) || 3000
const nextServer = new NextServer({
hostname: 'localhost',
port: currentPort,
dir: path.join(__dirname),
dev: false,
customServer: false,
conf: {...} // copied from `server.js` in there same folder
});
const requestHandler = nextServer.getRequestHandler();
// this is a AWS lambda handler that converts lambda event
// to http request that next server can process
const handler = serverless(async (req, res) => {
// const parsedUrl = parse(req.url, true);
try {
await requestHandler(req, res);
}catch(err){
console.error(err);
res.statusCode = 500
res.end('internal server error')
}
});
module.exports = {
handler
}
testing it locally with local-lambda, but getting similar results when test against AWS deployed lambda
what is confusing is that server.js (in .next/standalone) has a similar setup, it only involves http server on top of of it
update:
aws lambda logs show
ERROR [Error [CompileError]: WebAssembly.compile(): Compiling function #64 failed: invalid value type 'Simd128', enable with --experimental-wasm-simd #+3457 ]
update 2:
the first error was fixed by selecting Node 16 for AWS lambda, now getting this error
{
"errorType": "Error",
"errorMessage": "write after end",
"trace": [
"Error [ERR_STREAM_WRITE_AFTER_END]: write after end",
" at new NodeError (node:internal/errors:372:5)",
" at ServerlessResponse.end (node:_http_outgoing:846:15)",
" at ServerlessResponse.end (/var/task/node_modules/next/dist/compiled/compression/index.js:22:783)",
" at NodeNextResponse.send (/var/task/node_modules/next/dist/server/base-http/node.js:93:19)",
" at NextNodeServer.handleRequest (/var/task/node_modules/next/dist/server/base-server.js:332:47)",
" at processTicksAndRejections (node:internal/process/task_queues:96:5)",
" at async /var/task/index.js:34:5"
]
}
At the moment of writing Vercel's runtime: 'experimental-edge' seems to be unstable (run into multiple issues with it)
I ended up recreating #vercel/og lib without wasm and next.js dependencies, can be found here
and simply use it in AWS lambda. It depends on #resvg/resvg-js instead of wasm version, which uses binaries, so there should not be much perf loss comparing to wasm

"Unsupported Media Type" using serverless offline

I'm working on a small serverless offline assignment and I got error Unsupported Media Type when tried to invoke one lambda function in another.
I found a solution but when I tried to applied to my project was not working:
here in the link all the details. cloud anyone help me with that
https://github.com/dherault/serverless-offline/issues/1005#issue-632401297
there are three possible solutions.
Make sure that the lambda_A have the same port and host where the lambda_B is running.
Lambda_A:
const { Lambda } = require('aws-sdk');
const lambda = new Lambda({
region: 'us-east-1',
endpoint: 'http://localhost:3000',
});
module.exports.main = async (event, context) => {
// invoke
}
Lambda_B: Is running on http://localhost:3000
You have configured out serverless-offline in twice functions.
https://www.serverless.com/plugins/serverless-offline#usage-with-invoke
Lambda_A or Lambda_B have correctly stage?. Remember to use sls offline --stage local in both functions.

Examples of integrating moleculer-io with moleculer-web using moleculer-runner instead of ServiceBroker?

I am having fun with using moleculer-runner instead of creating a ServiceBroker instance in a moleculer-web project I am working on. The Runner simplifies setting up services for moleculer-web, and all the services - including the api.service.js file - look and behave the same, using a module.exports = { blah } format.
I can cleanly define the REST endpoints in the api.service.js file, and create the connected functions in the appropriate service files. For example aliases: { 'GET sensors': 'sensors.list' } points to the list() action/function in sensors.service.js . It all works great using some dummy data in an array.
The next step is to get the service(s) to open up a socket and talk to a local program listening on an internal set address/port. The idea is to accept a REST call from the web, talk to a local program over a socket to get some data, then format and return the data back via REST to the client.
BUT When I want to use sockets with moleculer, I'm having trouble finding useful info and examples on integrating moleculer-io with a moleculer-runner-based setup. All the examples I find use the ServiceBroker model. I thought my Google-Fu was pretty good, but I'm at a loss as to where to look to next. Or, can i modify the ServiceBroker examples to work with moleculer-runner? Any insight or input is welcome.
If you want the following chain:
localhost:3000/sensor/list -> sensor.list() -> send message to local program:8071 -> get response -> send response as return message to the REST caller.
Then you need to add a socket io client to your sensor service (which has the list() action). Adding a client will allow it to communicate with "outside world" via sockets.
Check the image below. I think it has everything that you need.
As a skeleton I've used moleculer-demo project.
What I have:
API service api.service.js. That handles the HTTP requests and passes them to the sensor.service.js
The sensor.service.js will be responsible for communicating with remote socket.io server so it needs to have a socket.io client. Now, when the sensor.service.js service has started() I'm establishing a connection with a remote server located at port 8071. After this I can use this connection in my service actions to communicate with socket.io server. This is exactly what I'm doing in sensor.list action.
I've also created remote-server.service.js to mock your socket.io server. Despite being a moleculer service, the sensor.service.js communicates with it via socket.io protocol.
It doesn't matter if your services use (or not) socket.io. All the services are declared in the same way, i.e., module.exports = {}
Below is a working example with socket.io.
const { ServiceBroker } = require("moleculer");
const ApiGateway = require("moleculer-web");
const SocketIOService = require("moleculer-io");
const io = require("socket.io-client");
const IOService = {
name: "api",
// SocketIOService should be after moleculer-web
// Load the HTTP API Gateway to be able to reach "greeter" action via:
// http://localhost:3000/hello/greeter
mixins: [ApiGateway, SocketIOService]
};
const HelloService = {
name: "hello",
actions: {
greeter() {
return "Hello Via Socket";
}
}
};
const broker = new ServiceBroker();
broker.createService(IOService);
broker.createService(HelloService);
broker.start().then(async () => {
const socket = io("http://localhost:3000", {
reconnectionDelay: 300,
reconnectionDelayMax: 300
});
socket.on("connect", () => {
console.log("Connection with the Gateway established");
});
socket.emit("call", "hello.greeter", (error, res) => {
console.log(res);
});
});
To make it work with moleculer-runner just copy the service declarations into my-service.service.js. So for example, your api.service.js could look like:
// api.service.js
module.exports = {
name: "api",
// SocketIOService should be after moleculer-web
// Load the HTTP API Gateway to be able to reach "greeter" action via:
// http://localhost:3000/hello/greeter
mixins: [ApiGateway, SocketIOService]
}
and your greeter service:
// greeter.service.js
module.exports = {
name: "hello",
actions: {
greeter() {
return "Hello Via Socket";
}
}
}
And run npm run dev or moleculer-runner --repl --hot services

CloudFormation: The runtime parameter of nodejs6.10 is no longer supported for creating or updating AWS Lambda functions

I'm trying to update a cloud formation template with a few lambda functions in it. The last version of the template was deployed a few years ago, and all the lambda functions currently have a runtime of node6.10.
I have updated the runtime for all functions to node10.x, but when I deploy the template, I get the following message:
The runtime parameter of nodejs6.10 is no longer supported for creating or updating AWS Lambda functions
I've created a change set, and reviewed it, and it includes an update to the runtime property for each lambda function, however Cloud Formation seems to be ignoring it.
Is there something I'm missing?
Context:
I assume that you encountered this issue - (you got "nodejs version not supported error" message after you tried to amplify push followed by amplify add auth)
Go to amplify -> backend -> auth -> cognito -> click cognito cloudformation ->
search for "Runtime: node"
change it to "Runtime: nodejs8.10" - whatever latest recommended in error message
re-run
$ amplify push
Unfortunately, I found I had to update the runtime of all functions in a template outside of Cloud Formation, to get the stacks to deploy. I used this script:
const AWS = require('aws-sdk')
const lambda = new AWS.Lambda(...)
main().catch(err => {
console.error(err)
process.exit(1)
})
async function main() {
const functions = await getFunctions()
await Promise.all(
functions
// filter only functions you want to update
.filter(...)
.filter(x => x.Runtime !== 'nodejs10.x')
.map(updateFunction)
)
}
async function updateFunction(func) {
await lambda
.updateFunctionConfiguration({
FunctionName: func.FunctionName,
Runtime: 'nodejs10.x'
})
.promise()
console.log(`function updated: ${func.FunctionName}`)
}
async function getFunctions() {
let marker
let functions = []
do {
const result = await lambda
.listFunctions({
Marker: marker
})
.promise()
functions = [...functions, ...result.Functions]
marker = result.NextMarker
} while (marker)
return functions
}

How to capture the transactions while doing testing using Mocha

I am in the process of writing unit/behavioural tests using Mocha for a particular blockchain network use-case. Based on what I can see, these tests are not hitting the actual fabric, in other words, they seem to be running in some kind of a simulated environment. I don't get to see any of the transactions that took place as a part of the test. Can someone please tell me if it is somehow possible to capture the transactions that take place as part of the Mocha tests?
Initial portion of my code below:
describe('A Network', () => {
// In-memory card store for testing so cards are not persisted to the file system
const cardStore = require('composer-common').NetworkCardStoreManager.getCardStore( { type: 'composer-wallet-inmemory' } );
let adminConnection;
let businessNetworkConnection;
let businessNetworkDefinition;
let businessNetworkName;
let factory;
//let clock;
// Embedded connection used for local testing
const connectionProfile = {
name: 'hlfv1',
'x-type': 'hlfv1',
'version': '1.0.0'
};
before(async () => {
// Generate certificates for use with the embedded connection
const credentials = CertificateUtil.generate({ commonName: 'admin' });
// PeerAdmin identity used with the admin connection to deploy business networks
const deployerMetadata = {
version: 1,
userName: 'PeerAdmin',
roles: [ 'PeerAdmin', 'ChannelAdmin' ]
};
const deployerCard = new IdCard(deployerMetadata, connectionProfile);
console.log("line 63")
const deployerCardName = 'PeerAdmin';
deployerCard.setCredentials(credentials);
console.log("line 65")
// setup admin connection
adminConnection = new AdminConnection({ cardStore: cardStore });
console.log("line 69")
await adminConnection.importCard(deployerCardName, deployerCard);
console.log("line 70")
await adminConnection.connect(deployerCardName);
console.log("line 71")
});
Earlier, my connection profile was using the embedded mode, which I changed to hlfv1 after looking at the answer below. Now, I am getting the error: Error: the string "Failed to import identity. Error: Client.createUser parameter 'opts mspid' is required." was thrown, throw an Error :). This is coming from
await adminConnection.importCard(deployerCardName, deployerCard);. Can someone please tell me what needs to be changed. Any documentation/resource will be helpful.
Yes you can use a real Fabric. Which means you could interact with the created transactions using your test framework or indeed other means such as REST or Playground etc.
In Composer's own test setup, the option for testing against an hlfv1 Fabric environment is used in its setup (ie whether you want to use embedded, web or real Fabric) -> see https://github.com/hyperledger/composer/blob/master/packages/composer-tests-functional/systest/historian.js#L120
Setup is captured here
https://github.com/hyperledger/composer/blob/master/packages/composer-tests-functional/systest/testutil.js#L192
Example of setting up artifacts that you would need to setup to use a real Fabric here
https://github.com/hyperledger/composer/blob/master/packages/composer-tests-functional/systest/testutil.js#L247
Also see this blog for more guidelines -> https://medium.com/#mrsimonstone/debug-your-blockchain-business-network-using-hyperledger-composer-9bea20b49a74

Resources