i want to publish a mqtt message from AWS lambda function, i tried this
`exports.handler = function(event, context) {
// TODO implement
context.done(null, 'Hello from Lambda');
const mqtt = require('mqtt');
const client = mqtt.connect('mqtt://broker.hivemq.com');
var state = 'closed';
client.on('connect', () => {
console.log("connected to broker");
// Inform controllers that garage is connected
client.publish('garage/connected', 'true')
})
};
`
You need to create a zip file with the node_modules directory and the source file for your lambda.
e.g.
lamdba.js
node_modules/mqtt
node_modules/mqtt/package.json
...
You should use npm to install the package locally.
The details are described in the Lambda documentation here
Related
I want to test the creation of an application that uses Node.js and Memgraph. What is the fastest and easiest way for me to test this type of setup?
The easiest way is to use Express.js. Here are the exact steps:
Create a new directory for your application, /MyApp and position yourself in it.
Create a package.json file using npm init
Install Express.js using npm install express --save and the Bolt driver using npm install neo4j-driver --save in the /MyApp directory. Both packages will be added to the dependencies list.
To make the actual program, create a program.js file with the following code:
const express = require("express");
const app = express();
const port = 3000;
var neo4j = require("neo4j-driver");
app.get("/", async (req, res) => {
const driver = neo4j.driver("bolt://localhost:7687");
const session = driver.session();
try {
const result = await session.writeTransaction((tx) =>
tx.run(
'CREATE (a:Greeting) SET a.message = $message RETURN "Node " + id(a) + ": " + a.message',
{
message: "Hello, World!",
}
)
);
const singleRecord = result.records[0];
const greeting = singleRecord.get(0);
console.log(greeting);
} finally {
await session.close();
}
// on application exit:
await driver.close();
});
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`);
});
Once you save the file, you can run your program using node program.js.
For additional details check the official Memgraph documentation.
I am trying to create a Lambda in AWS that serves as a producer to an MSK topic. All the AWS docs say to create a new EC2 instance, but as my Lambda is in the same VPC I feel like this should work. I am very new to this and I notice my log statement never hits in my producer.on function.
I am using nodejs and the kafka-node module. The code can be found below.
Essentially, I am just wondering if anyone knows how to do this and why the producer.on function is never hit when I run test through the Lambda? This is just some test code to see if I can get it to send, but if any more data is needed oy help please let me know and thanks in advance.
exports.handler = async (event, context,callback) => {
const kafka = require('kafka-node');
const bp = require('body-parser');
const kafka_topic = 'MyTopic';
const Producer = kafka.Producer;
var KeyedMessage = kafka.KeyedMessage;
const Client = kafka.Client;
const client = new kafka.KafkaClient({kafkaHost: 'myhost:9094'});
console.log('client :: '+JSON.stringify(client));
const producer = new Producer(client);
console.log('about to hit producer code');
producer.on('ready', function() {
console.log('Hello there!')
let message = 'my message';
let keyedMessage = new KeyedMessage('keyed', 'me keyed message');
producer.send([
{ topic: kafka_topic, partition: 0, messages: [message, keyedMessage], attributes: 0 }
], function (err, result) {
console.log(err || result);
process.exit();
});
});
producer.on('error', function (err) {
console.log('error', err);
});
}
return "success";
What you need is to be able to produce messages on your MSK cluster using REST API. Why not setup a REST proxy for MSK as detailed here and then call this API to produce your messages to MSK.
I'm trying to deploy an S3 static website and API gateway/lambda in a single stack.
The javascript in the S3 static site calls the lambda to populate an HTML list but it needs to know the API Gateway URL for the lambda integration.
Currently, I generate a RestApi like so...
const handler = new lambda.Function(this, "TestHandler", {
runtime: lambda.Runtime.NODEJS_10_X,
code: lambda.Code.asset("build/test-service"),
handler: "index.handler",
environment: {
}
});
this.api = new apigateway.RestApi(this, "test-api", {
restApiName: "Test Service"
});
const getIntegration = new apigateway.LambdaIntegration(handler, {
requestTemplates: { "application/json": '{ "statusCode": "200" }' }
});
const apiUrl = this.api.url;
But on cdk deploy, apiUrl =
"https://${Token[TOKEN.39]}.execute-api.${Token[AWS::Region.4]}.${Token[AWS::URLSuffix.1]}/${Token[TOKEN.45]}/"
So the url is not parsed/generated until after the static site requires the value.
How can I calculate/find/fetch the API Gateway URL and update the javascript on cdk deploy?
Or is there a better way to do this? i.e. is there a graceful way for the static javascript to retrieve a lambda api gateway url?
Thanks.
You are creating a LambdaIntegration but it isn't connected to your API.
To add it to the root of the API do: this.api.root.addMethod(...) and use this to connect your LambdaIntegration and API.
This should give you an endpoint with a URL
If you are using the s3-deployment module to deploy your website as well, I was able to hack together a solution using what is available currently (pending a better solution at https://github.com/aws/aws-cdk/issues/12903). The following together allow for you to deploy a config.js to your bucket (containing attributes from your stack that will only be populated at deploy time) that you can then depend on elsewhere in your code at runtime.
In inline-source.ts:
// imports removed for brevity
export function inlineSource(path: string, content: string, options?: AssetOptions): ISource {
return {
bind: (scope: Construct, context?: DeploymentSourceContext): SourceConfig => {
if (!context) {
throw new Error('To use a inlineSource, context must be provided');
}
// Find available ID
let id = 1;
while (scope.node.tryFindChild(`InlineSource${id}`)) {
id++;
}
const bucket = new Bucket(scope, `InlineSource${id}StagingBucket`, {
removalPolicy: RemovalPolicy.DESTROY
});
const fn = new Function(scope, `InlineSource${id}Lambda`, {
runtime: Runtime.NODEJS_12_X,
handler: 'index.handler',
code: Code.fromAsset('./inline-lambda')
});
bucket.grantReadWrite(fn);
const myProvider = new Provider(scope, `InlineSource${id}Provider`, {
onEventHandler: fn,
logRetention: RetentionDays.ONE_DAY // default is INFINITE
});
const resource = new CustomResource(scope, `InlineSource${id}CustomResource`, { serviceToken: myProvider.serviceToken, properties: { bucket: bucket.bucketName, path, content } });
context.handlerRole.node.addDependency(resource); // Sets the s3 deployment to depend on the deployed file
bucket.grantRead(context.handlerRole);
return {
bucket: bucket,
zipObjectKey: 'index.zip'
};
},
};
}
In inline-lambda/index.js (also requires archiver installed into inline-lambda/node_modules):
const aws = require('aws-sdk');
const s3 = new aws.S3({ apiVersion: '2006-03-01' });
const fs = require('fs');
var archive = require('archiver')('zip');
exports.handler = async function(event, ctx) {
await new Promise(resolve => fs.unlink('/tmp/index.zip', resolve));
const output = fs.createWriteStream('/tmp/index.zip');
const closed = new Promise((resolve, reject) => {
output.on('close', resolve);
output.on('error', reject);
});
archive.pipe(output);
archive.append(event.ResourceProperties.content, { name: event.ResourceProperties.path });
archive.finalize();
await closed;
await s3.upload({Bucket: event.ResourceProperties.bucket, Key: 'index.zip', Body: fs.createReadStream('/tmp/index.zip')}).promise();
return;
}
In your construct, use inlineSource:
export class TestConstruct extends Construct {
constructor(scope: Construct, id: string, props: any) {
// set up other resources
const source = inlineSource('config.js', `exports.config = { apiEndpoint: '${ api.attrApiEndpoint }' }`);
// use in BucketDeployment
}
}
You can move inline-lambda elsewhere but it needs to be able to be bundled as an asset for the lambda.
This works by creating a custom resource that depends on your other resources in the stack (thereby allowing for the attributes to be resolved) that writes your file into a zip that is then stored into a bucket, which is then picked up and unzipped into your deployment/destination bucket. Pretty complicated but gets the job done with what is currently available.
The pattern I've used successfully is to put a CloudFront distribution or an API Gateway in front of the S3 bucket.
So requests to https://[api-gw]/**/* are proxied to https://[s3-bucket]/**/*.
Then I will create a new Proxy path in the same API gateway, for the route called /config which is a standard Lambda-backed API endpoint, where I can return all sorts of things like branding information or API keys to the frontend, whenever the frontend calls GET /config.
Also, this avoids issues like CORS, because both origins are the same (the API Gateway domain).
With CloudFront distribution instead of an API Gateway, it's pretty much the same, except you use the CloudFront distribution's "origin" configuration instead of paths and methods.
I am trying to get the list of uploads in the AWS Device Farm. I tried to use the method "devicefarm.listUploads" from Lambda.
I am facing an issue when I am trying to filter it by type.
var uploadList = devicefarm.listUploads({ arn: deviceFarmProject, type: 'IOS_APP' }).promise()
.then(res => res)
.catch(err => err);
The expectation is to get data about all the iOS apps, but getting the bellow error.
{
"message": "Unexpected key 'type' found in params",
"code": "UnexpectedParameter",
"time": "2019-05-02T15:49:35.351Z"
}
~~I'm not sure why the type isn't recognized here~~
[Edit]
The reason for this error is due to the version of the aws-sdk in AWS Lambda.
https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
Node.js Runtimes
Name Identifier AWS SDK for JavaScript
Node.js 8.10
nodejs8.10
2.290.0
I created a Lambda layer with the following commands and applied it to my function through the web console.
npm init
npm install aws-sdk
mkdir nodejs
cp -r node-modules nodejs
zip -r aws-sdk-layer.zip nodejs
note the zip file structure needs to match the Lambda documentation example.
https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-path
Node.js – nodejs/node_modules, nodejs/node8/node_modules (NODE_PATH)
Example AWS X-Ray SDK for Node.js
xray-sdk.zip
└ nodejs/node_modules/aws-xray-sdk
after I applied the layer I was able to execute the function successfully.
but I used the following and it seemed to work though I didn't have any iOS uploads.
// assume we already executed `npm install aws-sdk`
var AWS = require('aws-sdk');
// Device Farm is only available in the us-west-2 region
var devicefarm = new AWS.DeviceFarm({ region: 'us-west-2' });
var params = {};
devicefarm.listProjects(params, function (err, projects) {
if (err) console.log(err, err.stack); // an error occurred
else{
project = projects.projects[0];
console.log("project: ", project);
uploadList = devicefarm.listUploads({ arn: project.arn, type: 'IOS_APP' }).promise()
.then(function(uploadList){
console.log("uploads: ",uploadList);
})
.catch(err => err);
}
});
code I executed in Lambda
// assume we already executed `npm install aws-sdk`
var AWS = require('aws-sdk');
// Device Farm is only available in the us-west-2 region
var devicefarm = new AWS.DeviceFarm({ region: 'us-west-2' });
exports.handler = async (event) => {
return new Promise(function (resolve, reject) {
var params = {};
devicefarm.listProjects(params, function (err, projects) {
if (err) reject(err); // an error occurred
else {
var project = projects.projects[0];
console.log("project: ", project);
resolve(project);
}
});
}).then(function(data){
console.log("in then function","data: ",data);
return new Promise(function(resolve,reject){
devicefarm.listUploads({ arn: data.arn, type: 'IOS_APP' }, function(err,uploads){
if (err) reject(err); // an error occurred
else {
resolve(uploads);
}
})
}).then(function(data){
console.log("uploads: ", data);
return data;
}).catch(function(data){
console.error("list uploads failed","data: ", data);
return data;
});
}).catch(function(data){
console.error("list projects failed","data: ",data);
return data;
});
};
It might be the case that the aws-sdk version in Lambda isn't up to date in which case you would need to apply a Lambda layer or include the aws-sdk in the code package.
Locally I executed this code and it provided the following output:
node sample.js
project: { arn: 'arn:aws:devicefarm:us-west-2:111122223333:project:00ec5d2a-9170-4c52-b65e-0e12986e4fc3',
name: 'web_app_test',
created: 2019-02-10T22:47:35.052Z }
uploads: { uploads: [] }
aws-sdk version: aws-sdk#2.448.0
node --version
v8.12.0
HTH
James
According to the DialogFLow Fulfillment docs, the WebhookClient constructor needs Express HTTP request and response objects.
However, in Lambda function, I receive only the event (the request). How do I create the Express request and response objects?
I have tried this so far:
const {WebhookClient} = require('dialogflow-fulfillment');
exports.dialogflowFulfillment = async (event) => {
let response = {};
const agent = new WebhookClient({ event, response });
function sayNiceThings(agent) {
agent.add(`Nice to meet you!`);
}
let intentMap = new Map();
intentMap.set('Say Nice Things', sayNiceThings);
agent.handleRequest(intentMap);
};
Create an NodeJS Express App
Install serverless-http package for adding AWS Lambda bridge
Install dialogflow-fulfillment and actions-on-google npm packages.
npm init -f
npm install --save express serverless-http
npm install dialogflow-fulfillment
npm install actions-on-google
Create index.js file:
index.js:
=========
const serverless = require('serverless-http');
const bodyParser = require('body-parser');
const express = require('express');
const app = express();
app.use(bodyParser.json({ strict: false }));
const {WebhookClient, Card, Suggestion} = require('dialogflow-fulfillment');
const request = require('request');
app.get('/', function (req, res) {
res.send('Hello World !!!\n');
console.log("Testing express lambda\n");
})
app.post('/', function (req, res) {
const agent = new WebhookClient({request: req, response: res});
function test_handler(agent) {
agent.add(`Welcome to my agent on AWS Lambda!`);
}
// Run the proper function handler based on the matched Dialogflow intent name
let intentMap = new Map();
intentMap.set('test-intent', test_handler);
agent.handleRequest(intentMap);
})
module.exports.handler = serverless(app);
Add configurations in serverless.yml:
serverless.yml
================
service: example-express-lambda
provider:
name: aws
runtime: nodejs8.10
stage: dev
region: ap-southeast-1
functions:
app:
handler: index.handler
events:
- http: ANY /
- http: 'ANY {proxy+}'
Then deploy the lambda function.
Add the endpoint url in the Dialogflow fulfilment webhook url.
Reference:
https://serverless.com/blog/serverless-express-rest-api/