I am using aws lambda with an application load balancer. I have been fighting to figure out why I am getting only a piece of my content (a few bytes).
I am try to return a PDF file as binary data. This works local and it works when my application is deployed to ec2 or anywhere other than lambda.
I have looked at answers like this one AWS Lambda fails to return PDF file
but these answers are all discussing lambda settings in regards to API gateway. I am not using API gateway I am using the application load balancer. Any ideas?
Found the answer! I am using serverless-http with express. I needed to add the binary parameter for my return content type.
const serverless = require('serverless-http');
const app = require("./index");
const handler = serverless(app, {
binary: ['application/pdf']
});
exports.handler = async (event, context) => {
return await handler(event, context);
};
Related
I have a Web Api that invokes another web api call to get some information. In order to make the app more resilient, I implemented a HttpTransientErrorDetectionStrategy following the steps at: https://alexandrebrisebois.wordpress.com/2013/02/21/defining-an-http-transient-error-detection-strategy-for-rest-calls/
After that, I use code like below to invoke the other web app:
RetryPolicy _retryPolicy = new RetryPolicy<HttpTransientErrorDetectionStrategy>(
new ExponentialBackoff(retryCount: 2, minBackoff: TimeSpan.FromSeconds(0), maxBackoff: TimeSpan.FromSeconds(10), deltaBackoff: TimeSpan.FromSeconds(2)));
var _httpClient = new HttpClient
{
BaseAddress = new Uri("http://www.microsoft.com")
};
HttpResponseMessage response = _retryPolicy.ExecuteAsync(async () => await _httpClient.GetAsync($"", HttpCompletionOption.ResponseContentRead)).Result;
The _httpClient.GetAsync call gets stuck, and I have no idea why. If I remove the _retryPolicy, and just use _httpClient.GetAsync directly, it returns in a matter of seconds.
I have similar code on a console app, to invoke the same web app, and that is working fine, so this seems to be specific to the way I am using it in my web API. This is intended to be an app on Azure, but it happens when I debug locally as well. Does anybody have any idea why this is getting stuck? How can I debug this?
Thank you!
I have similar code on a console app, to invoke the same web app, and that is working fine, so this seems to be specific to the way I am using it in my web API.
The code you posted is blocking right here:
HttpResponseMessage response = _retryPolicy.ExecuteAsync(...).Result;
Don't block on async code. Instead, use await:
HttpResponseMessage response = await _retryPolicy.ExecuteAsync(...);
If I remove the _retryPolicy, and just use _httpClient.GetAsync directly, it returns in a matter of seconds.
If your original code is blocking, and you must block on asynchronous code (for some reason), then you can either use the ConfigureAwait(false) hack:
HttpResponseMessage response = _retryPolicy.ExecuteAsync(async () => await _httpClient.GetAsync($"", HttpCompletionOption.ResponseContentRead).ConfigureAwait(false)).Result;
or elide async/await:
HttpResponseMessage response = _retryPolicy.ExecuteAsync(() => _httpClient.GetAsync($"", HttpCompletionOption.ResponseContentRead)).Result;
P.S. Check out DecorrelatedJitterBackoffV2.
I have an NestJS application which consist of many modules: databaseModule, userModule, devicesModule, etc. They all packaged to one module ApplicationModule, which handle server actions.
But now I need to add some lambda function to my project and I need to exec some methods from databaseModule, but I don't know how to do it.
Pseudo code that I imagine:
export const handler: Handler = (event: any, context: Context, callback) => {
const dbModule = DataBaseModule.build();
dbModule.get(UserService).createProject('my_project');
callback(null, event);
};
I think that nestjs should have similar functionality but I can't find it on official page.
P.S. I can't use just UserService because it depends on other services and providers in DatabaseModule. That is why I want this module to be fully configured and I can use its services
I found an answer https://docs.nestjs.com/application-context
We can use our submodules in the next way:
const app = await NestFactory.create(ApplicationModule);
const tasksService = app.get(TasksService);
You can use the Lifecycle events from NestJS (https://docs.nestjs.com/fundamentals/lifecycle-events).
That way, you can implement OnModuleInit() on the Service and set the desired function to run when the Module is loaded (it can be sync or async).
There are some other events that can also be useful, like onApplicationBootstrap()
I have an an AWS Lambda function that needs to connect to a remote TCP service. Is there any way to configure the Lambda function with the IP address of the remote service after the Lambda function has been deployed to AWS? Or do I have to bake the configuration into the packaged Lambda function before it's deployed?
I found a way that I use for supporting a test environment and a production environment that will help you.
For the test version of the function, I am calling it TEST-ConnectToRemoteTcpService and for the production version of the function I am naming the function PRODUCTION-ConnectToRemoteTcpService. This allows me pull out the environment name using a regular expression.
Then I am storing config/test.json and config/production.json in the zip file that I upload as code for the function. This zip file will be extracted into the directory process.env.LAMBDA_TASK_ROOT when the function runs. So I can load that file and get the config I need.
Some people don't like storing the config in the code zip file, which is fine - you can just load a file from S3 or use whatever strategy you like.
Code for reading the file from the zip:
const readConfiguration = () => {
return new Promise((resolve, reject) => {
let environment = /^(.*?)-.*/.exec(process.env.AWS_LAMBDA_FUNCTION_NAME)[1].toLowerCase();
console.log(`environment is ${environment}`);
fs.readFile(`${process.env.LAMBDA_TASK_ROOT}/config/${environment}.json`, 'utf8', function (err,data) {
if (err) {
reject(err);
} else {
var config = JSON.parse(data);
console.log(`configuration is ${data}`);
resolve(config);
}
});
});
};
Support for environment variables was added for AWS Lambda starting November 18, 2016. Adding a variable to an existing function can be done via command line as shown below or from the AWS Console.
aws lambda update-function-configuration \
--function-name MyFunction \
--environment Variables={REMOTE_SERVICE_IP=100.100.100.100}
Documentation can be found here.
You can invoke the Lambda function via SNS topic subscription and have it configure itself from the payload inside the SNS event.
Here's the official guide on how to do that Invoking Lambda via SNS.
A few options, depending on the use-case
If your config will not change then you can use S3 objects and access from Lambda or set your Lambda to trigger on new config changes. (Though this is the cheapest way, you are limited in what you can do compared to other alternatives)
If the config is changing constantly, then DynamoDB - Key/value is an alternative.
If DynamoDB is expensive for the frequent read/writes and not worth the value then you can have TCP service post config into a SQS queue. (or an SNS if you want to trigger when the service posts a new config)
Is there an API that allows access to Google's Mobile Friendly Test which can be seen at https://www.google.com/webmasters/tools/mobile-friendly/?
If you can't find one by googling, it probably doesn't exist.
A hacky solution would be to create a process with PhantomJS that inputs the url, submits it, and dirty-checks the dom for results.
PhantomJS is a headless WebKit scriptable with a JavaScript API.
However, if you abuse this, there is a chance that google will blacklist your ip address. Light use should be fine. Also be aware that google can change their dom structure or class names at any time, so don't be surprised if your tool suddenly breaks.
Here is some rough, untested code...
var url = 'https://www.google.com/webmasters/tools/mobile-friendly/';
page.open(url, function (status) {
// set the url
document.querySelector('input.jfk-textinput').value = "http://thesite.com";
document.querySelector('form').submit();
// check for results once in a while
setInterval(function(){
var results = getResults(); // TODO create getResults
if(results){
//TODO save the results
phantom.exit();
}
}, 1000);
});
There is an option in pagespeed api
https://www.googleapis.com/pagespeedonline/v3beta1/mobileReady?url={url}&key={api key}
key can be obtained form google cloud platform.
Acquire a PageSpeed Insights API KEY in https://console.developers.google.com/apis/api/pagespeedonline-json.googleapis.com/overview?project=citric-program-395&hl=pt-br&duration=P30D and create a credentials, follow the google's instructions.
In C# (6.0) and .NET 4.5.2, I did some like this:
(add in your project a reference for Newtonsoft.Json.)
String yourURL = "https://www.google.com.br";
HttpClient client = new HttpClient();
client.BaseAddress = new Uri("https://www.googleapis.com");
client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
var response = client.GetAsync($"/pagespeedonline/v3beta1/mobileReady?url={yourURL }&key=AIzaSyArsacdp79HPFfRZRvXaiLEjCD1LtDm3ww").Result;
string json = response.Content.ReadAsStringAsync().Result;
JObject obj = JObject.Parse(json);
bool isMobileFriendly = obj.Value<JObject>("ruleGroups").Value<JObject>("USABILITY").Value<bool>("pass");
There is an API (Beta) for the Mobile Friendly-Test. (Release Date: 31.01.2017).
The API test outputs has three statuses:
MOBILE_FRIENDLY_TEST_RESULT_UNSPECIFIED Internal error when running this test. Please try running the test again.
MOBILE_FRIENDLY The page is mobile friendly.
3.NOT_MOBILE_FRIENDLY The page is not mobile friendly.
Here are more informations: https://developers.google.com/webmaster-tools/search-console-api/reference/rest/v1/urlTestingTools.mobileFriendlyTest/run
I have a service that is successfully deployed to Google Endpoints and it is accessible through browser.
Now I am trying to load Google API javascript client library to call my services using javascript.
As far as I know, I should do this
gapi.client.load([MY_APP_NAME], 'v1', function() {
var request = gapi.client.[API_NAME].[SERVICE_NAME].[METHOD]();
request.execute(function(jsonResp, rawResp) {...});
);
But I always get an exception at run time complaining about gapi.client.[MY_API_NAME] is undefined. I do the same thing with any Google API (such as Plus) and it works fine. For example, If I load 'plus' API, I will have access to gapi.client.plus... and I can call methods.
Am I missing something? All samples and documents are about Google Service APIs and I could not find a sample for custom APIs (the one that developers write).
I even tried gapi.client.request with different paths (absolute path and relative path) but I get 404 - Not Found error in "status".
var request = gapi.client.request({'path':
'https://[APP_NAME].appspot.com/_ah/api/[SERVICE_NAME]/v1/[METHOD]'
, 'method': 'GET'});
request.execute(function(jsonResp, rawResp) {...});
var request = gapi.client.request({
'path':'/[SERVICE_NAME]/v1/[METHOD]',
'method': 'GET'});
request.execute(function(jsonResp, rawResp) {...});
The problem was a missing parameter in calling gapi.client.load().
I looked at the definition of gapi.client.load at this link https://developers.google.com/api-client-library/javascript/reference/referencedocs#gapiclientload
gapi.client.load(name, version, callback)
which then later I found out is not totally correct and an optional parameter is missing (app_api_root_url).
gapi.client.load(name, version, callback, app_api_root_url)
If the app_api_root_url is missing, the client is loaded for Google Service APIs only (app_api_root_url such as https://myapp.appspot.com/_ah/api)
You can find more details on how to use gapi.client.load() properly at this link https://developers.google.com/appengine/docs/java/endpoints/consume_js
As you can see in the following piece of code, I didn't have ROOT parameter when I was calling gapi.client.load and that is why Google by default was looking at its own service API and obviously could not find my APIs.
var ROOT = 'https://your_app_id.appspot.com/_ah/api';
gapi.client.load('your_api_name', 'v1', function() {
var request = gapi.client.your_api_name.your_method_name();
request.execute(function(jsonResp, rawResp) {
//do the rest of what you need to do
});
}, ROOT);
NOTE: your_app_id is used in ROOT parameter only to load the client script. After loading is done, you will have an object that is named after your API and not your app. That object is like your Java (service) class and you can use to invoke methods directly.