I'm a bit of a AWS newbie so forgive me if this is a stupid question. We have a legacy Classic ASP website (not hosted at Amazon) and I wanted to know if its possible to utilize Amazon Cloudwatch for custom logging. I know we could use a site like Loggly or Sumo Logic but their retention policies are a little short and the pricing a bit too expensive for our small site. But essentially, I'm trying to recreate their functionality in AWS.
There is quite a bit of detail on how to log from various Amazon services (EC2, etc...) but I'm not finding much on using Cloudwatch from a non-Amazon hosted site
Basically, I want to SERVERXMLHTTP POST a json log string from our Classic ASP site to Cloudwatch. However, I'm struggling to find the right endpoint and how to post to a particular Log Group that I've created. Do I use
https://monitoring.us-west-1.amazonaws.com/doc/2010-08-01/?Action=PutMetricData
or
https://logs.us-west-1.amazonaws.com/doc/2010-08-01/?Action=PutMetricData
to try to post json like this
[
{
"MetricName": "404 Error",
"Timestamp": "Wednesday, June 12, 2013 8:28:20 PM",
"Value": "http://example.com/badpage.asp",
"Unit": "Count"
}
]
Also, how do I post the json to a particular log group / log stream that I've created? Do I authenticate with some sort of
xxx.setRequestHeader "Authorization", "Basic " & [auth credentials]
?
Thanks for any help!
You can find the list of AWS endpoints here: http://docs.aws.amazon.com/general/latest/gr/rande.html
logs.us-west-1.amazonaws.com
The authentication is done with signed requests: http://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/making-api-requests.html
When you send HTTP requests to AWS, you sign the requests so that AWS can identify who sent them. You sign requests with your AWS access key, which consists of an access key ID and secret access key.
And about the specific API call to put the log probably is PutLogEvents: http://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html
Uploads a batch of log events to the specified log stream.
And remember there's a SDK for the browser: https://aws.amazon.com/es/sdk-for-browser/
Related
I have a redirect widget that calls the AWS Lambda using AWS API Gateway. it returns a twiml-gather then will call an external API base on the output I receive on the twiml-gather
for security reasons, I would like to make my AWS API Gateway have an OAuth or API key
right now, I'm not sure how can i achieve this given that using the redirect widget doesnt have an option to input a http-headers (can't use the Twilio function because of 10 seconds time limit)
You can make use of the X-Twilio-Signature here.
You also find some Twilio blog posts on this topic.
Validating Requests are coming from Twilio
https://www.twilio.com/docs/usage/security
If your application exposes sensitive data, or is possibly mutative to your data, then you may want to be sure that the HTTP requests to your web application are indeed coming from Twilio, and not a malicious third party. To allow you this level of security, Twilio cryptographically signs its requests.
I'm trying to figure out how to upload a file to google cloud storage using rest API , i don't want to use the client Library .
i read the documents but it was not helpful for a beginner in this flied ,
anyone can give me a step-by-step how to do this ? and how the URL/header/body format should look like , if also can give me an examples that would be very helpful .
If you're not going to use any of the helper libraries and are also a beginner, the hardest part of implementing an upload to GCS will likely be authenticating yourself. Let's ignore that for now.
The simplest way to upload an object to Google Cloud Storage is to make an HTTPS call to storage.googleapis.com that looks like this:
PUT /your-bucket-name/your-object.txt HTTP/1.1
Authorization: (YOUR ACCESS TOKEN GOES HERE)
Content-Length: 20
Content-Type: text/plain-or-whatever; charset=utf-8
Host: storage.googleapis.com
User-Agent: YourApplication/1.0
This is a test file
That will upload a file named "your-object.txt" of type "text/plain-or-whatever" to the bucket "your-bucket-name", with the contents "This is a test file."
If your bucket allows anonymous users to upload files (you shouldn't do that), then just don't include the Authorization line and you're done.
Now, since you really don't want to use any client libraries, and that presumably includes Google's OAuth libraries, you're going to need to implement authorization yourself, so let me give you an overview.
First, though, if you want to try this out immediately, install the "gcloud" tool, login with "gcloud auth login", and the print an access token with gcloud auth print-access-token. Then use the Authorization header Authorization: Bearer whatever.gcloudprintedout. That way you can be off and running with GCS quickly. But the token will only last an hour or so, so you'll need to implement OAuth for real.
Google Cloud APIs use OAuth to handle their requests, which is a powerful but not simple auth mechanism. There's extensive documentation on how OAuth with Google works: https://developers.google.com/identity/protocols/OAuth2
And there's also more general information on authorizing Google Cloud requests: https://cloud.google.com/docs/authentication
If you are running your application on a Google Cloud technology like App Engine or GCE, auth will be somewhat easier, but I will assume you're running this on your own machine. I will further assume that you want your application to have its own identity, rather than simply having you log in as part of the upload flow. For such a case, you'll need a service account, which will have an associated private key.
The basic flow for a service account is that you will create a JWT request for access credentials, then cryptographically sign that request with your private key, then send that signed request to Google. It will return you a token that may then be passed to your actual upload request later. You can keep using that token until it expires, at which time you'll need to build another JWT request to request another token.
Again, the client libraries entirely take care of this whole process for you. I am describing the approach of implementing everything exclusively on your own.
You can find the same example here:
https://stackoverflow.com/a/53955058/4345389
in which I already explained how to upload a file to google cloud storage using rest API.
Thanks
We're trying to set up an instagram app and have worked our way through the process. We've now got stuck when trying to create a subscription.
We have our app hosted on AWS API Gateway, which can only be deployed with HTTPS endpoints (it does not support unencrypted connections).
When we do the POST to get instagram to subscribe to a user using the following:
curl -F 'client_id=CLIENT-ID' \
-F 'client_secret=CLIENT-SECRET' \
-F 'object=user' \
-F 'aspect=media' \
-F 'verify_token=myVerifyToken' \
-F 'callback_url=https://YOUR-CALLBACK/URL' \
https://api.instagram.com/v1/subscriptions/
then we get the following error:
{
"meta": {
"error_type": "APISubscriptionError",
"code": 400,
"error_message": "Invalid response"
}
}
This happens whether we use our own HTTPS certificate that we have registered with API Gateway, or whether we use the stock AWS URL (https://xxxx.execute-api.us-east-1.amazonaws.com) as the callback (which also has a valid HTTPS certificate). I have verified that the certificates are 'good' using SSLLabs (they both get an A result). Our code NEVER gets called (so it's not the return of the hub.challenge parameter that's the problem. Instagram seems to reject the HTTPS certificate when initiating the connection.
Interestingly, if we use the same certificate that we use with AWS API Gateway on a normal machine (EC2 instance) and change the DNS records to point to this server, then it all works as expected and the subscription works.
Has anyone got Instagram Subscriptions working when using AWS API Gateway?
I came across this post and felt deceived by the current answer given that API gateway can still be used despite instagram's api not accepting the SNI.
"Instagram apps will not work if hosted on API Gateway unfortunately until Instagram upgrades their version of Python." is not true.
All you need to do is put a CloudFront instance in front of the api gateway and it will work fine without needing an expensive SSL cert or anything.
I included screenshots of my config so that it may help someone in the future.
Here is the first configuration screenshot
Behavior Settings
Origin Settings
We contacted Instagram about this. The version of Python (or their libraries) that Instagram uses does not support SNI which API Gateway uses (http://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-custom-domains.html) so Instagram apps will not work if hosted on API Gateway unfortunately until Instagram upgrades their version of Python.
As Garrett points out below, whilst Instagram cannot post directly to API Gateway, you can put a cloudfront distribution in front of your API Gateway endpoints and let Instagram point to that. This will work.
To verify if this is a API Gateway issue have you tried to issue the same challenge request that Instagram is making to your API endpoint directly ?
If you haven't already, you should also try to enable CloudWatch logging on your API stage to debug the call.
I have an application where I use OAuth2 to grab some info from a users spreadsheet in Google Drive. I have just completed the OAuth2 procedure, which I thought was the challenging part since there are no OAuth2 libraries for xquery, but now I am running into another problem.
I send a GET request to the Google Drive SDK, which I have turned ON in the Google API Console, and get the following output:
{
"error": {
"errors": [
{
"domain": "usageLimits",
"reason": "accessNotConfigured",
"message": "Access Not Configured"
}
],
"code": 403,
"message": "Access Not Configured"
}
}
I have found this question which seems to be the cause of my error :Google API returning Access Not Configured but I am unable to complete the solution to my problem which is to " Visit code.google.com/apis/console. First, make sure the service you want is enabled under "Services". Then go to "API Access" and specify your calling domains or IP addresses." When I try to add my domain to the notification endpoints, it doesn't do it.
In addition, I see that it says "Allows webhook notifications to be sent to external domains that are owned by the user. Google verifies that the user does in fact own each of the listed domains via Webmaster Tools" in the box where you add said endpoints. Since I am on an Amazon EC2 instance I am not sure that I will be able to do that. I really hope the fact that I am on an Amazon server is not stopping me from making my app.
Any help would be appreciated, thank you all.
You should enable Drive SDK and Drive API in the API console.
I'm stuck trying to get an API call to Google Places working. I'm using a server-side PHP proxy for the request - not because I want to, but because it's part of JQuery-POI-Mapper.
The server I'm using is an Amazon EC2 server without a static IP address. I'm out of stactic IP addresses at the moment, but I'll take the time to request more if people think that's the problem. I identified the current public IP address of my EC2 server by running this command on the server:
curl http://169.254.169.254/latest/meta-data/public-ipv4
Next, I went to https://code.google.com/apis/console and created a new API key. I selected a server-side API key and used my EC2 server's public IP address.
Under the services tab, I enabled every related API I could think of, including:
Google Maps API v2
Google Maps API v3
Google Maps Gelolocation API
Places API
Static Maps API
Here's a screen capture of my Services:
The proxy code running on the EC2 server is part of a commercial package, so I shouldn't post the entire code, but it's very short, and the important part is:
$json = file_get_contents($url);
The $url variable in my case was:
https://maps.googleapis.com/maps/api/place/search/json?location=-37.7133771,145.14891620000003&radius=2000&types=bakery&sensor=false&key=AIzaSyCCUV...
The response I get is:
{
"html_attributions" : [],
"results" : [],
"status" : "REQUEST_DENIED"
}
I checked to see if I had gone over my quota already, but everything looks OK. What's interesting is that Google is showing that I had made requests to the Places API today, so Google definitely knows that the requests are coming from me.
Here's a screen capture of my API Traffic Report, which is all from testing:
Any advice would be much appreciated.
Thanks,
Bret
You need a Static IP address to successfully use IP Locking with a Server API Key.