I am deploying a Next.js app using serverless component (serverless-next.js, not the plugin). After deploying, my API routes run smoothly, but some (not all) pages and all content in /public folder are not accessible.
On the web, requests to those resources end with a 503:
content-length: 1019
content-type: text/html
date: Tue, 27 Oct 2020 19:55:05 GMT
server: CloudFront
status: 503
via: 1.1 79ba346413d83ce62db11c8d0b05c22d.cloudfront.net (CloudFront)
x-amz-cf-id: RalFtcB6W4pm1_eTgzr0Y9LWCR-uwfqgW2alSgX_X72YVlUxlEkrKw==
x-amz-cf-pop: PRG50-C1
x-cache: LambdaExecutionError from cloudfront
I tracked down the error to Default Lambda#Edge for Next CloudFront distribution where I can see a record for every call in CloudWatch logs:
{
"errorType": "AccessDenied",
"errorMessage": "Access Denied",
"Code": "AccessDenied",
"RequestId": "75B8CCC0B1F960A7",
"HostId": "92cygvWjn3xLwT0PChJPc56ExzUFBDFTHyjLXQ/BWbT7pCVMb3I2ww15+Gzs2fSuXjHEPSfUIdg=",
"name": "AccessDenied",
"$fault": "client",
"$metadata": {
"httpStatusCode": 403,
"httpHeaders": {
"x-amz-request-id": "75B8CCC0B1F960A7",
"x-amz-id-2": "92cygvWjn3xLwT0PChJPc56ExzUFBDFTHyjLXQ/BWbT7pCVMb3I2ww15+Gzs2fSuXjHEPSfUIdg=",
"content-type": "application/xml",
"transfer-encoding": "chunked",
"date": "Tue, 27 Oct 2020 19:54:59 GMT",
"server": "AmazonS3"
},
"attempts": 1,
"totalRetryDelay": 0
},
"stack": [
"AccessDenied: Access Denied",
" at deserializeAws_restXmlGetObjectCommandError (/var/task/index.js:14166:41)",
" at processTicksAndRejections (internal/process/task_queues.js:97:5)"
]
}
I checked the origin S3 bucket and all resources are there. I tried to remove complete deployment and do a clean deploy to no avail. As I understand the problem lies in S3 permissions, I granted the policy under which the lambda is invoked full access to S3 but the problem persists.
Any help how to debug this further appreciated.
Related
Environment details
OS: macOS Big Sur Version 11.6 (Apple M1 Chip)
Node.js version: v16.4.1
npm version: 7.23.0
#google-cloud/talent version: v4
Intruduction
I'm having a hard time to get the job search in Google Cloud Talent Solution to work.
I already can create/read/update, tenants/companies/jobs indicating that the credentials are ok.
But I don't find any jobs searching them.
Facts
Currently I have one job stored in Google Cloud Talent Solution.
This ist the job export, done with the Google Console:
{
"name":"projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5/jobs/135317048994472646",
"requisition_id":"f9bffe6e-3c8c-40d5-b3c3-672d30485745",
"title":"IT-Berater"
}
This is the JSON stringifyed request, passed to "searchJobs":
{
"parent":"projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5",
"searchMode":"JOB_SEARCH",
"requestMetadata":{"domain":"insurancepunk.com","sessionId":"8f47bbab-5c15-4bd9-9008-60f79030ab3b","userId":"vCobKcXPFdf6zlVibjnb"},
"jobQuery":{"query":"IT-Berater"}
}
As you can see the project id and the tenant id match the exported job.
This is my very simple code:
const talent = require('#google-cloud/talent').v4;
const client = new talent.JobServiceClient();
client.searchJobs(request)
.then(responses => {
const resources = responses[0];
for (const resource of resources) {
console.log(`Job summary: ${resource.jobSummary}`);
console.log(`Job title snippet: ${resource.jobTitleSnippet}`);
const job = resource.job;
console.log(`Job name: ${job.name}`);
console.log(`Job title: ${job.title}`);
}
})
.catch(err => {
console.error(err);
});
The code enters the then-path, but "responses" is empty.
Google OAuth Play Ground
When testing it in Google OAuth Play Ground, i get this results.
Googel OAuth Play Ground: https://developers.google.com/oauthplayground/
Google Talent Solution Scope: [https://www.googleapis.com/auth/jobs2
The Scope was found here:
https://cloud.google.com/talent-solution/job-search/docs/reference/rpc/google.cloud.talent.v4
Output from Google OAuth Play Ground:
Request:
POST /v4/projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5/jobs:search HTTP/1.1
Host: jobs.googleapis.com
Content-length: 277
Content-type: application/json
Authorization: Bearer ya29.a0ARrda...
{
"parent":"projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5",
"searchMode":"JOB_SEARCH",
"requestMetadata":{"domain":"insurancepunk.com","sessionId":"8f47bbab-5c15-4bd9-9008-60f79030ab3b","userId":"vCobKcXPFdf6zlVibjnb"},
"jobQuery":{"query":"IT-Berater"}
}
Response:
HTTP/1.1 200 OK
Content-length: 117
X-xss-protection: 0
X-content-type-options: nosniff
Transfer-encoding: chunked
Vary: Origin, X-Origin, Referer
Server: ESF
-content-encoding: gzip
Cache-control: private
Date: Tue, 30 Nov 2021 09:44:40 GMT
X-frame-options: SAMEORIGIN
Alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Content-type: application/json; charset=UTF-8
{
"metadata": {
"requestId": "0e7330e1-ee15-404e-85d2-b8174679583f:APAb7ITvUURH6nrwLrbLLBCB9Zg6NKPMfg=="
}
}
Listing companies works well!
Request:
GET /v4/projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5/companies HTTP/1.1
Host: jobs.googleapis.com
Content-length: 0
Authorization: Bearer ya2...
Response:
HTTP/1.1 200 OK
Content-length: 1124
X-xss-protection: 0
Content-location: https://jobs.googleapis.com/v4/projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5/companies
X-content-type-options: nosniff
Transfer-encoding: chunked
Vary: Origin, X-Origin, Referer
Server: ESF
-content-encoding: gzip
Cache-control: private
Date: Tue, 30 Nov 2021 10:08:49 GMT
X-frame-options: SAMEORIGIN
Alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Content-type: application/json; charset=UTF-8
{
"companies": [
{
"displayName": "Pompadour GmbH",
"name": "projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5/companies/a6f34dd2-76f1-40e8-8175-1274f49f5977",
"headquartersAddress": "Am Burgweg 1, 97346 Iphofen",
"imageUri": "http://www.pompadour.info/bar",
"derivedInfo": {
"headquartersLocation": {
"locationType": "STREET_ADDRESS",
"postalAddress": {
"postalCode": "97346",
"regionCode": "DE",
"administrativeArea": "BY",
"addressLines": [
"Am Burgweg 1, 97346 Iphofen, Germany"
],
"locality": "Iphofen"
},
"radiusMiles": 6.892640659556388e-05,
"latLng": {
"latitude": 49.7102381,
"longitude": 10.254041
}
}
},
"externalId": "9a1ebd16-886c-40ac-ae0a-d5a4e288f867",
"websiteUri": "http://www.pompadour.info",
"hiringAgency": true
}
],
"metadata": {
"requestId": "5780a724-ca05-4f56-8a88-d74fdc04a24e:APAb7IS/J4Hs1KThU2G0nCZk5fOdBT3sJw=="
}
}
All hints are welcome.
Wow!
This don't quite smell like Artificial Intelligence...
My one and only job had the Title "IT-Berater".
Searching for "IT-Berater" returned an empty result set.
However, searching for "Berater" returned the job...
The results are equal no matter if I used the Node.js API ore the original Google HTTP-API...
I have a lambda function and API gateway deployed through terraform. The API gateway has multiple resources under the top-level root resource / such as /first and /second. The resource for /first is set up with lambda-proxy integration (ANY and /first and ANY on /first/{proxy+}). The /second resource points to a different integration.
My lambda is an ASP.NET Core 3.1 with Lambda Custom Runtime generated initially with dotnet new serverless.AspNetCore30WebApi but later updated to 3.1 when support was added to AWS.
My issue is that when the lambda proxy integration transforms the request to be sent to the lambda function, it includes the full resource name (i.e. /first/Product instead of just /Product). Since my REST API in the lambda function only knows about Products and doesn't have a concept of what first is, I get 404 results for anything forwarded through the API gateway. I can't seem to find a place to strip the resource name from the path before it's forwarded on to the lambda function, because with the lambda proxy integration, it doesn't seem like you have a chance to do any request mapping templates or substitutions.
Thu Jun 03 12:24:52 UTC 2021 : HTTP Method: GET, Resource Path: /first/Product
Thu Jun 03 12:24:52 UTC 2021 : Method request path: {proxy=Product}
Thu Jun 03 12:24:52 UTC 2021 : Method request query string: {}
Thu Jun 03 12:24:52 UTC 2021 : Method request headers: {}
...
Thu Jun 03 12:24:52 UTC 2021 : Endpoint request body after transformations: <<relevant snippet parts formatted below>>
{
"resource": "/first/{proxy+}",
"path": "/first/Product",
"httpMethod": "GET",
"pathParameters": {
"proxy": "Product"
},
"requestContext": {
"resourcePath": "/first/{proxy+}",
"httpMethod": "GET",
"path": "/first/{proxy+}",
"protocol": "HTTP/1.1"
}
}
If I go and unlock an achievement in game, and then run a reset, after running through the OAuth process for my user, posting to https://www.googleapis.com/games/v1management/achievements/reset, I receive a correct response,
{
"kind": "gamesManagement#achievementResetAllResponse",
"results": [
{
"definitionId": "CgkIh9rSofwUEAIQAQ",
"kind": "gamesManagement#achievementResetResponse",
"updateOccurred": true,
"currentState": "REVEALED"
}
]
}
If I post to https://www.googleapis.com/games/v1management/achievements/reset again, again I get a successful empty list response, as is expected,
{
"kind": "gamesManagement#achievementResetAllResponse",
"results": []
}
But when I go back into the game, check my achievements, it still shows that I have that achievement. No matter if I log out, log in, restart the app, makes no difference, I still have that achievement,
Even if I post the full reset, https://www.googleapis.com/games/v1management/achievements/resetAllForAllPlayers, I get the successful, no body response,
HTTP/1.1 204 No Content
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Vary: Origin,X-Origin
Server: GSE
Etag: "aHcpPO1q3J7BuZyftdPIkrurI0I/vyGp6PvFo4RvsFtPoIWeCReyIC8"
Pragma: no-cache
Cache-control: no-cache, no-store, max-age=0, must-revalidate
Date: Tue, 06 Oct 2015 15:14:31 GMT
I am attempting to use the Google JSON API - taskqueue/v1beta2
https://developers.google.com/apis-explorer/#p/taskqueue/v1beta2/taskqueue.tasks.list
I am not able to successfully authenticate for this service using the API explorer. I can use other JSON APIs like the storage API under the same project & account. When I go the the APIs tab in the developer console there is nothing listed for "TaskQueue" or "TaskQueue JSON API".
Edit: The oauth flow seems to work correctly, with scopes:
https://www.googleapis.com/auth/taskqueue
https://www.googleapis.com/auth/taskqueue.consumer
No matter what I do I get a 403 Forbidden response. How can I authorize my account to use the TaskQueue API? Here is my response from the API explorer:
403 Forbidden
cache-control: private, max-age=0
content-encoding: gzip
content-length: 146
content-type: application/json; charset=UTF-8
date: Mon, 19 Jan 2015 17:20:50 GMT
expires: Mon, 19 Jan 2015 17:20:50 GMT
server: GSE
vary: Origin, X-Origin
{
"error": {
"errors": [
{
"domain": "global",
"reason": "forbidden",
"message": "you are not allowed to make this api call"
}
],
"code": 403,
"message": "you are not allowed to make this api call"
}
}
The REST API only works for pull queues, and you need to also specify the email address of the user you are authorizing as in your queue.yaml. From the docs:
queue:
- name: pull-queue
mode: pull
acl:
- user_email: bar#foo.com # can list, get, lease, delete, and update tasks
- writer_email: user#gmail.com # can insert tasks
- writer_email: bar#foo.com # can insert tasks, in addition to rights granted by being a user_email above
Defining pull queues
https://cloud.google.com/appengine/docs/python/config/queue#Python_Defining_pull_queues
I've spent the past couple of days trying to access Google's Directory API in Ruby but haven't been able to get it working. According to this document the Directory API can be authorized using 2lo:
If your application has certain unusual authorization requirements, such as logging in at the same time as requesting data access (hybrid) or domain-wide delegation of authority (2LO), then you cannot currently use OAuth 2.0 tokens. In such cases, you must instead use OAuth 1.0 tokens and an API key. You can find your application's API key in the Google APIs Console, in the Simple API Access section of the API Access pane.
I currently have working code that can access the Provisioning API using 2lo. From the documentation it sounds that I can use this same code to access the Directory API by just adding an API access key parameter to the request and enabling a few permissions. However, it's not working and I have no idea why.
Here is the request code:
def self.get_user2(email)
#client = Google::APIClient.new(:authorization => :two_legged_oauth_1)
#client.authorization.client_credential_key = GOOGLE_APP_KEY
#client.authorization.client_credential_secret = GOOGLE_APP_SECRET
#directory = #client.discovered_api('admin', 'directory_v1')
result = #client.execute(
#directory.users.get,
'userKey' => email,
:key => GOOGLE_API_KEY
)
JSON.parse(result.body)
end
This gets me the response:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "authError",
"message": "Invalid Credentials",
"locationType": "header",
"location": "Authorization"
}
],
"code": 401,
"message": "Invalid Credentials"
}
}
I've added the required scope to my manifest file,
<Scope id="usersAPI">
<Url>https://www.googleapis.com/auth/admin.directory.user.readonly</Url>
<Reason>See all users in your company.</Reason>
</Scope>
and also enabled the Admin SDK for my project in the API console.
Here is the log output after adding Faraday.default_connection.response :logger to my development.rb file:
get https://www.googleapis.com/admin/directory/v1/users/brian#crushing.mygbiz.com?key=AIzaSyAHYBWlC_qiihRtTKTZleZlAw2ts8Q1WO8
User-Agent: "google-api-ruby-client/0.6.4 Mac OS X/10.8.4"
Authorization: "OAuth oauth_consumer_key=\"76548528623.apps.googleusercontent.com\", oauth_signature_method=\"HMAC-SHA1\", oauth_timestamp=\"1375899810\", oauth_nonce=\"de4d976eed6883a06b3f6084e3dd0db4\", oauth_version=\"1.0\", oauth_signature=\"3D7aqhBeaCYOYyaF8bWpaM9MA8U%3D\""
Cache-Control: "no-store"
Content-Type: "application/x-www-form-urlencoded"
401
www-authenticate: "AuthSub realm=\"https://www.google.com/accounts/AuthSubRequest\""
content-type: "application/json; charset=UTF-8"
date: "Wed, 07 Aug 2013 18:23:30 GMT"
expires: "Wed, 07 Aug 2013 18:23:30 GMT"
cache-control: "private, max-age=0"
x-content-type-options: "nosniff"
x-frame-options: "SAMEORIGIN"
x-xss-protection: "1; mode=block"
server: "GSE"
connection: "close"
After searching the internet for the past day I am out of ideas for why this isn't working. Ideas anyone?
It turns out I was missing the 'xoauth_requestor_id' field. Thanks to sqrrrl for answering my question on github