FineUploaderS3: socket connection to the server was not read from or written to within the timeout period - fine-uploader

We are using FineUploader 5.4.1's jQuery uploader. Periodically, our uploads to AWS S3 are failing. It's working consistently from one of our office locations and failing from the other.
Request:
Accept:/
Accept-Encoding:gzip, deflate
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:keep-alive
Content-Length:1129759
Content-Type:multipart/form-data; boundary=----WebKitFormBoundarySNlCNy1xiPquuzCl
Host:s3.amazonaws.com
Origin:
Pragma:no-cache
Referer:
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/47.0.2526.73 Chrome/47.0.2526.73 Safari/537.36
Request Payload
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="key"
virginia/original/b71bb640-c771-44d9-9d28-44a075197bc3.png
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="Content-Type"
image/png
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="success_action_status"
200
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="acl"
public-read
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="x-amz-meta-qqparentuuid"
3d343555-7821-43d3-a954-aba2ce17f597
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="x-amz-meta-qqparentsize"
983029
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="x-amz-meta-qquuid"
b71bb640-c771-44d9-9d28-44a075197bc3
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="x-amz-meta-qqfilename"
missing%20(o).png
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="AWSAccessKeyId"
AKIAIVACH6FJDZWZ5LYA
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="policy"
eyJleHBpcmF0aW9uIjoiMjAxNS0xMi0zMFQwNzoyMDowMy41NjBaIiwiY29uZGl0aW9ucyI6W3siYWNsIjoicHVibGljLXJlYWQifSx7ImJ1Y2tldCI6InZpcmdpbmlhLmlua2l2ZS5jb20ifSx7IkNvbnRlbnQtVHlwZSI6ImltYWdlXC9wbmcifSx7InN1Y2Nlc3NfYWN0aW9uX3N0YXR1cyI6IjIwMCJ9LHsia2V5IjoidmlyZ2luaWFcL29yaWdpbmFsXC9iNzFiYjY0MC1jNzcxLTQ0ZDktOWQyOC00NGEwNzUxOTdiYzMucG5nIn0seyJ4LWFtei1tZXRhLXFxcGFyZW50dXVpZCI6IjNkMzQzNTU1LTc4MjEtNDNkMy1hOTU0LWFiYTJjZTE3ZjU5NyJ9LHsieC1hbXotbWV0YS1xcXBhcmVudHNpemUiOiI5ODMwMjkifSx7IngtYW16LW1ldGEtcXF1dWlkIjoiYjcxYmI2NDAtYzc3MS00NGQ5LTlkMjgtNDRhMDc1MTk3YmMzIn0seyJ4LWFtei1tZXRhLXFxZmlsZW5hbWUiOiJtaXNzaW5nJTIwKG8pLnBuZyJ9LFsiY29udGVudC1sZW5ndGgtcmFuZ2UiLCIwIiwiMTUwMDAwMDAwIl1dfQ==
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="signature"
J7XwHe5kmIVXznw+8cIiGwiyLbE=
------WebKitFormBoundarySNlCNy1xiPquuzCl
Content-Disposition: form-data; name="file"; filename="blob"
Content-Type: image/png
------WebKitFormBoundarySNlCNy1xiPquuzCl--
Response:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>RequestTimeout</Code><Message>Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.</Message><RequestId>C7E8E596E8FF701A</RequestId><HostId>aIePbPl+NPEr3/6LZkV9KlFzVpPTapnROfoNgI/DN+/t1zrttudxSADiPcCdwVnBq15BDKeodwY=</HostId></Error>
On the same network, AWS CLI managed to upload the same file successfully multiple times.
Our FineUploader settings:
{
uploaderType: 'basic',
maxConnections: 3,
request: {
endpoint: <OUR BUCKET>,
accessKey: <OUR UPLOAD KEY>
},
signature: {
endpoint: <OUR UPLOAD SIGNATURE ENDPOINT>
},
uploadSuccess: {
endpoint: <OUR UPLOAD SUCCESS ENDPOINT>,
params: {
param1: "Hello",
param2: "World"
}
},
iframeSupport: {
localBlankPagePath: <HTML FILE URL>
},
retry: {
showButton: true
},
chunking: {
enabled: true
},
resume: {
enabled: true
},
validation: {
sizeLimit: 150000000
},
messages: {
typeError: '{file} has an invalid extension. Valid extension(s): {extensions}.'
}
};
Tried with and without chunking.

Sounds like a pretty clear-cut occasional network issue at one of your offices. You can read more about potential causes at https://github.com/aws/aws-sdk-js/issues/281. That case suggests that the Content-Length, which is set by the browser and not Fine Uploader, is larger than the actual number of bytes sent. This to me suggests that a network issue is preventing all bytes from being sent in some cases.

Related

SAM AWS Lambda Python: gateway api - handle binary form upload

SAM AWS Lambda Python: I create a function that receive a file with multipart/form-data
but when I send the following request
curl --location --request POST 'https://....execute-api.....amazonaws.com/Prod/ocrReceipt' \
--header 'Connection: keep-alive' \
--header 'Pragma: no-cache' \
--header 'Cache-Control: no-cache' \
--header 'Accept: application/json, text/plain, */*' \
--header 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36' \
--header 'Accept-Language: en-US,en;q=0.9,he;q=0.8' \
--form 'file=#"/D:/receipt.png"'
I get the event with missing Content-Length and "isBase64Encoded": false
and my code fail because I do fp = io.BytesIO(base64.b64decode(body)) # decode
when I access the same function with the Function URL I don't get this error (i.e. Content-Length is set and "isBase64Encoded": true
How can I config the gateway api to accept multipart/form-data properly?
I tried to set multipart/form-data and multipart/* in API > Settings > Binary Media Types
but it doesn't help
Not that is says You can configure binary support for your API by specifying which media types should be treated as binary types. API Gateway will look at the Content-Type and Accept HTTP headers to decide how to handle the body.
Did I set the correct value?
Can I fix it by some setting in the SAM template?
Event:
{"resource": "/ocrReceipt", "path": "/ocrReceipt", "httpMethod": "POST", "headers": {"Accept": "application/json, text/plain, */*", "Accept-Encoding": "gzip, deflate, br", "Accept-Language": "en-US,en;q=0.9,he;q=0.8", "Cache-Control": "no-cache", "CloudFront-Forwarded-Proto": "https", "CloudFront-Is-Desktop-Viewer": "true", "CloudFront-Is-Mobile-Viewer": "false", "CloudFront-Is-SmartTV-Viewer": "false", "CloudFront-Is-Tablet-Viewer": "false", "CloudFront-Viewer-ASN": "1680", "CloudFront-Viewer-Country": "IL", "Content-Type": "multipart/form-data; boundary=--------------------------592233465752962090703619", "Host": "6y5o0gb78g.execute-api.eu-west-3.amazonaws.com", "Origin": "https://doc2txt.com", "Postman-Token": "f39dea09-b71f-47bb-aa8a-57783df2ddf1", "Pragma": "no-cache", "Referer": "https://doc2txt.com/", "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36", "Via": "1.1 759e09affff41285e9585e1a31532bd4.cloudfront.net (CloudFront)", "X-Amz-Cf-Id": "5jCh7W91ayJWUf_75DfWr-IfCj2CvCobvTlshBjoqaElhVWsNmlFuA==", "X-Amzn-Trace-Id": "Root=1-63b7df99-5a343d1a16ca5cde72a47d66", "X-Forwarded-For": "176.12.157.166, 130.176.1.83", "X-Forwarded-Port": "443", "X-Forwarded-Proto": "https"}, "multiValueHeaders": {"Accept": ["application/json, text/plain, */*"], "Accept-Encoding": ["gzip, deflate, br"], "Accept-Language": ["en-US,en;q=0.9,he;q=0.8"], "Cache-Control": ["no-cache"], "CloudFront-Forwarded-Proto": ["https"], "CloudFront-Is-Desktop-Viewer": ["true"], "CloudFront-Is-Mobile-Viewer": ["false"], "CloudFront-Is-SmartTV-Viewer": ["false"], "CloudFront-Is-Tablet-Viewer": ["false"], "CloudFront-Viewer-ASN": ["1680"], "CloudFront-Viewer-Country": ["IL"], "Content-Type": ["multipart/form-data; boundary=--------------------------592233465752962090703619"], "Host": ["6y5o0gb78g.execute-api.eu-west-3.amazonaws.com"], "Origin": ["https://doc2txt.com"], "Postman-Token": ["f39dea09-b71f-47bb-aa8a-57783df2ddf1"], "Pragma": ["no-cache"], "Referer": ["https://doc2txt.com/"], "User-Agent": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36"], "Via": ["1.1 759e09affff41285e9585e1a31532bd4.cloudfront.net (CloudFront)"], "X-Amz-Cf-Id": ["5jCh7W91ayJWUf_75DfWr-IfCj2CvCobvTlshBjoqaElhVWsNmlFuA=="], "X-Amzn-Trace-Id": ["Root=1-63b7df99-5a343d1a16ca5cde72a47d66"], "X-Forwarded-For": ["176.12.157.166, 130.176.1.83"], "X-Forwarded-Port": ["443"], "X-Forwarded-Proto": ["https"]}, "queryStringParameters": null, "multiValueQueryStringParameters": null, "pathParameters": null, "stageVariables": null, "requestContext": {"resourceId": "7g6rs7", "resourcePath": "/ocrReceipt", "httpMethod": "POST", "extendedRequestId": "eT_iDFWXCGYFfyQ=", "requestTime": "06/Jan/2023:08:45:26 +0000", "path": "/Prod/ocrReceipt", "accountId": "899418482974", "protocol": "HTTP/1.1", "stage": "Prod", "domainPrefix": "6y5o0gb78g", "requestTimeEpoch": 1672994726594, "requestId": "8a23ce92-11fe-4e54-bbb0-5ad355130833", "identity": {"cognitoIdentityPoolId": null, "accountId": null, "cognitoIdentityId": null, "caller": null, "sourceIp": "176.12.157.166", "principalOrgId": null, "accessKey": null, "cognitoAuthenticationType": null, "cognitoAuthenticationProvider": null, "userArn": null, "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36", "user": null}, "domainName": "6y5o0gb78g.execute-api.eu-west-3.amazonaws.com", "apiId": "6y5o0gb78g"}, "body": "...", "isBase64Encoded": false}
template.yaml
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
python3.8
Sample SAM Template for ocrSam
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 3
MemorySize: 128
Resources:
Doc2txtFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
Timeout: 60
Role: arn:aws:iam::899418482974:role/service-role/ocrReceipt-role-v2edsg0i
PackageType: Image
Architectures:
- x86_64
Events:
Doc2txt:
Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /ocrReceipt
Method: ANY
Metadata:
Dockerfile: Dockerfile
DockerContext: ./doc2txt
DockerTag: python3.8-v1
Outputs:
# ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
# Find out more about other implicit resources you can reference within SAM
# https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
Doc2txtApi:
Description: "API Gateway endpoint URL for Prod stage for Doc2txt function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/ocrReceipt/"
Doc2txtFunction:
Description: "Doc2txt Lambda Function ARN"
Value: !GetAtt Doc2txtFunction.Arn
Doc2txtFunctionIamRole:
Description: "Implicit IAM Role created for Doc2txt function"
Value: arn:aws:iam::899418482974:role/service-role/ocrReceipt-role-v2edsg0i #!GetAtt Doc2txtFunctionRole.Arn
EDIT:
I was able to get "isBase64Encoded": true with the following:
Globals:
Api:
BinaryMediaTypes:
- "*/*"
I was able to get "isBase64Encoded": true with the following:
Globals:
Api:
BinaryMediaTypes:
- "*/*"
apparently the content-lenght is not necessary, so I just remove it from the code:
headers = {
"content-type": headers["Content-Type"],
# "content-length": headers["Content-Length"],
}
fs = cgi.FieldStorage(fp=fp, environ=environ, headers=headers)

Custom app in Microsoft Teams can't connect to it's own API from desktop app

I am trying to embed our app into Teams. We have it working from the web version of MS Teams. However when we try it from the MS Teams desktop app, our app's tab is unable to communicate with our API saying net::ERR_BLOCKED_BY_RESPONSE for all XHR requests the app tries to make to the API. The OPTIONS request seems to get some response headers back but is still listed as failed (see header below)
What is blocking the requests in the Desktop app? We have tried various security header changes and disabling our apps service worker among other things but to no avail. The application loads fine (we have the iframe enabling headers set in the CSP - see below). It's just the communication with the API that is blocked.
Application details that we are embedding in the MS Teams APP as a custom app
Progressive web app
Angular v13 with
Angular routing strategy (useHash = false)
API is REST based
NB: The app tab works in the mobile app version of teams on iOS as well as on the web version
Response headers that are returned by the API we are trying to talk to on the OPTIONS request
access-control-allow-headers: Content-Type, Authorization, V, D
access-control-allow-methods: OPTIONS, GET, POST, DELETE, PUT
access-control-allow-origin: https://**redacted**
access-control-max-age: 86400
cache-control: no-cache
content-length: 0
date: Wed, 12 Jan 2022 00:14:44 GMT
permissions-policy: geolocation=()
referrer-policy: no-referrer
server: CloudFront
status: 200
strict-transport-security: max-age=31536000; includeSubDomains
vary: Origin
via: 1.1 **redacted**.cloudfront.net (CloudFront)
x-amz-cf-id: **redacted**
x-amz-cf-pop: SYD62-P1
x-cache: Miss from cloudfront
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
Headers and CSP on the application (SPA)
age: 6
cache-control: no-cache
content-encoding: br
content-security-policy: default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data: https://*.**redacted**.com https://**redacted**.app https://*.**redacted**.app; connect-src https://*.**redacted**.com https://**redacted**.app https://*.**redacted**.app; object-src 'none'; frame-ancestors teams.microsoft.com *.office.com outlook.office.com outlook.office365.com
content-type: text/html
cross-origin-embedder-policy: require-corp
cross-origin-opener-policy: same-origin
cross-origin-resource-policy: same-origin
date: Wed, 12 Jan 2022 00:22:58 GMT
etag: **redacted**
expect-ct: max-age=0, report-uri="https://**redacted**.report-uri.com/r/d/ct/reportOnly"
last-modified: Tue, 11 Jan 2022 23:27:02 GMT
permissions-policy: geolocation=()
referrer-policy: no-referrer-when-downgrade
server: CloudFront
status: 304
strict-transport-security: max-age=31536000; includeSubDomains
vary: Accept-Encoding
via: 1.1 **redacted**.cloudfront.net (CloudFront)
x-amz-cf-id: **redacted**
x-amz-cf-pop: SYD62-P1
x-cache: Hit from cloudfront
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
The manifest, currently setup with a configure page that adds a tab after choosing our customer account name which generates the url for the tab.
{
"$schema": "https://developer.microsoft.com/en-us/json-schemas/teams/v1.11/MicrosoftTeams.schema.json",
"manifestVersion": "1.11",
"version": "4.1.8",
"id": "***",
"packageName": "app.***",
"developer": {
"name": "*** Pty Limited",
"websiteUrl": "https://www.***.com",
"privacyUrl": "https://www.***.com/privacy",
"termsOfUseUrl": "https://***.com/resources/20201019+***+Terms+and+Conditions.pdf"
},
"icons": {
"color": "color.png",
"outline": "outline.png"
},
"name": {
"short": "***",
"full": "*** For Microsoft Teams"
},
"description": {
"short": "***",
"full": "***"
},
"accentColor": "#F9F9FA",
"configurableTabs": [
{
"configurationUrl": "https://dev.***.app/ms-teams/teams-configure.html",
"canUpdateConfiguration": true,
"scopes": [
"team"
],
"context": [
"channelTab"
],
"supportedSharePointHosts": [
"sharePointFullPage"
]
}
],
"permissions": [
"identity",
"messageTeamMembers"
],
"validDomains": [
"dev.***.app",
"*.dev.***.app",
"*.teams.***.app",
"api.test.***.com",
"api.***.com",
"*.test.***.app",
"*.***.app"
]
}

Multiple Set-Cookie headers ignored by API Gateway in combination with Lambda integration and CloudFront

My setup looks like this:
|––––––––––––| |–––––––––––––| |–––––––––––––––––|
| | <- origin 1 -> | API Gateway | <-> | Lambda function |
| | |–––––––––––––| |–––––––––––––––––|
| CloudFront |
| | |–––––––––––––|
| | <- origin 2 -> | S3 bucket |
|––––––––––––| |–––––––––––––|
I need CloudFront in front of the API Gateway to get automatic http->https redirection.
I'm using a custom login.example.com subdomain w/ CloudFront.
API Gateway's generated URL is the origin 1 for CloudFront distribution.
This all works as expected.
I can even return one Set-Cookie header from the lambda function and it will get passed on until it reaches the browser.
{
"statusCode": 302,
"body": "",
"headers": {
"location": "/test",
"surrogate-control": "no-store",
"cache-control": "no-store, no-cache, must-revalidate, proxy-revalidate",
"pragma": "no-cache",
"expires": "0",
"content-length": "0",
"date": "Fri, 19 Feb 2021 17:25:56 GMT",
"connection": "keep-alive",
"set-cookie": "cookie1=68abcdbefbef7d84c26e68; Max-Age=2592000; Domain=example.com; Path=/; HttpOnly; Secure; SameSite=Strict"
},
"isBase64Encoded": false
}
Adding another one isn't working - as expected when you look at the docs:
https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#apigateway-multivalue-headers-and-parameters
https://aws.amazon.com/blogs/compute/support-for-multi-value-parameters-in-amazon-api-gateway/
{
"statusCode": 302,
"headers": {
"location": "/test",
"set-cookie": [
"cookie1=68abcdbefbef7d84c26e68; Max-Age=2592000; Domain=example.com; Path=/; HttpOnly; Secure; SameSite=Strict",
"cookie2-login=; Max-Age=0; Path=/; Expires=Thu, 01 Jan 1970 00:00:00 GMT; HttpOnly; Secure"
],
"surrogate-control": "no-store",
"cache-control": "no-store, no-cache, must-revalidate, proxy-revalidate",
"pragma": "no-cache",
"expires": "0",
"content-length": "0"
}
}
Both of these will be ignored/removed.
But even when I'm using the multiValueHeaders object to return more than one of the same kind like this:
{
"statusCode": 302,
"body": "",
"headers": {
"location": "/test",
"surrogate-control": "no-store",
"cache-control": "no-store, no-cache, must-revalidate, proxy-revalidate",
"pragma": "no-cache",
"expires": "0",
"content-length": "0",
"date": "Fri, 19 Feb 2021 17:25:56 GMT",
"connection": "keep-alive"
},
"isBase64Encoded": false,
"multiValueHeaders": {
"Set-Cookie": [
"cookie1=68abcdbefbef7d84c26e68; Max-Age=2592000; Domain=example.com; Path=/; HttpOnly; Secure; SameSite=Strict",
"cookie2-login=; Max-Age=0; Path=/; Expires=Thu, 01 Jan 1970 00:00:00 GMT; HttpOnly; Secure"
]
}
}
the API Gateway removes/ignores them from the response it passes on to CloudFront.
What am I doing wrong?
Do I have to map something in the API Gateway when using the multiValueHeaders?
Normal headers['set-cookie'] is passed on automatically but multiValueHeaders not?
Are the additional attributes a problem?
Is it a problem that I'm trying to set the cookie for the root-domain and not the login.example.com domain?
Not sure how Philipp missed this, as it was on the same page he cited:
To customize the response, your Lambda function should return a response with the following format.
{
"cookies" : ["cookie1", "cookie2"],
"isBase64Encoded": true|false,
"statusCode": httpStatusCode,
"headers": { "headername": "headervalue", ... },
"body": "Hello from Lambda!"
}
So just return cookies: ['name=value', 'name=value']
Source: https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html#http-api-develop-integrations-lambda.v2
Finally found the answer myself:
The new payload format (2.0) does not support multiValueHeaders.
Working with AWS Lambda proxy integrations for HTTP APIs
[...] Format 2.0 doesn't have multiValueHeaders or multiValueQueryStringParameters fields. Duplicate headers are combined with commas and included in the headers field. Duplicate query strings are combined with commas and included in the queryStringParameters field. [...]
https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html
So I'm rewriting the set-cookie headers to different spellings:
Set-cookie
sEt-cookie
seT-cookie
That was the way you had to do it before there were multiValueHeaders - but it seems it is still the only when you're using the new payload format :(

what is the correct way to get the origin header in a serverless deployed lambda fronted by api gateway?

I am using the serverless framework to deploy a simple lambda (written in node/ express) behind api gateway...
In a GET I can see the origin header, but in a POST I cannot - it's not passed into my lambda function!
Anyone know why not and how to make it pass?
Background:
I did notice that in the POST the origin appears to be appended to the query string in the logs:
originalUrl: '/dev/endpoint?Origin=MY%20ORIGIN%20HERE',
So, I could extract the origin from the query string values, but I want to know if this is the correct way to do things on the serverless framework or should I be setting something to allow the API Gateway to send the origin through on a POST like it does on a GET ? Why are the 2 verbs behaving differently?
My Code/ Config:
the function definition in the serverless.yml looks like:
myGetFunction:
handler: lambda/index.handler
events:
- http:
path: /endpoint
method: get
cors: true
myPostFunction:
handler: lambda/index.handler
events:
- http:
path: /endpoint
method: post
cors: true
If I just put the following in my index.handler:
app.use((req, res) => {
console.log('LOG REQUEST', req)
res.send('interesting')
})
In my handler function, I only want to get the origin request header that was sent. In the GET it's easy: the cloudwatch logs show that it's available in the GET request in the headers object:
headers:
{
accept: '*/*',
'accept-encoding': 'gzip, deflate',
'cache-control': 'no-cache',
'cloudfront-forwarded-proto': 'https',
'cloudfront-is-desktop-viewer': 'true',
'cloudfront-is-mobile-viewer': 'false',
'cloudfront-is-smarttv-viewer': 'false',
'cloudfront-is-tablet-viewer': 'false',
'cloudfront-viewer-country': 'GB',
host: 'X.execute-api.us-east-1.amazonaws.com',
origin: 'MY ORIGIN HERE',
BUT: in the POST all the other headers are there, but origin is blank.
I did a bit of digging on this and ran into a couple realizations.
By default, it seems that no "origin" header passed in the API Gateway events. (I created a fresh serverless project, and just echoed back exactly what the API gateway event was.) So this is coming from some other source. I figured it may be a custom domain, and tested that. No dice.
My only other guess is that you have this behind some other layer (CloudFront?) that is forwarding these headers for you. If that ends up being the case, I would suggest you look and see if you can make it forward these headers for the POST request as it is for the GET request.
My only other final thought if none of the above is true is that there is some magic going on in some of the express middleware. I doubt this is the case.
For reference, this was my full serverless.yml and handler.js I tested with as well as a full unaltered event object I got in the endpoint.
service: so-test
provider:
name: aws
runtime: nodejs8.10
functions:
myGetFunction:
handler: handler.hello
events:
- http:
path: /endpoint
method: get
cors: true
myPostFunction:
handler: handler.hello
events:
- http:
path: /endpoint
method: post
cors: true
And the nodejs code:
'use strict';
module.exports.hello = async (event, context) => {
return {
statusCode: 200,
body: JSON.stringify({
message: 'Go Serverless v1.0! Your function executed successfully!',
input: event,
}),
};
};
Finally the response object
{
"message": "Go Serverless v1.0! Your function executed successfully!",
"input": {
"resource": "/endpoint",
"path": "/test/endpoint",
"httpMethod": "GET",
"headers": {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.9",
"CloudFront-Forwarded-Proto": "https",
"CloudFront-Is-Desktop-Viewer": "true",
"CloudFront-Is-Mobile-Viewer": "false",
"CloudFront-Is-SmartTV-Viewer": "false",
"CloudFront-Is-Tablet-Viewer": "false",
"CloudFront-Viewer-Country": "US",
"Host": "so-test.serverless-examples.com",
"upgrade-insecure-requests": "1",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36",
"Via": "2.0 f92491812e422470607f365e923929b5.cloudfront.net (CloudFront)",
"X-Amz-Cf-Id": "6AwZPV3uCYxseJIAmsGzhApzRostCiLXwwM3XsbSJP4K8hQx11MSgw==",
"X-Amzn-Trace-Id": "Root=1-5c086dd9-bce03ab0c216116fa6de9786",
"X-Forwarded-For": "55.55.55.555, 70.132.32.155",
"X-Forwarded-Port": "443",
"X-Forwarded-Proto": "https"
},
"multiValueHeaders": {
"Accept": [
"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
],
"Accept-Encoding": [
"gzip, deflate, br"
],
"Accept-Language": [
"en-US,en;q=0.9"
],
"CloudFront-Forwarded-Proto": [
"https"
],
"CloudFront-Is-Desktop-Viewer": [
"true"
],
"CloudFront-Is-Mobile-Viewer": [
"false"
],
"CloudFront-Is-SmartTV-Viewer": [
"false"
],
"CloudFront-Is-Tablet-Viewer": [
"false"
],
"CloudFront-Viewer-Country": [
"US"
],
"Host": [
"so-test.serverless-examples.com"
],
"upgrade-insecure-requests": [
"1"
],
"User-Agent": [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
],
"Via": [
"2.0 f92491812e422470607f365e923929b5.cloudfront.net (CloudFront)"
],
"X-Amz-Cf-Id": [
"6AwZPV3uCYxseJIAmsGzhApzRostCiLXwwM3XsbSJP4K8hQx11MSgw=="
],
"X-Amzn-Trace-Id": [
"Root=1-5c086dd9-bce03ab0c216116fa6de9786"
],
"X-Forwarded-For": [
"55.55.55.555, 70.132.32.155"
],
"X-Forwarded-Port": [
"443"
],
"X-Forwarded-Proto": [
"https"
]
},
"queryStringParameters": null,
"multiValueQueryStringParameters": null,
"pathParameters": null,
"stageVariables": null,
"requestContext": {
"resourceId": "mftg6x",
"resourcePath": "/endpoint",
"httpMethod": "GET",
"extendedRequestId": "RdYZ7HaxoAMFQYQ=",
"requestTime": "06/Dec/2018:00:31:21 +0000",
"path": "/test/endpoint",
"accountId": "800708648372",
"protocol": "HTTP/1.1",
"stage": "dev",
"domainPrefix": "so-test",
"requestTimeEpoch": 1544056281163,
"requestId": "410632a3-f8ee-11e8-a7e2-7d886f93a0e4",
"identity": {
"cognitoIdentityPoolId": null,
"accountId": null,
"cognitoIdentityId": null,
"caller": null,
"sourceIp": "55.55.55.555",
"accessKey": null,
"cognitoAuthenticationType": null,
"cognitoAuthenticationProvider": null,
"userArn": null,
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36",
"user": null
},
"domainName": "so-test.serverless-examples.com",
"apiId": "txctij0cnp"
},
"body": null,
"isBase64Encoded": false
}
}

Why doesn’t the form not POST in Excel 2016 for Mac?

We are testing an add-in for Excel that has a form that is POSTed to https://httpbin.org/anything. The form will not POST in Excel 2016 version 15.39 (171010) for Mac (High Sierra ver 10.13.1). Here are the HTML form’s essentials:
<script type="text/javascript">
//submit form
$("#testForm").submit();
</script>
</head>
<body>
<form method="POST" id="testForm" action="https://httpbin.org/anything" accept-charset="UTF-8" target="_blank">
<div>
<input type='hidden' name='mergeDataFormat' value='csv'>
</div>
<div>
<input type="hidden" name="mergeData" id="mergeData" value='Name,Street,"City, State",ZIP Code'>
</div>
<input type="submit" value="Send" id="submitForm" />
</form>
AppDomain was set in the Manifest:
<AppDomains>
<AppDomain>https://httpbin.org/</AppDomain>
...
</AppDomains>
This is what we see in Charles HTTP monitor ver 4.2:
GET /anything HTTP/1.1
Host httpbin.org
Connection keep-alive
User-Agent Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36
Upgrade-Insecure-Reque
sts 1
Accept text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding gzip, deflate, br
Accept-Language en-US,en;q=0.9
Cookie _gauges_unique_day=1; _gauges_unique_month=1; _gauges_unique_year=1; _gauges_unique=1
HTTP/1.1 200 OK
Server: meinheld/0.6.1
Date: Wed, 15 Nov 2017 18:07:11 GMT
Content-Type: application/json
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
X-Powered-By: Flask
X-Processed-Time: 0.00142621994019
Content-Length: 726
Via: 1.1 vegur
Connection: Keep-alive
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.9",
"Connection": "close",
"Cookie": "_gauges_unique_day=1; _gauges_unique_month=1; _gauges_unique_year=1; _gauges_unique=1",
"Host": "httpbin.org",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36"
},
"json": null,
"method": "GET",
"origin": "<ip-address-here>",
"url": "https://httpbin.org/anything"
}
After pressing the Send button, the httpbin.org page pops up to show this:
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.9",
"Connection": "close",
"Cookie": "_gauges_unique_day=1; _gauges_unique_month=1; _gauges_unique_year=1; _gauges_unique=1",
"Host": "httpbin.org",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36"
},
"json": null,
"method": "GET",
"origin": "<ip-address-here>",
"url": "https://httpbin.org/anything"
}
It seems like there was no POST to the URL. No errors seen in Excel. The form can be POSTed successfully to the same URL in a browser on the Mac. This issue is seen only in Excel 2016 for Mac. The same code works fine in Excel 2016 for Windows (7 and 10) and Excel online. Any insights to help resolve this is much appreciated.
See related question Is the Excel add-in incompatible with Excel 2016 for Mac when using POST method for forms?
Update Excel 2016 on Mac to version 16.9.0 (18011602). We are able to POST from Excel 2016 for Mac! Thanks to the entire MS team.
Please refer this post for latest on this. Seems to be a duplicate issue.

Resources