esp32, esp32_https_server library, self-signed certificate, cors and 499 status code - https

i am working on an ESP32 project. one of my goals is to communicate with the ESP32 from a website using javascript fetch or XMLHttpRequest().
the ESP32 is connected to my local network and i am using the esp32_https_server library. it uses a self-signed certificate which the browser indicates as valid (but issues a warning, "Connection not protected" due to the self-signed certificate). the website has a CA certificate and is secure.
in testing, the esp32 is conected via USB to my computer, idealy i would like it to stand alone.
the problem i am experiencing is that i cannot seem to connect to the esp32. i keep getting status code 499 errors.
my questions are:
1) how do i successfully connect to the esp32 server from a secure website to get data frome the esp32?
2) how do i do this when the esp32 is not connected to my pc via the usb cable?
please see more info regarding the esp32 set up and responses below.
here's the esp32 code:
ResourceNode *nodeRoot = new ResourceNode("/", "GET", [](HTTPRequest *req, HTTPResponse *res) {
ResourceParameters *params = req->getParams();
std::string action = params->getRequestParameter("action");
String aksie = action.c_str();
Serial.println("Aksie: " + aksie);
if (aksie != "upload_data" && aksie != "upload_current_temp")
{
// this should be home page displayed
// Set the response status
res->setStatusCode(200);
res->setStatusText("success");
res->println("Secure Hello World!!!");
}
else
{
// either uploads..
processParams(aksie, res);
}
});
secureServer->registerNode(nodeRoot);
and here's the code that processes the "upload_current_temp" request:
if (action == "upload_current_temp")
{
// get random temperature
int currentTemp = random(0, 9);
String temp = String(currentTemp);
Serial.println("upload current temperature");
Serial.println("uploadCurrentTemp: " + temp);
std::string tem = temp.c_str();
// Set the response status
res->setStatusCode(200);
res->setStatusText("success current temperature");
StaticJsonDocument<200> doc;
doc["temperature"] = temp;
// Produce a minified JSON document
String output;
serializeJson(doc, output);
Serial.println("curent temp json output: " + output);
deserializeJson(doc, output);
// Set the content type of the response
res->setHeader("Content-Type", "application/json");
res->setHeader("Access-Control-Allow-Origin", "*");
res->setHeader("Access-Control-Allow-Methods", "GET,POST,OPTIONS");
// As HTTPResponse implements the Print interface, this works fine. Just remember
// to use *, as we only have a pointer to the HTTPResponse here:
serializeJson(doc, *res);
}
and also in setUp() i have this line:
secureServer->setDefaultHeader("Access-Control-Allow-Origin", "*"); //replace * with actual address
when using:
const xhr = new XMLHttpRequest();
const url = 'https://192.168.0.102/?action=upload_current_temp';
xhr.open('GET', url);
xhr.responseType = 'text';
xhr.onload = function () {
const data = xhr.response;
console.log(data);
if (this.readyState == 4 && this.status == 200) {
var obj = JSON.parse(this.responseText);
console.log("getCurTemp(), responseText: " + JSON.stringify(this.responseText, null, 2));
currentTemperature = obj.temperature;
console.log("current temperature: " + currentTemperature);
document.getElementById('currentTemp').innerHTML = currentTemperature;
}
};
xhr.send();
i get these errors (in opera):
499 (Request has been forbidden by antivirus)
has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
and in chrome:
has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
with these headers (opera):
Request URL: https://192.168.0.102/?action=upload_current_temp
Request Method: GET
Status Code: 499 Request has been forbidden by antivirus
Remote Address: 192.168.0.102:443
Referrer Policy: no-referrer-when-downgrade
Cache-Control: no-store, no-cache, must-revalidate, max-age=0
Connection: close
Content-Length: 52266
Content-Type: text/html; charset=utf-8
Expires: Mon, 04 Dec 1999 21:29:02 GMT
Pragma: no-cache
Accept: /
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Host: 192.168.0.102
Origin: https://istimuli.co.uk
Referer: https://istimuli.co.uk/?code=66b72f8e-400c-4adb-ad42-f4efec391d06
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36 OPR/67.0.3575.79
action: upload_current_temp
and when using :
var url = "https://192.168.0.102/?action=upload_current_temp";
var request = new Request(url, {
method: 'GET',
mode: 'cors', // no-cors, *cors, same-origin
headers: {
'Content-Type': 'application/json'
}
});
fetch(request).then(function (response) {
// Convert to JSON
return response.json();
}).then(function (data) {
console.log("temp: " + JSON.stringify(data));
return data;
}).catch(function (error) {
console.log('Request failed', error)
return 000;
});
i get these errors in opera:
499 (Request has been forbidden by antivirus)
has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
and in chrome:
has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
and these are the headers (opera):
1 requests
51.3 KB transferred
51.0 KB resources
Request URL: https://192.168.0.102/?action=upload_current_temp
Request Method: OPTIONS
Status Code: 499 Request has been forbidden by antivirus
Remote Address: 192.168.0.102:443
Referrer Policy: no-referrer-when-downgrade
Cache-Control: no-store, no-cache, must-revalidate, max-age=0
Connection: close
Content-Length: 52266
Content-Type: text/html; charset=utf-8
Expires: Mon, 04 Dec 1999 21:29:02 GMT
Pragma: no-cache
Accept: /
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Access-Control-Request-Headers: content-type
Access-Control-Request-Method: GET
Connection: keep-alive
Host: 192.168.0.102
Origin: https://istimuli.co.uk
Referer: https://istimuli.co.uk/?code=66b72f8e-400c-4adb-ad42-f4efec391d06
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36 OPR/67.0.3575.79
action: upload_current_temp

Related

Why CORS error happens even if Access-Control-Allow-Origin is present in OPTIONS response in AWS lambda trigger (API Gateway of REST API)

I'm mad. Why error No 'Access-Control-Allow-Origin' header is present on the requested resource still raises even if 'Access-Control-Allow-Origin' is present in preflight response?
this is my configure:
AWS lambda function backend code (nodejs):
'use strict';
require('dotenv').config();
const mysql = require('mysql2/promise');
exports.handler = async (event, context, callback) => {
if (event.httpMethod == 'OPTIONS') {
return {
headers: {
"foo": "bar",
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Headers": "Origin, X-Requested-With, Content-Type, Accept",
"Access-Control-Allow-Methods": "OPTIONS,POST,GET"
},
statusCode: 200,
}
}
if (!event.body) {
throw new Error("no body provided")
}
const body = JSON.parse(event.body)
if (!body.statement) {
throw new Error("no statement provided")
}
const connection = await mysql.createConnection(process.env.DATABASE_URL);
const [results] = await connection.query(body.statement)
connection.end();
let response = {
statusCode: 200,
headers: {
"Access-Control-Allow-Headers": "Origin, X-Requested-With, Content-Type, Accept",
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "OPTIONS,POST,GET"
},
contentType: "application/json",
body: JSON.stringify(results)
};
return response;
};
this is the trigger configure (API Gateway of REST API)
API Gateway: planetscale-API
arn:aws:execute-api:ap-northeast-1:288496770745:832iyue4j2/*/*/planetscale
API endpoint: https://832iyue4j2.execute-api.ap-northeast-1.amazonaws.com/default/planetscale
Details
API type: REST
Authorization: NONE
Method: ANY
Resource path: /planetscale
Service principal: apigateway.amazonaws.com
Stage: default
Statement ID: lambda-11567109-822a-4850-abcf-14a1b0565c54
and this is the js code:
await axios.post("https://832iyue4j2.execute-api.ap-northeast-1.amazonaws.com/default/planetscale", { statement:"select * from t" });
this is what happend in chrome network
1.preflight request
Request URL: https://832iyue4j2.execute-api.ap-northeast-1.amazonaws.com/default/planetscale
Request Method: OPTIONS
Status Code: 200
Remote Address: 127.0.0.1:64646
Referrer Policy: strict-origin-when-cross-origin
access-control-allow-headers: Origin, X-Requested-With, Content-Type, Accept
access-control-allow-methods: OPTIONS,POST,GET
access-control-allow-origin: *
content-length: 0
content-type: application/json
date: Tue, 09 Aug 2022 09:24:19 GMT
x-amz-apigw-id: WlsmmH0mNjMFZGA=
x-amzn-requestid: d25ff521-94ee-48d2-bc33-68bb92495b42
x-amzn-trace-id: Root=1-62f227c3-14547cf1741c86e83d846763;Sampled=0
xxxx: aaaa
:authority: 832iyue4j2.execute-api.ap-northeast-1.amazonaws.com
:method: OPTIONS
:path: /default/planetscale
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: zh-CN,zh;q=0.9
access-control-request-headers: content-type
access-control-request-method: POST
origin: http://localhost:8080
referer: http://localhost:8080/
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: cross-site
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36
2. POST request
Request URL: https://832iyue4j2.execute-api.ap-northeast-1.amazonaws.com/default/planetscale
Request Method: POST
Status Code: 502
Referrer Policy: strict-origin-when-cross-origin
content-length: 36
content-type: application/json
date: Tue, 09 Aug 2022 09:24:20 GMT
x-amz-apigw-id: WlsmpE_HtjMFmfw=
x-amzn-errortype: InternalServerErrorException
x-amzn-requestid: a3f991a8-1f6b-445f-b5db-697c7efaa1f8
:authority: 832iyue4j2.execute-api.ap-northeast-1.amazonaws.com
:method: POST
:path: /default/planetscale
:scheme: https
accept: application/json, text/plain, */*
accept-encoding: gzip, deflate, br
accept-language: zh-CN,zh;q=0.9
content-length: 50
content-type: application/json
origin: http://localhost:8080
referer: http://localhost:8080/
sec-ch-ua: ".Not/A)Brand";v="99", "Google Chrome";v="103", "Chromium";v="103"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: cross-site
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36
this is what happends in chrome console :
localhost/:1
Access to XMLHttpRequest at 'https://832iyue4j2.execute-api.ap-northeast-1.amazonaws.com/default/planetscale' from origin 'http://localhost:8080' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
App.vue?18cf:151 error aws lambda
xhr.js?ed5b:220
POST https://832iyue4j2.execute-api.ap-northeast-1.amazonaws.com/default/planetscale net::ERR_FAILED 502
dispatchXhrRequest # xhr.js?ed5b:220
xhrAdapter # xhr.js?ed5b:16
dispatchRequest # dispatchRequest.js?d0b9:58
request # Axios.js?3509:109
httpMethod # Axios.js?3509:144
wrap # bind.js?285c:9
Comment.query # models.js?a3ef:37
exec # modelsql.mjs?8304:626
download # App.vue?18cf:149
tryDownload # App.vue?18cf:200
invokeWithErrorHandling # vue.runtime.esm.js?3841:2988
invoker # vue.runtime.esm.js?3841:1786
original_1._wrapper # vue.runtime.esm.js?3841:7405

rxjs5 Observable.ajax ignores explicitly set HTTP headers

I'm getting my feet wet with redux-observable and OAuth2 authentication. I'm stuck at the point where I have to POST adding Authorization header to my HTTP request. The header is has not been added. Instead, I see any custom-set header names as values of Access-Control-Request-Headers, and that's it.
This is a redux-observable 'epic':
const epicAuth = function(action$){
return action$.ofType(DO_AUTHENTICATE)
.mergeMap(
action => Rx.Observable.ajax( authRequest(action.username, action.password))
.map( response => renewTokens(response))
.catch(error => Rx.Observable.of({
type: AJAX_ERROR,
payload: error,
error: true,
}))
)
}
This is my request object:
const authRequest = function(username, password){
return {
url: TOKEN_PROVIDER + '?grant_type=password&username=' + username + '&password=' + password,
method: 'POST',
responseType: 'json',
crossDomain: true,
withCredentials: true,
headers: {
'Authorization': 'Basic <base64-encoded-user#password>',
}
}
}
The HTTP headers captured:
http://localhost:8082/api/oauth/token?grant_type=password&username=xxx&password=yyy
OPTIONS /api/oauth/token?grant_type=password&username=xxx&password=yyy HTTP/1.1
Host: localhost:8082
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:56.0) Gecko/20100101 Firefox/56.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Access-Control-Request-Method: POST
Access-Control-Request-Headers: authorization
Origin: http://localhost:3000
DNT: 1
Connection: keep-alive
HTTP/1.1 401
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
WWW-Authenticate: Basic realm="MY_REALM/client"
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 1098
Date: Wed, 01 Nov 2017 17:57:38 GMT
It all ends up with 401 response, since the Authorization header was not sent. I have tested the Oauth2 endpoint manually with Postman tool, and all went well: I've got a valid access token, could renew it, etc. CORS is enabled on server side.
What am I missing here?
The client code is working correctly.
You've captured the OPTIONS cors request, which is asking the server if it is OK to POST the Authorization header (see the Access-Control-Request-Headers: authorization).
Make sure that you've configured CORS correctly on your server. It shouldn't be trying to authenticate OPTIONS calls. It should instead be sending a proper response which tells the browser if it is allowed to make the POST call.

Please help me understand Ajax request versus Backbone fetch()

My app can currently hit our API with a standard JQuery Ajax GET request and get good data back. CORS has been properly implemented on the remote server as far as I can see. Here are the response headers:
company_client_envelope_id: 88764736-6654-22e4-br344-a1w2239a892d
access-control-allow-headers: X-Requested-With, Cookie, Set-Cookie, Accept, Access-Control
Allow-Credentials, Origin, Content-Type, Request-Id , X-Api-Version, X-Request-Id,Authorization, COMPANY_AUTH_WEB
access-control-expose-headers: Location
response-time: 55
request-id: 88764736-6654-22e4-br344-a1w2239a892d
company_api_version: 0.01.09
server: localhost
transfer-encoding: chunked
connection: close
access-control-allow-credentials: true
date: Sun, 09 Feb 2014 14:44:05 GMT
access-control-allow-origin: *
access-control-allow-methods: GET, POST
content-type: application/json
However, using Backbone and calling the same GET request by using fetch() causes the following CORS error:
No 'Access-Control-Allow-Origin' header is present on the requested resource.
I cannot figure out what the difference is. Both requests are running from localhost.
In the case of the AJAX query, the following is being sent as requested by the API guys:
headers: {
"accept":"application/json"
}
And in the case of the model and collection declaration I am sending the headers like so:
MyApp.someCollection = Backbone.Collection.extend(
{
model:MyApp.someModel,
headers: {
'Accept':'application/json',
'withCredentials': 'true'
},
url: MYCOMPANY_GLOBALS.API + '/endpoint'
});
and my fetch is simply:
someCollection.fetch();
===============================
Added in response to: #ddewaele
These are the headers from the network tab:
Request URL:http://api-blah.com:3000/
Request Headers CAUTION: Provisional headers are shown.
Accept:application/json
Cache-Control:no-cache
Origin:http://localhost
Pragma:no-cache
Referer:http://localhost/blah/blah/main.html
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107Safari/537.36
There is no pre-flight or remote headers from the API server:
many thanks,
Wittner
I've recommended to you rewrite Backbone.sync method, because in your app you have some security field for example and other reason.
var oldBackboneSync = Backbone.sync;
// Override Backbone.Sync
Backbone.sync = function (method, model, options) {
if (method) {
if (options.data) {
// properly formats data for back-end to parse
options.data = JSON.stringify(options.data);
}
// transform all delete requests to application/json
options.contentType = 'application/json';
}
return oldBackboneSync.apply(this, [method, model, options]);
}
You can add different headers as you want.

OPTIONS request while adding Authorization header

Here's my problem, I'm trying to add the following Header to my GET ajax request :
Authorization: Basic XXXXXXX
So I wrote this :
function() {
var storage = window.localStorage;
var currentUsername = storage.getItem("username");
var currentPassword = storage.getItem("password");
var auth = makeBasicAuth(currentUsername, currentPassword);
$.ajaxSetup ({
beforeSend: function(request) {
request.setRequestHeader( "Authorization", auth );
}
});
I already have check the value of auth which is correct. And when I try to send my request, I get that :
OPTIONS <myurl> Error 401
Here are the headers of the request and of its answer :
Request :
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding gzip, deflate
Accept-Language fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Access-Control-Request-He... authorization
Access-Control-Request-Me... GET
Connection keep-alive
Host <myhost>
Origin null
User-Agent Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:22.0) Gecko/20100101 Firefox/22.0
Answer :
Access-Control-Allow-Head... Authorization
Access-Control-Allow-Meth... *
Access-Control-Allow-Orig... *
Access-Control-Max-Age 3628800
Connection Keep-Alive
Content-Encoding gzip
Content-Length 440
Content-Type text/html;charset=utf-8
Date Tue, 23 Jul 2013 13:00:58 GMT
Keep-Alive timeout=15, max=100
Server Apache-Coyote/1.1
Set-Cookie JSESSIONID=03A48D00EA04DE16396824E804E914AC; Path=/...
Vary Accept-Encoding
WWW-Authenticate Basic realm="Name Of Your Realm"
Note : I get this result whatever is the header I try to add.
Where is the problem ?
Thanks ^^.

Only in Opera: JSON.parse: Unterminated string

In Opera only I receive "JSON.parse: Unterminated string" when going to http://www.underfashion.nl/babys
The string is indeed unterminated, does not end with "]}.
In the other browsers (IE, FF, Chrome) it works fine and receives the entire string.
The string is very long: 217529 chars. Is that possibly the problem? The other browsers receive 220374 chars ending with "]}
I have tried 3 AJAXways to get the data, all with the same strings as result:
The first:
var value = (function () {
var val = null;
$.ajax({'async': false, 'global': false, 'url': uf_urlsearch,
'success': function (data) { val = data;
alert("Data Loaded: " + data.slice(-100) + "<br/>Numofchars: " + data.length);
}
});
return val;
})();
The second:
$.get(uf_urlsearch, function(data){
alert("Data Loaded: " + data.slice(-100));
});
The third:
uf_XMLHttpProductlist.onreadystatechange=function(){
if (uf_XMLHttpProductlist.readyState==4 && uf_XMLHttpProductlist.status==200){
//Get the returned menu-items in Responsetext, expected to look like this:
...
};//if (uf_XMLHttp.readyState==4 && uf_XMLHttp.status==200){
};//uf_XMLHttp.onreadystatechange=function()
uf_urlsearch = "http://www.underfashion.nl/php/get_productlist.php?"+uf_PHPsearchstring;
uf_XMLHttpProductlist.open("GET",uf_urlsearch,true);
uf_XMLHttpProductlist.send();
};
Anyone see any solution?
Best regards,
To inspect the network activity, Go to Opera Menu -> Tools -> Advanced -> Opera Dragonfly. Then enter the URL in your addressbar.
In the Network Tab you can see the list of resources. Select the XHR button, and you will see the get_productlist.php resource. For what is worth, I didn't have any issue with your Web site. The HTTP Request was:
GET /php/get_productlist.php?afdeling=babys HTTP/1.1
User-Agent: Opera/9.80 (Macintosh; Intel Mac OS X 10.7.4; U; fr) Presto/2.10.289 Version/12.00
Host: www.underfashion.nl
Accept-Language: fr,en;q=0.9,en-US;q=0.8,ja;q=0.7,pt;q=0.6,de;q=0.5,zh-CN;q=0.4,es;q=0.3,it;q=0.2,nl;q=0.1,sv;q=0.1,nb;q=0.1,da;q=0.1,fi;q=0.1,zh-TW;q=0.1,ko;q=0.1,pl;q=0.1,pt-PT;q=0.1,ru;q=0.1,ar;q=0.1,cs;q=0.1,hu;q=0.1,tr;q=0.1,ca;q=0.1,el;q=0.1,he;q=0.1,hr;q=0.1,ro;q=0.1,sk;q=0.1,th;q=0.1,uk;q=0.1
Accept-Encoding: gzip, deflate
Referer: http://www.underfashion.nl/babys
Cookie: JSESSIONID=9ABC3B0357487E01298EBC7A02B5FDCD; __atuvc=1%7C25; __utma=137714676.906129982.1340200451.1340200451.1340200451.1; __utmb=137714676.1.10.1340200451; __utmc=137714676; __utmz=137714676.1340200451.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmv=
Connection: Keep-Alive
X-Requested-With: XMLHttpRequest
Accept: */*
Now the HTTP Response is interesting:
HTTP/1.1 200 OK
Date: Wed, 20 Jun 2012 13:54:11 GMT
Server: Apache/2.2.14 (Ubuntu)
X-Powered-By: PHP/5.3.2-1ubuntu4.15
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 11469
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html
Then the json content. Do you see what is wrong in the HTTP response above? YUP.
Content-Type: text/html
The mime type for JSON is defined in RFC 4627. Please send with JSON content the following mime type.
Content-Type: application/json
That said You are saying that you still have the issue (I don't) on some specific URIs. Could you share which one?

Resources