Missing headers in a 301 redirect response - response-headers

It has been flagged by a pen tester for one of our clients that the site we built does not return the correct headers for a 302, 500 and 403 response. I have tested this and in fiddler I can see that the headers returned from a 302 are indeed different to a normal 200 response. I've checked our config file and can see that all the headers are in the config file as expected.
Why are the headers different and what do I need to do to fix this?
The response from the pen tester was as follows:
The following HTTP security headers were found to be missing from 403, 302 and 500 responses:
• HTTP Strict Transport Security (HSTS) – The header ensures that supported browsers should only interact with it using HTTPS protocol, rejecting the insecure HTTP protocol, protecting against protocol downgrade attacks and cookie hijacking.
• X-Frame-Options – The header ensures that the browser must not display the transmitted content in frames of other web pages, protecting against Clickjacking attacks.
• X-XSS-Protection – The header will force the browser to enable any available Cross-Site Scripting filter, providing an additional defence against Cross-Site Scripting attacks.
• X-Content-Type-Options – The header will prevent the browser from interpreting files as something else other than what is declared by the content type, which can help protect against some Cross-Site Scripting attacks.
• Referrer Policy – The header governs which referrer information is sent in the Referer header along with requests.
My web config file is:
<add name="Vary" value="Accept-Encoding"/>
<add name="X-UA-Compatible" value="IE=edge"/>
<add name="P3P" value="policyref="/w3c/p3p.xml", CP="This is not a privacy policy!""/>
<add name="E-TAG" value=""/>
<add name="Arr-Disable-Session-Affinity" value="True"/>
<add name="Access-Control-Allow-Origin" value="*"/>
<add name="Access-Control-Allow-Methods" value="*"/>
<add name="Access-Control-Allow-Headers" value="*"/>
<add name="Strict-Transport-Security" value="max-age=31536000;includeSubDomains"/>
<add name="Referrer-Policy" value="strict-origin"/>
<add name="x-Content-Type-Options" value="nosniff"/>
<nwebsec>
<httpHeaderSecurityModule xsi:noNamespaceSchemaLocation="NWebsecConfig/HttpHeaderSecurityModuleConfig.xsd" xmlns="http://nwebsec.com/HttpHeaderSecurityModuleConfig.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<redirectValidation enabled="false" />
<securityHttpHeaders>
<x-Frame-Options policy="Deny" />
<strict-Transport-Security max-age="365" includeSubdomains="true" httpsOnly="true" preload="true" />
<x-Content-Type-Options enabled="true" />
<x-Download-Options enabled="true" />
<x-XSS-Protection policy="FilterEnabled" blockMode="true" />
<content-Security-Policy enabled="false" />
</securityHttpHeaders>
</httpHeaderSecurityModule>
</nwebsec>
and a 200 response returns:
Vary: Accept-Encoding
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-Frame-Options: Deny
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-XSS-Protection: 1; mode=block
X-UA-Compatible: IE=edge
P3P: policyref="/w3c/p3p.xml", CP="This is not a privacy policy!"
E-TAG: True
Arr-Disable-Session-Affinity: *
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: *
Access-Control-Allow-Headers: max-age=31536000;includeSubDomains
Strict-Transport-Security: strict-origin
Referrer-Policy: nosniff
however a 302 returns :-
Vary: Accept-Encoding
X-UA-Compatible: IE=edge
P3P: policyref="/w3c/p3p.xml", CP="This is not a privacy policy!"
E-TAG: True
Arr-Disable-Session-Affinity: *
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: *
Access-Control-Allow-Headers: max-age=31536000;includeSubDomains
Strict-Transport-Security: strict-origin
Referrer-Policy: nosniff
Seems to me that all the x-* headers are missing?

Not an authoritative answer but: I don't see the point of the X headers with a 302, or in normal cases with 403 or 500. They are instructing your browser on how to handle aspects of the returned page, and there is (normally) no page returned with those response codes.
Same general idea as answered on https://security.stackexchange.com/questions/188134/x-frame-options-header-on-redirect
I have not found any decent discussion of application of HSTS to these responses. I would suggest it is practically irrelevant but theoretically ideal to include HSTS in most cases (with exceptions like an URL shortener that is always a redirect so most visitors never get the HSTS directive if you don't include it with a 30X).
Per https://www.w3.org/TR/referrer-policy/#set-requests-referrer-policy-on-redirect it would seem Referrer-Policy should be included on a redirect. 403 and 500 would seem to be irrelevant (again, unless you include a page with those responses).
Again, not an authority, and despite searching a fair bit I haven't found a convincing answer on most of this, so take it as an opinion only.

Related

Cannot bypass Dynamic status cache for my APIs in Cloudflare

After defining the following configuration to avoid dynamic caching for an API : Cloudflare for your API
My calls are still in a Dynamic cache status. You can see the received reponse headers :
access-control-allow-methods: GET,OPTIONS
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400
cache-control: no-cache, no-store, max-age=0, must-revalidate
cf-cache-status: DYNAMIC
cf-ray: 698eeac5ae5640cf-CDG
client-control: max-age=43200, s-max-age=43200
content-encoding: br
content-type: application/json
date: Mon, 04 Oct 2021 13:57:03 GMT
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
expires: 0
pragma: no-cache
server: cloudflare
strict-transport-security: max-age=31536000; includeSubDomains; preload
vary: Origin,Access-Control-Request-Method,Access-Control-Request-Headers, Accept-Encoding
x-content-type-options: nosniff
x-frame-options: DENY
x-xss-protection: 1; mode=block
Here's the configuration i've done
Do you find a way to get a 'BYPASS' cache Clouflare Status Headers
If your goal is to avoid having your API calls cached, then the above page rule looks correct. You are using a Cache Level directive to Bypass the caching logic.
This means that all the calls matching the URL pattern will always be pulled from your origin server. This is confirmed by the cf-cache-status response header valued with DYNAMIC
From the documentation
DYNAMIC: Cloudflare does not consider the asset eligible to cache and your Cloudflare settings do not explicitly instruct Cloudflare to cache the asset. Instead, the asset was requested from the origin web server.
The response header cf-cache-status can also have a value of BYPASS, but this happens in other scenarios (also described in the documentation)

amp-list from remote location?

I would like to use amp-list on a website that is otherwise valid AMP to avoid using an amp-iframe embed with JS. I just finished reading the cors docs by Google AMP at https://www.ampproject.org/docs/fundamentals/amp-cors-requests and am still confused - is it possible to have the json source for the amp-list from a remote domain?
The thing I need is to have a source of URLs+titles generated and updated outside of the main jekyll website because the main website takes too long to build.
I am testing it with a valid JSON and headers as follows and am getting nothing in console and the list is not rendered, so I presume what I am trying to do is not possible?
/source.json
Content-type: application/json; charset=utf-8
X-Frame-Options: ALLOW
Access-Control-Allow-Headers: Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: https://fakename.netlify.com/
AMP-Access-Control-Allow-Source-Origin: https://fakename.netlify.com/
Access-Control-Allow-Methods: POST, GET, OPTIONS
Access-Control-Expose-Headers: AMP-Access-Control-Allow-Source-Origin

AWS S3 Not Sending Access-Control-Allow-Origin header when Origin header on request is present

I have an AWS S3 bucket with the following CORS policy:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
<ExposeHeader>Access-Control-Allow-Origin</ExposeHeader>
</CORSRule>
</CORSConfiguration>
In my app, I load a page of images from the bucket. The images appear on the page as expected. When I click on one of the images to open it in the Adobe Creative SDK, the SDK fails to load the image because it is blocked by CORS. The SDK takes the url of the image and loads it via AJAX. The SDK has an option to enable CORS and appears to be sending the proper Origin header (See screenshot), however AWS is not including the Access-Control-Allow-Origin-Header, causing the image to be blocked by CORS.
I've scoured every question on here about this subject and it seems like no one has a straight answer. There are dozens of questions on this subject that are unanswered, however in many of those cases it appears that the AJAX request isn't sending the Origin header or that the bucket isn't properly configured. In this case, neither of those are true, which is why this is not a duplicate.
As you can see in the screenshot, the request includes the Origin header, but the response does not include the Access-Control-Allow-Origin header. What I would like to know is:
1. Why isn't S3 sending the proper header if my bucket is properly configured?
2. Is it possible that when the Adobe Creative SDK requests the image via AJAX, the browser sees that the image is already cached and tried to serve the cached image (which doesn't have the Origin header) instead?
3. If #2 is the case, how can I forcibly add the Origin header to the image request? In the app, the image is the background of a div (css background-image property).
I created a test bucket, enabled CORS, and used your CORS rules.
Note that I am not convinced that <ExposeHeader>Access-Control-Allow-Origin</ExposeHeader> is necessary, but it should not do any harm by being present, so I've retained it.
The behavior I observe in testing is not consistent with what you are showing in your browser's request/response headers.
Using curl, I set the Origin: header to http://example.com. (Yes, that's actually what I set it to... this was not modified in the output below).
$ curl -v http://xxxxxxxxxxxx.s3.amazonaws.com/index.txt -H 'Origin: http://example.com'
* Hostname was NOT found in DNS cache
* Trying 54.231.98.120...
* Connected to xxxxxxxxxxxx.s3.amazonaws.com (54.231.98.120) port 80 (#0)
> GET /index.txt HTTP/1.1
> User-Agent: curl/7.35.0
> Host: xxxxxxxxxxxx
> Accept: */*
> Origin: http://example.com
>
< HTTP/1.1 200 OK
< x-amz-id-2: pF39K26ii42SzxSU2Dt0KT2z7+xmfyiP4yekp9s4DCYJo0jlRwCTDg6QO6f0HMIL4H9b640zq7U=
< x-amz-request-id: 3B18A563CFF4E485
< Date: Sat, 28 May 2016 21:49:21 GMT
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET
< Access-Control-Expose-Headers: Access-Control-Allow-Origin
< Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method
< Cache-Control: no-cache
< Last-Modified: Sat, 28 May 2016 21:40:57 GMT
< ETag: "cd21a8c7268dc6af728d180f9a3a81d7"
< Accept-Ranges: bytes
< Content-Type: text/plain
< Content-Length: 71
* Server AmazonS3 is not blacklisted
< Server: AmazonS3
Why is this interesting?
S3, with CORS configured and a <CORSRule> matching your request, always returns the CORS response headers and a Vary: header (which means "If you vary one of the following headers in your request, I may vary something about my response.") The headers from the above output that I am referring to, specifically, are these.
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET
Access-Control-Expose-Headers: Access-Control-Allow-Origin
Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method
There is only one exception I can find to this, which is when there is not a matching CORS rule. To demonstrate that, I first changed my CORS configuration to <AllowedOrigin>http://example.com</AllowedOrigin> and then repeated the request. Note that the response is almost identical, except that S3 uses the exact origin in the response, and adds Access-Control-Allow-Credentials.
< Access-Control-Allow-Origin: http://example.com
< Access-Control-Allow-Methods: GET
< Access-Control-Expose-Headers: Access-Control-Allow-Origin
< Access-Control-Allow-Credentials: true
< Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method
...however, if I -- still using the more restrictive CORS rule that specifies a specific origin -- then send a request for Origin: example.org, which is an origin that is not mentioned in my CORS configuration and has no wildcard <AllowedOrigin> to match, S3 responds as though CORS was not configured at all.
< HTTP/1.1 200 OK
< x-amz-id-2: WJ0QmIZ6jTQYefFi8GjlDQkZKFHDX8/5cmejeulhG1ov3/NdoSXbsTKetYpxvXPML8aUnPNZ/ac=
< x-amz-request-id: 15A03D8B1E08A830
< Date: Sat, 28 May 2016 21:58:34 GMT
< Cache-Control: no-cache
...etc.
This takes me back to my initial conclusion that the request you've shown your browser making does not match the timing of when you configured the bucket for CORS, and may have been cached before your bucket's CORS settings were in the correct state, or before they matched those you've posted in the question.
If an Origin: header was present on that request, as the browser shows that it was, then S3 should have added the CORS response headers, whether or not the request was actually a cross-origin request.
Your next steps would be to try this with a new object at a new path that there is no possibility of being cached, or try the common browser cache-busting tactic of adding ?some-random=thing-here to the object's URL when making the request. Failing that, you should consider proving or disproving correct behavior by your bucket using a tool like curl that shows exactly what's happening in the request/response.

should we close the connection of a pre-flight Cors request while sending response?

As I know that if cors request comes with some extra headers set, first server needs to process it.
With CORS, the server must send the Access-Control-Allow-Headers header to allow uncommon request headers from the client.
Access-Control-Allow-Headers ... - Comma-delimited list of the supported request headers.
e.g suppose my pre-flight request is
OPTIONS /cors HTTP/1.1
Origin: http://api.bob.com
Access-Control-Request-Method: PUT
Access-Control-Request-Headers: X-Custom-Header
Host: api.alice.com
Accept-Language: en-US
Connection: keep-alive
User-Agent: Mozilla/5.0...
Then from server-side I will send response
Access-Control-Allow-Origin: http://api.bob.com
Access-Control-Allow-Methods: GET, POST, PUT
Access-Control-Allow-Headers: X-Custom-Header
Content-Type: text/html; charset=utf-8
My question is -
should I close the connection on server side while we send pre-flight response to client?
One more thing how can I cached pre-flight request for all other distinct subsequent requests?
Thanks
You could cache the OPTIONS request using the
Access-Control-Max-Age
header.
Attach it to the headers collection of the OPTIONS response.
But nevertheless an initial OPTIONS request by the user agent (browser) has to be made, you cannot avoid this.
But all further OPTIONS requests are cached and not issued to the server.
No need to close the connection.
Access-Control-Allow-Origin: http://hello-world.example
Access-Control-Max-Age: 3628800
Access-Control-Allow-Methods: PUT
as explained here, search for
could have the following headers specified
to get to the designated text section.

How to remove unwanted WWW-Authenticate headers

From an MVC app, I'm sourcing an iCal subscription with authentication following the answer to this SO question:
Serving an iCalendar file in ASPNET MVC with authentication
The iCal stream is being created dynamically from events in the DB using the DDay.iCal library.
This solution works fine on the local development server: both OSX Calendar and Outlook can subscribe to and receive updates from the app.
However, on the shared server at my web host, the authentication fails for both Calendar and Outlook. That is, they both keep asking me for user & password after the (correct) ones fail.
EDIT: If I point a browser at the calendar URL it also fails authentication.
EDIT: Getting weirder—Firefox authenticates and gets the iCal file. Safari, Chrome and IE fail authentication.
If I point curl at the calendar URL with the same credentials I'm successful (i.e. I get the desired iCal file). And, of course, the same credentials can be used to login to the MVC app.
EDIT — I think I know what's going on, but I don't know how to fix it. In my OnAuthorization() I add only WWW-Authentication Basic but with Fiddler I can see that three types of authentication are offered:
HTTP/1.1 401 Unauthorized
WWW-Authenticate: Basic realm="Secure Calendar"
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
... etc ...
At this point only Firefox responds with Basic Authorization, which succeeds.
GET <<URL>> HTTP/1.1
...
Authorization: Basic <<encoded credentials>>
IE responds with Negotiate, which fails
GET <<URL>> HTTP/1.1
...
Authorization Negotiate <<encoded stuff>>
Who is adding the other two and how can I make it stop? Here's more detail from the server response:
HTTP/1.1 401 Unauthorized
Cache-Control: private
Transfer-Encoding: chunked
Content-Type: text/html
Server: Microsoft-IIS/7.5
X-AspNetMvc-Version: 3.0
WWW-Authenticate: Basic realm="Secure Calendar"
X-AspNet-Version: 4.0.30319
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
X-Powered-By: ASP.NET
X-Powered-By-Plesk: PleskWin
Date: Tue, 23 Oct 2012 13:27:48 GMT
Thanks,
Eric
Ha ha, the answer lay in IIS configuration.
I asked the admins at my host to turn off the other authentications, which broke everything but the iCal feed.
Now they've turned a couple back on again and the MVC site works as well as the calendar feed with authentication... whew! Very, very big smile.
Here's the IIS configuration we ended up with:
Name Status Response Type
Anonymous Authentication Enabled
ASP.NET Impersonation Disabled
Basic Authentication Disabled HTTP 401 Challenge
Digest Authentication Disabled HTTP 401 Challenge
Forms Authentication Enabled HTTP 302 Login/Redirect
Windows Authentication Enabled HTTP 401 Challenge
I'm not sure why this works—or what else might break—but today I'm happy.
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
are used by Windows authentication. Since you finally enabled anonymous authentication, all WWW-Authenticate headers will not appear.
Easy way :
If you want this "X-Powered-By-Plesk" Header to be removed from EVERY NEWLY created domains, you can create a default web.config file within the "httpdocs" folder of the "Default Host Template".
This default website template is usually located under : "C:\inetpub\vhosts.skel\0\httpdocs".
That web.config file will be used by default when you create a new website.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<httpProtocol>
<customHeaders>
<remove name="X-Powered-By-Plesk" />
</customHeaders>
</httpProtocol>
</system.webServer>
</configuration>
TIP 1 : You can use this method to remove any unwanted Custom header (In order to not tell too much to bad guys about your server) :
<remove name="X-Powered-By"/>
<remove name="X-Powered-By-Plesk"/>
<remove name="X-AspNet-Version"/>
<remove name="X-AspNetMvc-Version"/>
TIP 2 : If you want to remove any Dynamic header (like the famous "Server" header), you will need to operate with outboundRules :
<configuration>
<system.webServer>
<rewrite>
<outboundRules>
<rule name="StripHeader_Server" patternSyntax="Wildcard">
<match serverVariable="RESPONSE_SERVER" pattern="*"/>
<action type="Rewrite" value=""></action>
</rule>
<rule name="StripHeader_ETag">
<match serverVariable="RESPONSE_ETag" pattern=".+" />
<action type="Rewrite" value="" />
</rule>
</outboundRules>
</rewrite>
</system.webServer>
</configuration>
TIP 3 : Additionally, you can use this default web.config file to set all configuration parameters you want to use for every new website (in example : to define a list of default documents for your websites, as explained on this Plesk Help article : https://support.plesk.com/hc/en-us/articles/213364049-How-to-configure-global-default-document-settings-in-Parallels-Plesk )
As a belated answer to this, you could also handle this by creating a custom message handler.
The message handler would be inheriting from DelegatingHandler and has to be added to the HttpConfiguration its MessageHandlers
A way this could look would be the following:
public class EnsureNoAuthenticationHeaderHandler : DelegatingHandler
{
async protected override Task<HttpResponseMessage> SendAsync( HttpRequestMessage request, CancellationToken cancellationToken )
{
var response = await base.SendAsync( request, cancellationToken );
if ( response.StatusCode == System.Net.HttpStatusCode.Unauthorized )
{
response.Headers.Remove( "WWW-Authenticate" );
}
return response;
}
}
And then register it in the HttpConfiguration somewhat like the following
private void Register( HttpConfiguration configuration )
{
configuration.MessageHandlers.Add( new EnsureNoAuthenticationHeaderHandler() );
}
Which you would probably call from your global configuration. A message handler can also be attached to a route directly, so if you don't want it to be available everywhere, just have a looked at the linked article on MSDN for more explanation
I had the same problem.
The response included 3 WWW-Authenticate headers and only Firefox worked correctly. Chrome, Bing and IE prompted for username and password but after that they did not send the Authenticate Header to the server.
I just changed IIS Authentication settings and it was solved:
Anonymous Authentication Enabled
ASP.NET Impersonation Disabled
Basic Authentication Disabled HTTP 401 Challenge
Forms Authentication Disabled HTTP 302 Login/Redirect
Windows Authentication Disabled HTTP 401 Challenge

Resources