After defining the following configuration to avoid dynamic caching for an API : Cloudflare for your API
My calls are still in a Dynamic cache status. You can see the received reponse headers :
access-control-allow-methods: GET,OPTIONS
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400
cache-control: no-cache, no-store, max-age=0, must-revalidate
cf-cache-status: DYNAMIC
cf-ray: 698eeac5ae5640cf-CDG
client-control: max-age=43200, s-max-age=43200
content-encoding: br
content-type: application/json
date: Mon, 04 Oct 2021 13:57:03 GMT
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
expires: 0
pragma: no-cache
server: cloudflare
strict-transport-security: max-age=31536000; includeSubDomains; preload
vary: Origin,Access-Control-Request-Method,Access-Control-Request-Headers, Accept-Encoding
x-content-type-options: nosniff
x-frame-options: DENY
x-xss-protection: 1; mode=block
Here's the configuration i've done
Do you find a way to get a 'BYPASS' cache Clouflare Status Headers
If your goal is to avoid having your API calls cached, then the above page rule looks correct. You are using a Cache Level directive to Bypass the caching logic.
This means that all the calls matching the URL pattern will always be pulled from your origin server. This is confirmed by the cf-cache-status response header valued with DYNAMIC
From the documentation
DYNAMIC: Cloudflare does not consider the asset eligible to cache and your Cloudflare settings do not explicitly instruct Cloudflare to cache the asset. Instead, the asset was requested from the origin web server.
The response header cf-cache-status can also have a value of BYPASS, but this happens in other scenarios (also described in the documentation)
Related
I need help or any suggestion. I have no idea how to do it?
Request URL: https://www.vizofly.com/NTU/Stress/StreamingAssets/Schools.json
Request Method: GET
Status Code: 200 (from disk cache)
Remote Address: 172.66.43.59:443
Referrer Policy: strict-origin-when-cross-origin
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
cf-cache-status: DYNAMIC
cf-ray: 6d4c32aacdb1926d-FRA
content-encoding: br
content-type: application/json
date: Fri, 28 Jan 2022 18:14:00 GMT
etag: W/"1261-61f2caaf-3e2d8e;;;"
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
last-modified: Thu, 27 Jan 2022 16:39:11 GMT
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
report-to: {"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v3?s=aVZAHndifoZtrY0MH3O1WlauF71saxbdUuS7eBS0tReoUi5fGDG3zSlxFCTvbIwJxvGeVeiQyjT%2FVIUWKfUpxNbRT1jUi%2F9VEOvJnaBBRtJKapsW8RBKeLUxqP%2FusLzYEHQ%3D"}],"group":"cf-nel","max_age":604800}
server: cloudflare
x-content-type-options: nosniff
x-frame-options: ALLOW-FROM https://www.ntu.edu.sg
If you're asking about how to configure JMeter to send the request:
Add Thread Group to your Test plan
Add HTTP Request sampler and set it up like:
You may also want to add HTTP Cache Manager to represent browser cache, HTTP Cookie Manager to automatically handle cookies and so on in order to configure JMeter to behave more like a real browser
I have to, sadly, poll an endpoint and update another system when the data changes. I wrote a loop (with a sleep statement so I don’t DOS the server):
require 'nokogiri'
require 'open-uri'
desired_data = 'foo'
data = nil
url = nil
while data != desired_data do
sleep(2)
url = "https://elections.wi.gov/index.php/elections-voting/statistics"
doc = Nokogiri::HTML.parse(open(url))
puts doc
# do some nokogiri stuff to extract the information I want.
# store information to `data` variable.
end
# if control is here it means the data changed
This works fine except when the server updates, open(url) still returns the old content (even if I restart the script).
It seems like there may be some caching at play. How do I disable it?
Here are the HTTP headers returned:
HTTP/2 200
date: Fri, 02 Oct 2020 14:00:44 GMT
content-type: text/html; charset=UTF-8
set-cookie: __cfduid=dd8fca84d468814dd199dfc08d45c98831601647244; expires=Sun, 01-Nov-20 14:00:44 GMT; path=/; domain=.elections.wi.gov; HttpOnly; SameSite=Lax; Secure
x-powered-by: PHP/7.2.24
cache-control: max-age=3600, public
x-drupal-dynamic-cache: MISS
link: <https://elections.wi.gov/index.php/elections-voting/statistics>; rel="canonical"
x-ua-compatible: IE=edge
content-language: en
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
expires: Sun, 19 Nov 1978 05:00:00 GMT
last-modified: Fri, 02 Oct 2020 12:47:38 GMT
vary: Cookie
x-generator: Drupal 8 (https://www.drupal.org)
x-drupal-cache: HIT
x-speed-cache: HIT
x-speed-cache-key: /index.php/elections-voting/statistics
x-nocache: Cache
x-this-proto: https
x-server-name: elections.wi.gov
access-control-allow-origin: *
x-xss-protection: 1; mode=block
cache-control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
cf-cache-status: DYNAMIC
cf-request-id: 058b368b9f00002ff234177200000001
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 5dbef38c3b6a2ff2-ORD```
If it matters, I’m using Ruby 2.7 on macOS Big Sur.
It might be a problem on the Drupal 8 website itself as it has its own cache manager - and it seems like there's a cache per user somewhere if you have new content using a web browser.
It is easy to see which cache contexts a certain page varies by and which cache tags it is invalidated by: one must only look at the X-Drupal-Cache-Contexts and X-Drupal-Cache-Tags headers!
But those headers are not available in your list. If you're in touch with the website's developers ask them to do the following:
You can debug cacheable responses (responses that implement this interface, which may be cached by Page Cache or Dynamic Page Cache) by setting the http.response.debug_cacheability_headers container parameter to true, in your services.yml. Followed by a container rebuild, which is necessary when changing a container parameter.
That will cause Drupal to send X-Drupal-Cache-Tags, X-Drupal-Cache-Contexts headers.
It seems I can't use Google Cloud CDN with GKE Ingress in https requests, because the Ingress controller adds a Vary http header: x-forwarded-proto, which CDN won't cache (see https://cloud.google.com/cdn/docs/caching#non-cacheable_content). I think it's correct that the header is added, because it tells the application serving the content that it's being requested as https. But I'm not sure why CDN won't cache pages with this header.
Am I missing something or is there a workaround?
Example of header when page is requested as http and is cached:
Age: 1713
Cache-Control: public, max-age=2000
Content-Encoding: gzip
Content-Length: 8503
Content-Type: text/html; charset=UTF-8
Date: Thu, 13 Aug 2020 20:56:23 GMT
Server: Apache/2.4.38 (Debian)
Vary: Accept-Encoding
Via: 1.1 google
X-Powered-By: PHP/7.4.9
Example of header when page is requested as http and isn't cached:
alt-svc: clear
cache-control: public, max-age=2000
content-encoding: gzip
content-length: 8499
content-type: text/html; charset=UTF-8
date: Thu, 13 Aug 2020 21:25:37 GMT
server: Apache/2.4.38 (Debian)
status: 200
vary: X-Forwarded-Proto,Accept-Encoding
via: 1.1 google
x-powered-by: PHP/7.4.9
Many thanks,
Sam
As you already confirmed the Cloud CDN won't cache it due to this header. This would be expected.
Regarding that header which is added, you will need to review why it's added and find a way to remove it if you want to cache this content with Cloud CDN. I found a few article that you can review:
https://github.com/kubernetes/ingress-gce/issues/51
https://serverfault.com/questions/912168/how-to-make-google-cloud-load-balancer-respect-the-received-x-forwarded-proto
https://trac.nginx.org/nginx/ticket/1181
You would have to find a solution to remove this header based on your deployment.
I get the following exception when I POST the login credentials for my Spring Boot app.
org.springframework.security.web.firewall.RequestRejectedException: The request was rejected because the URL contained a potentially malicious String ";"
It is posting to /dologin and then redirecting to /home with the jsessionid appended to the end. It does also set the session cookie too. I did not change any session settings and there is no mention of session in application.properties.
I tried to set
server.servlet.session.cookie.http-only=true
server.servlet.session.tracking-modes=cookie
As mentioned in https://stackoverflow.com/a/31792535/148844, but it didn't work.
I added
#Bean
public ServletContextInitializer servletContextInitializer() {
return new ServletContextInitializer() {
#Override
public void onStartup(ServletContext servletContext) throws ServletException {
servletContext.setSessionTrackingModes(Collections.singleton(SessionTrackingMode.COOKIE));
SessionCookieConfig sessionCookieConfig=servletContext.getSessionCookieConfig();
sessionCookieConfig.setHttpOnly(true);
}
};
}
But now it just POSTs, sets the cookie, and redirects back to the login screen. It's as if it can't access the session.
I set server.session.tracking-modes=cookie (instead of server.servlet...) and it is only using cookies now, but the Chrome browser is not sending the cookie back to the server after login! /home action will only re-display the login page if user attribute in the session is null.
POST /dologin HTTP/1.1
Host: localhost:8080
Origin: http://localhost:8080
Upgrade-Insecure-Requests: 1
Referer: http://localhost:8080/home
HTTP/1.1 302
Set-Cookie: JSESSIONID=3B82AAA40CE94FF490FBF7B4DBD837DD; Path=/; HttpOnly
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
Location: http://localhost:8080/home
GET /home HTTP/1.1
Host: localhost:8080
Upgrade-Insecure-Requests: 1
Referer: http://localhost:8080/home
HTTP/1.1 200
Set-Cookie: JSESSIONID=B60BF649068F7E85346691FD2F5D119B; Path=/; HttpOnly
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
Content-Type: text/html;charset=UTF-8
Content-Language: en-US
Content-Length: 2742
Date: Sat, 29 Sep 2018 17:41:55 GMT
Notice the cookies are different and Chrome did not send the cookie back to the server? Why?
Spring Boot 1.5.13, Chrome Version 69.0.3497.100 (Official Build) (64-bit)
OK changing server.servlet.session.cookie.http-only=true to server.session.tracking-modes=cookie and changing http://localhost:8080 to http://127.0.0.1:8080/ worked. I found this answer:
Chrome localhost cookie not being set
It seems Chrome keeps flipping from allowing localhost to disallowing localhost. It was working about a month or three ago. localhost is working for a Rails app and Chrome is sending the cookies.
In fact, Chrome is also sending the _mt_rails_session Rails cookie for localhost to the Spring Boot app, but never the JSESSIONID cookie.
I suspect, but have not confirmed, it may be due to setting up HTTPS on port 8080 for an unrelated 3rd Spring Boot app, and there may be some HSTS setting cached in Chrome internals. It's probably a bug in Chrome.
I have scan the Laravel Project using AppScan tool, I am facing security issue Permanent Cookie Contains Sensitive Session Information issue in AppScan Security Document.
Here is My Network information of Header:
Cache-Control: no-cache, private
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 3692
Content-Type: text/html; charset=UTF-8
Date: Thu, 16 Aug 2018 06:01:02 GMT
Expires: 0
Pragma: no-cache
Server: Apache
Set-Cookie: XSRF-TOKEN=eyJpdiI6Ik9XVzJyOXRSZ0JkRnhaTXdkZlQzeVE9PSIsInZhbHVlIjoibUxEKzBLUHdCM2VwWVc4MzhPNjR3TmpcL2VZdWJXRjJmeVZklSaGtnb2RRblVBblpXNEVDS1wvMXExKzQwaE9NWlQwVFRpUTZiTHB3b1ZRMlcwZz09IiwibWFjIjoiNzhmMDg4MjUzN2YzMDA3MDg3MTJiOGNkMTQ4MTZlNWIyZWZiZTkxYjgwNzI1NzVmMzBmNWQyYjE1ZThjZDc3MiJ9; expires=Thu, 16-Aug-2018 08:01:02 GMT; Max-Age=7200; path=/; secure;HttpOnly;HttpOnly;Secure
Set-Cookie: laravel_session=eyJpdiI6InJpK3FNeHhTMGFEUUgxSGxKUkc1Rmc9PSIsInZhbHVlIjoieVRGaGpMcnJ3bnBrWCtOTzZcL2dzVFRKNDl5U29jUTlvb0tSZE54d2YxbTFvSHZ3TW5jc2FXcFwvcjhLaXlFN2R5eEo0Tlpoa1FxY0VUWlJ5bUjIjoiYTY2MjkwODU3YWQyMTFmZTJkNmZlMWUzMTgyYzk2NjA2OTBiMDgyODMzM2ZlZjg4NDE4MDBjZDFkYmIyNmEyMCJ9; expires=Thu, 16-Aug-2018 08:01:02 GMT; Max-Age=7200; path=/; secure; httponly;HttpOnly;HttpOnly;Secure
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Vary: Accept-Encoding
X-Content-Security-Policy: allow 'self';
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Also When i stop/ remove cookies from server side. I will block the csrf token.
How can i fix this security issue.?
Please Help!!
Thanks in Advance
I think it's a bug in the AppScan application. The message is already correct, session information is stored in the cookie. This serves to transfer this token with every request and to send back a new CSRF token afterwards.
Since the cookie has a Max-Age=7200; the cookie lives only 2 hours. Therefore, AppScan's statement that the cookie exists permanently is false.
And remember, with each request a new CSRF-Token is assigned to you, whereby the attacker would have to manipulate your locally sent request to be able to use this method.