I'm a bit unsure what it happening here but ill try to explain what is happening and maybe write a better question once i figured out what i'm actually asking.
I have just installed Varnish which seems awesome for my request times. It is a Magneto 2 store which I have followed the default configuration within dev docs for varnish.
My Issue
Currently my issue is that the browser seems to be caching the page until i click refresh. I believe i am successfully flushing / purging the cache with magento / varnish. As when using Curl to request the page i can see a new page is generated each time i flush cache and just serves cached page if i don't.
Within chrome and firefox however on my client pc however the whole page markup seems to be cached (when clicking a link to page or pasting url in browser) until clicking refresh which seems to reload the real page. When deploying new static files etc as the old resources are still in the cached markup and the new location for resources is signed e.g. version1234/styles.css and not matching the markup i get CSS less pages until client clicks refresh and loads the actual markup from server?
How can i setup caching so that this does not happen?
Curl -IL result of URL:
HTTP/1.1 200 OK
Date: Fri, 24 Nov 2017 12:08:32 GMT
Strict-Transport-Security: max-age=63072000; includeSubdomains
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
Expires: Sun, 26 Nov 2017 15:55:17 GMT
Cache-Control: max-age=186400, public, s-maxage=186400
Pragma: cache
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Vary: Accept-Encoding
X-UA-Compatible: IE=edge
Content-Type: text/html; charset=UTF-8
X-Magento-Cache-Control: max-age=186400, public, s-maxage=186400
X-Magento-Cache-Debug: HIT
Grace: none
age: 0
Accept-Ranges: bytes
Connection: keep-alive
Browser caching takes please because of these headers being sent:
Expires: Sun, 26 Nov 2017 15:55:17 GMT
Cache-Control: max-age=186400, public, s-maxage=186400
You should adjust your server configuration so that those are not sent for PHP. Most likely you have a configuration block in nginx or .htaccess that applies to the whole website, as opposed to just static files.
Related
I have to, sadly, poll an endpoint and update another system when the data changes. I wrote a loop (with a sleep statement so I don’t DOS the server):
require 'nokogiri'
require 'open-uri'
desired_data = 'foo'
data = nil
url = nil
while data != desired_data do
sleep(2)
url = "https://elections.wi.gov/index.php/elections-voting/statistics"
doc = Nokogiri::HTML.parse(open(url))
puts doc
# do some nokogiri stuff to extract the information I want.
# store information to `data` variable.
end
# if control is here it means the data changed
This works fine except when the server updates, open(url) still returns the old content (even if I restart the script).
It seems like there may be some caching at play. How do I disable it?
Here are the HTTP headers returned:
HTTP/2 200
date: Fri, 02 Oct 2020 14:00:44 GMT
content-type: text/html; charset=UTF-8
set-cookie: __cfduid=dd8fca84d468814dd199dfc08d45c98831601647244; expires=Sun, 01-Nov-20 14:00:44 GMT; path=/; domain=.elections.wi.gov; HttpOnly; SameSite=Lax; Secure
x-powered-by: PHP/7.2.24
cache-control: max-age=3600, public
x-drupal-dynamic-cache: MISS
link: <https://elections.wi.gov/index.php/elections-voting/statistics>; rel="canonical"
x-ua-compatible: IE=edge
content-language: en
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
expires: Sun, 19 Nov 1978 05:00:00 GMT
last-modified: Fri, 02 Oct 2020 12:47:38 GMT
vary: Cookie
x-generator: Drupal 8 (https://www.drupal.org)
x-drupal-cache: HIT
x-speed-cache: HIT
x-speed-cache-key: /index.php/elections-voting/statistics
x-nocache: Cache
x-this-proto: https
x-server-name: elections.wi.gov
access-control-allow-origin: *
x-xss-protection: 1; mode=block
cache-control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
cf-cache-status: DYNAMIC
cf-request-id: 058b368b9f00002ff234177200000001
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 5dbef38c3b6a2ff2-ORD```
If it matters, I’m using Ruby 2.7 on macOS Big Sur.
It might be a problem on the Drupal 8 website itself as it has its own cache manager - and it seems like there's a cache per user somewhere if you have new content using a web browser.
It is easy to see which cache contexts a certain page varies by and which cache tags it is invalidated by: one must only look at the X-Drupal-Cache-Contexts and X-Drupal-Cache-Tags headers!
But those headers are not available in your list. If you're in touch with the website's developers ask them to do the following:
You can debug cacheable responses (responses that implement this interface, which may be cached by Page Cache or Dynamic Page Cache) by setting the http.response.debug_cacheability_headers container parameter to true, in your services.yml. Followed by a container rebuild, which is necessary when changing a container parameter.
That will cause Drupal to send X-Drupal-Cache-Tags, X-Drupal-Cache-Contexts headers.
When I make a request to a Google Api font (e.g. https://fonts.googleapis.com/css?family=Roboto:400), the last-modified header is always changing to the current time.
Caching doesn't work as a result and the file has to be downloaded every load. Is there a reason for this? Should I download the file and host it on my server?
Should I download the file and host it on my server?
Absolutely not, because the content of the CSS files is dynamic and has different content for every user agent. This is because not all browsers support all font formats. Some require WOFF/WOFF2, others require EOT, TTF or SVG. By downloading and serving the file statically you will break font support for all other browsers.
Interestingly though, I do not see a last-modified header at all:
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Type: text/css
Alt-Svc: clear
Alternate-Protocol: 443:quic,p=0
X-XSS-Protection: 1; mode=block
Server: GSE
Expires: Mon, 14 Dec 2015 09:14:21 GMT
Timing-Allow-Origin: *
Cache-Control: private, max-age=86400
Date: Mon, 14 Dec 2015 09:14:21 GMT
Content-Length: 222
Connection: close
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
The Expires header indeed is the same as the Date - so it expires at the time the ressource was loaded. The max-age option of Cache-Control though has higher precedence. So the file should be cached by the browser for 1 day.
Issue:
We were having issue with one of the ajax request in webkit browsers ( Chrome, Firefox, and Safari).
scenarios working:
1. If we configure chrome/firefox with Fiddler and enable https decryption in fiddler it works as expected.
2. Works properly in internet explorer.
We were able to see response on page, but if you go to networking tab and check response it's null and request is not complete it's spinning. we were thinking it's more of webkit browser decoding issue.
Please share if you have any idea how to fix or what might be causing this issue, any input will be appreciated.
Raw HTTP Header
HTTP/1.1 200 OK
content-encoding: gzip
content-language: en-US
content-type: application/json;charset=UTF-8
date: Wed, 19 Aug 2015 18:38:23 GMT
p3p: CP="NON CUR OTPi OUR NOR UNI"
vary: X-Forwarded-Host
transfer-encoding: chunked
server-name: app2
cache-control: private, must-revalidate, max-age: 0
x-powered-by: Servlet/3.0
x-ua-compatible: IE=edge
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-frame-options: SAMEORIGIN
expires: -1
The issue was server side, i was getting extra data in JSON response and data binding was not happening properly in chrome, firefox and safari.
Whereas IE is having loose binding policy i guess that's reason it was working in IE
I am working on a web application which has user management in place. I find a concerning issue in firefox related to Work Offline. Following are the steps describing the scenario:
User logs in to the application
User performs some action and logs out of the application
If the user now enables Work Offline mode in firefox, he/she can use browser back to access the last page. However, this page is supposed to be secure.
In my opinion this is a data security issue as any other user can apply this technique to fetch valuable information of the last user.
I have used cache control headers to communicate to the browser that HTML content should not be cached. Following are the response headers used:
HTTP/1.1 200 OK
Date: Tue, 05 May 2015 10:39:30 GMT
Server: Apache/2.4.9 (Unix) OpenSSL/0.9.8za
Cache-Control: no-cache, no-store
Expires: Wed, 31 Dec 1969 23:59:59 GMT
Content-Type: text/html;charset=UTF-8
Content-Language: en
Vary: Accept-Encoding
Content-Encoding: gzip
X-Frame-Options: SAMEORIGIN
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
I have used
Cache-Control: no-cache, no-store
Expires: Wed, 31 Dec 1969 23:59:59 GMT
I have noted this vulnerability in applications like Facebook. Is this resolvable? Thank you.
I am not using the SBT, but making direct REST calls with Abdera to the current version of Connections on IBM SmartCloud. REST URL in question: https://apps.na.collabserv.com/search/serviceconfigs
Observations
When testing from my laptop (using Firefox and the REST client add-on,) this works as expected. I get back an ATOM feed.
When testing from a server (on a different network,) using the same method (Firefox + REST client,) I get back HTML that is a log-in page.
In addition, I get this same result when I call the URL from a Java program on the same server.
In all cases, I am using the same credentials with basic authentication.
Update: If I log into SmartCloud first, on a separate tab in Firefox on the server, then call the URL as before, from another tab, it works. I get the ATOM feed as desired. Naturally, this is unsuitable as a solution, but I present it as additional information that could lead to an actual solution.
Update: Further testing shows that the local (laptop) log-in exhibits the same behavior as the server. A form-based log-in is required from the same browser, then subsequent REST calls work.
Update: Here is a relevant simplified code snippet:
private static Abdera ABDERA = new Abdera();
private static AbderaClient ABDERA_CLIENT = new AbderaClient(ABDERA);
...
String host = "https://apps.na.collabserv.com";
ABDERA_CLIENT.addCredentials(host, AuthScope.ANY_REALM, "basic", new UsernamePasswordCredentials("user", "password"));
...
ClientResponse response = ABDERA_CLIENT.get("https://apps.na.collabserv.com/search/serviceconfigs");
Summary
It appears that something about the originating server or the call is causing SmartCloud to respond with a log-in page. Whereas, the same call and credentials from my laptop, work as expected.
Question
Where should I start to trouble-shoot this? How can I build the client credentials to allow programmatic log-in?
Response Headers
If it helps, here are the response headers that I get back in each case.
Unsuccessful
Status Code: 200 OK
Cache-Control: no-cache
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 1850
Content-Type: text/html
Date: Tue, 08 Oct 2013 14:15:03 GMT
Pragma: no-cache
Server: WebSEAL/6.1.1.3 (Build 110428)
Set-Cookie: PD-H-SESSION-ID=4_0_IR4***masked***oRKlJI;secure; Path=/; HttpOnly BIGipServerE3A-WebSEAL-80-fe=2132806922.20480.0000;secure; path=/
Vary: Accept-Encoding
p3p: CP="NON CUR OTPi OUR NOR UNI"
Successful
Status Code: 200 OK
Cache-Control: public, max-age=86400, s-maxage=86400, no-cache=set-cookie, private, must-revalidate
Content-Encoding: gzip
Content-Language: en-US
Content-Length: 1164
Content-Type: application/atom+xml; charset=UTF-8
Date: Mon, 07 Oct 2013 17:21:12 GMT
Expires: Tue, 08 Oct 2013 17:21:12 GMT
Server: WebSphere Application Server/8.0
Vary: Accept-Encoding
p3p: CP="NON CUR OTPi OUR NOR UNI"
x-lconn-auth: true
x-powered-by: Servlet/3.0
#Grant is your login using SAML? I could see this redirect happening. also could be TFIM related... maybe you should grab the auth on a different page, store the cookies, and then try connecting to the endpoint above.