I do a request to my Squid through CloudFront and receive a miss:
< content-type: text/html
< date: Thu, 17 Mar 2022 08:22:46 GMT
< server: Apache/2.4.48 (Ubuntu)
< cache-control: max-age=3153600000, public, only-if-cached, immutable
< last-modified: Thu, 01 Jan 1970 00:00:00 GMT
< x-cache-lookup: MISS from ip-172-31-68-17:3128
< via: 1.1 ip-172-31-68-17 (squid/4.13), 1.1 c929a0b0be95dbd556dd38270accc062.cloudfront.net (CloudFront)
< content-security-policy: default-src * 'unsafe-eval' 'unsafe-inline' 'unsafe-dynamic' data: blob: ws: wss:; img-src http: https: data: mediastream: 'unsafe-eval' 'unsafe-hashes' 'unsafe-inline' blob: ws: wss:; frame-ancestors 'self'; frame-ancestors https://allfilesnews.com https://*.allfilesnews.com
< x-frame-options: SAMEORIGIN
< link: <https://Reb1lBkV3d9kbe7qlaaC7nZDFGNEErQKHQBk51bdA6c.allfilesnews.com/Reb1lBkV3d9kbe7qlaaC7nZDFGNEErQKHQBk51bdA6c/VvaYNP0fJiVQfefYmC>; rel="canonical"
< expires: Thu, 15 Apr 2110 20:00:00 GMT
< vary: Origin
< x-cache: Miss from cloudfront
< x-amz-cf-pop: TLV50-C1
< x-amz-cf-id: t6P2Rr1HqwCmSDmzC4oVh2fzq25ooHzPY4OZPwWras-9tBbMqinr3Q==
You see that max-age is very long and Last-Modified is in deep past. Why then this (repeated, I already queried this URL no more than a few days ago) query to Squid gives a MISS?
Here is my Squid config:
read_timeout 60 minutes
http_port 3128 accel vhost allow-direct
cache_dir ufs /Files/main/cache 120000 16 256
cache_peer 127.0.0.1 parent 9003 0 no-query default
acl all src 0.0.0.0/0.0.0.0
http_access allow all
never_direct allow all
forward_max_tries 5
Note that 120GB of the cache space is nearly not used:
$ df -h /Files/main/
Filesystem Size Used Avail Use% Mounted on
Files/main 125G 187M 125G 1% /Files/main
Related
I have made a GitHub action in which I am having my dockerfile as :
FROM alpine:latest
RUN apk update
RUN apk --no-cache add curl git python3 py3-pip
COPY entrypoint.sh /entrypoint.sh
RUN ["chmod", "+x", "/entrypoint.sh"]
ENTRYPOINT ["/entrypoint.sh"]
In the entrypoint.sh I am using curl to request the GitHub API to change one repo's default branch. the command I am using is :
curl -i -u "$USER_NAME:$GITHUB_KEY" --request PATCH -d '{"default_branch": "master"}' https://api.github.com/repos/$USER_NAME/$REPO_NAME
Everything is Well Defined.
But in the response I am getting :
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 74 0 38 100 36 255 241 --:--:-- --:--:-- --:--:-- 496
HTTP/2 403
server: GitHub.com
date: Wed, 23 Feb 2022 07:20:13 GMT
content-type: text/plain; charset=utf-8
vary: X-PJAX, X-PJAX-Container
permissions-policy: interest-cohort=()
cache-control: no-cache
set-cookie: _gh_sess=%2BEichIQiS6mPGOQezauQXt5LDjh8PQrowD3A5%2FzFXiDuXxvgFxJViurljc6uElUDKarOEnWSLlxeDz0RmL4MCusqQMitCn4v2UFd4IQ%3D%3D--QEwYz3UHkEiL1NGR--jjbCBNRKQEQOA%3D%3D; path=/; secure; HttpOnly; SameSite=Lax
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 0
referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin
expect-ct: max-age=2592000, report-uri="https://api.github.com/_private/browser/errors"
content-security-policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; child-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com objects-origin.githubusercontent.com www.githubstatus.com collector.githubapp.com collector.github.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com cdn.optimizely.com logx.optimizely.com/v1/events translator.github.com wss://alive.github.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src render.githubusercontent.com viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: github.githubassets.com identicons.github.com collector.githubapp.com collector.github.com github-cloud.s3.amazonaws.com secured-user-images.githubusercontent.com/ *.githubusercontent.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; worker-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/
vary: Accept-Encoding, Accept, X-Requested-With
x-github-request-id: 07C7:12E1:C19D08:14B9C02:6215E02D
Cookies must be enabled to use GitHub.
I am not getting out to get out of it.
Please help me to get out of it. or please tell me any way using git to change default branch of a repo.
Apache2.4 cache control header is missing when the content is not cached, is it because the request is HEAD instead of GET , this behavior was different on 2.2 were head and post was having cache control header.
curl -I https://localhost/test.html
HTTP/1.1 200 OK
Date: Sat, 14 Nov 2020 00:10:39 GMT
Server: Apache
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Frame-Options: SAMEORIGIN
Content-Security-Policy: frame-ancestors 'self'
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Type: text/html;charset=utf-8
Content-Language: en
curl -I https://localhost/test.html
HTTP/1.1 200 OK
Date: Sat, 14 Nov 2020 00:25:24 GMT
Server: Apache
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Frame-Options: SAMEORIGIN
Content-Security-Policy: frame-ancestors 'self'
Last-Modified: Sat, 14 Nov 2020 00:25:19 GMT
Accept-Ranges: bytes
Content-Length: 206004
Expires: Sun, 15 Nov 2020 00:25:24 GMT
Vary: Accept-Encoding
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Cache-Control: public, max-age=86400
Content-Type: text/html
Content-Language: en
Default behavior on Apache2.4 , when data is stored inside apache then header info is displayed or else not.
I have installed Varnish and configured it as per as the guideline.
And when I try it curl -I https://d-o-m-a-i-n.com I get following which seems its working correctly.
HTTP/1.1 200 OK
Date: Sat, 28 Mar 2020 03:17:02 GMT
Server: Apache/2.4.18 (Ubuntu)
Expires: Sun, 29 Mar 2020 03:17:03 GMT
Cache-Control: max-age=86400, public, s-maxage=86400
Pragma: cache
X-Magento-Tags: cms_b_porto_homeslider_3,store,cms_b,cms_p_91,cms_b_porto_custom_notice_new,cat_p,cat_c_p_30,cat_p_22,cat_p_1,cat_p_34,cat_p_21,cat_p_41,cat_p_11,cat_p_39,cat_p_35,cat_p_33,cms_b_porto_footer_top_1_for_5,cms_b_porto_footer_middle_1_for_5,cms_b_porto_footer_middle_2_for_5
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Vary: Accept-Encoding
X-UA-Compatible: IE=edge
Content-Type: text/html; charset=UTF-8
X-Varnish: 33268 3
Age: 13957
Via: 1.1 varnish (Varnish/5.2)
X-Cache: cached
Accept-Ranges: bytes
But when I check in Google Chrome header response I get followings where it shows Varnish is not caching and age is 0?
Accept-Ranges: bytes
Age: 0
Cache-Control: max-age=86400, public, s-maxage=86400
Connection: Keep-Alive
Content-Encoding: gzip
Content-Length: 20027
Content-Type: text/html; charset=UTF-8
Date: Sat, 28 Mar 2020 07:13:35 GMT
Expires: Sun, 29 Mar 2020 07:13:35 GMT
Keep-Alive: timeout=5, max=100
Pragma: cache
Server: Apache/2.4.18 (Ubuntu)
Vary: Accept-Encoding
Via: 1.1 varnish (Varnish/5.2)
X-Cache: uncached
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Magento-Tags: cms_b_porto_homeslider_3,store,cms_b,cms_p_91,cms_b_porto_custom_notice_new,cat_p,cat_c_p_30,cat_p_1,cat_p_22,cat_p_21,cat_p_11,cat_p_34,cat_p_41,cat_p_39,cat_p_35,cat_p_33,cms_b_porto_footer_top_1_for_5,cms_b_porto_footer_middle_1_for_5,cms_b_porto_footer_middle_2_for_5
X-UA-Compatible: IE=edge
X-Varnish: 1704417
X-XSS-Protection: 1; mode=block
Do you have any ideas why that is?
How are you generating the key for caching(vcl_hash)?
Check it and make sure there is no user agent involved.
I'm running curl on a resource on the server that is using Cloudflare as it's CDN. However, I'm not sure if it's caching or not.
Here's the output from A - This is the one I'm not sure is caching:
Resource A:
< HTTP/1.1 530
< Date: Wed, 22 Mar 2017 12:47:06 GMT
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: close
< Set-Cookie: __cfduid=<numbers>; expires=Thu, 22-Mar-18 12:47:05 GMT; path=/; domain=.<domain>; HttpOnly
< Cache-Control: max-age=6
< Expires: Wed, 22 Mar 2017 12:47:12 GMT
< X-Frame-Options: SAMEORIGIN
< Server: cloudflare-nginx
< CF-RAY: <numbers>-LHR
Resource B: - this is clearly caching as it has a CF-Cache-Status: HIT header
< HTTP/1.1 200 OK
< Date: Wed, 22 Mar 2017 12:47:56 GMT
< Content-Type: image/png
< Content-Length: 3853
< Connection: keep-alive
< Set-Cookie: __cfduid=<numbers>; expires=Thu, 22-Mar-18 12:47:56 GMT; path=/; domain=<domain>; HttpOnly
< Cf-Bgj: imgq:100
< Cf-Polished: pngoptimizer, origSize=5384
< Etag: "<etag>"
< Last-Modified: Fri, 17 Mar 2017 00:07:20 GMT
< X-Amz-Id-2: <numbers>=
< X-Amz-Meta-S3b-Last-Modified: 20170316T132941Z
< X-Amz-Request-Id: <numbers>
< CF-Cache-Status: HIT
< Expires: Wed, 22 Mar 2017 13:47:56 GMT
< Cache-Control: public, max-age=3600
< Server: cloudflare-nginx
< CF-RAY: <numbers>-LHR
Note I've run these curl requests several times.
Any suggestions?
CloudFlare does not cache HTML resources by default. Since Resource A has Content-Type: text/html, it is likely a .html file and therefore is not cached.
Images, however, are cached by default. That's why the Content-Type of image/png (likely actually a .png file) of Resource B shows CF-Cache-Status: HIT.
Note that CloudFlare caches based on file extension, not by MIME type. So, without seeing the full url of your requests with their file extensions, I am only assuming that the extensions of resources A and B actually match the MIME type in the curl responses.
If you want to cache HTML files, you can configure page rules to do so.
I hope that helps!
I create one proxy cache using (tor + privoxy + squid) follow
How to create an advanced proxy server using Tor, Privoxy, and Squid
I try call to Proxy cache
curl -x 'my-ip:3400' -I 'https://github.global.ssl.fastly.net/images/modules/dashboard/bootcamp/octocat_setup.png'
curl -x 'my-ip:3400' -I http://hadoop.apache.org/images/hadoop-logo.jpg
https request is ok (through Pivoxy)
HTTP/1.1 200 Connection established
Proxy-Agent: Privoxy/3.0.21
HTTP/1.1 200 OK
Date: Sat, 08 Mar 2014 04:38:37 GMT
Server: GitHub.com
Content-Type: image/png
Last-Modified: Fri, 16 Aug 2013 09:42:18 GMT
Expires: Sun, 08 Mar 2015 04:38:37 GMT
Cache-Control: max-age=31536000, public
Timing-Allow-Origin: https://github.com
Content-Length: 10722
Accept-Ranges: bytes
Via: 1.1 varnish
Age: 0
X-Served-By: cache-am72-AMS
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1394253517.449177980,VS0,VE97
Vary: Accept-Encoding
but http requets not working
HTTP/1.0 200 OK
Date: Sat, 08 Mar 2014 04:39:19 GMT
Server: Apache/2.4.8-dev (Unix) mod_wsgi/3.4 Python/2.7.5 OpenSSL/1.0.1e
Last-Modified: Tue, 12 Feb 2008 17:38:57 GMT
ETag: "24e3-445f987fe69d8"
Accept-Ranges: bytes
Content-Length: 9443
Content-Type: image/jpeg
X-Cache: MISS from SVR226L-696.localdomain
X-Cache-Lookup: MISS from SVR226L-696.localdomain:3128
Via: 1.0 SVR226L-696.localdomain (squid)
Connection: keep-alive
Please suggest a specific solution