I'm trying to install and configure varnish in mey srever, and I think I've done it, altought when I run the teste i get:
curl -I http://localhost
HTTP/1.1 503 Backend fetch failed
Date: Mon, 26 Aug 2019 17:16:50 GMT
Server: Varnish
Content-Type: text/html; charset=utf-8
Retry-After: 5
X-Varnish: 32770
Age: 0
Via: 1.1 varnish (Varnish/5.2)
Connection: keep-alive
Can any expert help me?
For what i see i get this problem "HTTP/1.1 503 Backend fetch failed", I've already locate more memory but the problem persists...
Best regards,
Leonel Nunes
BereqProtocol HTTP/1.0
BereqHeader Host: www.test.com
BereqHeader X-Forwarded-For: 145.239.78.207
BereqHeader grace: none
BereqHeader X-Varnish: 30
VCL_call BACKEND_FETCH
VCL_return fetch
BackendOpen 29 boot.default 145.239.78.207 8080 145.239.78.207 56942
BackendStart 145.239.78.207 8080
Timestamp Bereq: 1568108230.028067 0.000121 0.000121
BerespProtocol HTTP/1.0
Timestamp Beresp: 1568108230.030388 0.002443 0.002321
BerespProtocol HTTP/1.0
BerespStatus 200
BerespReason OK
BerespHeader Date: Tue, 10 Sep 2019 09:37:10 GMT
BerespHeader Server: Apache
BerespHeader Last-Modified: Fri, 23 Aug 2019 01:46:14 GMT
BerespHeader Accept-Ranges: bytes
BerespHeader Content-Length: 7725
BerespHeader Cache-Control: max-age=31536000
BerespHeader Expires: Wed, 09 Sep 2020 09:37:10 GMT
BerespHeader X-Frame-Options: SAMEORIGIN
BerespHeader Connection: close
BerespHeader Content-Type: image/jpeg
VCL_call BACKEND_RESPONSE
TTL VCL -1 259200 0 1568108230
TTL VCL 120 259200 0 1568108230
VCL_return deliver
Storage malloc Transient
ObjProtocol HTTP/1.0
ObjStatus 200
ObjReason OK
ObjHeader Date: Tue, 10 Sep 2019 09:37:10 GMT
ObjHeader Server: Apache
ObjHeader Last-Modified: Fri, 23 Aug 2019 01:46:14 GMT
ObjHeader Accept-Ranges: bytes
ObjHeader Content-Length: 7725
ObjHeader Cache-Control: max-age=31536000
ObjHeader Expires: Wed, 09 Sep 2020 09:37:10 GMT
ObjHeader X-Frame-Options: SAMEORIGIN
ObjHeader Content-Type: image/jpeg
Fetch_Body 3 length stream
BackendClose 29 boot.default
Timestamp BerespBody: 1568108230.030469 0.002524 0.000082
Length 7725
BereqAcct 192 0 192 309 7725 8034
End
<< Request >> 38
Begin req 37 rxreq
Timestamp Start: 1568108230.176665 0.000000 0.000000
Timestamp Req: 1568108230.176665 0.000000 0.000000
ReqStart 145.239.78.207 40768
ReqMethod GET
ReqURL /pub/static/version1567874318/frontend/Swissup/absolute/pt_PT/images/payments/ww.png
ReqProtocol HTTP/1.0
ReqHeader Host: www.test.com
ReqHeader Connection: close
ReqHeader X-Forwarded-For: 145.239.78.207
VCL_call RECV
ReqHeader grace: none
ReqURL /pub/static/version1567874318/frontend/Swissup/absolute/pt_PT/images/payments/ww.png
VCL_return pass
VCL_call HASH
VCL_return lookup
VCL_call PASS
VCL_return fetch
Link bereq 39 pass
Timestamp Fetch: 1568108230.178857 0.002191 0.002191
RespProtocol HTTP/1.0
RespStatus 200
RespReason OK
RespHeader Date: Tue, 10 Sep 2019 09:37:10 GMT
RespHeader Server: Apache
RespHeader Last-Modified: Tue, 03 Sep 2019 16:11:49 GMT
RespHeader Accept-Ranges: bytes
RespHeader Content-Length: 1003
RespHeader Cache-Control: max-age=31536000, public
RespHeader Expires: Wed, 09 Sep 2020 09:37:10 GMT
RespHeader X-Frame-Options: SAMEORIGIN
RespHeader Content-Type: image/png
RespProtocol HTTP/1.1
RespHeader X-Varnish: 38
RespHeader Age: 0
RespHeader Via: 1.1 varnish (Varnish/5.2)
VCL_call DELIVER
RespUnset Age: 0
VCL_return deliver
Timestamp Process: 1568108230.178877 0.002212 0.000021
RespHeader Connection: close
Timestamp Resp: 1568108230.178912 0.002246 0.000035
ReqAcct 155 0 155 363 1003 1366
End
<< Session >> 37
Begin sess 0 HTTP/1
SessOpen 145.239.78.207 40768 a0 145.239.78.207 80 1568108230.176609 26
Link req 38 rxreq
SessClose REQ_CLOSE 0.002
End
<< BeReq >> 98340
Begin bereq 98339 pass
Timestamp Start: 1568108230.180024 0.000000 0.000000
BereqMethod GET
BereqURL /pub/static/version1567874318/frontend/Swissup/absolute/pt_PT/images/payments/paypal.png
BereqProtocol HTTP/1.0
BereqHeader Host: www.test.com
BereqHeader X-Forwarded-For: 145.239.78.207
BereqHeader grace: none
BereqHeader X-Varnish: 98340
VCL_call BACKEND_FETCH
VCL_return fetch
BackendOpen 29 boot.default 145.239.78.207 8080 145.239.78.207 56974
BackendStart 145.239.78.207 8080
Timestamp Bereq: 1568108230.180132 0.000108 0.000108
BerespProtocol HTTP/1.0
Timestamp Beresp: 1568108230.181992 0.001968 0.001860
BerespProtocol HTTP/1.0
BerespStatus 200
BerespReason OK
BerespHeader Date: Tue, 10 Sep 2019 09:37:10 GMT
BerespHeader Server: Apache
BerespHeader Last-Modified: Tue, 03 Sep 2019 16:11:49 GMT
BerespHeader Accept-Ranges: bytes
BerespHeader Content-Length: 1757
BerespHeader Cache-Control: max-age=31536000, public
BerespHeader Expires: Wed, 09 Sep 2020 09:37:10 GMT
BerespHeader X-Frame-Options: SAMEORIGIN
BerespHeader Connection: close
BerespHeader Content-Type: image/png
VCL_call BACKEND_RESPONSE
TTL VCL -1 259200 0 1568108230
TTL VCL 120 259200 0 1568108230
VCL_return deliver
Storage malloc Transient
ObjProtocol HTTP/1.0
ObjStatus 200
ObjReason OK
ObjHeader Date: Tue, 10 Sep 2019 09:37:10 GMT
ObjHeader Server: Apache
ObjHeader Last-Modified: Tue, 03 Sep 2019 16:11:49 GMT
ObjHeader Accept-Ranges: bytes
ObjHeader Content-Length: 1757
ObjHeader Cache-Control: max-age=31536000, public
ObjHeader Expires: Wed, 09 Sep 2020 09:37:10 GMT
ObjHeader X-Frame-Options: SAMEORIGIN
ObjHeader Content-Type: image/png
Fetch_Body 3 length stream
BackendClose 29 boot.default
Timestamp BerespBody: 1568108230.182080 0.002055 0.000088
Length 1757
BereqAcct 204 0 204 316 1757 2073
End
<< Request >> 98339
Begin req 98338 rxreq
Timestamp Start: 1568108230.179969 0.000000 0.000000
Timestamp Req: 1568108230.179969 0.000000 0.000000
ReqStart 145.239.78.207 40772
ReqMethod GET
ReqURL /pub/static/version1567874318/frontend/Swissup/absolute/pt_PT/images/payments/paypal.png
ReqProtocol HTTP/1.0
ReqHeader Host: www.test.com
ReqHeader Connection: close
ReqHeader X-Forwarded-For: 145.239.78.207
VCL_call RECV
ReqHeader grace: none
ReqURL /pub/static/version1567874318/frontend/Swissup/absolute/pt_PT/images/payments/paypal.png
VCL_return pass
VCL_call HASH
VCL_return lookup
VCL_call PASS
VCL_return fetch
Link bereq 98340 pass
Timestamp Fetch: 1568108230.182061 0.002092 0.002092
RespProtocol HTTP/1.0
RespStatus 200
RespReason OK
RespHeader Date: Tue, 10 Sep 2019 09:37:10 GMT
RespHeader Server: Apache
RespHeader Last-Modified: Tue, 03 Sep 2019 16:11:49 GMT
RespHeader Accept-Ranges: bytes
RespHeader Content-Length: 1757
RespHeader Cache-Control: max-age=31536000, public
RespHeader Expires: Wed, 09 Sep 2020 09:37:10 GMT
RespHeader X-Frame-Options: SAMEORIGIN
RespHeader Content-Type: image/png
RespProtocol HTTP/1.1
RespHeader X-Varnish: 98339
RespHeader Age: 0
RespHeader Via: 1.1 varnish (Varnish/5.2)
VCL_call DELIVER
RespUnset Age: 0
VCL_return deliver
Timestamp Process: 1568108230.182087 0.002118 0.000026
RespHeader Connection: close
Timestamp Resp: 1568108230.182146 0.002177 0.000058
ReqAcct 159 0 159 366 1757 2123
End
Not enough detail for a succinct answer.
To cut the problem in-half, try to browse http address of the backend server?
Another important consideration, Have you properly configured, /etc/varnish/user.vcl ?
Here is a Varnish tutorial... https://www.linode.com/docs/websites/varnish/getting-started-with-varnish-cache/
Finally, find the location of the Varnish log file to get more details.
I will post a complete log:
varnishlog
* << BeReq >> 84
- Begin bereq 83 fetch
- Timestamp Start: 1566844978.393604 0.000000 0.000000
- BereqMethod GET
- BereqURL /index.php/giantpointsadmin/adminhtml_earning_rate/index
- BereqProtocol HTTP/1.1
- BereqHeader Host: www.test.pt
- BereqHeader User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36
- BereqHeader X-Forwarded-For: 51.89.7.90
- BereqHeader grace: none
- BereqHeader Accept-Encoding: gzip
- BereqHeader X-Varnish: 84
- VCL_call BACKEND_FETCH
- VCL_return fetch
- FetchError no backend connection
- Timestamp Beresp: 1566844978.393680 0.000077 0.000077
- Timestamp Error: 1566844978.393687 0.000083 0.000007
- BerespProtocol HTTP/1.1
- BerespStatus 503
- BerespReason Service Unavailable
- BerespReason Backend fetch failed
- BerespHeader Date: Mon, 26 Aug 2019 18:42:58 GMT
- BerespHeader Server: Varnish
- VCL_call BACKEND_ERROR
- BerespHeader Content-Type: text/html; charset=utf-8
- BerespHeader Retry-After: 5
- VCL_return deliver
- Storage malloc Transient
- ObjProtocol HTTP/1.1
- ObjStatus 503
- ObjReason Backend fetch failed
- ObjHeader Date: Mon, 26 Aug 2019 18:42:58 GMT
- ObjHeader Server: Varnish
- ObjHeader Content-Type: text/html; charset=utf-8
- ObjHeader Retry-After: 5
- Length 279
- BereqAcct 0 0 0 0 0 0
- End
<< Request >> 83
Begin req 82 rxreq
Timestamp Start: 1566844978.384062 0.000000 0.000000
Timestamp Req: 1566844978.393264 0.009202 0.009202
ReqStart 51.89.7.90 55715
ReqMethod GET
ReqURL /index.php/giantpointsadmin/adminhtml_earning_rate/index
ReqProtocol HTTP/1.1
ReqHeader Host: www.test.pt
ReqHeader Connection: keep-alive
ReqHeader Accept-Encoding: gzip,deflate
ReqHeader User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36
ReqHeader X-Forwarded-For: 51.89.7.90
VCL_call RECV
ReqHeader grace: none
ReqURL /index.php/giantpointsadmin/adminhtml_earning_rate/index
ReqUnset Accept-Encoding: gzip,deflate
ReqHeader Accept-Encoding: gzip
VCL_return hash
VCL_call HASH
VCL_return lookup
VCL_call MISS
VCL_return fetch
Link bereq 84 fetch
Timestamp Fetch: 1566844978.393832 0.009770 0.000568
RespProtocol HTTP/1.1
RespStatus 503
RespReason Backend fetch failed
RespHeader Date: Mon, 26 Aug 2019 18:42:58 GMT
RespHeader Server: Varnish
RespHeader Content-Type: text/html; charset=utf-8
RespHeader Retry-After: 5
RespHeader X-Varnish: 83
RespHeader Age: 0
RespHeader Via: 1.1 varnish (Varnish/5.2)
VCL_call DELIVER
VCL_return deliver
Timestamp Process: 1566844978.393857 0.009795 0.000025
RespHeader Content-Length: 279
RespHeader Connection: keep-alive
Timestamp Resp: 1566844978.393929 0.009868 0.000072
ReqAcct 266 0 266 247 279 526
End
<< Session >> 82
Begin sess 0 HTTP/1
SessOpen 51.89.7.90 55715 a0 145.239.78.207 80 1566844978.383924 25
Link req 83 rxreq
SessClose REM_CLOSE 0.020
End
<< BeReq >> 32855
Begin bereq 32854 pass
Timestamp Start: 1566844978.423125 0.000000 0.000000
BereqMethod POST
BereqURL /index.php/giantpointsadmin/adminhtml_earning_rate/index/
BereqProtocol HTTP/1.1
BereqHeader Host: www.test.pt
BereqHeader Accept-Encoding: gzip,deflate
BereqHeader Content-Type: multipart/form-data; boundary=----------------7O12ZqAnhn621bBI
BereqHeader Content-Length: 326
BereqHeader User-Agent: Opera/9.80 (Windows NT 6.0); U) Presto/2.12.388 Version/12.16
BereqHeader Cookie: adminhtml=86g8r80a2uhrf66b95f2lnslo0
BereqHeader X-Forwarded-For: 51.89.7.90
BereqHeader X-Varnish: 32855
VCL_call BACKEND_FETCH
VCL_return fetch
FetchError no backend connection
Timestamp Beresp: 1566844978.423157 0.000033 0.000033
Timestamp Error: 1566844978.423165 0.000040 0.000008
BerespProtocol HTTP/1.1
BerespStatus 503
BerespReason Service Unavailable
BerespReason Backend fetch failed
BerespHeader Date: Mon, 26 Aug 2019 18:42:58 GMT
BerespHeader Server: Varnish
VCL_call BACKEND_ERROR
BerespHeader Content-Type: text/html; charset=utf-8
BerespHeader Retry-After: 5
VCL_return deliver
Storage malloc Transient
ObjProtocol HTTP/1.1
ObjStatus 503
ObjReason Backend fetch failed
ObjHeader Date: Mon, 26 Aug 2019 18:42:58 GMT
ObjHeader Server: Varnish
ObjHeader Content-Type: text/html; charset=utf-8
ObjHeader Retry-After: 5
Length 282
BereqAcct 0 0 0 0 0 0
End
<< Request >> 32854
Begin req 32853 rxreq
Timestamp Start: 1566844978.413495 0.000000 0.000000
Timestamp Req: 1566844978.422953 0.009458 0.009458
ReqStart 51.89.7.90 55717
ReqMethod POST
ReqURL /index.php/giantpointsadmin/adminhtml_earning_rate/index/
ReqProtocol HTTP/1.1
ReqHeader Host: www.test.pt
ReqHeader Connection: keep-alive
ReqHeader Accept-Encoding: gzip,deflate
ReqHeader Content-Type: multipart/form-data; boundary=----------------7O12ZqAnhn621bBI
ReqHeader Content-Length: 326
ReqHeader User-Agent: Opera/9.80 (Windows NT 6.0); U) Presto/2.12.388 Version/12.16
ReqHeader Cookie: adminhtml=86g8r80a2uhrf66b95f2lnslo0
ReqHeader X-Forwarded-For: 51.89.7.90
VCL_call RECV
VCL_return pass
VCL_call HASH
VCL_return lookup
VCL_call PASS
VCL_return fetch
Link bereq 32855 pass
Timestamp Fetch: 1566844978.423294 0.009799 0.000341
Storage malloc Transient
Timestamp ReqBody: 1566844978.423314 0.009819 0.000020
RespProtocol HTTP/1.1
RespStatus 503
RespReason Backend fetch failed
RespHeader Date: Mon, 26 Aug 2019 18:42:58 GMT
RespHeader Server: Varnish
RespHeader Content-Type: text/html; charset=utf-8
RespHeader Retry-After: 5
RespHeader X-Varnish: 32854
RespHeader Age: 0
RespHeader Via: 1.1 varnish (Varnish/5.2)
VCL_call DELIVER
VCL_return deliver
Timestamp Process: 1566844978.423336 0.009841 0.000022
RespHeader Content-Length: 282
RespHeader Connection: keep-alive
Timestamp Resp: 1566844978.423392 0.009897 0.000056
ReqAcct 374 326 700 250 282 532
End
<< Session >> 32853
Begin sess 0 HTTP/1
SessOpen 51.89.7.90 55717 a0 145.239.78.207 80 1566844978.413439 24
Link req 32854 rxreq
SessClose REM_CLOSE 0.020
End
Best regards,
Leonel Nunes
Related
I have installed Varnish and configured it as per as the guideline.
And when I try it curl -I https://d-o-m-a-i-n.com I get following which seems its working correctly.
HTTP/1.1 200 OK
Date: Sat, 28 Mar 2020 03:17:02 GMT
Server: Apache/2.4.18 (Ubuntu)
Expires: Sun, 29 Mar 2020 03:17:03 GMT
Cache-Control: max-age=86400, public, s-maxage=86400
Pragma: cache
X-Magento-Tags: cms_b_porto_homeslider_3,store,cms_b,cms_p_91,cms_b_porto_custom_notice_new,cat_p,cat_c_p_30,cat_p_22,cat_p_1,cat_p_34,cat_p_21,cat_p_41,cat_p_11,cat_p_39,cat_p_35,cat_p_33,cms_b_porto_footer_top_1_for_5,cms_b_porto_footer_middle_1_for_5,cms_b_porto_footer_middle_2_for_5
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Vary: Accept-Encoding
X-UA-Compatible: IE=edge
Content-Type: text/html; charset=UTF-8
X-Varnish: 33268 3
Age: 13957
Via: 1.1 varnish (Varnish/5.2)
X-Cache: cached
Accept-Ranges: bytes
But when I check in Google Chrome header response I get followings where it shows Varnish is not caching and age is 0?
Accept-Ranges: bytes
Age: 0
Cache-Control: max-age=86400, public, s-maxage=86400
Connection: Keep-Alive
Content-Encoding: gzip
Content-Length: 20027
Content-Type: text/html; charset=UTF-8
Date: Sat, 28 Mar 2020 07:13:35 GMT
Expires: Sun, 29 Mar 2020 07:13:35 GMT
Keep-Alive: timeout=5, max=100
Pragma: cache
Server: Apache/2.4.18 (Ubuntu)
Vary: Accept-Encoding
Via: 1.1 varnish (Varnish/5.2)
X-Cache: uncached
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Magento-Tags: cms_b_porto_homeslider_3,store,cms_b,cms_p_91,cms_b_porto_custom_notice_new,cat_p,cat_c_p_30,cat_p_1,cat_p_22,cat_p_21,cat_p_11,cat_p_34,cat_p_41,cat_p_39,cat_p_35,cat_p_33,cms_b_porto_footer_top_1_for_5,cms_b_porto_footer_middle_1_for_5,cms_b_porto_footer_middle_2_for_5
X-UA-Compatible: IE=edge
X-Varnish: 1704417
X-XSS-Protection: 1; mode=block
Do you have any ideas why that is?
How are you generating the key for caching(vcl_hash)?
Check it and make sure there is no user agent involved.
I'm working on a server-side HTTP/2 implementation.
Sometimes clients disconnect with a GOAWAY.COMPRESSION_ERROR in response to the server headers.
According to the Via header, in all of those cases, the client is an intermediary, 'Websense Web Security Gateway v7.6' (looks like a transparent caching solution)
The RFC says that COMPRESSION_ERROR can be sent in 2 cases.
1) A decoding error in a header block MUST be treated as a connection error of type COMPRESSION_ERROR
2) An endpoint receiving HEADERS, PUSH_PROMISE, or CONTINUATION frames needs to reassemble header blocks and perform decompression even if
the frames are to be discarded. A receiver MUST terminate the connection with a connection error of type COMPRESSION_ERROR if it does not decompress a header block
So in my case it's either an error on my side (bad headers encoding) or it's not an error but the client just discarded the response and sent a COMPRESSION_ERROR as specified by RFC.
If it's the latter then I don't need to fix anything.
But if it's the former then it's a bug in our implementation.
I'd like to rule out the bug case.
Below is a sample of such a session terminated with a COMPRESSION_ERROR.
It's hex dumps of data with my interpretation in {}.
S>C stands for server->client direction, C>S - client->server direction.
Can someone review the dump and tell me if it has some obvious issues with S>C frames or my interpretation of the data?
{preface stripped}
========== S>C frame: ============
00000000: 000024040000000000 ..$......
{type=SETTINGS stream=0 flags=NO_FLAGS(0) len=36 data={...}}
00000000: 00020000000000010000100000030000006400047fffffff000600007f380005 .................d...........8..
00000020: 00007ff7 ....
==================================
========== C>S frame: ============
00000000: 000012040000000000 .........
{type=SETTINGS stream=0 flags=NO_FLAGS(0) len=18 data={...}}
00000000: 000200000000000300000064000600008000 ...........d......
==================================
========== S>C frame: ============
00000000: 000000040100000000 .........
{type=SETTINGS stream=0 flags=ACK(1) len=0 data={}}
==================================
========== C>S frame: ============
00000000: 000004080000000000 .........
{type=WINDOW_UPDATE stream=0 flags=NO_FLAGS(0) len=4 data={inc=10485760}}
00000000: 00a00000 ....
==================================
========== C>S frame: ============
00000000: 000000040100000000 .........
{type=SETTINGS stream=0 flags=ACK(1) len=0 data={}}
==================================
========== C>S frame: ============
00000000: 00015c01050000000d ..\......
{type=HEADERS stream=13 flags=END_STREAM|END_HEADERS(5) len=348 data={...}}
00000000: 418aa4b2186cae824952fdea82449360d4cccb02005f603109a637e36a7c475e A....l..IR...D.`....._`1..7.j|G^
00000020: baa687539e352398ac782c75fd1a91cc56075d537d1a91cc563e7ebe58f9fbed ...S.5#..x,u....V.]S}...V>~.X...
00000040: 00177b73dc9d29ad171863c78f0ba0b54875fbd4c7f96a698a92c86b698072ca ..{s..)...c.....Hu....ji...ki.r.
00000060: 6a0c3497e2d4d3120f6d842c03210a8426c1c85f81cb2c4834903cefede170ed j.4......m.,.!..&.._..,H4.<...p.
00000080: 665435fb875c9b1a7b2dfceda35f7a67967e7ea3a19fde8dd02fdbf97d5e2ee9 fT5..\..{-..._zg.~~....../..}^..
000000a0: c97ad5d07f66a281b0dae053fae46aa43f8429a77a8102e0fb5396ae4e35fda9 .z...f.....S..j.?.).z....S..N5..
000000c0: 0d75d05e458f3192c36cbabb2e29fd66c7bf467fa5283752a988a4ea7fed4bd3 .u.^E.1..l...).f..F..(7R......K.
000000e0: d87a4ac3acae05d971e65708195370e51d8661b65d5d973f508d9bd9abfa5242 .zJ.....q.W..Sp...a.]].?P.....RB
00000100: cb40d25fa523b351b82d4b70ddf45abefb4005dffaee6fbed00177bebdcff7da .#._.#.Q.-Kp..Z..#....o...w.....
00000120: 002eefd7b9d6bda7efb4005dcfaf73ad8f37df6800bb7f5ee75b7f2fbed00176 ...........]..s..7.h...^.[./...v
00000140: bf7c99e458d05a9055391635371496d864fa5310d25f03f4a7775773 .|..X.Z.U9.57...d.S.._...wWs
==================================
========== S>C frame: ============
00000000: 00007c01040000000d ..|......
{type=HEADERS stream=13 flags=END_HEADERS(4) len=124 data={...}}
00000000: 3fe11f887684aa6355e76196c361be940bea435d8a08017d4006e32edc684a62 ?...v..cU.a..a....C]...}#....hJb
00000020: d1bf5f87352398ac5754df0f0d83644f3f6c96df3dbf4a09a5328ea50400bea0 .._.5#..WT....dO?l..=.J..2......
00000040: 1ab826ee32da98b46f628cfe5b234fb2566558923ffcff0f1596dd6d5f4a09e5 ..&.2...ob..[#O.VeX.?......m_J..
00000060: 21aec50400bea05ab8cbf700253168df588aa47e561cc581c034f001 !......Z....%1h.X..~V....4..
==================================
========== S>C frame: ============
00000000: 000cd900010000000d .........
{type=DATA stream=13 flags=END_STREAM(1) len=3289 data={...}}
{data stripped}
==================================
========== C>S frame: ============
00000000: 000014070000000000 .........
{type=GOAWAY stream=0 flags=NO_FLAGS(0) len=20 data={...}}
00000000: 0000000000000009000000000000033200000333 ...............2...3
{last_stream_id=0, error=COMPRESSION_ERROR}
==================================
========== S>C frame: ============
00000000: 000008070000000000 .........
{type=GOAWAY stream=0 flags=NO_FLAGS(0) len=8 data={...}}
00000000: 0000000d00000000 ........
==================================
C>S headers decoded:
:path: /img3/2019/1_24/TH4hGo.png HTTP/2.0
:authority: media3.scdn.vn
Accept: image/webp,image/apng,image/*,*/*;q=0.8
Referer: https://www.sendo.vn/?utm_medium=affiliate&utm_source=accesstrade&aff_sid=...
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,vi;q=0.8,zh;q=0.7,zh-CN;q=0.6,zh-HK;q=0.5,zh-TW;q=0.4
Via: Websense Web Security Gateway v7.6
S>C HEADERS frame decoded:
3fe11f
{set dyn table size 4096}
{dyn table size=0/4096}
88
{indexed header ix=8 :status: 200}
{dyn table not changed}
7684aa6355e7
{literal indexed header ix=54 Server: nginx }
{dyn table size=43/4096:
ix=62 off=0 size=43 Server: nginx
}
6196c361be940bea435d8a08017d4006e32edc684a62d1bf
{literal indexed header ix=33 Date: Fri, 19 Apr 2019 01:37:42 GMT }
{dyn table size=108/4096:
ix=62 off=0 size=65 Date: Fri, 19 Apr 2019 01:37:42 GMT
ix=63 off=65 size=43 Server: nginx
}
5f87352398ac5754df
{literal indexed header ix=31 Content-Type: image/png}
{dyn table size=161/4096:
ix=62 off=0 size=53 Content-Type: image/png
ix=63 off=53 size=65 Date: Fri, 19 Apr 2019 01:37:42 GMT
ix=64 off=118 size=43 Server: nginx
}
0f0d83644f3f
{literal not indexed header ix=28 Content-Length: 3289}
{dyn table not changed}
6c96df3dbf4a09a5328ea50400bea01ab826ee32da98b46f
{literal indexed header ix=44 Last-Modified: Thu, 24 Jan 2019 04:25:35 GMT }
{dyn table size=235/4096:
ix=62 off=0 size=74 Last-Modified: Thu, 24 Jan 2019 04:25:35 GMT
ix=63 off=74 size=53 Content-Type: image/png
ix=64 off=127 size=65 Date: Fri, 19 Apr 2019 01:37:42 GMT
ix=65 off=192 size=43 Server: nginx
}
628cfe5b234fb2566558923ffcff
{literal indexed header ix=34 ETag: "5c493e3f-cd9"}
{dyn table size=285/4096:
ix=62 off=0 size=50 ETag: "5c493e3f-cd9"
ix=63 off=50 size=74 Last-Modified: Thu, 24 Jan 2019 04:25:35 GMT
ix=64 off=124 size=53 Content-Type: image/png
ix=65 off=177 size=65 Date: Fri, 19 Apr 2019 01:37:42 GMT
ix=66 off=242 size=43 Server: nginx
}
0f1596dd6d5f4a09e521aec50400bea05ab8cbf700253168df
{literal not indexed header ix=36 Expires: Sun, 28 Apr 2019 14:39:02 GMT}
{dyn table not changed}
588aa47e561cc581c034f001
{literal indexed header ix=24 Cache-Control: max-age=604800}
{dyn table size=344/4096:
ix=62 off=0 size=59 Cache-Control: max-age=604800
ix=63 off=59 size=50 ETag: "5c493e3f-cd9"
ix=64 off=109 size=74 Last-Modified: Thu, 24 Jan 2019 04:25:35 GMT
ix=65 off=183 size=53 Content-Type: image/png
ix=66 off=236 size=65 Date: Fri, 19 Apr 2019 01:37:42 GMT
ix=67 off=301 size=43 Server: nginx
}
The decoded S>C headers:
:status: 200
Server: nginx
Date: Fri, 19 Apr 2019 01:37:42 GMT
Content-Type: image/png
Content-Length: 3289
Last-Modified: Thu, 24 Jan 2019 04:25:35 GMT
ETag: "5c493e3f-cd9"
Expires: Sun, 28 Apr 2019 14:39:02 GMT
Cache-Control: max-age=604800
I added an hpack decoder to h2get here: https://github.com/deweerdt/h2get/pull/18/files and i'm seeing the same results as you are:
$ ./hpack_decoder 3fe11f887684aa6355e76196c361be940bea435d8a08017d4006e32edc684a62d1bf5f87352398ac5754df0f0d83644f3f6c96df3dbf4a09a5328ea50400bea01ab826ee32da98b46f628cfe5b234fb2566558923ffcff0f1596dd6d5f4a09e521aec50400bea05ab8cbf700253168df588aa47e561cc581c034f001
dyn table resize to 4096
[:status] => [200]
[server] => [nginx]
[date] => [Fri, 19 Apr 2019 01:37:42 GMT]
[content-type] => [image/png]
[content-length] => [3289]
[last-modified] => [Thu, 24 Jan 2019 04:25:35 GMT]
[etag] => ["5c493e3f-cd9"]
[expires] => [Sun, 28 Apr 2019 14:39:02 GMT]
[cache-control] => [max-age=604800]
I'm unsure as to why the VCL is calculating the TTL to be 0 when I am setting the appropriate cache headers. I am hoping someone with a bit more experience may be able to spot something really obvious.
Here are the logs
varnish_1 | * << BeReq >> 18
varnish_1 | - Begin bereq 17 pass
varnish_1 | - Timestamp Start: 1522327212.212730 0.000000 0.000000
varnish_1 | - BereqMethod GET
varnish_1 | - BereqURL /routes/%2F
varnish_1 | - BereqProtocol HTTP/1.1
varnish_1 | - BereqHeader Host: localhost:8081
varnish_1 | - BereqHeader accept: application/ld+json
varnish_1 | - BereqHeader User-Agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Mobile Safari/537.36
varnish_1 | - BereqHeader DNT: 1
varnish_1 | - BereqHeader Referer: http://localhost:8081/
varnish_1 | - BereqHeader Accept-Encoding: gzip, deflate, br
varnish_1 | - BereqHeader Accept-Language: en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7
varnish_1 | - BereqHeader Cookie: _ga=GA1.1.917775181.1515688563; Phpstorm-53bb5460=b5be55f5-d679-4bb9-b2b2-108bdac978eb; PHPSESSID=3gc3fulg7kdv0k5hiaitmlaq7e
varnish_1 | - BereqHeader If-None-Match: "90cd705e5c80bbb83aa64439f9b18c62"
varnish_1 | - BereqHeader X-Forwarded-For: 172.18.0.1
varnish_1 | - BereqHeader X-Varnish: 18
varnish_1 | - VCL_call BACKEND_FETCH
varnish_1 | - VCL_return fetch
varnish_1 | - BackendOpen 23 boot.default 172.18.0.4 80 172.18.0.5 34084
varnish_1 | - BackendStart 172.18.0.4 80
varnish_1 | - Timestamp Bereq: 1522327212.212948 0.000218 0.000218
varnish_1 | - Timestamp Beresp: 1522327217.902208 5.689478 5.689260
varnish_1 | - BerespProtocol HTTP/1.1
varnish_1 | - BerespStatus 200
varnish_1 | - BerespReason OK
varnish_1 | - BerespHeader Server: nginx/1.13.9
varnish_1 | - BerespHeader Content-Type: application/ld+json; charset=utf-8
varnish_1 | - BerespHeader Transfer-Encoding: chunked
varnish_1 | - BerespHeader Connection: keep-alive
varnish_1 | - BerespHeader X-Powered-By: PHP/7.2.3
varnish_1 | - BerespHeader Vary: Accept
varnish_1 | - BerespHeader Vary: Content-Type
varnish_1 | - BerespHeader Vary: Authorization
varnish_1 | - BerespHeader X-Content-Type-Options: nosniff
varnish_1 | - BerespHeader X-Frame-Options: deny
varnish_1 | - BerespHeader Cache-Control: max-age=0, public, s-maxage=3600
varnish_1 | - BerespHeader Date: Thu, 29 Mar 2018 12:40:17 GMT
varnish_1 | - BerespHeader Link: <http://localhost:8081/docs.jsonld>; rel="http://www.w3.org/ns/hydra/core#apiDocumentation"
varnish_1 | - BerespHeader ETag: "fe8fa0778f4ee8045999017ccf82c34f"
varnish_1 | - BerespHeader Cache-Tags: /routes/%252F,/pages/3d07e6ba3a1d4ba1860a1846c0c1436b,/layouts/72ecbecdf0c140db8ac146dc69d70e33,/nav_bars/ef102fd0830442348273c924ff62249f,/component_groups/e0cbdc6f83f3444ba0392d20497c19e0,/component_groups/bd7c54c561a34442869369d9d655f08f,/
varnish_1 | - TTL RFC 3600 10 -1 1522327218 1522327218 1522327217 0 3600
varnish_1 | - VCL_call BACKEND_RESPONSE
varnish_1 | - BerespHeader url: /routes/%2F
varnish_1 | - TTL VCL -1 3600 0 1522327218
varnish_1 | - BerespHeader X-Cacheable: NO:Not Cacheable
varnish_1 | - BerespHeader X-Cache-TTL: 0.000
varnish_1 | - TTL VCL 120 3600 0 1522327218
varnish_1 | - VCL_return deliver
varnish_1 | - Storage malloc Transient
varnish_1 | - ObjProtocol HTTP/1.1
varnish_1 | - ObjStatus 200
varnish_1 | - ObjReason OK
varnish_1 | - ObjHeader Server: nginx/1.13.9
varnish_1 | - ObjHeader Content-Type: application/ld+json; charset=utf-8
varnish_1 | - ObjHeader X-Powered-By: PHP/7.2.3
varnish_1 | - ObjHeader Vary: Accept, Content-Type, Authorization
varnish_1 | - ObjHeader X-Content-Type-Options: nosniff
varnish_1 | - ObjHeader X-Frame-Options: deny
varnish_1 | - ObjHeader Cache-Control: max-age=0, public, s-maxage=3600
varnish_1 | - ObjHeader Date: Thu, 29 Mar 2018 12:40:17 GMT
varnish_1 | - ObjHeader Link: <http://localhost:8081/docs.jsonld>; rel="http://www.w3.org/ns/hydra/core#apiDocumentation"
varnish_1 | - ObjHeader ETag: "fe8fa0778f4ee8045999017ccf82c34f"
varnish_1 | - ObjHeader Cache-Tags: /routes/%252F,/pages/3d07e6ba3a1d4ba1860a1846c0c1436b,/layouts/72ecbecdf0c140db8ac146dc69d70e33,/nav_bars/ef102fd0830442348273c924ff62249f,/component_groups/e0cbdc6f83f3444ba0392d20497c19e0,/component_groups/bd7c54c561a34442869369d9d655f08f,/
varnish_1 | - ObjHeader url: /routes/%2F
varnish_1 | - ObjHeader X-Cacheable: NO:Not Cacheable
varnish_1 | - ObjHeader X-Cache-TTL: 0.000
varnish_1 | - Fetch_Body 2 chunked stream
varnish_1 | - BackendReuse 23 boot.default
varnish_1 | - Timestamp BerespBody: 1522327217.902374 5.689644 0.000166
varnish_1 | - Length 3195
varnish_1 | - BereqAcct 588 0 588 1972 3195 5167
varnish_1 | - End
varnish_1 |
varnish_1 | * << Request >> 17
varnish_1 | - Begin req 16 rxreq
varnish_1 | - Timestamp Start: 1522327212.212645 0.000000 0.000000
varnish_1 | - Timestamp Req: 1522327212.212645 0.000000 0.000000
varnish_1 | - ReqStart 172.18.0.1 50692
varnish_1 | - ReqMethod GET
varnish_1 | - ReqURL /routes/%2F
varnish_1 | - ReqProtocol HTTP/1.1
varnish_1 | - ReqHeader Host: localhost:8081
varnish_1 | - ReqHeader Connection: keep-alive
varnish_1 | - ReqHeader accept: application/ld+json
varnish_1 | - ReqHeader User-Agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Mobile Safari/537.36
varnish_1 | - ReqHeader DNT: 1
varnish_1 | - ReqHeader Referer: http://localhost:8081/
varnish_1 | - ReqHeader Accept-Encoding: gzip, deflate, br
varnish_1 | - ReqHeader Accept-Language: en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7
varnish_1 | - ReqHeader Cookie: _ga=GA1.1.917775181.1515688563; Phpstorm-53bb5460=b5be55f5-d679-4bb9-b2b2-108bdac978eb; PHPSESSID=3gc3fulg7kdv0k5hiaitmlaq7e
varnish_1 | - ReqHeader If-None-Match: "90cd705e5c80bbb83aa64439f9b18c62"
varnish_1 | - ReqHeader X-Forwarded-For: 172.18.0.1
varnish_1 | - VCL_call RECV
varnish_1 | - VCL_return pass
varnish_1 | - VCL_call HASH
varnish_1 | - VCL_return lookup
varnish_1 | - VCL_call PASS
varnish_1 | - VCL_return fetch
varnish_1 | - Link bereq 18 pass
varnish_1 | - Timestamp Fetch: 1522327217.902388 5.689743 5.689743
varnish_1 | - RespProtocol HTTP/1.1
varnish_1 | - RespStatus 200
varnish_1 | - RespReason OK
varnish_1 | - RespHeader Server: nginx/1.13.9
varnish_1 | - RespHeader Content-Type: application/ld+json; charset=utf-8
varnish_1 | - RespHeader X-Powered-By: PHP/7.2.3
varnish_1 | - RespHeader Vary: Accept, Content-Type, Authorization
varnish_1 | - RespHeader X-Content-Type-Options: nosniff
varnish_1 | - RespHeader X-Frame-Options: deny
varnish_1 | - RespHeader Cache-Control: max-age=0, public, s-maxage=3600
varnish_1 | - RespHeader Date: Thu, 29 Mar 2018 12:40:17 GMT
varnish_1 | - RespHeader Link: <http://localhost:8081/docs.jsonld>; rel="http://www.w3.org/ns/hydra/core#apiDocumentation"
varnish_1 | - RespHeader ETag: "fe8fa0778f4ee8045999017ccf82c34f"
varnish_1 | - RespHeader Cache-Tags: /routes/%252F,/pages/3d07e6ba3a1d4ba1860a1846c0c1436b,/layouts/72ecbecdf0c140db8ac146dc69d70e33,/nav_bars/ef102fd0830442348273c924ff62249f,/component_groups/e0cbdc6f83f3444ba0392d20497c19e0,/component_groups/bd7c54c561a34442869369d9d655f08f,/
varnish_1 | - RespHeader url: /routes/%2F
varnish_1 | - RespHeader X-Cacheable: NO:Not Cacheable
varnish_1 | - RespHeader X-Cache-TTL: 0.000
varnish_1 | - RespHeader X-Varnish: 17
varnish_1 | - RespHeader Age: 0
varnish_1 | - RespHeader Via: 1.1 varnish-v4
varnish_1 | - VCL_call DELIVER
varnish_1 | - RespUnset url: /routes/%2F
varnish_1 | - RespHeader X-Cache: MISS
varnish_1 | - VCL_return deliver
varnish_1 | - Timestamp Process: 1522327217.902479 5.689834 0.000091
varnish_1 | - RespHeader Accept-Ranges: bytes
varnish_1 | - RespHeader Content-Length: 3195
varnish_1 | - Debug "RES_MODE 2"
varnish_1 | - RespHeader Connection: keep-alive
varnish_1 | - Timestamp Resp: 1522327217.902553 5.689908 0.000074
varnish_1 | - ReqAcct 568 0 568 2086 3195 5281
varnish_1 | - End
varnish_1 |
varnish_1 | * << Session >> 16
varnish_1 | - Begin sess 0 HTTP/1
varnish_1 | - SessOpen 172.18.0.1 50692 :80 172.18.0.5 80 1522327212.212513 21
varnish_1 | - Link req 17 rxreq
varnish_1 | - SessClose RX_TIMEOUT 10.695
varnish_1 | - End
The TTL from RFC looks OK to me (I think). But then the TTL from the VCL is -1
Perhaps I've got a long header or something but if anyone can give a little advice I'd really appreciate it.
Thanks,
EDIT:
Here is my config (thought it may be helpful)
vcl 4.0;
import std;
backend default {
.host = "api";
.port = "80";
# Health check
#.probe = {
# .url = "/";
# .timeout = 5s;
# .interval = 10s;
# .window = 5;
# .threshold = 3;
#}
}
# Hosts allowed to send BAN requests
acl ban {
"localhost";
"php";
}
sub vcl_backend_response {
# Ban lurker friendly header
set beresp.http.url = bereq.url;
# Add a grace in case the backend is down
set beresp.grace = 1h;
if (bereq.http.Cookie ~ "(UserID|_session)") {
set beresp.http.X-Cacheable = "NO:Got Session";
set beresp.uncacheable = true;
} elsif (beresp.ttl <= 0s) {
# Varnish determined the object was not cacheable
set beresp.http.X-Cacheable = "NO:Not Cacheable";
set beresp.http.X-Cache-TTL = beresp.ttl;
} elsif (beresp.http.set-cookie) {
# You don't wish to cache content for logged in users
set beresp.http.X-Cacheable = "NO:Set-Cookie";
set beresp.uncacheable = true;
} elsif (beresp.http.Cache-Control ~ "private") {
# You are respecting the Cache-Control=private header from the backend
set beresp.http.X-Cacheable = "NO:Cache-Control=private";
set beresp.uncacheable = true;
} else {
# Varnish determined the object was cacheable
set beresp.http.X-Cacheable = "YES";
}
}
sub vcl_deliver {
# Don't send cache tags related headers to the client
unset resp.http.url;
# Uncomment the following line to NOT send the "Cache-Tags" header to the client (prevent using CloudFlare cache tags)
#unset resp.http.Cache-Tags;
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT";
} else {
set resp.http.X-Cache = "MISS";
}
}
sub vcl_recv {
# Remove the "Forwarded" HTTP header if exists (security)
unset req.http.forwarded;
# To allow API Platform to ban by cache tags
if (req.method == "BAN") {
if (client.ip !~ ban) {
return(synth(405, "Not allowed"));
}
if (req.http.ApiPlatform-Ban-Regex) {
ban("obj.http.Cache-Tags ~ " + req.http.ApiPlatform-Ban-Regex);
return(synth(200, "Ban added"));
}
return(synth(400, "ApiPlatform-Ban-Regex HTTP header must be set."));
}
}
# From https://github.com/varnish/Varnish-Book/blob/master/vcl/grace.vcl
sub vcl_hit {
if (obj.ttl >= 0s) {
# Normal hit
return (deliver);
} elsif (std.healthy(req.backend_hint)) {
# The backend is healthy
# Fetch the object from the backend
return (fetch);
} else {
# No fresh object and the backend is not healthy
if (obj.ttl + obj.grace > 0s) {
# Deliver graced object
# Automatically triggers a background fetch
return (deliver);
} else {
# No valid object to deliver
# No healthy backend to handle request
# Return error
return (synth(503, "API is down"));
}
}
}
After a chat on the IRC (user fgs was very helpful) I can read the logs a bit better and understand why this is happening.
vcl_recv is returning pass instead of lookup because cookies are sent in the request.
I missed the docs here:
https://varnish-cache.org/docs/4.1/users-guide/increasing-your-hitrate.html#cookies
After a lot of unsuccessful research, I'm wondering if it's possible to use nginx or varnish cache function to check data freshness it contains using etag attribute in the header.
This scenario is easier to explain with a diagram, please see below :
http://image.noelshack.com/fichiers/2014/49/1417684577-cache-nginx.png
Do you think this use case is feasible ?
Thank you and sorry for my english :s
You can do something similar with varnish, stale-while-revalidate, but not quite the same. The idea with the front caches is that when the backend sends an object to the cache it sets an Expires header that will allow the cache to serve this content without bothering the backend. If the cache would check every request with the backend then there wouldn't be much speed to gain, as you would end up waiting for the backend on each request.
To solve this, you should set your Expires headers accordingly, and/or purge/ban the urls if suddenly you need to get rid of stale content before Expires. That way the cache would send the content from its own cache, or get from the backend if it doesn't have it or it's stale.
Edit:
I have tested with Varnish 4.0.1, it supports conditional requests to the backend.
My test setup has been an Apache virtual hosts with ExpiresByType text/html "access plus 5 seconds"
The result:
* << BeReq >> 65539
- Begin bereq 65538 bgfetch
- Timestamp Start: 1418403624.141695 0.000000 0.000000
- BereqMethod GET
- BereqURL /prueba.html
- BereqProtocol HTTP/1.1
- BereqHeader User-Agent: lwp-request/6.03 libwww-perl/6.05
- BereqHeader X-Forwarded-For: 127.0.0.1
- BereqHeader Host: localtest
- BereqHeader Surrogate-Capability: key=ESI/1.0
- BereqHeader Accept-Encoding: gzip
- BereqHeader If-Modified-Since: Thu, 01 Dec 2011 10:37:55 GMT
- BereqHeader If-None-Match: "3e8be5-46-4b30571f606c0"
- BereqHeader X-Varnish: 65539
- VCL_call BACKEND_FETCH
- VCL_return fetch
- Backend 17 vdir server1(127.0.0.1,,80)
- Timestamp Bereq: 1418403624.141869 0.000174 0.000174
- Timestamp Beresp: 1418403624.143205 0.001510 0.001336
- BerespProtocol HTTP/1.1
- BerespStatus 304
- BerespReason Not Modified
- BerespHeader Date: Fri, 12 Dec 2014 17:00:24 GMT
- BerespHeader Server: Apache
- BerespHeader ETag: "3e8be5-46-4b30571f606c0"
- BerespHeader Expires: Fri, 12 Dec 2014 17:00:29 GMT
- BerespHeader Cache-Control: max-age=5
- TTL RFC 5 -1 -1 1418403624 1418403624 1418403624 1418403629 5
- BerespProtocol HTTP/1.1
- BerespStatus 200
- BerespReason OK
- BerespHeader Last-Modified: Thu, 01 Dec 2011 10:37:55 GMT
- BerespHeader Content-Type: text/html
- VCL_call BACKEND_RESPONSE
- TTL VCL 5 21600 0 1418403624
- VCL_return deliver
- Storage malloc s0
- ObjProtocol HTTP/1.1
- ObjStatus 200
- ObjReason OK
- ObjHeader Date: Fri, 12 Dec 2014 17:00:24 GMT
- ObjHeader Server: Apache
- ObjHeader ETag: "3e8be5-46-4b30571f606c0"
- ObjHeader Expires: Fri, 12 Dec 2014 17:00:29 GMT
- ObjHeader Cache-Control: max-age=5
- ObjHeader Last-Modified: Thu, 01 Dec 2011 10:37:55 GMT
- ObjHeader Content-Type: text/html
- BackendReuse 17 server1(127.0.0.1,,80)
- Timestamp BerespBody: 1418403624.143465 0.001770 0.000260
- Length 70
- BereqAcct 289 0 289 181 0 181
- End
Here you can see that I ask for a stale object, so varnish goes to fetch it, and request it this way:
- BereqURL /prueba.html
- BereqHeader If-Modified-Since: Thu, 01 Dec 2011 10:37:55 GMT
- BereqHeader If-None-Match: "3e8be5-46-4b30571f606c0"
The backend answers with:
- BerespStatus 304
- BerespReason Not Modified
- BerespHeader Date: Fri, 12 Dec 2014 17:00:24 GMT
- BerespHeader ETag: "3e8be5-46-4b30571f606c0"
- BerespHeader Expires: Fri, 12 Dec 2014 17:00:29 GMT
- BerespHeader Cache-Control: max-age=5
So no body bytes are transferred, and varnish refreshes it's object with the newly updated headers.
Perhaps that's the behavior you wanted.
Keep in mind that a GET request that does validate the conditions (If-Modified-Since and If-None-Match) would only return a 304 Not Modified, some headers and no body (exactly the same as if it would be a HEAD request with the same conditions), if it doesn't validate it will return the headers and the new body. But a HEAD requests that does not validate would return 200 OK, and NO body, forcing you to re-request it with a GET.
HTTP/1.1 Method Definitions RFC:
The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation
contained in the HTTP headers in response to a HEAD request SHOULD be
identical to the information sent in response to a GET request.
I create one proxy cache using (tor + privoxy + squid) follow
How to create an advanced proxy server using Tor, Privoxy, and Squid
I try call to Proxy cache
curl -x 'my-ip:3400' -I 'https://github.global.ssl.fastly.net/images/modules/dashboard/bootcamp/octocat_setup.png'
curl -x 'my-ip:3400' -I http://hadoop.apache.org/images/hadoop-logo.jpg
https request is ok (through Pivoxy)
HTTP/1.1 200 Connection established
Proxy-Agent: Privoxy/3.0.21
HTTP/1.1 200 OK
Date: Sat, 08 Mar 2014 04:38:37 GMT
Server: GitHub.com
Content-Type: image/png
Last-Modified: Fri, 16 Aug 2013 09:42:18 GMT
Expires: Sun, 08 Mar 2015 04:38:37 GMT
Cache-Control: max-age=31536000, public
Timing-Allow-Origin: https://github.com
Content-Length: 10722
Accept-Ranges: bytes
Via: 1.1 varnish
Age: 0
X-Served-By: cache-am72-AMS
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1394253517.449177980,VS0,VE97
Vary: Accept-Encoding
but http requets not working
HTTP/1.0 200 OK
Date: Sat, 08 Mar 2014 04:39:19 GMT
Server: Apache/2.4.8-dev (Unix) mod_wsgi/3.4 Python/2.7.5 OpenSSL/1.0.1e
Last-Modified: Tue, 12 Feb 2008 17:38:57 GMT
ETag: "24e3-445f987fe69d8"
Accept-Ranges: bytes
Content-Length: 9443
Content-Type: image/jpeg
X-Cache: MISS from SVR226L-696.localdomain
X-Cache-Lookup: MISS from SVR226L-696.localdomain:3128
Via: 1.0 SVR226L-696.localdomain (squid)
Connection: keep-alive
Please suggest a specific solution