I have in nginx a config file like
server {
root /var/www/releaser/site/web/;
location ~ \.php$ {
include snippets/fastcgi-php.conf;
fastcgi_pass unix:/var/run/php5-fpm.sock;
}
location ^~ /images/ {
alias /var/www/releaser/site/web/img/;
}
# Media: images, icons, video, audio, HTC
location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
expires 1M;
access_log off;
add_header Cache-Control "public";
}
include snippets/nginx-basics.conf;
error_log /var/log/nginx/site-error.log;
access_log /var/log/nginx/site-access.log;
server_name site.test;
}
So, when I do this (the extra headers for images are missing):
$ curl -I site.test/images/one.jpg
HTTP/1.1 200 OK
Server: nginx/1.9.3 (Ubuntu)
Date: Tue, 10 Nov 2015 05:29:59 GMT
Content-Type: image/jpeg
Content-Length: 185547
Last-Modified: Fri, 24 Jul 2015 22:12:20 GMT
Connection: keep-alive
Accept-Ranges: bytes
But, if I do this (the extra headers for images are included):
$ curl -I site.test/img/one.jpg
HTTP/1.1 200 OK
Server: nginx/1.9.3 (Ubuntu)
Date: Tue, 10 Nov 2015 05:29:36 GMT
Content-Type: image/jpeg
Content-Length: 185547
Last-Modified: Fri, 24 Jul 2015 22:12:20 GMT
Connection: keep-alive
Expires: Thu, 10 Dec 2015 05:29:36 GMT
Cache-Control: max-age=2592000
Cache-Control: public
Accept-Ranges: bytes
How can I fix this without putting a copy of the extra headers inside the alias' location section?
Thanks a lot!
One solution is to replace the alias directive with a rewrite directive:
location ^~ /images/ {
rewrite ^/images(.*)$ /img$1;
}
Related
I have installed Varnish and configured it as per as the guideline.
And when I try it curl -I https://d-o-m-a-i-n.com I get following which seems its working correctly.
HTTP/1.1 200 OK
Date: Sat, 28 Mar 2020 03:17:02 GMT
Server: Apache/2.4.18 (Ubuntu)
Expires: Sun, 29 Mar 2020 03:17:03 GMT
Cache-Control: max-age=86400, public, s-maxage=86400
Pragma: cache
X-Magento-Tags: cms_b_porto_homeslider_3,store,cms_b,cms_p_91,cms_b_porto_custom_notice_new,cat_p,cat_c_p_30,cat_p_22,cat_p_1,cat_p_34,cat_p_21,cat_p_41,cat_p_11,cat_p_39,cat_p_35,cat_p_33,cms_b_porto_footer_top_1_for_5,cms_b_porto_footer_middle_1_for_5,cms_b_porto_footer_middle_2_for_5
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Vary: Accept-Encoding
X-UA-Compatible: IE=edge
Content-Type: text/html; charset=UTF-8
X-Varnish: 33268 3
Age: 13957
Via: 1.1 varnish (Varnish/5.2)
X-Cache: cached
Accept-Ranges: bytes
But when I check in Google Chrome header response I get followings where it shows Varnish is not caching and age is 0?
Accept-Ranges: bytes
Age: 0
Cache-Control: max-age=86400, public, s-maxage=86400
Connection: Keep-Alive
Content-Encoding: gzip
Content-Length: 20027
Content-Type: text/html; charset=UTF-8
Date: Sat, 28 Mar 2020 07:13:35 GMT
Expires: Sun, 29 Mar 2020 07:13:35 GMT
Keep-Alive: timeout=5, max=100
Pragma: cache
Server: Apache/2.4.18 (Ubuntu)
Vary: Accept-Encoding
Via: 1.1 varnish (Varnish/5.2)
X-Cache: uncached
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Magento-Tags: cms_b_porto_homeslider_3,store,cms_b,cms_p_91,cms_b_porto_custom_notice_new,cat_p,cat_c_p_30,cat_p_1,cat_p_22,cat_p_21,cat_p_11,cat_p_34,cat_p_41,cat_p_39,cat_p_35,cat_p_33,cms_b_porto_footer_top_1_for_5,cms_b_porto_footer_middle_1_for_5,cms_b_porto_footer_middle_2_for_5
X-UA-Compatible: IE=edge
X-Varnish: 1704417
X-XSS-Protection: 1; mode=block
Do you have any ideas why that is?
How are you generating the key for caching(vcl_hash)?
Check it and make sure there is no user agent involved.
I'm seeing the following warning in Chrome Developer Tools when I access my JHipster application at http://www.jhipster-book.com. I don't see this error when running locally.
Failed to decode downloaded font: http://www.jhipster-book.com/assets/fonts/sourcesanspro/sourcesanspro.woff
I'm using these same fonts on a client's project (that runs on GAE) and it works fine. It looks like the content-type header is not being set on Heroku. Here's what I see locally:
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/2.7.9
Date: Fri, 12 Jun 2015 21:57:23 GMT
Content-type: application/x-font-woff
Content-Length: 27248
Last-Modified: Fri, 12 Sep 2014 16:22:14 GMT
And on Heroku:
HTTP/1.1 200 OK
Connection: keep-alive
Server: Apache-Coyote/1.1
X-Application-Context: application:prod,heroku:56079
Cache-Control: max-age=2678400000, public
Pragma: cache
Expires: Mon, 13 Jul 2015 21:59:11 GMT
Last-Modified: Fri, 12 Jun 2015 21:47:41 GMT
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
Content-Length: 27247
Date: Fri, 12 Jun 2015 21:59:11 GMT
Via: 1.1 vegur
Is there something I have to configure in Heroku for this to work?
I have configured Hadoop HDFS with security enabled using Kerberos (CDH4.3). It works fine, I can do all the operations using command line tools. There are some other non hadoop cluster nodes need to access HDFS file system thru REST API, so I have webhdfs enabled with Kerberos configured.
My secure webhdfs works fine for operations that don't involve temporary redirect: ie LISTSTATUS, create directory, remove directory, remove files, all work fine. But for two step operations like create file, open file, it failed with below info:
[DEV][root#namenode ~]# curl -i --negotiate -u : -X
PUT "http: //namenode:50070/webhdfs/v1/tmp/hosts.txt?op=CREATE"
HTTP/1.1 401
Date: Fri, 25 Apr 2014 02:45:48 GMT
Pragma: no-cache
Date: Fri, 25 Apr 2014 02:45:48 GMT
Pragma: no-cache
WWW-Authenticate: Negotiate
Set-Cookie: hadoop.auth=;Path=/;Expires=Thu, 01-Jan-1970 00:00:00 GMT
Content-Length: 0
Server: Jetty(6.1.26.cloudera.2)
HTTP/1.1 307 TEMPORARY_REDIRECT
Cache-Control: no-cache
Expires: Thu, 01-Jan-1970 00:00:00 GMT
Date: Fri, 25 Apr 2014 02:45:48 GMT
Pragma: no-cache
Date: Fri, 25 Apr 2014 02:45:48 GMT
Pragma: no-cache
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs#UNIXKRB&t=kerberos&e=1398429948267&s=XhEp/tfs5Pfp04Dp
9yy1moFPnEo=";Path=/
Location: http: //datanode.net:1006/webhdfs/v1/tmp/hosts.txt?
op=CREATE&delegation=HgAEaGRmcwRoZGZzAIoBRZbGSGyKAUW60sxsWo4BAhTfG8IRyDJjJ2BSR7K
hveMEo3V4dxJXRUJIREZTIGRlbGVnYXRpb24NMC4wLjAuMDo1MDA3MA&namenoderpcaddress=
namenode.net:8020&overwrite=false
Content-Type: application/octet-stream
Content-Length: 0
Server: Jetty(6.1.26.cloudera.2)
[DEV][root#namenode~]# curl -i -X PUT -T hosts.txt
http: //datanode.net:1006/webhdfs/v1/tmp/hosts.txt?
op=CREATE&delegation=HgAEaGRmcwRoZGZzAIoBRZbGSGyKAUW60sxsWo4BAhTfG8IRyDJjJ2
BSR7KhveMEo3V4dxJXRUJIREZTIGRlbGVnYXRpb24NMC4wLjAuMDo1MDA3MA&namenoderpcaddress=
namenode.net:8020&overwrite=false
[2] 19047
[3] 19048
[4] 19049
[DEV][root#namenode~]# HTTP/1.1 100 Continue
HTTP/1.1 401 Unauthorized
Cache-Control: no-cache
Expires: Fri, 25 Apr 2014 02:46:26 GMT
Date: Fri, 25 Apr 2014 02:46:26 GMT
Pragma: no-cache
Expires: Fri, 25 Apr 2014 02:46:26 GMT
Date: Fri, 25 Apr 2014 02:46:26 GMT
Pragma: no-cache
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(6.1.26.cloudera.2)
{"RemoteException":
{"exception":"SecurityException","javaClassName":"java.lang.SecurityException",
"message":"Failed to obtain user group information: java.io.IOException:
Security enabled but user not authenticated by filter"}}
[2] Done curl -i -X PUT -T hosts.txt
http://datanode.net:1006/webhdfs/v1/tmp/hosts.txt?op=CREATE
[3] Done
delegation=HgAEaGRmcwRoZGZzAIoBRZbGSGyKAUW60sxsWo4BAhTfG8IRyDJjJ2BSR7KhveM
Eo3V4dxJXRUJIREZTIGRlbGVnYXRpb24NMC4wLjAuMDo1MDA3MA
[4]- Done namenoderpcaddress=namenode.net:8020
[DEV][root#namenode ~]#
Is there anyone who can shed some lights?
Thanks a lot,
In my case it was working fine as below:
$ curl -i --negotiate -u : -X PUT "http://nn30.my.dom:50070/webhdfs/v1/tmp/hosts.txt?op=CREATE"
HTTP/1.1 401
Date: Wed, 30 Apr 2014 12:39:24 GMT
Pragma: no-cache
Date: Wed, 30 Apr 2014 12:39:24 GMT
Pragma: no-cache
WWW-Authenticate: Negotiate
Set-Cookie: hadoop.auth=;Path=/;Expires=Thu, 01-Jan-1970 00:00:00 GMT
Content-Length: 0
Server: Jetty(7.6.10.v20130312)
HTTP/1.1 307 Temporary Redirect
Date: Wed, 30 Apr 2014 12:39:24 GMT
Pragma: no-cache
Cache-Control: no-cache
Date: Wed, 30 Apr 2014 12:39:24 GMT
Pragma: no-cache
Set-Cookie: hadoop.auth="u=gpadmin&p=gpadmin#MY.DOM&t=kerberos&e=1398897564735&s=p8haj7KXAsUqj5A2WzoE5VxJYA8=";Path=/
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Location: http ://dn33.my.dom:1006/webhdfs/v1/tmp/hosts.txt?op=CREATE&delegation=IgAHZ3BhZG1pbgdncGFkbWluAIoBRbKli0OKAUXWsg9DAxIUt8i6BG2xt5V4wB4vPpXSPm8MMVMSV0VCSERGUyBkZWxlZ2F0aW9uEzE5Mi4xNjguMTAuMzA6NTAwNzA&namenoderpcaddress=nn30.my.dom:8020&overwrite=false
Content-Type: application/octet-stream
Content-Length: 0
Server: Jetty(7.6.10.v20130312)
$ curl -i -X PUT -T hosts.txt "http://dn33.my.dom:1006/webhdfs/v1/tmp/hosts.txt?op=CREATE&delegation=IgAHZ3BhZG1pbgdncGFkbWluAIoBRbKli0OKAUXWsg9DAxIUt8i6BG2xt5V4wB4vPpXSPm8MMVMSV0VCSERGUyBkZWxlZ2F0aW9uEzE5Mi4xNjguMTAuMzA6NTAwNzA&namenoderpcaddress=nn30.my.dom:8020&overwrite=false"
HTTP/1.1 100 Continue
HTTP/1.1 201 Created
Expires: Wed, 30 Apr 2014 12:42:24 GMT
Date: Wed, 30 Apr 2014 12:42:24 GMT
Pragma: no-cache
Cache-Control: no-cache
Expires: Wed, 30 Apr 2014 12:42:24 GMT
Date: Wed, 30 Apr 2014 12:42:24 GMT
Pragma: no-cache
Location: webhdfs ://nn30.my.dom:50070/tmp/hosts.txt
Content-Type: application/octet-stream
Content-Length: 0
Server: Jetty(7.6.10.v20130312)
NOTE - if your screen capture was not the edited/modified you had few syntax issues ("") and typo things. I bet you missed "" for the http input from first command.
check here for your info - http ://hadoop.apache.org/docs/r1.0.4/webhdfs.html
BTW, I used Pivotal HD, there shouldn't be that much difference though.
I create one proxy cache using (tor + privoxy + squid) follow
How to create an advanced proxy server using Tor, Privoxy, and Squid
I try call to Proxy cache
curl -x 'my-ip:3400' -I 'https://github.global.ssl.fastly.net/images/modules/dashboard/bootcamp/octocat_setup.png'
curl -x 'my-ip:3400' -I http://hadoop.apache.org/images/hadoop-logo.jpg
https request is ok (through Pivoxy)
HTTP/1.1 200 Connection established
Proxy-Agent: Privoxy/3.0.21
HTTP/1.1 200 OK
Date: Sat, 08 Mar 2014 04:38:37 GMT
Server: GitHub.com
Content-Type: image/png
Last-Modified: Fri, 16 Aug 2013 09:42:18 GMT
Expires: Sun, 08 Mar 2015 04:38:37 GMT
Cache-Control: max-age=31536000, public
Timing-Allow-Origin: https://github.com
Content-Length: 10722
Accept-Ranges: bytes
Via: 1.1 varnish
Age: 0
X-Served-By: cache-am72-AMS
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1394253517.449177980,VS0,VE97
Vary: Accept-Encoding
but http requets not working
HTTP/1.0 200 OK
Date: Sat, 08 Mar 2014 04:39:19 GMT
Server: Apache/2.4.8-dev (Unix) mod_wsgi/3.4 Python/2.7.5 OpenSSL/1.0.1e
Last-Modified: Tue, 12 Feb 2008 17:38:57 GMT
ETag: "24e3-445f987fe69d8"
Accept-Ranges: bytes
Content-Length: 9443
Content-Type: image/jpeg
X-Cache: MISS from SVR226L-696.localdomain
X-Cache-Lookup: MISS from SVR226L-696.localdomain:3128
Via: 1.0 SVR226L-696.localdomain (squid)
Connection: keep-alive
Please suggest a specific solution
with Open::URI, I can do the following:
require 'open-uri'
#check status
open('http://google.com').status
#get entire html
open('http://google.com').read
Is it possible to get the HTTP headers of a request so things can be debugged, something like Curls' curl -I http://google.com?
$ curl -I google.com
HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Mon, 17 Dec 2012 14:28:17 GMT
Expires: Wed, 16 Jan 2013 14:28:17 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 219
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Is this possible?
Use the meta method of the virtual filehandle:
open('http://google.com'){|f| pp f.meta }
{"x-frame-options"=>"SAMEORIGIN",
"expires"=>"-1",
"p3p"=>
"CP=\"This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info.\"",
"content-type"=>"text/html; charset=ISO-8859-1",
"date"=>"Mon, 17 Dec 2012 14:37:29 GMT",
"server"=>"gws",
"x-xss-protection"=>"1; mode=block",
"set-cookie"=>
"PREF=ID=d2fb8a93d369bcd2:FF=0:TM=1355755049:LM=1355755049:S=ONVSP6n2jtluFgll; expires=Wed, 17-Dec-2014 14:37:29 GMT; path=/; domain=.google.com, NID=67=OFEvvHCOa3C6wScQCUIKfu_89oL9MSmnFjwN-u5LX_foP8NLsX7G9dq48NLVrf4WUXhqOA1jb38s0e9qeRp1Iwx_LT_N8IuF0Qi6dXVtR2zdvA86INqtfg5uNrKvxJfJ; expires=Tue, 18-Jun-2013 14:37:29 GMT; path=/; domain=.google.com; HttpOnly",
"cache-control"=>"private, max-age=0",
"transfer-encoding"=>"chunked"}
http://www.ruby-doc.org/stdlib-1.9.3/libdoc/open-uri/rdoc/OpenURI/Meta.html