I'm writing a simple application that will download some piece of data from a website then I can use it later for any purpose.
The following is the request and response copied from Firebug as the browser did that.
When you type http://x5.travian.com.sa/ajax.php?f=k7&x=18&y=-186&xx=12&yy=-192
you will get a PHP file has some data.
But when I make a request with HttpWebRequest I get wrong data (some unknown letters)
Can anyone help me in that? Do I have to make some encodings or what?
Response
1. Server nginx
2. Date Tue, 04 Jan 2011 23:03:49 GMT
3. Content-Type application/json; charset=UTF-8
4. Transfer-Encoding chunked
5. Connection keep-alive
6. X-Powered-By PHP/5.2.8
7. Expires Mon, 26 Jul 1997 05:00:00 GMT
8. Last-Modified Tue, 04 Jan 2011 23:03:49 GMT
9. Cache-Control no-store, no-cache, must-revalidate, post-check=0, pre-check=0
10. Pragma no-cache
11. Content-Encoding gzip
12. Vary Accept-Encoding
Request
1. Host x5.travian.com.sa
2. User-Agent Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13)
3. Gecko/20101203 Firefox/3.6.13
4. Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
5. Accept-Language en-us,en;q=0.5
6. Accept-Encoding gzip,deflate
7. Accept-Charset ISO-8859-1,utf-8;q=0.7,*;q=0.7
8. Keep-Alive 115
9. Connection keep-alive
10. Cookie CAD=57878984%231292375897%230%230%23%230; T3E=%3DImYykTN2EzMmhjO5QTM2QDN2oDM1ITOyoDOxIjM4EDN5ITM6gjO4MDOxIWZyQWMipTZu9metl2ctl2c6MDNxADN6MDNxADNjMDNxADNjMDNxADN; orderby_b1=0; orderby_b=0; orderby2=0; orderby=0
I used wget on unix to hit the site, and I got back a page that had arabic characters in it. As per the response headers the page is encoded as UTF8.
If you can show the code you are using, we could provide feedback. Maybe there is a bug in your code in the way you are interpreting the response stream.
Finally it is solved, the response was encoded with the gzib encode, so I just decode it with the gzibdecodestream...
Related
I'm stumped, my website was working fine and now on Firefox suddenly the redirects stopped working.
I've tested IE and Chrome and going to /login redirects me to /dashboard however on Firefox the page is blank (no output sent) and no errors are logged. So this is why I'm assuming it to be a browser related issue. It might be due to a firefox update, but not sure how to confirm that.
Here are the headers:
Request Headers
GET /login HTTP/1.1
Host: local.example.com
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0 FirePHP/0.7.4
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Cookie: __utma=34805930.947644602.1372214584.1380730296.1380733154.30; __utmz=34805930.1378700053.15.2.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); __utma=214248714.242656582.1377296111.1380047082.1380734348.30; __utmz=214248714.1377296111.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __qca=P0-705514134-1378344178153; __utmc=34805930; __utmb=34805930.15.10.1380733154; __utmb=214248714.5.10.1380734348; __utmc=214248714; PHPSESSID=lli8i30qkhvohfm9ufkbdvbki0
x-insight: activate
Connection: keep-alive
Response Headers
HTTP/1.1 302 Found
Date: Wed, 02 Oct 2013 17:30:58 GMT
Server: Apache/2.4.3 (Win32) OpenSSL/1.0.1c PHP/5.4.7
X-Powered-By: PHP/5.4.7
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Location: /dashboard
Content-Length: 0
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8
It all looks pretty standard to me, however FF stays stuck on /login Am I missing something?
This behaviour is both on my local windows host and my remote amazon Linux web-server. The body is empty...
How could I go about debugging this?
The Expires header field in the response is really off. Firefox probably does not bother to render stale responses.
Please check the system time in your server. It is possible it is an Amazon problem, but it is also possible that one of the server users set the system time.
You can look into setting up a Network Time Protocol (NTP) client to run regularly (with ntpd), if you don't have that yet.
I would fire up Fiddler to see what bits actually went over the wire. Among other information, Fiddler will show what content type is actually used during the HTTP request / response.
This could be related to the fact that there is no extension. Firefox could be having trouble determining if this is a document or folder. Try firebug and see what URL Firefox tries to request after the redirect.
firefox do not send the header : if-modified-since after receive Last-modified from the response.
eg:
visiting load a website in webview in Firefox, and after several times load this page，press F12，in the console show below screenshot.
Response :
Age 123613
Cache-Control public, max-age=31536000
Content-Length 93868
Content-Type text/javascript; charset=UTF-8
Date Fri, 16 Mar 2012 04:17:48 GMT
Expires Thu, 24 May 2012 14:47:48 GMT
Last-Modified Wed, 23 Nov 2011 21:10:59 GMT
Server sffe
Vary Accept-Encoding
Via 1.1 HSPROXY08
Warning 113 HSPROXY08
X-Content-Type-Options nosniff
X-XSS-Protection 1; mode=block
Request:
Accept */*
Accept-Encoding gzip, deflate
Accept-Language zh-cn,zh;q=0.5
Host ajax.googleapis.com
Proxy-Connection keep-alive
Referer https://stackoverflow.com/questions/5066087/load-a-website-in-webview
User-Agent Mozilla/5.0 (Windows NT 5.1; rv:10.0.2) Gecko/20100101 Firefox/10.0.2
I find there is no "if-modified-since" in request header, that's why?
and I thought the response status code should be 304,but it is 200.
thanks..
before sending your 304 header,
1/ you need to send the same exact headers (except for the date & expires values obviously) you've sent with the file the first time in the exact same order
2/ you add a 'Content-Length:0' header
then you send that 304 header and then you're good.
I've encountered quite unexpected problem using Tomcat and CAS authorization. I just cannot logout in Firefox. I'm redirected to the logout page, but as soon as I reenter application url in the address bar, it is opened as if I'm logged (and I'm logged actually!).
First I've take a notable amount of attempts to fix something in tomcat config, then I've read logs, but nothing helped me actually before it comes up to my mind to check logout behavior in other browsers.
In other browsers everything work just as expected.
And I'm just stuck and would appreciate if one will give me a hint.
I guess [this question][1] is in some way relative with mine, but, helas, disabling caching on the page which should me logouted doesn't help either.
UPD: Some debug information. Firefox's version is 7.0.1, unfortunately, it is not a public application and I can not provide any url. It looks like response.sendRedirect output is something that Firefox is missing. Here is minimal code that works in any browser except Firefox.
session.invalidate();
response.sendRedirect("https://app:8552/cas/logout");
HEADERS
1st REQUEST - which invalidates session and redirect to CAS logout page
REQUEST HEADERS
Host: dev.service.net
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Referer: http://dev.service.net/
Cookie: JSESSIONID=53B9469EFE9F130E9694F7406BFAB755
RESPONSE HEADERS
Server: nginx/1.0.4
Date: Thu, 20 Oct 2011 09:20:45 GMT
Content-Type: text/html
Content-Length: 184
Location: https://dev:8552/cas/logout
2nd REQUEST - cas logout page itself
REQUEST HEADERS
Host: dev:8552
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Referer: http://dev.service.net/
Cookie: JSESSIONID=8A68F008825A0F0D14C6BF803E1332CF; GUEST_LANGUAGE_ID=en_US; COOKIE_SUPPORT=true
RESPONSE HEADERS
Server: Apache-Coyote/1.1
Pragma: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: no-cache, no-store
Content-Type: text/html;charset=UTF-8
Content-Language: en-US
Content-Length: 1226
Date: Thu, 20 Oct 2011 15:53:57 GMT
3rd REQUEST - we are retuninig to the page which actually should
redirect us to login page, but it does not.
REQUEST HEADERS
Host: dev.service.net
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Cookie: JSESSIONID=53B9469EFE9F130E9694F7406BFAB755
RESPONSE HEADERS
Server: Apache-Coyote/1.1
Pragma: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: no-cache, no-store
Content-Type: text/html;charset=UTF-8
Content-Language: en-US
Content-Length: 1226
Date: Thu, 20 Oct 2011 13:30:51 GMT
According to the headers, you're maintaining two different sessions on two different hosts. When you request a logout on the first host, you're redirected to the second host (which uses a different session cookie). The session cookie of the second host is in turn indeed invalidated (according to the presence of the Set-Cookie header). But based on the last request, the session has not been recreated on the server side (there is no Set-Cookie header). This means that session.invalidate() before response.sendRedirect() has failed somehow, or that the page is actually requested from the browser cache.
In Firebug you should be able to see if the page is requested from the browser cache by checking the text color of the request in the Net tab. If it's grayed out, then it means that it's been served from the browser cache. For Firefox, the must-revalidate header is actually mandatory next to the no-cache, no-store headers. You need to configure your server to add that entry to the header, or to change/create a Filter for that.
See also:
How to control web page caching, across all browsers?
I used config given in this for page level caching of php content. The problem is that the cached page is saving in gzip format and it's returning the same gzip content to browser.
I need the o/p like this "12:15:37 12:15:47" (Its coming for 1st time when the page is not cached) after that if request is resend it is returning ‹������34²26±24à23Œ¸¸�`Î9”��� (gzip response as I tried zcat its returning fine)
Response Headers
Server nginx/0.8.34
Date Wed, 17 Mar 2010 07:04:58 GMT
Content-Type text/html
Last-Modified Wed, 17 Mar 2010 07:04:20 GMT
Transfer-Encoding chunked
Connection keep-alive
Vary Accept-Encoding
Expires Wed, 17 Mar 2010 07:04:58 GMT
Cache-Control max-age=0
Content-Encoding gzip
Request Headers
Host localhost
User-Agent Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.18) Gecko/2010021501
Ubuntu/9.04 (jaunty) Firefox/3.0.18 GTB6
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language en-us,en;q=0.5
Accept-Encoding gzip,deflate
Accept-Charset ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive 300
Connection keep-alive
Its possible you missed adding the gzip_static option which will serve gzipped content correctly.
However, I have now posted a new article based on Nginx 0.7+ which is better for use as a proxy cache. It doesn't need that option anymore.
http://www.webtatic.com/blog/2010/04/page-level-caching-with-nginx/
How can I convince Firefox (3.0.1, if it matters) to send an If-Modified-Since header in an HTTPS request? It sends the header if the request uses plain HTTP and my server dutifully honors it. But when I request the same resource from the same server using HTTPS instead (i.e., simply changing the http:// in the URL to https://) then Firefox does not send an If-Modified-Since header at all. Is this behavior mandated by the SSL spec or something?
Here are some example HTTP and HTTPS request/response pairs, pulled using the Live HTTP Headers Firefox extension, with some differences in bold:
HTTP request/response:
http://myserver.com:30000/scripts/site.js
GET /scripts/site.js HTTP/1.1
Host: myserver.com:30000
User-Agent: Mozilla/5.0 (...) Gecko/2008070206 Firefox/3.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
If-Modified-Since: Tue, 19 Aug 2008 15:57:30 GMT
If-None-Match: "a0501d1-300a-454d22526ae80"-gzip
Cache-Control: max-age=0
HTTP/1.x 304 Not Modified
Date: Tue, 19 Aug 2008 15:59:23 GMT
Server: Apache/2.2.8 (Unix) mod_ssl/2.2.8 OpenSSL/0.9.8
Connection: Keep-Alive
Keep-Alive: timeout=5, max=99
Etag: "a0501d1-300a-454d22526ae80"-gzip
HTTPS request/response:
https://myserver.com:30001/scripts/site.js
GET /scripts/site.js HTTP/1.1
Host: myserver.com:30001
User-Agent: Mozilla/5.0 (...) Gecko/2008070206 Firefox/3.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
HTTP/1.x 200 OK
Date: Tue, 19 Aug 2008 16:00:14 GMT
Server: Apache/2.2.8 (Unix) mod_ssl/2.2.8 OpenSSL/0.9.8
Last-Modified: Tue, 19 Aug 2008 15:57:30 GMT
Etag: "a0501d1-300a-454d22526ae80"-gzip
Accept-Ranges: bytes
Content-Encoding: gzip
Content-Length: 3766
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/javascript
UPDATE: Setting browser.cache.disk_cache_ssl to true did the trick (which is odd because, as Nickolay points out, there's still the memory cache). Adding a "Cache-control: public" header to the response also worked. Thanks!
HTTPS requests are not cached so sending an If-Modified-Since doesn't make any sense. The not caching is a security precaution.
The not caching on disk is a security pre-caution, but it seems it indeed affects the If-Modified-Since behavior (glancing over the code).
Try setting the Firefox preference (in about:config) browser.cache.disk_cache_ssl to true. If that helps, try sending Cache-Control: public header in your response.
UPDATE: Firefox behavior was changed for Gecko 2.0 (Firefox 4) -- HTTPS content is now cached.
HTTPS requests are not cached so sending an If-Modified-Since doesn't make any sense. The not caching is a security precaution.