Having a problem with Varnish 3.x probe page from a SpringBoot application (1.4). Varnish is failing to detect the probe page (returns 503 SERVICE NOT AVAILABLE) and consequently fails to route.
When I manually ping the probe URL, it works fine, but Varnish is flagging the probe page as being down.
Removing the probe page, everything works fine.
Pointing to a static probe page (my.css) or any other static or dynamic URL fails.
Looking at the logs, the response header looks like this:
HTTP/1.1 200
Content-Type: text/plain;charset=utf-8
Content-Length: 72
Date: Wed, 01 Feb 2017 15:20:48 GMT
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Comparing this to other working (non Spring Boot) applications, the only difference is that the working applications have an OK after the response, and the bad ones don't:
HTTP/1.1 200 OK
Does that mean anything?
For example, here is a good one:
HTTP/1.1 200 OK
Date: Wed, 01 Feb 2017 14:04:18 GMT
Last-Modified: Fri, 11 Nov 2016 22:00:02 GMT
Content-Type: text/css
Content-Length: 129
Server: Jetty(9.3.z-SNAPSHOT)
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Is this a SpringBoot issue? Not sure where else to look!
Any clues?
Varnish support has replied
You found the issue: https://github.com/varnishcache/varnish-cache/issues/2069
This is "fixed" in master. But maybe you can fix your backend?
Now to see how to change Spring Boot response message!
Related
It seems I can't use Google Cloud CDN with GKE Ingress in https requests, because the Ingress controller adds a Vary http header: x-forwarded-proto, which CDN won't cache (see https://cloud.google.com/cdn/docs/caching#non-cacheable_content). I think it's correct that the header is added, because it tells the application serving the content that it's being requested as https. But I'm not sure why CDN won't cache pages with this header.
Am I missing something or is there a workaround?
Example of header when page is requested as http and is cached:
Age: 1713
Cache-Control: public, max-age=2000
Content-Encoding: gzip
Content-Length: 8503
Content-Type: text/html; charset=UTF-8
Date: Thu, 13 Aug 2020 20:56:23 GMT
Server: Apache/2.4.38 (Debian)
Vary: Accept-Encoding
Via: 1.1 google
X-Powered-By: PHP/7.4.9
Example of header when page is requested as http and isn't cached:
alt-svc: clear
cache-control: public, max-age=2000
content-encoding: gzip
content-length: 8499
content-type: text/html; charset=UTF-8
date: Thu, 13 Aug 2020 21:25:37 GMT
server: Apache/2.4.38 (Debian)
status: 200
vary: X-Forwarded-Proto,Accept-Encoding
via: 1.1 google
x-powered-by: PHP/7.4.9
Many thanks,
Sam
As you already confirmed the Cloud CDN won't cache it due to this header. This would be expected.
Regarding that header which is added, you will need to review why it's added and find a way to remove it if you want to cache this content with Cloud CDN. I found a few article that you can review:
https://github.com/kubernetes/ingress-gce/issues/51
https://serverfault.com/questions/912168/how-to-make-google-cloud-load-balancer-respect-the-received-x-forwarded-proto
https://trac.nginx.org/nginx/ticket/1181
You would have to find a solution to remove this header based on your deployment.
I am not using the SBT, but making direct REST calls with Abdera to the current version of Connections on IBM SmartCloud. REST URL in question: https://apps.na.collabserv.com/search/serviceconfigs
Observations
When testing from my laptop (using Firefox and the REST client add-on,) this works as expected. I get back an ATOM feed.
When testing from a server (on a different network,) using the same method (Firefox + REST client,) I get back HTML that is a log-in page.
In addition, I get this same result when I call the URL from a Java program on the same server.
In all cases, I am using the same credentials with basic authentication.
Update: If I log into SmartCloud first, on a separate tab in Firefox on the server, then call the URL as before, from another tab, it works. I get the ATOM feed as desired. Naturally, this is unsuitable as a solution, but I present it as additional information that could lead to an actual solution.
Update: Further testing shows that the local (laptop) log-in exhibits the same behavior as the server. A form-based log-in is required from the same browser, then subsequent REST calls work.
Update: Here is a relevant simplified code snippet:
private static Abdera ABDERA = new Abdera();
private static AbderaClient ABDERA_CLIENT = new AbderaClient(ABDERA);
...
String host = "https://apps.na.collabserv.com";
ABDERA_CLIENT.addCredentials(host, AuthScope.ANY_REALM, "basic", new UsernamePasswordCredentials("user", "password"));
...
ClientResponse response = ABDERA_CLIENT.get("https://apps.na.collabserv.com/search/serviceconfigs");
Summary
It appears that something about the originating server or the call is causing SmartCloud to respond with a log-in page. Whereas, the same call and credentials from my laptop, work as expected.
Question
Where should I start to trouble-shoot this? How can I build the client credentials to allow programmatic log-in?
Response Headers
If it helps, here are the response headers that I get back in each case.
Unsuccessful
Status Code: 200 OK
Cache-Control: no-cache
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 1850
Content-Type: text/html
Date: Tue, 08 Oct 2013 14:15:03 GMT
Pragma: no-cache
Server: WebSEAL/6.1.1.3 (Build 110428)
Set-Cookie: PD-H-SESSION-ID=4_0_IR4***masked***oRKlJI;secure; Path=/; HttpOnly BIGipServerE3A-WebSEAL-80-fe=2132806922.20480.0000;secure; path=/
Vary: Accept-Encoding
p3p: CP="NON CUR OTPi OUR NOR UNI"
Successful
Status Code: 200 OK
Cache-Control: public, max-age=86400, s-maxage=86400, no-cache=set-cookie, private, must-revalidate
Content-Encoding: gzip
Content-Language: en-US
Content-Length: 1164
Content-Type: application/atom+xml; charset=UTF-8
Date: Mon, 07 Oct 2013 17:21:12 GMT
Expires: Tue, 08 Oct 2013 17:21:12 GMT
Server: WebSphere Application Server/8.0
Vary: Accept-Encoding
p3p: CP="NON CUR OTPi OUR NOR UNI"
x-lconn-auth: true
x-powered-by: Servlet/3.0
#Grant is your login using SAML? I could see this redirect happening. also could be TFIM related... maybe you should grab the auth on a different page, store the cookies, and then try connecting to the endpoint above.
Good Day.
I'm trying to figure out a caching issue. We are currently using a CMS with built in caching. We can then delete the cached copy via the control panel and the uncached page will be served until we cache that page again.
Long story short, we know that a either a proxy server or load balancer was put in place, and we think that it is caching the pages as well as the CMS. Our specific issue is that when we un-cache the page in the CMS, it's taking 15 minutes (timed) to show up un-cached (even after using different browser, clearing the browser cache, etc.), when prior to the network appliance being introduced, the un-cached page would show up immediately. Unfortunately we don't have any historical response headers saved anywhere.
When we believe that the page is being cached by the proxy/lb, the Response header is returning:
HTTP/1.1 304 Not Modified
Server: Apache/2.0.59 (Unix) JRun/4.0 mod_ssl/2.0.59 OpenSSL/0.9.8k PHP/5.2.6
Last-Modified: Fri, 03 Aug 2012 13:29:12 GMT
Etag: "92fe-18f7-837ada00"
Accept-Ranges: bytes
Keep-Alive: timeout=5, max=100
Content-Type: text/html
Content-Encoding: gzip
Connection: Keep-Alive
Date: Mon, 06 Aug 2012 13:49:40 GMT
X-Cntnt-Length: 6391
When it's not being cached by the CMS, the response headers are:
HTTP/1.0 200 OK
Date: Mon, 06 Aug 2012 14:03:59 GMT
Server: Apache/2.0.59 (Unix) JRun/4.0 mod_ssl/2.0.59 OpenSSL/0.9.8k PHP/5.2.6
X-Powered-By: PHP/5.2.6
Set-Cookie: blah-blah-blah
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Last-Modified: Mon, 06 Aug 2012 14:04:04 GMT
Pragma: no-cache
Connection: close
Content-Type: text/html
I guess my question is, can Apache2 be configured to return both HTTP 1.0 and 1.1 ?
I know the Etag indicates a cached page, and believe it's not available in HTTP/1.0.
Thanks for any insights.
WebClient.DownloadStringAsync does cache the server response.
After once getting a response from the server I get a response even without internet connection!
Is WebClient caching smart enough to determine from the server response how long to cache?
Or is it buggy and I should disable caching.
Backgound info:
Url: http://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml
Fiddler trace:
GET /stats/eurofxref/eurofxref-daily.xml HTTP/1.1
Accept: /
Referer: file:///Applications/Install/4D0DF1F7-1481-45CA-86BE-C14FF5CCD955/Install/
Accept-Encoding: identity
User-Agent: NativeHost
Host: www.ecb.europa.eu
Connection: Keep-Alive
HTTP/1.1 200 OK
Date: Sun, 25 Mar 2012 08:54:40 GMT
Server: Apache/2.2.3 (Linux/SUSE)
Last-Modified: Fri, 23 Mar 2012 13:31:39 GMT
ETag: "19d4e5-6a9-4bbe90b5904c0"
Accept-Ranges: bytes
Content-Length: 1705
Keep-Alive: timeout=3, max=200
Connection: Keep-Alive
Content-Type: text/xml
Set-Cookie: BIGipServerPOOL.www.ecb.europa.eu_HTTP=2684883628.16415.0000; path=/
...
Disabling caching via Headers does not work:
.Headers("cache-control") = "no-cache"
.Headers("HttpRequestHeader.IfModifiedSince") = DateTime.UtcNow.ToString()
Disabling caching via appending uniqa parameter works:
"http://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml" & "?MakeRequestUnique=" & Environment.TickCount
The integrated cache isn't smart at all. So if you expect different results when querying the page, you have to bypass it. I say 'bypass' because there's no way I know of to disable it with the WebClient (I don't think it's enabled if you directly use the HttpRequest class).
So if you want to use the WebClient, the best way is to append a random parameter to the request.
We're having some really strange problems with IE7, and only IE7. When trying to replicate the issue it only happens with native IE7, not when running IE7 mode in IE8/9, so please remember that if you try to replicate this issue.
The problem is the following:
We're polling for a response from the server with AJAX. The user posts something that the server may have to work on for a bit, so every 5 seconds or so a request is sent check if the server is done. This works fine in every browser, except native IE7. The problem is that it never stops "loading". When checking the requests with Fiddler2 we see that it does two requests and then nothing more. It stops doing anything when still in PENDING mode. In a sain browser it keeps on polling, and then stops when it gets "CONFIRMED". The really weird thing is that it does its final request and returns as normal, only, and really only, when you open a new tab.
It's not that the page needs focus or anything, clicking around randomly does nothing. I'm asking here because I can't even reproduce the issue using a local instance of the project.
Here are the headers sent back by the server:
These are the headers for the response on the production machine. This was the last response gotten. It loads indefinitely until you open a new tab (just an empty new tab!), then the final request is made and everything works out.
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 0
Cache-Control: max-age=0, private, must-revalidate
Content-Type: application/json; charset=utf-8
Date: Tue, 04 Oct 2011 07:37:45 GMT
ETag: "867dafc628c43b6ca8a73d1977669250"
P3P: CP="ALL DSP COR CURa ADMa DEVa OUR IND COM NAV"
Server: nginx/1.0.6
Set-Cookie: _web_session=COOKIE; path=/; expires=Tue, 04-Oct-2011 10:37:45 GMT; HttpOnly
Vary: Accept-Encoding
Via: 1.1 varnish
X-Cache: MISS
X-Runtime: 0.062794
X-UA-Compatible: IE=Edge,chrome=1
X-Varnish: 55900984
Content-Length: 145
Connection: keep-alive
{"direct_publishing_settings_id":9970,"confirmed":"PENDING","errors":{},"username":"************","url":"","blog_id":44606,"platform":"blogg_se"}
These are the headers for the same request on my local server. This does not stall up the requests.
HTTP/1.1 200 OK
X-Ua-Compatible: IE=Edge
Etag: "253c934246a69c9ca821464f80f400b3"
P3p: CP="ALL DSP COR CURa ADMa DEVa OUR IND COM NAV"
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Date: Tue, 04 Oct 2011 07:34:22 GMT
Server: WEBrick/1.3.1 (Ruby/1.8.7/2010-01-10)
X-Runtime: 0.459232
Content-Length: 137
Cache-Control: max-age=0, private, must-revalidate
Set-Cookie: _web_session=COOKIE; path=/; expires=Tue, 04-Oct-2011 10:34:22 GMT; HttpOnly
{"direct_publishing_settings_id":10,"confirmed":"PENDING","url":"","blog_id":29,"errors":{},"username":"fsasaffas","platform":"blogg_se"}
If you want to try it you can go to videofy.me, get a new account (it's really easy), when logged in go to videofy.me/blogger/settings/direct_publishing. Choose a blog platform in the first dropdown, press "activate" and write something into the username/password fields that appear, then press the green button and wait forever. After 45 seconds or a random amount of time open a new tab, and see that the request is magically finished.
I'm guessing it has to do something with IE7 cacheing the request, and then something is released if a new tab is opened. But it's just a guess, and googling returns nothing related. I'm posting here because it's so obscure that I hope someone here knows anything about it.