I am setting up apache traffic server as forward proxy and want to push some objects in the cache from out side. I tried following instructions given here.
when I try to push an object following is the output I get
telnet 127.0.0.1 8080
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
PUSH http://www.company.com HTTP/1.0
Content-length: 84
HTTP/1.0 200 OK
Content-type: text/html
Content-length: 17
<HTML>
a
</HTML>
HTTP/1.0 400 Response Not Cachable
Date: Sun, 02 Aug 2015 16:55:41 GMT
Server: ATS/3.2.4
Cache-Control: no-store
Content-Type: text/html
Content-Language: en
Content-Length: 200
<HEAD><TITLE>Response Not Cachable</TITLE></HEAD>
<BODY BGCOLOR="white" FGCOLOR="black">
<FONT FACE="Helvetica,Arial"><B>
</B></FONT>
<!-- default "Response Not Cachable" response (400) -->
</BODY>
Connection closed by foreign host.
Yeah, you have to submit the PUSH request with a correct, cacheable response header. Depending on your ATS settings, that could either be something as simple as a Last-Modified header, or, more likely (ATS defaults) a properly specified Cache-Control: max-age= header.
Edit: Take a look at the traffic_primer script that comes with the ATS source, or from github at https://github.com/apache/trafficserver/blob/master/tools/traffic_primer . It doesn't do exactly what you are asking for, but what it does is to "replay" a request from an origin to other ATS boxes, using PUSH.
Related
According to QuickBooks Ruby API documentation, it provides an example of using curl to submit a POST request to the authorization url obtained from Step 1, but right underneath it, it shows that if the web application doesn't support browsers, to use Playground or a web component.
My question is – is it actually possible to use the curl command as shown in the example? If I take the exact URL and try to establish a POST request, I get the following results:
bash-3.2$ curl -X POST "https://appcenter.intuit.com/connect/oauth2?client_id=[redacted]&redirect_uri=http%3A%2F%2Flocalhost&response_type=code&scope=com.intuit.quickbooks.accounting&state=be17472c59724eb46bfe2690"
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
bash-3.2$
Obviously it's trying to redirect to another URL, but the API documentation doesn't show the usage of the -L parameter to follow redirects. If I try to provide the -L parameter for curl to follow redirects, then I get the following response:
bash-3.2$ curl -X POST "https://appcenter.intuit.com/connect/oauth2?client_id=[redacted]&redirect_uri=http%3A%2F%2Flocalhost&response_type=code&scope=com.intuit.quickbooks.accounting&state=be17472c59724eb46bfe2690" -L
<HTML><HEAD>
<TITLE>Bad Request</TITLE>
</HEAD><BODY>
<H1>Bad Request</H1>
Your browser sent a request that this server could not understand.<P>
Reference #7.2500e8ac.1592267832.14229c52
</BODY>
</HTML>
Again, doesn't work even following redirects. If I append -I to the curl command, then I get the following:
bash-3.2$ curl -X POST "https://appcenter.intuit.com/connect/oauth2?client_id=[redacted]&redirect_uri=http%3A%2F%2Flocalhost&response_type=code&scope=com.intuit.quickbooks.accounting&state=be17472c59724eb46bfe2690" -LI
HTTP/2 301
date: Tue, 16 Jun 2020 00:37:17 GMT
content-type: text/html
content-length: 162
location: https://appcenter.intuit.com/app/connect/oauth2?client_id=[redacted]&redirect_uri=http%3A%2F%2Flocalhost&response_type=code&scope=com.intuit.quickbooks.accounting&state=be17472c59724eb46bfe2690
server: nginx
cache-control: no-store, no-cache, must-revalidate
strict-transport-security: max-age=3156000; includeSubDomains; preload
HTTP/2 302
date: Tue, 16 Jun 2020 00:37:17 GMT
content-type: text/plain;charset=utf-8
content-length: 406
location: https://accounts.intuit.com/index.html?partner_uid_button=google&offering_id=Intuit.sbg-fms.ippdevx&redirect_url=https%3A%2F%2Fappcenter.intuit.com%2Fapp%2Fconnect%2Foauth2%3Fclient_id%3D[redacted]%26redirect_uri%3Dhttp%253A%252F%252Flocalhost%26response_type%3Dcode%26scope%3Dcom.intuit.quickbooks.accounting%26state%3Dbe17472c59724eb46bfe2690
server: nginx
strict-transport-security: max-age=15552000
intuit_tid: 1-5ee8143d-29a68cec2ec922da5c6be528
x-spanid: ad76586b-a5ac-41bd-b2df-022148a5a78b
x-amzn-trace-id: Self=1-5ee8143d-3aa839e8b2cf25d846078238;Root=1-5ee8143d-29a68cec2ec922da5c6be528
x-dns-prefetch-control: off
x-download-options: noopen
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
cache-control: private,no-cache,no-store,pre-check=0,post-check=0,must-revalidate
expires: -1
pragma: no-cache
set-cookie: hosted-shell=%7B%22clientId%22%3A%22a4676677-5191-4eca-873e-34a04b5b1dd4%22%7D; Path=/; Expires=Fri, 14 Jun 2030 00:37:17 GMT; Secure
vary: Accept, Accept-Encoding
cache-control: no-store, no-cache, must-revalidate
strict-transport-security: max-age=3156000; includeSubDomains; preload
HTTP/2 411
server: AkamaiGHost
mime-version: 1.0
content-type: text/html
content-length: 223
expires: Tue, 16 Jun 2020 00:37:17 GMT
date: Tue, 16 Jun 2020 00:37:17 GMT
If I take the same URL and request it via a web browser, then it just takes me to the redirect callback URL as it's supposed to. I can't figure out why curl isn't doing the same thing. I don't understand what it means by "web component" if I can't even replicate the same thing via curl. I also have another web module that I'm using for GET and POST requests, and they're bringing me to an HTTP200 but not the callback URL.
Any idea how I could accomplish getting this authorization token from the redirect callback URL like it does in the web browser? Here's another example of me trying to do this via the web component in the Ruby script:
[3] pry(#<QuickBooksAPI>)> grant_url
=> "https://appcenter.intuit.com/connect/oauth2?client_id=[redacted]&redirect_uri=http%3A%2F%2Flocalhost&response_type=code&scope=com.intuit.quickbooks.accounting&state=be17472c59724eb46bfe2690"
[4] pry(#<QuickBooksAPI>)> response = WebRequest.new.get_request(grant_url)
=> #<Net::HTTPMovedPermanently 301 Moved Permanently readbody=true>
[5] pry(#<QuickBooksAPI>)> response['Location']
=> "https://appcenter.intuit.com/app/connect/oauth2?client_id=[redacted]&redirect_uri=http%3A%2F%2Flocalhost&response_type=code&scope=com.intuit.quickbooks.accounting&state=be17472c59724eb46bfe2690"
[6] pry(#<QuickBooksAPI>)> response = WebRequest.new.get_request(response['Location'])
=> #<Net::HTTPFound 302 Found readbody=true>
[7] pry(#<QuickBooksAPI>)> response['Location']
=> "https://accounts.intuit.com/index.html?partner_uid_button=google&offering_id=Intuit.sbg-fms.ippdevx&redirect_url=https%3A%2F%2Fappcenter.intuit.com%2Fapp%2Fconnect%2Foauth2%3Fclient_id%3D[redacted]%26redirect_uri%3Dhttp%253A%252F%252Flocalhost%26response_type%3Dcode%26scope%3Dcom.intuit.quickbooks.accounting%26state%3Dbe17472c59724eb46bfe2690"
[8] pry(#<QuickBooksAPI>)> response = WebRequest.new.get_request(response['Location'])
=> #<Net::HTTPOK 200 OK readbody=true>
[9] pry(#<QuickBooksAPI>)> response.uri
=> #<URI::HTTPS https://accounts.intuit.com/index.html?partner_uid_button=google&offering_id=Intuit.sbg-fms.ippdevx&redirect_url=https%3A%2F%2Fappcenter.intuit.com%2Fapp%2Fconnect%2Foauth2%3Fclient_id%3D[redacted]%26redirect_uri%3Dhttp%253A%252F%252Flocalhost%26response_type%3Dcode%26scope%3Dcom.intuit.quickbooks.accounting%26state%3Dbe17472c59724eb46bfe2690>
This time it sends me to a 200 but it never actually sends me to the callback/redirect URL.
I'm just simply trying to complete step 2 but cannot do so without a browser it seems, even using "web components" and curl.
Am I just going to have to manually use OAuth Playground and continuously just refresh my token since my API is all backend and no front-end/user redirect, etc.?
The OAuth grant flow needs your end user to grant your authenticate your application before you call API using access tokens. So here is what you process would look like
Present your user with a Connect to Intuit (or equivalent) button
User Clicks on the button and goes through authentication on intuit website (seems like the first bit of code you have inserted in your question. This cannot be automated)
Intuit redirects to the redirect url with a authorization_code
Rest is automated - Your browser or server takes the authorization_code from the url, combines it with a few other parameters and requests for an access_token
You can then continue to use the access_token to make your calls on behalf of the end user.
From Step3. onwards you can use curl to process all of it. Hope this helps :)
EDIT
No, In your case you would use OAuth differently. Using the Oauth Playground will take create an initial access_token for you. Steps 1-4 are taken care of.
Now you can use this token to authenticate all your API calls. However, the token will expire. So when you do get an expired response then you have to do a curl to fetch a new token using the info here...
https://developer.intuit.com/app/developer/qbo/docs/develop/authentication-and-authorization/oauth-2.0#refresh-the-token
Here's the curl -I response to my Javascript file:
HTTP/1.1 200 OK
Content-Type: text/javascript
Content-Length: 72640
Connection: keep-alive
Date: Sat, 18 Feb 2017 16:12:06 GMT
Cache-Control: 86400
Last-Modified: Wed, 15 Feb 2017 15:09:28 GMT
ETag: "a6ee06ff5e49a4290bb2aabe5e0f9029"
Server: AmazonS3
Vary: Accept-Encoding
Age: 1173
X-Cache: Hit from cloudfront
Via: 1.1 3b17302562f1709d8b6c9f7be1.cloudfront.net (CloudFront)
I can see the Cache-Control tag there. Not sure what the Vary and the ETag are doing, but so be it. Does this somehow specify to a user's browser NOT to cache this file? Why are Pingdom or Goog PageSpeed not recognising this as a browser-cacheable file?
Your Cache-Control header is present, but the value is not actually valid. The correct format looks like this:
Cache-Control: max-age=86400
The number, by itself, is meaningless.
ETag: is the entity tag -- an opaque value that uniquely identifies the current content of a given URL. If the content changes, the ETag will also change. A browser with a cached copy may use this value for subsequent requests to ask that the server only return the content if it differs, by sending an If-None-Match: request header, including the last-seen ETag.
Vary: tells the browser that certain changes to the request may generate a different response. Unlike browsers, curl doesn't advertise its ability to support gzipped payload unless you specify the --compressed option. Adding that option when invoking curl triggers the addition of Accept-Encoding: gzip to the request, which may trigger the response to be compressed if you have that option enabled in CloudFront.
Having a problem with Varnish 3.x probe page from a SpringBoot application (1.4). Varnish is failing to detect the probe page (returns 503 SERVICE NOT AVAILABLE) and consequently fails to route.
When I manually ping the probe URL, it works fine, but Varnish is flagging the probe page as being down.
Removing the probe page, everything works fine.
Pointing to a static probe page (my.css) or any other static or dynamic URL fails.
Looking at the logs, the response header looks like this:
HTTP/1.1 200
Content-Type: text/plain;charset=utf-8
Content-Length: 72
Date: Wed, 01 Feb 2017 15:20:48 GMT
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Comparing this to other working (non Spring Boot) applications, the only difference is that the working applications have an OK after the response, and the bad ones don't:
HTTP/1.1 200 OK
Does that mean anything?
For example, here is a good one:
HTTP/1.1 200 OK
Date: Wed, 01 Feb 2017 14:04:18 GMT
Last-Modified: Fri, 11 Nov 2016 22:00:02 GMT
Content-Type: text/css
Content-Length: 129
Server: Jetty(9.3.z-SNAPSHOT)
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Is this a SpringBoot issue? Not sure where else to look!
Any clues?
Varnish support has replied
You found the issue: https://github.com/varnishcache/varnish-cache/issues/2069
This is "fixed" in master. But maybe you can fix your backend?
Now to see how to change Spring Boot response message!
I'm trying to build a simple script using the Pocket API and I'm getting a 400 bad request when following the documentation on the site. Here is what I get:
POST /v3/oauth/request?consumer_key=xxxxxx-
xxxxxxxxxxxxxxxxxxx&redirect_uri=http://localhost:3000/callback HTTP/1.1
Host: getpocket.com
Cache-Control: no-cache
Content-Type: application/x-www-form-urlencoded
I am using Postman and I am running the redirect_uri from the rails server.
Additional information:
Cache-Control →private
Connection →keep-alive
Content-Length →15
Content-Type →text/html; charset=UTF-8
Date →Mon, 21 Apr 2014 01:17:05 GMT
Expires →Thu, 19 Nov 1981 08:52:00 GMT
P3P →policyref="/w3c/p3p.xml", CP="ALL CURa ADMa DEVa OUR IND UNI COM NAV INT STA PRE"
Pragma →no-cache
Server →Apache
Status →400 Bad Request
X-Error →Missing consumer key.
X-Error-Code →138
X-Source →Pocket
In a search to find out which x-error code was giving me a 400, I found that it's giving me a missing consumer key. I'm at a loss since this is a brand new key and I am certainly using it correctly. Any thoughts?
You are sending the consumer_key and redirect_uri as url parameters ?consumer_key=xxxxxx..
The correct way is to send them as form-data parameters:
I'm stumped, my website was working fine and now on Firefox suddenly the redirects stopped working.
I've tested IE and Chrome and going to /login redirects me to /dashboard however on Firefox the page is blank (no output sent) and no errors are logged. So this is why I'm assuming it to be a browser related issue. It might be due to a firefox update, but not sure how to confirm that.
Here are the headers:
Request Headers
GET /login HTTP/1.1
Host: local.example.com
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0 FirePHP/0.7.4
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Cookie: __utma=34805930.947644602.1372214584.1380730296.1380733154.30; __utmz=34805930.1378700053.15.2.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); __utma=214248714.242656582.1377296111.1380047082.1380734348.30; __utmz=214248714.1377296111.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __qca=P0-705514134-1378344178153; __utmc=34805930; __utmb=34805930.15.10.1380733154; __utmb=214248714.5.10.1380734348; __utmc=214248714; PHPSESSID=lli8i30qkhvohfm9ufkbdvbki0
x-insight: activate
Connection: keep-alive
Response Headers
HTTP/1.1 302 Found
Date: Wed, 02 Oct 2013 17:30:58 GMT
Server: Apache/2.4.3 (Win32) OpenSSL/1.0.1c PHP/5.4.7
X-Powered-By: PHP/5.4.7
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Location: /dashboard
Content-Length: 0
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8
It all looks pretty standard to me, however FF stays stuck on /login Am I missing something?
This behaviour is both on my local windows host and my remote amazon Linux web-server. The body is empty...
How could I go about debugging this?
The Expires header field in the response is really off. Firefox probably does not bother to render stale responses.
Please check the system time in your server. It is possible it is an Amazon problem, but it is also possible that one of the server users set the system time.
You can look into setting up a Network Time Protocol (NTP) client to run regularly (with ntpd), if you don't have that yet.
I would fire up Fiddler to see what bits actually went over the wire. Among other information, Fiddler will show what content type is actually used during the HTTP request / response.
This could be related to the fact that there is no extension. Firefox could be having trouble determining if this is a document or folder. Try firebug and see what URL Firefox tries to request after the redirect.