I want to remove some headers that my webserver injects into the header response but provides no ability to remove
I only see helper functions to add headers, but no way to remove them in Grape and I cant seem to find the variable where headers are contained
I specifically want to strip out X-Powered-By which gets injected by my Passenger server, passenger only seems to provide the ability to remove the version number but not the ability to remove it completely... which seems insecure...
i'd like to do this in my before block
before do
## I see i can easily add headers, but not remove
header 'X-Robots-Tag', 'noindex'
## how do I remove headers here?
## ...
end
## ... some routes
get '/' do
## ...
end
Many libraries provide the ability to remove headers, for instance:
ExpressJS
app.use(function (req, res, next) {
res.header('Pragma', 'no-cache');
res.removeHeader('Pragma');
next();
});
Rails
response.headers['Connection'] = 'Closed'
remove_keys = %w(X-Runtime Cache-Control Server Etag Set-Cookie)
response.headers.delete_if{|key| remove_keys.include? key}
While a bit awkward in my opinion the Docs do show you can delete a header by just passing the key and not the value (or passing the value as nil)
For your Example:
before do
## I see i can easily add headers, but not remove
header 'X-Robots-Tag', 'noindex'
## how do I remove headers here?
header 'X-Powered-By'
end
I guess the concept is that a header set to a non value is not really a header at all.
Related
I would like to be able to add some logic to my tritium project based on the incoming request header. Is it possible to access the header information and then perform match() with() logic?
My plan is to take an existing URL (that can be accessed via a normal GET request) and give it a second mode of functionality so that it can be turned into an AJAX API. When the JavaScript makes the API request, I could set a custom header flag so that the platform knows to interpret the request differently.
You should be able to access headers in the incoming HTTP request using the global variable syntax. For example, to access the site's hostname:
$host
# => yourwebsite.com
I believe that most of the standard headers are accessible as global variables in Tritium. However, I'm not sure if all headers are accessible as global vars.
Inside your project folder, on your development machine, there should be a tmp folder that contains the HTTP request/response bundles. Each bundle should be time stamped with the request's date and time. I think if you peek inside one of these folders, you should see a bunch of files:
incoming_request
incoming_response
outgoing_request
outgoing_response
And possibly a fifth file. I can't remember if this is still the case in the current version of the platform, but there's a chance you'll find a fifth file containing the global variables that the Tritium server creates to store HTTP request header values. So you can peek inside that file (if it exists) and find out what variable name your HTTP headers are using.
Hope that helps!
I'm late on this one, but I figured I would lend a hand to anyone else who needs help on this one.
you need to create two files in your scripts directory, one called
request_main.ts
and
response_main.ts
You can then use things such as the parse_headers function, which iterates through the request/ response headers, depending on the file which you put the code in.
parse_headers() { # iterate over all the incoming/outgoing headers
log(name()) # log the name of the current cookie in the iteration
log(value()) # log the value of the current cookie in the iteration
}
parse_headers(/Set-Cookie/) { # iterate over the Set-Cookie headers only.
log(this())
}
This will log all of your header names, to make modifications, you can then use "setter" functions, which you can read about here:
http://developer.moovweb.com/docs/local/configuration/headers
Good luck.
My application uses a bookmarklet, and I need to allow CORS for MyRouteR so my bookmarklet code can use this route for AJAX requests.
In my first draft of config/routes I gave MyRouteR support for only one request method, PUT. But it turned out (duh) that I'd need to support the OPTIONS method as well, which browsers use for CORS preflight requests.
I ended up with the following in config/routes:
/myroute MyRouteR PUT OPTIONS
I was kind of hoping there would be some relevant machinery in the Template Haskell that processes config/routes so that the addition of OPTIONS to this route's method list would automagically result in CORS support, but no dice. Not the end of the world, but it would have made sense and felt elegant that way.
To make CORS work, I gave the route an OPTIONS handler:
optionsMyRouteR :: Handler RepPlain
optionsMyRouteR = do
addHeader "Access-Control-Allow-Origin" "*"
addHeader "Access-Control-Allow-Methods" "PUT, OPTIONS"
return $ RepPlain $ toContent ("" :: Text)
putMyRouteR :: Handler RepJson
putMyRouteR = do
addHeader "Access-Control-Allow-Origin" "*"
-- more stuff ...
This works, but it feels slightly un-Yesodic because it's so boilerplate. So, two questions:
Do we have a better adjective than Yesodic?
Is there another, better way to let a route support cross-origin requests?
UPDATE:
Someone else published some generic middleware for this: http://hackage.haskell.org/package/wai-cors.
I am currently working on the same thing and haven't yet implemented a solution, however I imagine it can be done via a WAI Middleware similar to the sample code on the wiki page Allowing WOFF fonts to be accessed from other domains (CORS). This should allow you from writing the CORS code once without repeating yourself.
Sample code from the link above to add cross-origin access for WOFF fonts:
addCORStoWOFF :: W.Middleware
addCORStoWOFF app = fmap updateHeaders . app
where
updateHeaders (W.ResponseFile status headers fp mpart) = W.ResponseFile status (new headers) fp mpart
updateHeaders (W.ResponseBuilder status headers builder) = W.ResponseBuilder status (new headers) builder
updateHeaders (W.ResponseSource status headers src) = W.ResponseSource status (new headers) src
new headers | woff = cors : headers
| otherwise = headers
where woff = lookup HT.hContentType headers == Just "application/font-woff"
cors = ("Access-Control-Allow-Origin", "*")
I'm using Goliath and Grape. On my goliath server it calls the grape api like so:
when '/posts' then FrameworksAPI::API.call(env)
On my grape api class, my method is as simple as this:
get '/:id' do
Post.find(params[:id])
end
I'd like to modify the headers - specifically the 'Content-Length' but unsure how to.
Also i'd like to ask an additional question. How do i create callback/filters specifically before the method GET returns the result i'd like to modify the result.
Grape has a header helper for a few versions now.
header 'Content-Length`, 42.to_s
For your second question on modifying the body, try using after do ... at API level.
The return from your FrameworksAPI::API.call(env) method will be a triplet [status_code, headers, body]. So, instead of just returning that from your case you'd do something like:
when '/posts' then
status, headers, body = FrameworksAPI::API.call(env)
headers['whatever'] = blah
[status, headers, body]
You can also change the body, just be careful as the body maybe an array.
There is also a Content-Length middleware that is provided by Goliath. Content-Length is loaded by default although if you set a custom Content-Length it will take precedence. Just be carefull that other middlewares like the formattings don't change the body after you set your content-length.
I have a rewrite rule which redirects to / if no Accept-Language header is present and someone attempts to visit ?lang=en. It works fine, except for the headers returned. Vary: Accept-Language is missing from the response.
RewriteCond %{HTTP:Accept-Language} ^$
RewriteCond %{QUERY_STRING} ^lang=en
RewriteRule ^$ http://www.example.com/? [R=301,L]
The Apache documentation specifies:
If a HTTP header is used in a condition this header is added to the Vary header of the response in case the condition evaluates to to true for the request. It is not added if the condition evaluates to false for the request.
The conditions are definitely matching and redirecting, so I don't understand why Apache isn't adding the language vary. One can see why this would be a real problem if a proxy were to cache that ?lang=en and always redirect to / regardless of the Accept-Language header sent.
After peeking into the seedy underbelly of Apache's request handling system, it turns out that the documentation is somewhat misleading...But before I get into the explanation, from what I can tell you're at the mercy of Apache on this one.
The Client Problem
First, the header name will not be added to the Vary response header if it is not sent by the client. This is due to how mod_rewrite constructs the value for that header internally.
It looks up the header by name using apr_table_get(), the request's header table, and the name that you provided:
const char *val = apr_table_get(ctx->r->headers_in, name);
If name is not a key in the table, this function will return NULL. This is a problem, because immediately after this is a check against val:
if (val) {
// Set the structure member ctx->vary_this
}
ctx->vary_this is used on a per-RewriteCond basis to accumulate header names that should be assembled into the final Vary header*. Since no assignment or appending will occur if there is no value, a referenced (but not sent) header will never appear in Vary. The documentation doesn't explicitly state this, so it may or may not have been what you expected.
*As an aside, the NV (no vary) flag and ignore-on-failure functionality is implemented by setting ctx->vary_this to NULL, preventing its addition to the response header.
However, it's possible that you sent Accept-Language, but it was blank. In this case, the empty string will pass the above check, and the header name will be added to Vary by mod_rewrite from what's described above. Keeping this in mind, I used the following request to diagnose what was going on:
User-Agent: Fiddler
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language:
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Host: 129.168.0.123
This doesn't work either, but why? mod_rewrite definitely sets the headers when the rule and condition match (ctx->vary is an aggregate of ctx->vary_this across all checked conditions):
if (ctx->vary) {
apr_table_merge(r->headers_out, "Vary", ctx->vary);
}
This can be verified with a log statement, and r->headers_out is the variable used when generating the response headers. Given something is definitely going wrong though, there must be trouble after the rules are executed.
The .htaccess Problem
Currently, you appear to be defining your rules in .htaccess, or a <Directory> section. This means that mod_rewrite is operating in Apache's fixup phase, and the mechanism it uses to actually perform rewrites here is very messy. Let's assume for a second there's no external redirection, since you had problem a even without it (and I'll get to the issue with the redirect later).
After you perform a rewrite, it's far too late in the request processing for the module to actually map to a file. What it does instead is assign itself as the request's "content" handler and when the request reaches that point, it performs a call to ap_internal_redirect(). This leads to the creation of a new request object, one that does not contain the headers_out table from the original.
Assuming that mod_rewrite causes no further redirects, the response is generated from the new request object, which will never have the appropriate (original) headers assigned to it. It is possible to get around this by working in a per-server context (in the main configuration or in a <VirtualHost>), but...
The Redirect Problem
Unfortunately, it turns out that it's largely irrelevant anyway, since even if we do use mod_rewrite in a server context, the path the response takes in the event of a redirect still causes the headers that the module set to be tossed out.
When the request is received by Apache, through a chain of function calls it makes its way to ap_process_request(). This in turn calls ap_process_request_internal(), where the bulk of the important request parsing steps occur (including the invocation of mod_rewrite). It returns an integer status code, which in the case of your redirect happens to be set to 301.
Most requests return OK (which has a value of 0), leading immediately to ap_finalize_request_protocol(). However, that's not the case here:
if (access_status == OK) {
ap_finalize_request_protocol(r);
}
else {
r->status = HTTP_OK;
ap_die(access_status, r);
}
ap_die() does some additional manipulation (like returning the response code back to 301), and in this particular case ends with a call to ap_send_error_response().
Luckily, this is finally root of the problem. Though it might seem like it, things are not "assbackwards", and this causes the destruction of the original headers. There's even a comment about it in the source:
if (!r->assbackwards) {
apr_table_t *tmp = r->headers_out;
/* For all HTTP/1.x responses for which we generate the message,
* we need to avoid inheriting the "normal status" header fields
* that may have been set by the request handler before the
* error or redirect, except for Location on external redirects.
*/
r->headers_out = r->err_headers_out;
r->err_headers_out = tmp;
apr_table_clear(r->err_headers_out);
if (ap_is_HTTP_REDIRECT(status) || (status == HTTP_CREATED)) {
if ((location != NULL) && *location) {
apr_table_setn(r->headers_out, "Location", location);
}
//...
}
//...
}
Take note that r->headers_out is replaced, and the original table is cleared. That table had all of the information that was expected to show up in the response, so now it is lost.
Conclusion
If you don't redirect and you define the rules in a per-server context, everything does seem to work correctly. However, this is not what you want. I can see a potential workaround, but I'm not sure if it would be acceptable, not to mention the need to recompile the server.
As for the Vary: Accept-Encoding, I can only assume it comes from a different module that behaves in a way that allows the header to sneak through. I'm also not sure why Gumbo didn't have an issue when trying it.
For reference, I was looking at the 2.2.14 and 2.2 trunk source code, and I was modifying and running Apache 2.2.15. There doesn't appear to be any significant differences between the versions in the related code sections.
You may want to try something like the following as a workaround:
<LocationMatch "^.*lang\=">
Header onsuccess merge Vary "Accept-Language"
</LocationMatch>
To specifically set the Vary: Accept-Language HTTP response header on the redirect response only (which is what's expected here), you would need to set an environment variable (eg. VARY_ACCEPT_LANGUAGE) as part of the redirect rule and use this to set the header conditionally with the Header directive.
You also need to use the always condition (as opposed to the default onsuccess) with the Header directive in order to set this on the 3xx response (ie. non-200 reponses).
For example:
# Redirect requests that have an empty Accept-Language header and "lang=en" is present
RewriteCond %{HTTP:Accept-Language} ^$
RewriteCond %{QUERY_STRING} ^lang=en
RewriteRule ^$ /? [E=VARY_ACCEPT_LANGUAGE:1,R=301,L]
# Set/Merge "Vary" header on Accept-Language redirect
Header always merge Vary "Accept-Language" env=VARY_ACCEPT_LANGUAGE
HOWEVER, the Vary header shouldn't only be set on the redirect response (when the Accept-Language header is empty), it needs to be set on all responses to requests for /?lang=en, regardless of what the Accept-Language HTTP request header is actually set to. So, relying on Apache to set this header using only the redirect would not be sufficient anyway (even if it did set the header on the response as initially expected).
In order to set the appropriate Vary header on all responses to requests for /?lang=en, including the redirect then do it like this:
# Set env var if "/?lang=en" is requested
RewriteCond %{QUERY_STRING} ^lang=en
RewriteRule ^$ - [E=VARY_ACCEPT_LANGUAGE:1]
# Redirect requests that have an empty Accept-Language header and "lang=en" is present
RewriteCond %{HTTP:Accept-Language} ^$
RewriteCond %{QUERY_STRING} ^lang=en
RewriteRule ^$ /? [R=301,L]
# Set/Merge "Vary" header on all responses from "/?lang=en"
Header always merge Vary "Accept-Language" env=VARY_ACCEPT_LANGUAGE
Note, however, that if you have additional internal rewrite directives that cause the rewrite engine to start over then the env var VARY_ACCEPT_LANGUAGE is renamed to REDIRECT_VARY_ACCEPT_LANGUAGE and the above Header directive will not be successful. You'll probably need an additional directive to handle this. For example:
Header always merge Vary "Accept-Language" env=REDIRECT_VARY_ACCEPT_LANGUAGE
I would like to store the cookies from one open-uri call and pass them to the next one. I can't seem to find the right docs for doing this. I'd appreciate it if you could tell me the right way to do this.
NOTES: w3.org is not the actual url, but it's shorter; pretend cookies matter here.
h1 = open("http://www.w3.org/")
h2 = open("http://www.w3.org/People/Berners-Lee/", "Cookie" => h1.FixThisSpot)
Update after 2 nays: While this wasn't intended as rhetorical question I guarantee that it's possible.
Update after tumbleweeds: See (the answer), it's possible. Took me a good while, but it works.
I thought someone would just know, but I guess it's not commonly done with open-uri.
Here's the ugly version that neither checks for privacy, expiration, the correct domain, nor the correct path:
h1 = open("http://www.w3.org/")
h2 = open("http://www.w3.org/People/Berners-Lee/",
"Cookie" => h1.meta['set-cookie'].split('; ',2)[0])
Yes, it works. No it's not pretty, nor fully compliant with recommendations, nor does it handle multiple cookies (as is).
Clearly, HTTP is a very straight-forward protocol, and open-uri lets you at most of it. I guess what I really needed to know was how to get the cookie from the h1 request so that it could be passed to the h2 request (that part I already knew and showed). The surprising thing here is how many people basically felt like answering by telling me not to use open-uri, and only one of those showed how to get a cookie set in one request passed to the next request.
You need to add a "Cookie" header.
I'm not sure if open-uri can do this or not, but it can be done using Net::HTTP.
# Create a new connection object.
conn = Net::HTTP.new(site, port)
# Get the response when we login, to set the cookie.
# body is the encoded arguments to log in.
resp, data = conn.post(login_path, body, {})
cookie = resp.response['set-cookie']
# Headers need to be in a hash.
headers = { "Cookie" => cookie }
# On a get, we don't need a body.
resp, data = conn.get(path, headers)
Thanks Matthew Schinckel your answer was really useful. Using Net::HTTP I was successful
# Create a new connection object.
site = "google.com"
port = 80
conn = Net::HTTP.new(site, port)
# Get the response when we login, to set the cookie.
# body is the encoded arguments to log in.
resp, data = conn.post(login_path, body, {})
cookie = resp.response['set-cookie']
# Headers need to be in a hash.
headers = { "Cookie" => cookie }
# On a get, we don't need a body.
resp, data = conn.get(path, headers)
puts resp.body
Depending on what you are trying to accomplish, check out webrat. I know it is usually used for testing, but it can also hit live sites, and it does a lot of the stuff that your web browser would do for you, like store cookies between requests and follow redirects.
you would have to roll your own cookie support by parsing the meta headers when reading and adding a cookie header when submitting a request if you are using open-uri. Consider using httpclient http://raa.ruby-lang.org/project/httpclient/ or something like mechanize instead http://mechanize.rubyforge.org/ as they have cookie support built in.
There is a RFC 2109 and RFC 2965 cookie jar implementation to be found here for does that want standard compliant cookie handling.
https://github.com/dwaite/cookiejar