I can use the wininet function HttpOpenRequest and specify the lpszVerb argument as "OPTIONS" and the lpszObjectName as "/" to perform a
OPTIONS / HTTP/1.1
request to a web server.
However, I'd prefer this syntax (to query the general servers options):
OPTIONS * HTTP/1.1
But when I specify the lpszObjectName argument as "*" the query will be this:
OPTIONS /* HTTP/1.1
Is there a way to make this function send the asterisk only, without prepending a slash?
Thanks
Related
I have a website where the React frontend resides in (let's say) app.mydomain.com and the api on api.mydomain.com.
The user submits a login request to the API and, upon logging in successfully, receives a nice cookie to be used later. The front-end talks only and directly to the API so the domain on the cookie is simply set to api.mydomain.com.
I use Axios to perform the requests with the withCredentials flag set to true, in order to receive the cookie.
The headers on the server to allow CORS are as follows:
Access-Control-Allow-Origin: http://app.mydomain.com
Access-Control-Allow-Methods: GET,POST,DELETE,PUT,OPTIONS
Access-Control-Allow-Headers: *
Access-Control-Allow-Credentials: true
In this situation this is the response from Firefox:
But, as soon as the Access-Control-Allow-Headers value is set more specifically, say to
Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, everything works.
Mozilla says they do not allow "wildcarding" the Origin value, not the Headers one, same in this page, where nothing is mentioned.
Why is Firefox behaving like this and why is it not mentioned anywhere that I can find?
In a preflight response that includes Access-Control-Allow-Credentials: true, an asterisk used as the value of header Access-Control-Allow-Headers is interpreted literally, not as a wildcard (i.e. "all headers allowed").
It's true that the main MDN page about CORS doesn't explicitly state this rule. However, the more specific MDN page about the Access-Control-Allow-Headers header does so explicitly:
The value "*" only counts as a special wildcard value for requests without credentials (requests without HTTP cookies or HTTP authentication information). In requests with credentials, it is treated as the literal header name "*" without special semantics. Note that the Authorization header can't be wildcarded and always needs to be listed explicitly.
Edit (2023): As sideshowbarker pointed out in his comments, the MDN Web Docs page about CORS has since been updated and now states the following:
When responding to a credentialed request: [...]
The server must not specify the "*" wildcard for the Access-Control-Allow-Headers response-header value, but must instead specify an explicit list of header names; for example, Access-Control-Allow-Headers: X-PINGOTHER, Content-Type
Here are clarifying quotes from the more authoritative Fetch standard:
For Access-Control-Expose-Headers, Access-Control-Allow-Methods, and Access-Control-Allow-Headers response headers, the value * counts as a wildcard for requests without credentials.
Access-Control-Expose-Headers, Access-Control-Allow-Methods, and Access-Control-Allow-Headers response headers can only use * as value when request’s credentials mode is not "include".
(my emphasis)
The relevant normative requirement for browsers that the spec states is in the main fetch algorithm at https://fetch.spec.whatwg.org/#main-fetch, in step 13, substep 2:
If request's credentials mode is not
"include" and headerNames contains *, then set response’s CORS-exposed header-name list to all unique header names in response’s header list.
In other words, the browser behavior that spec statement requires for * is:
if the credentials mode is not "include", the browser is required to examine the actual list of response headers in the response, and allow all of them
if the credentials mode is "include", the browser is required to only allow any response header with the literal name "*"
At the end of that step, there’s actually a note saying pretty much the same thing:
One of the headerNames can still be * at this point, but will only match a header whose name is *.
Currently, my software has the following workflow
User performs an search through a REST API and selects an item
Server performs the same search again to validate the user's selection
In order to implement step 2, the user has to send the URL params that he used for his search as a string (ex. age=10&gender=M).
The server will then http_get(url + "?" + params_str_submitted_by_user)
Can a malicious user make the server connect to an unintended server by manipulating params_str_submitted_by_user?
What is the worst case scenario if even newlines are left in and the user can arbitrarily manipulate the HTTP headers?
As you are appending params_str_submitted_by_user to the base URL after the ? delimiter, you are safe from this type of attack used where the context of the domain is changed to a username or password:
Say URL was http://example.com and params_str_submitted_by_user was #evil.com and you did not have the / or ? characters in your URL string concatenation.
This would make your URL http://example.com#evil.com which actually means username example.com at domain evil.com.
However, the username cannot contain the ? (nor slash) character, so you should be safe as you are forcing the username to be concatenated. In your case URL becomes:
http://example.com?#evil.com
or
http://example.com/?#evil.com
if you include the slash in your base URL (better practise). These are safe as all it does is pass your website evil.com as a query string value because #evil.com will no longer be interpretted as a domain by the parser.
What is the worst case scenario if even newlines are left in and the user can arbitrarily manipulate the HTTP headers?
This depends on how good your http_get function is at sanitizing values. If http_get does not strip newlines internally it could be possible for an attacker to control the headers sent from your application.
e.g. If http_get internally created the following request
GET <url> HTTP/1.1
Host: <url.domain>
so under legitimate use it would work like the following:
http_get("https://example.com/foo/bar")
generates
GET /foo/bar HTTP/1.1
Host: example.com
an attacker could set params_str_submitted_by_user to
<space>HTTP/1.1\r\nHost: example.org\r\nCookie: foo=bar\r\n\r\n
this would cause your code to call
http_get("https://example.com/" + "?" + "<space>HTTP/1.1\r\nHost: example.org\r\nCookie: foo=bar\r\n\r\n")
which would cause the request to be
GET / HTTP/1.1
Host: example.org
Cookie: foo=bar
HTTP/1.1
Host: example.com
Depending on how http_get parses the domain this might not cause the request to go to example.org instead of example.com - it is just manipulating the header (unless example.org was another site on the same IP address as your site). However, the attacker has managed to manipulate headers and add their own cookie value. The advantage to the attacker depends on what can be gained under your particular setup from them doing this - there is not necessarily any general advantage, it would be more of a logic flaw exploit if they could trick your code into behaving in an unexpected way by causing it to make requests under the control of the attacker.
What should you do?
To guard against the unexpected and unknown, either use a version of http_get that handles header injection properly. Many modern languages now deal with this situation internally.
Or - if http_get is your own implementation, make sure it sanitizes or rejects URLs that contain invalid characters like carriage returns or line feeds and other parameters that are invalid in a URL. See this question for list of valid characters.
In my application I'm making some javascript requests to my Api Controllers to get some html formatted strings. When those requests are made with Accept: */* HTTP header (jQuery $.get method), so by default JsonMediaTypeFormatter is used and the data is returned with Content-Type: application/json in JSON format.
What I would like is to handle */* requests as text/html. So I tried to create a custom MediaTypeFormatter that supports */* media type, but it gives me the following error
The 'MediaTypeHeaderValue' of */* cannot be used as a supported
media type because it is a media range.`
Alternatively I could always provide correct expected data types in my requests, but I'm curious if there's a way to handle */* media types.
The above behavior is due to the following:
The default con-neg algorithm in Web API has the following precedence order of choosing the formatter for response:
Formatter match based on Media Type Mapping.
Formatter match based on Request Accept header's media type.
Formatter match based on Request Content-Type header's media type.
Formatter match based on if it can serialize the response data’s Type.
Now, JsonMediaTypeFormatter comes with a built-in media type mapping called XmlHttpRequestHeaderMapping which inspects an incoming request and sees if the request has the header x-requested-with: XMLHttpRequest and also if there is no accept header or if the Accept header is only having */*.
Since your request is mostly probably looking like below, according to the precedence order JsonMediaTypeFormatter is chosen as the one writing the response:
GET /api/something
Accept: */*
x-requested-with: XMLHttpRequest
A solution for your issue would be is to explicitly ask for "text/html" as this is what you are expecting.
GET /api/something
Accept: text/html
x-requested-with: XMLHttpRequest
Couple of very old blog posts about Content negotiation that I wrote:
http://blogs.msdn.com/b/kiranchalla/archive/2012/02/25/content-negotiation-in-asp-net-mvc4-web-api-beta-part-1.aspx
http://blogs.msdn.com/b/kiranchalla/archive/2012/02/27/content-negotiation-in-asp-net-mvc4-web-api-beta-part-2.aspx
Great question.
You can't set */* to be a supported media type, but what you can do is set your formatter to be the first one. Web API will pick the first formatter in the formatter collection that can write out the type if there is no Accept header or if the Accept header is */*.
So you'd want to configure your Web API like this:
config.Formatters.Insert(0, new MyHtmlFormatter());
Example:
I want to create one AutoResponse rule that will map all calls to one host to another host, but preserve the urls. Examples
http://hostname1/foo.html -> http://hostname2/foo.html
and
http://hostname1/js/script.js -> http://hostname2/js/script.js
in one rule.
For now, I've accomplished this by creating aN AutoResponse rule for every URL my project calls, but I'm sure there must be a way to right one rule using the right wildcards. I looked at http://www.fiddler2.com/Fiddler2/help/AutoResponder.asp, but I couldn't see how to do it. The wild cards all seem to be around the matching and not the action.
Full context: I'm developing on a beta platform and Visual Studio is borked in such away that it is sending all the requests to http://localhost:24575 when my project is actually running on http://localhost:56832
This is how I configured Fiddler2 :
I want to redirect all requests from http://server-name/vendor-portal-html/ to http://localhost/vendor-portal-html/
My configuration is as follows:
REGEX:.*/vendor-portal-html/(.*) to http://127.0.0.1/vendor-portal-html/$1
Thanks to EricLaw for above comment.
To map from one host to another, don't use AutoResponder. Instead, click Tools > Hosts.
Alternatively, you can click Rules > Customize Rules, scroll to OnBeforeRequest and write a bit of code:
if (oSession.HostnameIs("localhost") && (oSession.port == 24575)) oSession.port = 56832;
Because this was way harder to find than it should have been to use Fiddler to redirect all requests for one to host to another host:
Use the AutoResponder tab to set a rule such that any request matching your old host will redirect to your new host with the path and query string appended.
Match with regex options ix to make it case-insensitive and ignore whitespace. Leave off the n option as it requires explicitly named capture groups.
Capture the path and query string of the request and append it to the redirect response using the variable $1, where the path+query is the first capture group. You can use capture groups $1-$n if your regex has more.
Fiddler will then issue an HTTP 307 redirect response.
Request: regex:^(?ix)http://old.host.com/(.*)$ #Match HTTP host
Response: *redir:http://new.host.com/$1
Request
GET http://old.host.com/path/to/file.html HTTP/1.1
Host: old.host.com
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Response
HTTP/1.1 307 AutoRedir
Content-Length: 0
Location: http://new.host.com/path/to/file.html
Cache-Control: max-age=0, must-revalidate
Mapping requests with Fiddler Autoresponder using Regular Expressions is possible.
This can be done with rexexp rules. However this doesn't seem to be documented anywhere.
If you add a rule and use regular expressions within parenthesis, these matches can be used in the desired mapping when using the placeholders ... $n
each number corresponds to the matched regexp in the rule.
Example of Rule: regex:http://server1/(\w*) -> http://server2/
This will result in the following mapping: http://server1/foo.html -> http://server2/foo.html
I have a rewrite rule which redirects to / if no Accept-Language header is present and someone attempts to visit ?lang=en. It works fine, except for the headers returned. Vary: Accept-Language is missing from the response.
RewriteCond %{HTTP:Accept-Language} ^$
RewriteCond %{QUERY_STRING} ^lang=en
RewriteRule ^$ http://www.example.com/? [R=301,L]
The Apache documentation specifies:
If a HTTP header is used in a condition this header is added to the Vary header of the response in case the condition evaluates to to true for the request. It is not added if the condition evaluates to false for the request.
The conditions are definitely matching and redirecting, so I don't understand why Apache isn't adding the language vary. One can see why this would be a real problem if a proxy were to cache that ?lang=en and always redirect to / regardless of the Accept-Language header sent.
After peeking into the seedy underbelly of Apache's request handling system, it turns out that the documentation is somewhat misleading...But before I get into the explanation, from what I can tell you're at the mercy of Apache on this one.
The Client Problem
First, the header name will not be added to the Vary response header if it is not sent by the client. This is due to how mod_rewrite constructs the value for that header internally.
It looks up the header by name using apr_table_get(), the request's header table, and the name that you provided:
const char *val = apr_table_get(ctx->r->headers_in, name);
If name is not a key in the table, this function will return NULL. This is a problem, because immediately after this is a check against val:
if (val) {
// Set the structure member ctx->vary_this
}
ctx->vary_this is used on a per-RewriteCond basis to accumulate header names that should be assembled into the final Vary header*. Since no assignment or appending will occur if there is no value, a referenced (but not sent) header will never appear in Vary. The documentation doesn't explicitly state this, so it may or may not have been what you expected.
*As an aside, the NV (no vary) flag and ignore-on-failure functionality is implemented by setting ctx->vary_this to NULL, preventing its addition to the response header.
However, it's possible that you sent Accept-Language, but it was blank. In this case, the empty string will pass the above check, and the header name will be added to Vary by mod_rewrite from what's described above. Keeping this in mind, I used the following request to diagnose what was going on:
User-Agent: Fiddler
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language:
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Host: 129.168.0.123
This doesn't work either, but why? mod_rewrite definitely sets the headers when the rule and condition match (ctx->vary is an aggregate of ctx->vary_this across all checked conditions):
if (ctx->vary) {
apr_table_merge(r->headers_out, "Vary", ctx->vary);
}
This can be verified with a log statement, and r->headers_out is the variable used when generating the response headers. Given something is definitely going wrong though, there must be trouble after the rules are executed.
The .htaccess Problem
Currently, you appear to be defining your rules in .htaccess, or a <Directory> section. This means that mod_rewrite is operating in Apache's fixup phase, and the mechanism it uses to actually perform rewrites here is very messy. Let's assume for a second there's no external redirection, since you had problem a even without it (and I'll get to the issue with the redirect later).
After you perform a rewrite, it's far too late in the request processing for the module to actually map to a file. What it does instead is assign itself as the request's "content" handler and when the request reaches that point, it performs a call to ap_internal_redirect(). This leads to the creation of a new request object, one that does not contain the headers_out table from the original.
Assuming that mod_rewrite causes no further redirects, the response is generated from the new request object, which will never have the appropriate (original) headers assigned to it. It is possible to get around this by working in a per-server context (in the main configuration or in a <VirtualHost>), but...
The Redirect Problem
Unfortunately, it turns out that it's largely irrelevant anyway, since even if we do use mod_rewrite in a server context, the path the response takes in the event of a redirect still causes the headers that the module set to be tossed out.
When the request is received by Apache, through a chain of function calls it makes its way to ap_process_request(). This in turn calls ap_process_request_internal(), where the bulk of the important request parsing steps occur (including the invocation of mod_rewrite). It returns an integer status code, which in the case of your redirect happens to be set to 301.
Most requests return OK (which has a value of 0), leading immediately to ap_finalize_request_protocol(). However, that's not the case here:
if (access_status == OK) {
ap_finalize_request_protocol(r);
}
else {
r->status = HTTP_OK;
ap_die(access_status, r);
}
ap_die() does some additional manipulation (like returning the response code back to 301), and in this particular case ends with a call to ap_send_error_response().
Luckily, this is finally root of the problem. Though it might seem like it, things are not "assbackwards", and this causes the destruction of the original headers. There's even a comment about it in the source:
if (!r->assbackwards) {
apr_table_t *tmp = r->headers_out;
/* For all HTTP/1.x responses for which we generate the message,
* we need to avoid inheriting the "normal status" header fields
* that may have been set by the request handler before the
* error or redirect, except for Location on external redirects.
*/
r->headers_out = r->err_headers_out;
r->err_headers_out = tmp;
apr_table_clear(r->err_headers_out);
if (ap_is_HTTP_REDIRECT(status) || (status == HTTP_CREATED)) {
if ((location != NULL) && *location) {
apr_table_setn(r->headers_out, "Location", location);
}
//...
}
//...
}
Take note that r->headers_out is replaced, and the original table is cleared. That table had all of the information that was expected to show up in the response, so now it is lost.
Conclusion
If you don't redirect and you define the rules in a per-server context, everything does seem to work correctly. However, this is not what you want. I can see a potential workaround, but I'm not sure if it would be acceptable, not to mention the need to recompile the server.
As for the Vary: Accept-Encoding, I can only assume it comes from a different module that behaves in a way that allows the header to sneak through. I'm also not sure why Gumbo didn't have an issue when trying it.
For reference, I was looking at the 2.2.14 and 2.2 trunk source code, and I was modifying and running Apache 2.2.15. There doesn't appear to be any significant differences between the versions in the related code sections.
You may want to try something like the following as a workaround:
<LocationMatch "^.*lang\=">
Header onsuccess merge Vary "Accept-Language"
</LocationMatch>
To specifically set the Vary: Accept-Language HTTP response header on the redirect response only (which is what's expected here), you would need to set an environment variable (eg. VARY_ACCEPT_LANGUAGE) as part of the redirect rule and use this to set the header conditionally with the Header directive.
You also need to use the always condition (as opposed to the default onsuccess) with the Header directive in order to set this on the 3xx response (ie. non-200 reponses).
For example:
# Redirect requests that have an empty Accept-Language header and "lang=en" is present
RewriteCond %{HTTP:Accept-Language} ^$
RewriteCond %{QUERY_STRING} ^lang=en
RewriteRule ^$ /? [E=VARY_ACCEPT_LANGUAGE:1,R=301,L]
# Set/Merge "Vary" header on Accept-Language redirect
Header always merge Vary "Accept-Language" env=VARY_ACCEPT_LANGUAGE
HOWEVER, the Vary header shouldn't only be set on the redirect response (when the Accept-Language header is empty), it needs to be set on all responses to requests for /?lang=en, regardless of what the Accept-Language HTTP request header is actually set to. So, relying on Apache to set this header using only the redirect would not be sufficient anyway (even if it did set the header on the response as initially expected).
In order to set the appropriate Vary header on all responses to requests for /?lang=en, including the redirect then do it like this:
# Set env var if "/?lang=en" is requested
RewriteCond %{QUERY_STRING} ^lang=en
RewriteRule ^$ - [E=VARY_ACCEPT_LANGUAGE:1]
# Redirect requests that have an empty Accept-Language header and "lang=en" is present
RewriteCond %{HTTP:Accept-Language} ^$
RewriteCond %{QUERY_STRING} ^lang=en
RewriteRule ^$ /? [R=301,L]
# Set/Merge "Vary" header on all responses from "/?lang=en"
Header always merge Vary "Accept-Language" env=VARY_ACCEPT_LANGUAGE
Note, however, that if you have additional internal rewrite directives that cause the rewrite engine to start over then the env var VARY_ACCEPT_LANGUAGE is renamed to REDIRECT_VARY_ACCEPT_LANGUAGE and the above Header directive will not be successful. You'll probably need an additional directive to handle this. For example:
Header always merge Vary "Accept-Language" env=REDIRECT_VARY_ACCEPT_LANGUAGE