mod_rewrite condition and rule effects POST vars in a bad way - mod-rewrite

I have 2 domains mydomain.com and mydomain.org. The site lives at mydomain.org so I want any attempt to mydomain.com to resolve to mydomain.org.
The following mod_rewrite rule that works to a degree.
RewriteEngine on
RewriteCond %{HTTP_HOST} ^mydomain\.com$
RewriteRule ^ http://mydomain.org%{REQUEST_URI} [L,R=301]
After I implemented it and tested it I felt it was doing everything I needed until I submitted a form with method="post".
For some reason, this mod_rewrite trashes my _POST vars
I am working solely from mydomain.org (which is the TLD I want the site to resolve from and where I submitted the form from).
Does anyone know of an adjustment to my condition and rule to not lose the _POST vars?
I identified something interesting. I plugged-in the HTTP Live Headers add-on in Firefox. When I use the mod_rewrite I get an "HTTP/1.1 404 Not Found" and when I turn off mod_rewrite I get a "HTTP/1.1 200 OK". The same page and PHP code behind is used. Again, when I have the mod_rewrite directives turned off, the _POST data comes through. When I turn on the mod_rewrite directives, the _POST data does not come through.
MOD_REWRITE Turned Off:
http://dashausmuseum.org/subscribe.html
POST /subscribe.html HTTP/1.1
Host: dashausmuseum.org
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Referer: http://dashausmuseum.org/subscribe.html
Cookie: __utma=74430599.461726749.1312575846.1312897084.1312899646.5; __utmz=74430599.1312575846.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=1.480711209.1312899756.1312979975.1312981669.5; __utmz=1.1312981669.5.5.utmcsr=dashausmuseum.com|utmccn=(referral)|utmcmd=referral|utmcct=/directions.html; __utmb=1.9.10.1312981669; PHPSESSID=7f4a74d7fde56cf901aa85511410b7f6; __utmc=1
Content-Type: application/x-www-form-urlencoded
Content-Length: 60
email=abc&firstName=&lastName=&address=&phone=&submit=Submit
HTTP/1.1 200 OK
Date: Wed, 10 Aug 2011 13:18:31 GMT
Server: Apache/2.2.17 (Unix) mod_ssl/2.2.17 OpenSSL/0.9.8e-fips-rhel5 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 PHP/5.3.4
X-Powered-By: PHP/5.3.4
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 5420
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html
MOD_REWRITE Turned On:
http://dashausmuseum.org/subscribe.html
POST /subscribe.html HTTP/1.1
Host: dashausmuseum.org
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Referer: http://dashausmuseum.org/subscribe.html
Cookie: __utma=74430599.461726749.1312575846.1312897084.1312899646.5; __utmz=74430599.1312575846.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=1.480711209.1312899756.1312979975.1312981669.5; __utmz=1.1312981669.5.5.utmcsr=dashausmuseum.com|utmccn=(referral)|utmcmd=referral|utmcct=/directions.html; __utmb=1.10.10.1312981669; PHPSESSID=7f4a74d7fde56cf901aa85511410b7f6; __utmc=1
Content-Type: application/x-www-form-urlencoded
Content-Length: 60
email=xyz&firstName=&lastName=&address=&phone=&submit=Submit
HTTP/1.1 404 Not Found
Date: Wed, 10 Aug 2011 13:20:32 GMT
Server: Apache/2.2.17 (Unix) mod_ssl/2.2.17 OpenSSL/0.9.8e-fips-rhel5 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 PHP/5.3.4
X-Powered-By: PHP/5.3.4
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 5329
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html
Does anyone see anything in the HTTP Headers that gives a clue as to what is going on?

Something else should be wrong, rewrite rules can't destroy _POST vars.
How are you setting/calling them in your code?

There is some discussion about what should happen to a POST transaction in response to a 301 (or other) redirect. Here is one example where the author suggests it is "messy".
Is it possible your browser is converting the POST to a GET request? Is it also possible the behavior is different in different browsers?

Related

admin-ajax.php returning 404 status code sporadically

I have a new Wordpress site on a shared hosting environment that is not allowing me to save changes that I'm making in the theme customization panel. When I try to save, it pops up a window saying, "Looks like something’s gone wrong. Wait a couple seconds, and then try again."
I've used this panel to build this site up to this point. At first I was having sporadic failures to save. Then I would just wait a bit and it would save my changes. Now it's not saving any changes.
Looking at my network traffic, I see an obvious problem; admin-ajax.php is not found when I'm trying to save. Watching the network activity on that page, I can see that as I'm working, it's requesting this file (admin-ajax.php) over and over again. Some of the requests go through fine and some of them don't (either code 200 or 404). Those repeated accesses are apparently due to either an autosaving feature, a check to see that the user is logged in, or both. It also attempts to access that file on saving my working in the customization panel.
I've tried to compare the request and response headers as well as the parameters that are sent along with these requests to see if there is any difference that stands out and could lead me to a solution. So far, I don't see anything but you might.
Unsuccessful request:
Request headers:
Host: redacted.com
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://redacted.com/wp-admin/customize.php?return=%2Fwp-admin%2F
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Content-Length: 981
DNT: 1
Connection: keep-alive
Cookie: wordpress_3ae72dd3e5fcadd24ee59ff6b2db800f=redacted%7C1556173739%7CqfdnSqJucEmaB10OiEp18ejd33JajsNNqcoKByCFudSTg%7Cbc8d5c3384113c40964d2de489696aadf6c98c0f612e15dd2b13df8c3579818f; wordpress_test_cookie=WP+Cookie+check; wordpress_logged_in_3ae72dd3de5fcadd24ee59ff6b2db800f=redacted%7C1556173739%7CqfnSqJucEmaB10OiEp18ejd33JajsNNqcoKByCFuSTg%7C14064d03cf74c98cff037dcd73be1c8e40ec296c3b9d944755e4d0fdcacd7ddb08b; wp-settings-1=libraryContent%3Dbrowse; wp-settings-time-1=1556000940
Pragma: no-cache
Cache-Control: no-cache
----------
Response headers:
HTTP/1.1 404 Not Found
Server: nginx/1.14.2
Date: Tue, 23 Apr 2019 16:51:31 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control: no-cache, must-revalidate, max-age=0
Link: <http://redacted.com/wp-json/>; rel="https://api.w.org/"
Content-Encoding: gzip
Parameters:
wp_customize on
customize_theme twentyseventeen
nonce [redacted]
customize_changeset_uuid [redacted]
customize_autosaved on
customize_changeset_data {"nav_menu_item[-3955120765714645000]":{"value":{"object_id":49,"object":"page","menu_item_parent":0,"position":1,"type":"post_type","title":"Welcome","url":"http://mastermobilerepair.com/","target":"","attr_title":"","description":"","classes":"","xfn":"","status":"publish","original_title":"Welcome","nav_menu_term_id":2,"_invalid":false,"type_label":"Page"}},"nav_menu_item[-4393172170029273000]":{"value":false},"nav_menu_item[-6184586432959251000]":{"value":false},"nav_menu_item[-112932972895796220]":{"value":false}}
customize_changeset_autosave true
action customize_save
customize_preview_nonce [redacted]
And here's a successful request:
Request headers:
Host: redacted.com
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://redacted.com/wp-admin/customize.php?return=%2Fwp-admin%2F
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Content-Length: 230
DNT: 1
Connection: keep-alive
Cookie: wordpress_3ae72dd3e5fcadd24ede59ff6b2db800f=redacted%7C1d556173739%7CqfnSqJucEmaB10OiEp18ejd33JajsNNqcoKByCFuSTg%7Cbc8d5c3384113c40964dd2de489696aadf6c98c0f612e15dd2b13df8c3579818f; wordpress_test_cookie=WP+Cookie+check; wordpress_logged_in_3ae72dd3e5fcadd24ee59ff6b2db800f=redacted%7Cd1556173739%7CqfnSqJucEmaB10OiEp18ejd33JajsNNqcoKByCFuSTg%7C1406403cf74c98cff0d37dcd73be1c8e40ec296c3b9d944755e4d0fddcacd7b08b; wp-settings-1=libraryContent%3Dbrowse; wp-settings-time-1=1556000940
Pragma: no-cache
Cache-Control: no-cache
------------
Response headers:
HTTP/1.1 200 OK
Server: nginx/1.14.2
Date: Tue, 23 Apr 2019 16:52:24 GMT
Content-Type: application/json; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Robots-Tag: noindex
X-Content-Type-Options: nosniff
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control: no-cache, must-revalidate, max-age=0
X-Frame-Options: SAMEORIGIN
Referrer-Policy: strict-origin-when-cross-origin
Parameters:
data[check_changeset_lock] true
data[changeset_uuid] [redacted]
interval 60
_nonce [redacted]
action heartbeat
screen_id customize
has_focus true
wp_customize on
customize_preview_nonce [redacted]
I've also tried using other web browsers. I've tried Chrome (v. 73), Firefox (v. 66), and an older version of Safari. That didn't help.
I suspect that there's a Bayesian algorithm that's been triggered to recognize some of the parameters content and block it. I talked to my host and they said that mod_security was the problem and that they "whitelisted the rule ID manually from [their] end." I don't know what that means but I thought they would have made a change to my .htaccess file. I didn't see any changes when I looked today, so they may have been overwritten or otherwise I'm not looking in the right place.
My .htaccess contents are:
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress
I've tried adding
<IfModule mod_security.c>
SecFilterEngine Off
</IfModule>
to the top of the file (after # BEGIN WordPress), which also didn't help. So I'm lost. Any help is much appreciated.
This was resolved by my host. They whitelisted it for mod_security and that solved it.

Why cache doesn't work (cache control)

Actually we send a cache control and expire header (normally cache control override the expire header) but we got a 304 response. Why does the navigator make this validation? Normally the navigator should not create a 304 because there is a cache control and it should use the cache. (browser is firefox)
GET page
Response is:
304 Not Modified
Accept-Ranges: bytes
Age: 16059
Cache-Control: public, max-age=21600
Connection: keep-alive
Content-Encoding: gzip
Content-Language: fr
Content-Type: text/html; charset=utf-8
Date: Fri, 22 Apr 2016 08:30:37 GMT
Etag: "1461297779-11"
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Last-Modified: Fri, 22 Apr 2016 04:02:59 GMT
Server: Apache
Vary: Cookie,Accept-Encoding
Via: 1.1 varnish
X-BeResp-ttl: 21600.000
X-Country-Code: YT
X-Drupal-Cache: MISS
X-Forwarded-Region: 0
X-Varnish: 386264387 386223490
X-Varnish-Cache: HIT
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
Connection: keep-alive
Host: www.domain.com
If-None-Match: "1461297779-11"
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0

Chrome not setting cookie on AJAX POST

I do an AJAX POST to my webservice, and it sets 2 cookies in the response, but Chrome does not set them. Safari and Firefox do, however.
Here's the request:
POST /api/login HTTP/1.1
Host: 0.0.0.0:8080
Connection: keep-alive
Content-Length: 50
accept: application/json
Origin: http://0.0.0.0:8080
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36
content-type: application/json
Referer: http://0.0.0.0:8080/form
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8,nl;q=0.6,de;q=0.4,fr;q=0.2,pl;q=0.2
Response:
HTTP/1.1 200 OK
X-Powered-By: Express
Vary: X-HTTP-Method-Override, Accept-Encoding
x-frame-options: sameorigin
set-cookie: keystone.uid=s%3A55efe88923f753865f7a0985%3Ac5aT64aih9lxXi%2BNiSMr1rUJW4kzWyyNUforvUOrckk.JovuV%2FqeoQ32PiuyNPAZ7JcbIxXBcBvj%2FWFp8vf3SQQ; Path=/; HttpOnly
set-cookie: keystone.sid=s%3ADcv5el-TjLRkOSH9vNbvxQoOai-SQj-3.ZTfPFwEZp5mdVHSDZTukO%2FnrDnSpGU3OMW3tQu%2FSz7U; Path=/; HttpOnly
Content-Type: application/json; charset=utf-8
Content-Length: 224
ETag: W/"e0-B6OeRPdDEP0WPVdlZHqarA"
Date: Fri, 06 Nov 2015 14:39:37 GMT
Connection: keep-alive
I'm out of ideas. This doesn't work with a fully qualified domain name on port 80 either.
Found the solution:
You have send the request with credentials (XMLHttpRequest.withCredentials or e.g. credentials: 'include' for whatwg fetch).
Even though this is pointless since you're logging in and don't have any/have invalid cookies, it makes Chrome store the cookies from the returned answer. ¯\_(ツ)_/¯
As pointed out by #Anne, the XMLHTTPRequest specification actually requires user agents to disregard returned cookies unless withCredentials is specified. http://www.w3.org/TR/XMLHttpRequest/#the-withcredentials-attribute

Firefox/Mozilla personas errors -- URL change?

Has the location for Firefox/Mozilla personas changed? Looking at the error console I see:
GET https://addons.cdn.mozilla.net/en-US/firefox/_files/245568/firefoxtom.jpg [HTTP/1.1 404 Not Found 299ms]
Redownloading the persona gives:
GET https://addons.cdn.mozilla.net/user-media/addons/245568/firefoxtom.jpg [HTTP/1.1 200 OK 756ms]
For all my current personas, I only see the background color, not the image. Has something changed recently? Is there a workaround? FF 31. Thanks!
Hearing nothing, I'll assume it's a web server misconfiguration on Mozilla's part and that no one cares. Just for posterity, here is the request/response that I tried to post in comments above, but that didn't look very good.
https://addons.cdn.mozilla.net/_files/245568/firefoxtom.jpg?1288084876
Request Method: GET
Status Code: HTTP/1.1 301 Moved Permanently
Request Headers 09:25:39.000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:32.0) Gecko/20100101 Firefox/32.0
Host: addons.cdn.mozilla.net
Connection: keep-alive
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Accept: image/png,image/*;q=0.8,*/*;q=0.5
Response Headers Δ305ms
X-Frame-Options: DENY
X-Backend-Server: web7
Vary: Accept-Language, User-Agent, X-Mobile
Strict-Transport-Security: max-age=31536000
Server: nginx
Location: https://addons.cdn.mozilla.net/en-US/firefox/_files/245568/firefoxtom.jpg?1288084876
Date: Sun, 07 Sep 2014 15:25:41 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 0
Cache-Control: max-age=31536000

Firefox sending two get request for a website

Clicking a link results in two calls for the page to the server. I install livehttp and inspected the header but can't figure out why it's sending the second request.
http://example.com/schedule?delete=290376
GET /schedule?delete=290376 HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20110207 Firefox/3.6.13
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://example.com/schedule
Cookie: Code=XXX; CodeHash=XXXXX
HTTP/1.1 200 OK
Date: Tue, 01 Mar 2011 22:09:51 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Set-Cookie: Code=XXXX; expires=Wed, 02-Mar-2011 00:09:52 GMT; path=/
Set-Cookie: CodeHash=XXXX; expires=Wed, 02-Mar-2011 00:09:52 GMT; path=/
Keep-Alive: timeout=2, max=200
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html
----------------------------------------------------------
http://example.com/schedule?delete=290376
GET /schedule?delete=290376 HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20110207 Firefox/3.6.13
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://example.com/schedule
Cookie: Code=XXXX; CodeHash=XXXXX
HTTP/1.1 302 Moved Temporarily
Date: Tue, 01 Mar 2011 22:09:55 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Set-Cookie: Code=XXX; expires=Wed, 02-Mar-2011 00:09:55 GMT; path=/
Set-Cookie: CodeHash=XXX; expires=Wed, 02-Mar-2011 00:09:55 GMT; path=/
Location: http://example.org/schedule?errors=5
Keep-Alive: timeout=2, max=200
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html
----------------------------------------------------------
In case you didn't find your solution:
I have stumbled on that same issue, and it seems to be related to the page encoding. If FireFox downloads a page containing invalid characters (for example, utf-8 chars inside a page for which the Content-type header is something else), then it will download the page a second time and parse it in the encoding it has tried to guess from the invalid chars it detected in the first page.
So make sure your page either returns the correct Content-type header, or contains a meta http-equiv header with the correct encoding.
You don't happen to be using firefox, have the web developer toolbar and also have the display page validation on do you?
I am guessing in the dark here as to your enviro but my team and I have been able to demonstrate that having that tool bar installed in firefox and having page validation set to display actually duplicates the POSTs and GETs as it sends that same page data to the validation service.

Resources