URL being stored in SCRIPT_NAME on subsequent requests in IIRF? - url-rewriting

Question

I'm having an issue with IIRF (Ionics Isapi Rewrite Filter) on IIS6 (although in this case not sure that's relevant), and it appears to be working on initial requests, but on a standard refresh (i.e. F5 not CTRL + F5), most of the time it will just 404. Although it appears to be intermittment. My re-write rule itself seems correctly, and I've tested it on various RegEx testers, as well as the fact it does work fine on a clear-cache refresh. I'm not an expert, but it appears to relate to the fact that on the times it doesn't work, the URL of the page I'm trying to hit is being fed through in the SCRIPT_NAME HTTP variable, rather than coming via the URL, which in this case appears to be the path I want to re-write to (although like I say, it 404's so it doesn't appear to actually be going to this path in these cases). I'm sure you'll see quite quickly that I'm just essentially doing extensionless URL re-writing. I've tried adding rules to re-write on the SCRIPT_NAME but no such luck so far.
My config is:
RewriteLog iirf
RewriteLogLevel 5
RewriteEngine ON
IterationLimit 5
UrlDecoding OFF
# Rewrite all extensionless URLs to index.html
RewriteRule ^[^.]*$ /appname/index.html
See the log below - this is a case of it NOT working. I'm hitting /appname/task/5 but it appears to store that in the SCRIPT_NAME. Strangely the URL it appears to want to re-write is actually the URL I want it to rewrite to.. Again, this is only on subsequent requests. On the first request it almost always re-writes without issue and page loads fine.
Tue Jul 12 10:17:33 - 4432 - Cached: DLL_THREAD_DETACH
Tue Jul 12 10:17:33 - 4432 - Cached: DLL_THREAD_DETACH
Tue Jul 12 10:17:33 - 4432 - HttpFilterProc: SF_NOTIFY_URL_MAP
Tue Jul 12 10:17:33 - 4432 - HttpFilterProc: cfg= 0x01C8CC60
Tue Jul 12 10:17:33 - 4432 - HttpFilterProc: SF_NOTIFY_AUTH_COMPLETE
Tue Jul 12 10:17:33 - 4432 - DoRewrites
Tue Jul 12 10:17:33 - 4432 - GetServerVariable_AutoFree: getting 'QUERY_STRING'
Tue Jul 12 10:17:33 - 4432 - GetServerVariable_AutoFree: 1 bytes
Tue Jul 12 10:17:33 - 4432 - GetServerVariable_AutoFree: result ''
Tue Jul 12 10:17:33 - 4432 - GetHeader_AutoFree: getting 'method'
Tue Jul 12 10:17:33 - 4432 - GetHeader_AutoFree: 4 bytes ptr:0x000D93A8
Tue Jul 12 10:17:33 - 4432 - GetHeader_AutoFree: 'method' = 'GET'
Tue Jul 12 10:17:33 - 4432 - DoRewrites: Url: '/appname/index.html'
Tue Jul 12 10:17:33 - 4432 - EvaluateRules: depth=0
Tue Jul 12 10:17:33 - 4432 - GetServerVariable: getting 'SCRIPT_NAME'
Tue Jul 12 10:17:33 - 4432 - GetServerVariable: 16 bytes
Tue Jul 12 10:17:33 - 4432 - GetServerVariable: result '/appname/task/5'
Tue Jul 12 10:17:33 - 4432 - EvaluateRules: no RewriteBase
Tue Jul 12 10:17:33 - 4432 - EvaluateRules: Rule 1: pattern: ^[^.]*$ subject: /appname/index.html
Tue Jul 12 10:17:33 - 4432 - EvaluateRules: Rule 1: -1 (No match)
Tue Jul 12 10:17:33 - 4432 - EvaluateRules: returning 0
Tue Jul 12 10:17:33 - 4432 - DoRewrites: No Rewrite
Any help is much appreciated!
Thanks

Answer 1

I'm not sure what the reason for it being stored in the SCRIPT_NAME variable was, but I've written an extra rule to cater for it, which fixed it for me:
RewriteEngine ON
IterationLimit 5
UrlDecoding OFF
# Rewrite all extensionless URLs to index.html
RewriteRule ^[^.]*$ /appname/index.html
# Subsequent requests may store the URL in the SCRIPT_NAME variable for some reason
RewriteCond %{SCRIPT_NAME} ^[^.]*$ # If SCRIPT_NAME variable contains an extensionless URL
RewriteRule .*\S.* /appname/index.html [L] # Rewrite all URLs to index if RewriteCond met

Answer 2

I've had this issue for a while, and to compound it I have it running on an Integrated Windows Authentication secure website. I think the problem affects more than just the requested uri. My guess is somehow IIRF is caching the previous request's directive set, and using that.
For testing, firstly I ensured that the HTTP response headers forced the browser not to cache any content, so the browser should always request and receive updated content. This does work as expected.
I had the IIRF directives parse the request through various destination script files I could flip between. What I found, after updating and saving both the IIRF.ini directives and the php files, was that each request, essentially after a soft refresh (F5), would indeed receive updated content (IIS was parsing and serving the scripts), but the directives executed were the previous ones.
eg, if a request resulted in a legitimate 404 error via the directives, then the next URL request, whether to an existing script or not, would also produce the 404 error. Getting a new full request (generally the Ctrl-F5 hard refresh, or a first request) would actually execute the proper/current directives.
After some header analysis, it seems that the requests showing problems have unsynchronized server variables, on first run through the directives:
SCRIPT_NAME holds the current URL being requested
REQUEST_URI showed the previous URL requested (the only variable with the previous URL)
That meant that the RewriteRule directives would be making use of the previous request_uri, which didn't hold the correct url. That seems to explain the 'old' incorrect end result on subsequent requests. I added the following directives, which seems to have solved this, at least on a basic level:
RewriteCond %{REQUEST_URI}::%{SCRIPT_NAME} !^([^\?]*?)(\?.*)?::\1$
RewriteRule ^ %{SCRIPT_NAME} [R=301,L,QSA]
the first check makes sure the pre-querystring value of request_uri matches the script_name value
the 301 forces the browser to a hard refresh, which should then synchronize the URLs (and any querystring is also resent). It should be the first directive, with the rest following.
This doesn't work, however, if the same url is requested a 2nd time. The newly saved directives won't take until the hard refresh is sent. I haven't yet found a way to determine if the latest directives are being parsed on soft refresh of the same URL.
In context of my secure site, it also didn't seem to want to work with requests that didn't include the Authorization HTTP header. Each of the browsers does the login handshake process properly, but once authorized, if the browser doesn't send that header, it can cause cached 401 errors on requests that shouldn't produce it. So I determined that the same solution can apply to these requests - forcing the refresh as long as the browser has not sent the Authorization header:
RewriteCond %{REQUEST_URI}::%{SCRIPT_NAME} !^([^\?]*?)(\?.*)?::\1$ [OR]
RewriteCond %{HTTP_Authorization} !^.+$
RewriteRule ^ %{SCRIPT_NAME} [R=301,L,QSA]
I think this solution addresses the same-url soft refresh problem above, since it appears that the soft refresh doesn't send the Authorization header. Effectively, every request becomes a hard refresh, ensuring sync'd variables and Auth header for the secure content.
For false positives, in theory this solution may be insufficient if the same url is requested on a soft refresh after the directives have changed and the Authorization header is sent. Secure content will be most recent, but directives used will still be the previous version.
It may also be insufficient for any request being parsed with differing yet valid request_uri and script_name values (usually this is on second itartion though). One step may be to set an environment variable on first iteration match, and only do the sync check and redirect if it's the first iteration of the directives. But IIRF doesn't seem to support the [E] modifier.
In short, I think this describes the problem:
Via IIRF, for any valid first request iteration, ^%{REQUEST_URI}(\?.+)?$ should always match %{SCRIPT_NAME}, but an untrappable old-directives parsing would occur if the variables are identical on first iteration, unless there's a way to check the cached (currently used) directives against the saved directives.
At any rate, hopefully this is one step closer to determining a universal workaround for this seemingly cached previous-request-and-directives IIRF bug.