Team test: Failing load. Request failed: The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF - visual-studio

The folk in the QA department use visual studio team test (2008 IIRC) to run load tests against our web application.
The latest set of tests have failed on several pages. The error reported is
Request failed: The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF
Searching for this using google yields quite a few results. it would appear that this error message is generated from the .Net framework WebRequest class (i.e. it is not a visual studio specific message). The most useful result is this one, which details my exact problem and how to suppress the error.
But of course, I want to get to the bottom of why this error occurs in the first place. Here are some more facts: -
This error never used to occur when the tests were run against an older version of the web app. The web app. host OS and web server (Win 2003 and IIS 6) are identical in both cases.
Not all the pages generate this error - only some.
The only significant change to these pages (that I can think of) is that they now use some AJAX whereas before they did not (IIRC)
In order to narrow down the problem, I created the simplest page that I could to replicate the problem. Luckily, that was not too hard. I then inspected the bytes in the header using Fiddler but I could not find an occurrence of a CR (0x0D) that was not followed by a LF (0x0A).
The raw HTTP response (as stored from Fiddler by response saving bytes - so its encoding should not have been altered during the save) is here as text if you don't believe me!
So now I am left thinking that the supposed error might be a false alarm. Does anyone else have experience of this/can help shed light?

This is definitely not a false alarm - I've been getting this error in my app a lot while trying to communicate with Facebook API.
I've just stumbled upon this response from Steven Cheng - http://www.velocityreviews.com/forums/t302174-why-do-i-get-the-server-committed-a-protocol-violation.html - and let me quote him:
From your description, you're using
the HttpWebRequest component to send
some http request to some external web
resource in your ASP.NET web
application. However, you're always
getting the "The server committed a
protocol violation.
Section=ResponseStatusLine" error
unless you set the following section
in the web.config file:
<system.net>
<settings>
<httpWebRequest useUnsafeHeaderParsing="true" />
</settings>
</system.net>
And you're wondering the cause of this behavior, correct?
As for this issue, I've performed some
research on this and found that the
problem is actually caused by the
critical http header
parsing/validating of the
HttpWebRequest component. According to
the Http Specification(http1.1), the
HTTP header keys shoud specifically
not include any spaces in their names.
However, some web servers do not fully
respect standards they're meant to.
Applications running on the Dotnet
framework and making heavy use of http
requests usually use the
httpWebRequest class, which
encapsulates everything a web oriented
developer could dream of. With all the
recently issues related to security,
the "httpWebRequest" class provides a
self protection mechanism preventing
it to accept HTTP answers which not
fully qualify to the specifications.
The common case is having a space in
the "content-length" header key. The
server actually returns a "content
length" key, which, assuming no spaces
are allowed, is considered as an
attack vector (HTTP response split
attack), thus, triggering a "HTTP
protocol violation error" exception.
Will try if this helps right now and post results later

Related

Version 4.0 localhost testing getting 3228 : The ThreeDSNotificationURL field format is invalid

I am currently testing the changes for version 4.0 of the protocol for PSD2 using Direct Integration.
I am running under Visual Studio using a localhost website address.
When calling the SagePay payment endpoint with ThreeDSNotificationURL set as a localhost address (http://localhost:15536/Payments/ThreeDResponse) I receive the following error '3228 : The ThreeDSNotificationURL field format is invalid.'
If I change this field to a fully qualified domain (http://www.google.com) I no longer receive the error, but can't complete my testing.
Using localhost for the termurl in version 3.0 of the protocol works as expected.
I was attempting to work locally like your self and receiving the same issue. After speaking with support they confirmed that they will not accept "localhost". Also, the documentation suggests that HTTPS is a requirement, so this might also be a blocking factor.
I think someone suggested using ngrok as a means of tunneling external requests into your localhost, which is a good method to continue development locally whilst also being visible externally to services like SagePay.
Once I got passed the above issue, I got several more errors for other missing required fields as listed here; https://www.sagepay.co.uk/support/38/psd2-under-direct-integration (note that if BrowserJavascriptEnabled is true all conditional fields are then required)
Did you URL encode the ThreeDSNotificationURL in your post?
I send it like that and it is ok:
sb.Append(HttpUtility.UrlEncode("https://www.clientdomian.com/ac/ThreeDSNotificationURL.aspx"));
I run the site on my local IIS for development.
I recently had this issue and I wanted to document it here, because searching for this issue gives very little in terms of results.
I was getting this error code when i switched my Opayo/SagePay extension (MageNest SagePay for Magento) to 3ds2.
As it turns out, the full URL wasn't being sent. It was trying to send sagepay/direct/postBack?form_key=HZuYxgiEq9w2CNFB and NOT https://www.example.com/sagepay/direct/postBack?form_key=HZuYxgiEq9w2CNFB. It's partly my fault because there was a domain field in the config which was empty (it's not like the domain can't be retrieved automatically, huh) and partly the vendors fault because it was very badly documented.
So while this is a different problem for a different platform, I hope this helps someone.

413 request entity too large jetty server

I am trying to make a POST request to an endpoint served using jetty server. The request errors out saying 413 Request entity too large. But the content-length is only 70KB which I see is way below the default limit of 200KB.
I have tried serving via ngnix server and add client_max_body_size to desired level but that didn't work. I have set the setMaxFormContentSize of WebContext and that didn't help either. I have followed https://wiki.eclipse.org/Jetty/Howto/Configure_Form_Size and that didn't helped me either.
Does anyone have any solution to offer?
wiki.eclipse.org is OLD and is only for Jetty 7 and Jetty 8 (long ago EOL/End of Life). The giant red box at the top of the page that you linked it even tell you this, and gives you a link to the up to date documentation.
If you see a "413 Request entity too large" from Jetty, then it refers the the Request URI and Request Headers.
Note: some 3rd party libraries outside of Jetty's control can also use HttpServletResponse.sendError(413) which would result in the same response status message as you reported.
Judging by your screenshot, which does not include all of the details, (it's really better to copy/paste the text when making questions on stackoverflow, screenshots often hide details that are critical in getting a direct answer), your Cookie header is massive and is causing the 413 error by pushing the Request Headers over 8k in size.

IIS : Exception from HRESULT: 0xC00CEF03

I have my website hosted on IIS with window 2016 OS. When I am going to disable the Form Authentication under IIS, it is giving me the following error.
Form Authentication.
Exception from HRESULT: 0xC00CEF03
i had a similar issue but it occurred when i was trying to disable the 'Machine key' - 'automatically generate at runtime' and then Applying the same.
For your error, please verify if the authentication is enabled/disabled for Default Web-site. Change that setting and then try disabling you specific websites status. It should ideally work.
I've seen this error stem from three different controls on the authentication method:
If configuring the authentication method on an application, and the authentication method you're trying to disable is enabled on the containing site.
If configuring the authentication method on a site, and the authentication method you're trying to disable is enabled on the Default Web Site.
If configuring the authentication method that has been defined in a web.config file. You should adjust it in the config file.
0xC00CEF03 error code translates to WR_E_NSPREFIXWITHEMPTYNSURI which refers to “Writer: cannot use prefix with empty namespace URI” (Reference).
This error occurs when there is something wrong with the web.config file. It’s probably corrupted or there are incompatible tags and parameters. As a result of this corruption and incompatibility, IIS is not able to read this file. Therefore, it can’t make a change.
The issue mostly happens after migration because a piece of configuration that works in the older version of IIS (and .NET Framework) probably became unsupported in the newer version.
The shortest way of finding what part of the web.config is causing the issue is to remove sections one by one and test. High-level steps:
Remove a section from the bottom of the file (Let’s say you removed system.serviceModel section)
If the issue goes away, add the section back and start removing the subsections of it one by one. (For example, ws2007HttpBinding subsection in system.serviceModel section)
If the issue goes away again, bring the subsection back and go through each lines. Prefixes like wsid:, asm:, and trust: are the most common causes of this issue
Source: 0xC00CEF03 error (Cannot use prefix with empty namespace URI)

ASP.NET MVC "Potentially dangerous Request.Path" with valid URL

On my production ASP.NET MVC 3 site, I've been noticing the occasional "A potentially dangerous Request.Path value was detected from the client (%)." unhandled exception in the Windows application log.
While these can be perfectly valid under regular site usage (ie/ random web bots), a number of the requests appear to be from valid, local ISP users.
In the exception's request details, the Request URL is different than the Request path:
Request URL: http://www.somesite.com/Images/Image With Space.jpg
Request path: /Images/Imagehttp://www.somesite.com/Images/Image With Space.jpgWithhttp://www.somesite.com/Images/Image With Space.jpgSpace.jpg
Notice that in the "request path", any place there is a "space" in the path is replaced with an exact copy of the request url!
Within the site, the actual link looks like this:
<img src="/Images/Image%20With%20Space.jpg" />
Any idea what might be causing this? I tried to look at the documentation for Request.Path and Request.Url, but I can't figure out why they would be different. Hitting the Request URL directly brings up the resource correctly.
Update: I managed to get a trace of one of the malfunctioning requests by using IIS 7.0's Failed Request Tracing feature:
Referer: Google search
User-Agent: Mozilla/5.0 (iPad; CPU OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3
RequestURL: http://www.somesite.com:80/Images/Image%20With%20Space.jpg
Typing the URL manually into my iOS 5.1.1 brings up the image correctly. Searching for the image in Google Images brings up the image correctly. Still no successful reproduction.
Partway down the trace I see:
MODULE_SET_RESPONSE_ERROR_STATUS Warning. ModuleName="RequestFilteringModule", Notification="BEGIN_REQUEST", HttpStatus="404", HttpReason="Not Found", HttpSubStatus="11",
According to IIS' documentation, 404.11 from the Request Filtering module is a "double encoding" error in the URL. Experimenting a bit, if I purposefully create a double encoded url such as http://www.somesite.com/Images/Image%2520With%2520Space.jpg I get the exact error in the event log, complete with malformed Request Path.
The malformed Request Path in the event log error appears to be a bug in ASP.NET 4.0.
It doesn't, however, explain why I'm getting the error in the first place. I checked a large number of failed request logs - the only common factor is that they're all using AppleWebKit. Could it be a bug in Safari?
The httpRuntime section of the Web.Config can be modified to adjust the URL validation. ASP MVC projects are usually running in the validation mode 2.0 and the default invalid characters (separated by commas) are listed below.
<httpRuntime requestValidationMode="2.0" requestPathInvalidCharacters="<,>,*,%,:,&,\" />
As you can see the % sign is considered invalid. A space can be encoded to %20 causing the validation error. You can just add the requestPathInvalidCharacters attribute to the httpRuntime section in your Web.Config file and copy the values I listed below except for the "%," part.
Scott Hanselman has a blog post about this issue:
http://www.hanselman.com/blog/ExperimentsInWackinessAllowingPercentsAnglebracketsAndOtherNaughtyThingsInTheASPNETIISRequestURL.aspx
I can't help thinking that - given the restricted user-agent - this might represent incorrect handling of the URL by that browser on IOS 5.1.1. I don't personally own such a device so I can't test this - but it would be interesting to investigate how it behaves with a url that actually has spaces in it instead.
I have a feeling that it's seeing the %20 in the url from the page source and double-encoding it, thinking it's being helpful. The problem there being that IIS will decode it back (before ASP.net kicks in) and throw it's rattle out of it's pram because now it sees a literal %20 instead of a space.
I personally don't recommend modifying your servers' security settings; however it would be the easiest solution so I dare say that's what you will do.
Rather, I think if you can confirm this 'bug' (I'm already on the road, finding a safe hiding place from Apple's lawyers), find a format that works for this device; or take all the spaces out of your resource urls. A - is the best alternative.

BITS error codes

I'm writing an application updater that pulls installation package from our distribution web site to the user's PC using the background intelligent download service facility.
More or less everything is working fine now but I'm having a bit of problem getting the application react well to all recoverable errors. Specifically, I'd like the application to handle properly the case of proxy authentication.
In HTTP, it's simple: make a request, get a "407" HTTP response code, prompt for user name/password and repeat until you ether go through or the user press "cancel".
With BITS, it's not that simple. I don't get the HTTP status code. I get a couple of codes: the context (which should be BG_ERROR_CONTEXT_REMOTE_FILE in my case) and an "ErrorCode" that is supposed to depend on the context.
If I request the textual description of the error through GetErrorDescription, I get the correct "407 proxy authentication require" text. But the error code I have is 0x80190197 which is nowhere near 407.
So, does anyone know where I can get a full list of the BITS error code ? Failing that, partial list with the most common errors would be nice.
0x80190197 is not strictly speaking a BITS error, it's an HTTP stack error. The list is available here: Errors (019) FACILITY_HTTP

Resources