Kestrel server slow on "bad request data" - asp.net-web-api

I have an IOT device (black box, can't reprogram it) that sends http POST requests (136 bytes of JSON, a string) over wired ethernet to my .NET core 2.2 very simple server console application.
I just output the string to the console.
[HttpPost]
public void Post([FromBody] RootObject root)
{
string adv = root.prt;
Console.WriteLine(adv);
}
I get to display less than 1 line per second, sometimes 2.
Using Fiddler as a reverse proxy, instead I receive between 5 and 10 http req per second, that is the correct behavior of the device.
So I enabled Information logs, and get this, every second, with that "Invalid request line" error:
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
Request starting HTTP/1.1 POST http://192.168.0.92/api/values application/json 136
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
Executing endpoint 'BluepycWeb.Controllers.ValuesController.Post (BluepycWeb)'
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
Route matched with {action = "Post", controller = "Values"}. Executing action BluepycWeb.Controllers.ValuesController.Post (BluepycWeb)
02003F002293831000010033FF0006EFAA256B6D1A001E010201061AFF4C000215476C6F62616C2D54616700000000000000000000CD0101CC0001FF001FCC
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
Executing action method BluepycWeb.Controllers.ValuesController.Post (BluepycWeb) with arguments (BluepycWeb.Controllers.RootObject) - Validation state: Valid
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
Executed action method BluepycWeb.Controllers.ValuesController.Post (BluepycWeb), returned result Microsoft.AspNetCore.Mvc.EmptyResult in 0.2158ms.
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
Executed action BluepycWeb.Controllers.ValuesController.Post (BluepycWeb) in 9.4032ms
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
Executed endpoint 'BluepycWeb.Controllers.ValuesController.Post (BluepycWeb)'
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
Request finished in 24.5625ms 200
info: Microsoft.AspNetCore.Server.Kestrel[17]
Connection id "0HLMO99UNQBGM" bad request data: "Invalid request line: '\x0D\x0A'"
Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Invalid request line: '\x0D\x0A'
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpParser`1.RejectRequestLine(Byte* requestLine, Int32 length)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpParser`1.GetUnknownMethod(Byte* data, Int32 length, Int32& methodLength)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpParser`1.ParseRequestLine(TRequestHandler handler, Byte* data, Int32 length)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpParser`1.ParseRequestLine(TRequestHandler handler, ReadOnlySequence`1& buffer, SequencePosition& consumed, SequencePosition& examined)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpParser`1.Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.IHttpParser<TRequestHandler>.ParseRequestLine(TRequestHandler handler, ReadOnlySequence`1& buffer, SequencePosition& consumed, SequencePosition& examined)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1Connection.TakeStartLine(ReadOnlySequence`1 buffer, SequencePosition& consumed, SequencePosition& examined)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1Connection.ParseRequest(ReadOnlySequence`1 buffer, SequencePosition& consumed, SequencePosition& examined)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1Connection.TryParseRequest(ReadResult result, Boolean& endConnection)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequestsAsync[TContext](IHttpApplication`1 application)
If I send the same JSON paylod through Postman continuosly, no errors, no delays.
That error, that I don't know how to avoid nor by what is originated (but it seems it doesn't distub Fiddler) seems to hang my server for a second.
Any suggestion on how can I solve this problem?
Discard/correct the error?
Keep the error but not slowing down the data receive?
UPDATE:
Hosting with IISExpress, no problem, no "Invalid request line: '\x0D\x0A'" error, very fast.
Problem is only using Kestrel.
Thanks!

Yesterday I had same issue with my ASP .NETCore service.
Then realized I was sending HTTPS requests to my Kestrel server while it isn't configured to listen at HTTPS protocol, that's why Kestrel complaining.
If you want to use HTTPS in your kestrel you have multiple options to configure it, for example :
With assigned server certificate :
var host = new WebHostBuilder()
.UseKestrel(options =>
options.Listen(IPAddress.Any, 443, listenOptions =>
listenOptions.UseHttps("MyCert.pfx")))
With multiple hostnames and certificates :
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>()
.ConfigureKestrel((context, options) =>
{
options.ListenAnyIP(5005, listenOptions =>
{
listenOptions.UseHttps(httpsOptions =>
{
var localhostCert = CertificateLoader.LoadFromStoreCert(
"localhost", "My", StoreLocation.CurrentUser,
allowInvalid: true);
var exampleCert = CertificateLoader.LoadFromStoreCert(
"example.com", "My", StoreLocation.CurrentUser,
allowInvalid: true);
var subExampleCert = CertificateLoader.LoadFromStoreCert(
"sub.example.com", "My", StoreLocation.CurrentUser,
allowInvalid: true);
var certs = new Dictionary<string, X509Certificate2>(
StringComparer.OrdinalIgnoreCase);
certs["localhost"] = localhostCert;
certs["example.com"] = exampleCert;
certs["sub.example.com"] = subExampleCert;
httpsOptions.ServerCertificateSelector = (connectionContext, name) =>
{
if (name != null && certs.TryGetValue(name, out var cert))
{
return cert;
}
return exampleCert;
};
});
});
});

Related

Ajax.BeginForm OnFailure return empty result when connecting via HTTPS

When connecting via a non-secured protocol (http), my MVC 5 web application returns the error message properly. But the moment I connect via https, the error message is blank. Thought? NOTE: I try json return also. Thanks!
Browser debugger output
HTTP
HTTPS
MVC Controller:
string message = string.Join(" ", errors);
Response.StatusCode = (int)HttpStatusCode.RequestedRangeNotSatisfiable;
return new HttpStatusCodeResult(Response.StatusCode, string.Join(" ", message));
Script:
function ajaxOnFailure(response, status, error) {
debugger;
if (response.status == 500) {
error = "An internal error has occurred. Please try again. If the problem persists, please contact IT."
}
toastr.error("", error);
}
I ended up using a workaround. Instead of return a fail response, I return OK response and determine the type of message via jquery. It would have been nice if a fail response return the message via https.

Requests to same url blocked in http nio thread pool

I simply created a RestController in springboot(1.5.4.RELEASE) application and test how it works when multiple requests come in. What confused me is:
same url : the 2nd request blocked until the 1st request executed
different url: non-block, 2 requests executed almost at the same time
My question is who is blocking the 2nd requst and why?
Test Code:
#GetMapping(value = "/sleep")
public String sleep(HttpServletRequest request, #RequestParam boolean status)
{
String requestId = request.toString();
logger.info("request [{}] in and status = {}.", requestId, status);
String result;
if (status)
{
Thread.currentThread().sleep(10 * 1000);
result = "slept";
}
else
{
result = "stay up";
}
logger.info("response [{}] out and result = [{}].", requestId, result);
return result;
}
Test Result:
Different url:do not block, almost start executing at the same time.
http://localhost:20002/sleep?status=false AND http://localhost:20002/sleep?status=true
2018-08-14 15:04:14.139 [http-nio-20002-exec-5]: connection [RequestFacade#46515328] in and status = true.
2018-08-14 15:04:16.452 [http-nio-20002-exec-6]: connection [RequestFacade#1140f857] in and status = false.
2018-08-14 15:04:16.452 [http-nio-20002-exec-6]: connection [RequestFacade#1140f857] out and result = [stay up].
2018-08-14 15:04:24.139 [http-nio-20002-exec-5]: connection [RequestFacade#46515328] out and result = [slept].
Same url: block, the 2nd requst doesn't execute until the 1st requst is done.
http://localhost:20002/sleep?status=true AND http://localhost:20002/sleep?status=true
2018-08-14 15:10:29.943 [http-nio-20002-exec-9]: connection [RequestFacade#46515328] in and status = true.
2018-08-14 15:10:39.944 [http-nio-20002-exec-9]: connection [RequestFacade#46515328] out and result = [slept].
2018-08-14 15:10:39.960 [http-nio-20002-exec-1]: connection [RequestFacade#1140f857] in and status = true.
2018-08-14 15:10:49.960 [http-nio-20002-exec-1]: connection [RequestFacade#1140f857] out and result = [slept].
I debug the code of tomcat-embed-core and find that when request with different url comes in, the Poller thread could catch it immediately and process it; while request with the same url come in, the Poller could not get it until the 1st connection is returned.
Thanks to #M.Deinum, the blocking has nothing to do with NIO. It's the chrome who blocked the 2nd request, and for reason ,I find some comment below:
this behavior is due to Chrome locking the cache and waiting to see the result of one
request before requesting the same resource again.
https://stackoverflow.com/a/27514611/10222882
And It's proved by chrome Network -> Timing, the 2nd request is in [stalled] status until the 1st response is returned.
connection_stalled

No headers available in xhr object when an UNAUTHORIZED error occurs - IE10

I have a CORS Ajax call to a web api. I have a message handler which throws an:
var msg = new HttpResponseMessage(HttpStatusCode.Unauthorized) { ReasonPhrase = "Oops!!!" };
throw new HttpResponseException(msg);
On the client I can't get no response headers, tried:
error: function (xhr, error) {
var result = xhr.getResponseHeader("Response");
but result is null and no other heades are available.
The Debugger shows correctly a 401!
Should I return the Unauthorized Exception differently from the server?
Update:
I forgot the add the Origin header to my HttpResponseException, in order to get the headers.
But in IE10 I don't get any headers, only error message
"error"
How can I know what happend, when using IE10?
Here is related question.
As I cannot get the Headers when returning a 401 response.
No instead I return NoContent StatusCode, which triggers the AJAX success eventhandler.
Now I have to check the response header "response" (which I manually add on the server side) in the success eventhandler, in order to make it work in IE 10:
Server:
var msg = new HttpResponseMessage(HttpStatusCode.NoContent) { ReasonPhrase = "UNAUTHORIZED" };
msg.Headers.Add("RESPONSE", "401");
AddCORSHeaders(msg.Headers, request);
return await Task.FromResult(msg);
Client:
error: function (jqXhr, error) {
var isAuth= jqXhr.getResponseHeader("Response");
Now I get a "response" header with value 401.
IF SOMEONE HAS A BETTER APPROACH, I'LL EXCEPT THE ANSWER !
In IE10 the ajax error eventhandler you get readyState=0 and status="".
In Chrome you get a 401 and readyState=4.

System.OutOfMemoryException when uploading a large file (~600MB) using .NET 4.0 & HttpClient

I am getting this error only on my local workstation and prod server.
In Dev and Cert it is working fine.
local workstation - 20 GB memory, Win 7 64 bit, IIS Express, VS
2013 dev, cert & prod - 8 GB memory , 2008 R2 64 Bit, IIS 7.5
I have a web api (.net 4.0) which takes the incoming request body and uploads it to a storage server. configured web api as per this website.
I have these in my web.config
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="2147483648" />
</requestFiltering>
</security>
</system.webServer>
<system.web>
<httpRuntime maxRequestLength="2097152" />
</system.web>
I also have an implementation of IHostBufferPolicySelector which returns false for PUT & POST requests. so the request to this web api for PUt & POST are not buffered.
for any files < ~350 MB it is working fine. but web api is throwing out of memory exceptions when file size >= ~ 400 MB and this is happening only on Local workstation and Prod server.
Web Api controller calls below code to stream the request to the destination server
public async Task<HttpResponseMessage> StoreObjectAsync(Uri namespaceUrl, string userName, string password, string objectName, Stream objectContent, string contentType = "application/octet-stream", IDictionary<string, string> systemMetadata = null)
{
Uri namespaceRootUrl = Utilities.GetNamespaceRootUrl(namespaceUrl);
using (var request = new HttpRequestMessage() { Method = HttpMethod.Put })
{
request.RequestUri = Utilities.CreateRequestUri(namespaceRootUrl, objectName);
request.Content = new StreamContent(objectContent);
request.Content.Headers.ContentType = new MediaTypeHeaderValue(contentType);
HttpResponseMessage response;
response = await this.httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
return response;
}
}
After doing some research online, i understand from this link & this link that HttpClient on .Net 4.0 buffers the request body and because of that behavior it seemed to me that it is throwing outofmemory exception
so I changed my code to below this time using HttpWebRequest using which I have the control to specify that request should be streamed but not buffered.
public async Task<HttpResponseMessage> StoreObjectAsync(Uri namespaceUrl, string userName, string password, string objectName, Stream content, long contentLength, string contentType = "application/octet-stream", IDictionary<string, string> systemMetadata = null)
{
Uri namespaceRootUrl = Utilities.GetHCPNamespaceRootUrl(namespaceUrl);
HttpWebRequest httpWebRequest = ((HttpWebRequest)WebRequest.Create(requestUri));
httpWebRequest.Method = "PUT";
httpWebRequest.KeepAlive = true;
httpWebRequest.AllowWriteStreamBuffering = false;
httpWebRequest.ContentType = contentType;
httpWebRequest.ContentLength = contentLength;
using (Stream requestStream = await httpWebRequest.GetRequestStreamAsync())
{
await content.CopyToAsync(requestStream);
}
var webResponse = await httpWebRequest.GetResponseAsync();
HttpWebResponse httpWebResponse = (HttpWebResponse)webResponse;
Stream httpWebResponseContent = httpWebResponse.GetResponseStream();
HttpResponseMessage response = new HttpResponseMessage()
{
StatusCode = httpWebResponse.StatusCode,
ReasonPhrase = httpWebResponse.StatusDescription,
Content = new StreamContent(httpWebResponseContent)
};
return response;
}
Now it is working fine on my local machine. I am able to upload files around 1GB without any errors or memory exceptions. Havent pushed this to Prod yet.
But I still dont understand why the same code using HttpClient on .net 4.0 worked on Dev and Cert servers but not on Prod and my local.
please help me in understanding
How to find out why it worked on Dev and Cert? What system/server
configurations will affect the memory allocations to this api?

AJAX 504 when calling ASP.NET Web API

My AJAX call is returning a 504 error when calling an ASP.NET Web API action.
More info:
Here's my API action:
public HttpResponseMessage Get(string fileName, int feedID)
{
try
{
// create file...
return new HttpResponseMessage { Content = new StringContent("Complete."), StatusCode = HttpStatusCode.OK };
}
catch (Exception ex)
{
Log.WriteError(ex);
throw new HttpResponseException(new HttpResponseMessage
{
StatusCode = HttpStatusCode.InternalServerError,
Content = new StringContent("An error has occurred.")
});
}
}
Here's my AJAX call:
$.ajax({
url: url,
type: 'GET',
success: function () {
$("#lblProgressDownload").hide();
window.open("Previews/" + fileName);
},
error: function (xhr, status, error) {
$("#lblProgressDownload").hide();
alert("Error downloading feed preview: " + error);
}
});
I get a 504 error (viewed in fiddler/ chrome console) when the file takes too long to create. The "error" parameter in the error callback doesn't return anything.
I only get the 504 error when it's hosted - on my dev it works fine.
How do I prevent this 504 error?
Note, I already tried changing the executionTimeout property in my web.config, as well as the ajax timeout. Neither worked.
HTTP error 504 is a gateway timeout:
The server, while acting as a gateway or proxy, did not receive a timely response from the upstream server specified by the URI [...] in attempting to complete the request.
I suspect that means there is a proxy or gateway somewhere between you and the production server, but not your dev server, which is why it fails on the one but not the other.
Your choice is either to make your server code fast enough that it doesn't trigger the timeout, or get whoever is running the proxy server to relax their timeout restrictions (assuming it's something that you or your company controls).

Resources