I'm implementing an application that communicates with the GitHub API in order to infer some statistics about projects developed currently. I chose to make the requests asynchronously, using HttpAsyncClient.
My problem is that after I execute all requests and get all the responses from the API (around 150 of them) and I try to read the content with
String content = EntityUtils.toString(response.getEntity());
I'm getting following SSLException after ~120 responses read:
Exception in thread "main" javax.net.ssl.SSLException: SSL peer shut down incorrectly
at sun.security.ssl.InputRecord.readV3Record(InputRecord.java:557)
at sun.security.ssl.InputRecord.read(InputRecord.java:509)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:883)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:840)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:94)
at org.apache.http.impl.io.AbstractSessionInputBuffer.read(AbstractSessionInputBuffer.java:204)
at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:182)
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:138)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.Reader.read(Reader.java:140)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:224)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:264)
at pl.xsolve.githubmetrics.github.OwnGitHubClient.extractBodyFromResponse(OwnGitHubClient.java:117)
The problem disappears when I decrease the number of requests significantly (for instance, by half). Also, all the responses contain HTTP/1.1 200 OK - I've checked it with response.getStatusLine() and it works until the very end of the response list.
The problem persists even if I remove httpClient.shutdown() in the finally block (which is executed before reading the content).
From the stack trace, I've concluded that the exception is thrown on the line
while((l = reader.read(tmp)) != -1)
Is the entity in the HttpResponse getting somehow outdated? Do you see an error in my reasoning? What can be the reason why first 120 responses are parsed properly and then SSLException is thrown?
Any help will be greatly appreciated :)
This could also be dependent upon your version of openssl. On a python project I maintain, we've seen errors in openssl 0.9.8 when people make large numbers of SSL requests in a short period of time. Admittedly it was nothing like your error message, but upgrading to openssl 1.0 might help.
Related
I am using express graphql in my node app . and it graphql always sends 500 Internal server error for any thrown error from resolver. Please suggest any solution so i will get proper response and status code
The express-graphql sends the HTTP error 500 whenever detects there is no data returned. Definitely, in case of exception is thrown there won't be a returning data. Although it looks like an obvious bug the developers have their own opinion on that. They seemed to decide providing an option to disable the feature so that the server shouldn't set the 500 error. At least the open issue on that is still there.
Checking out the sources I've found these options to get around the problem:
To set any other response code than 200 (e.g. 299 (custom)) in your resolver.
To set the response code in your error formatting handler. This seems to be the most optimal solution.
Just to pick up some other library =)
I am trying to make a POST request to an endpoint served using jetty server. The request errors out saying 413 Request entity too large. But the content-length is only 70KB which I see is way below the default limit of 200KB.
I have tried serving via ngnix server and add client_max_body_size to desired level but that didn't work. I have set the setMaxFormContentSize of WebContext and that didn't help either. I have followed https://wiki.eclipse.org/Jetty/Howto/Configure_Form_Size and that didn't helped me either.
Does anyone have any solution to offer?
wiki.eclipse.org is OLD and is only for Jetty 7 and Jetty 8 (long ago EOL/End of Life). The giant red box at the top of the page that you linked it even tell you this, and gives you a link to the up to date documentation.
If you see a "413 Request entity too large" from Jetty, then it refers the the Request URI and Request Headers.
Note: some 3rd party libraries outside of Jetty's control can also use HttpServletResponse.sendError(413) which would result in the same response status message as you reported.
Judging by your screenshot, which does not include all of the details, (it's really better to copy/paste the text when making questions on stackoverflow, screenshots often hide details that are critical in getting a direct answer), your Cookie header is massive and is causing the 413 error by pushing the Request Headers over 8k in size.
Getting a 416 error when trying to GET a website with HTTParty. Works just fine in the browser.
I have never gotten this error before, so I went online and found this:
It occurs when the server is unable to fulfill the request. This may
be, for example, because the client asked for the 800th-900th bytes of
a document, but the document is only 200 bytes long.
The request includes a Range request-header field, and not any of the
range-specifier values in this field overlaps the current extent of
the selected resource, and also the request does not include an
If-Range request-header field.
Wondering if anyone has gotten 416 with HTTParty before and if there is a way to prevent this form happening. Thanks
Example website where error occurs:
http://www.bizjournals.com/jacksonville/blog/morning-edition/2014/07/teens-make-up-less-of-summer-workforce-than-ever.html
It appears that bizjournals is able to detect you are a bot (not accessing in the browser) and therefore returns a 416.
irb(main):005:0> HTTParty.get('http://www.bizjournals.com/jacksonville/blog/morning-edition/2014/07/teens-make-up-less-of-summer-workforce-than-ever.html').body
=> "........As you were browsing <strong>http://www.bizjournals.com</strong> something about your browser made us think you were a bot. There are a few reasons this might happen........"
You could either ask bizjournals to allow you to make requests or try to change the headers to make bizjournals think you are not a bot.
I have a Classic ASP/VBScript site running on IIS 6, and a program called SmarterStats (found via an earlier question) providing various stats and info from the server logs. This is an internal site, and we do use integrated authentication (AUTH_TYPE = Negotiate).
I'm looking at the report on response codes, and it says that in the last 7 days there have been 255 responses with a 5xx error code... naturally not the most useful stat I could get back.
I have a custom error page for 500 and 500.100 errors that logs the specifics for me to follow up on - and that log is virtually empty. I've tested with an intentionally broken page, and a standard user account, and it is working as expected - custom message, log entry made. (and checked with "friendly error messages" on and off - works either way)
The only entry I have in the log is a single entry for an error -2147024843 (authentication related, took too long to get a response), and that's it... what other errors could be happening and how can I find/track them?
Should I route all the different types of 5xx errors through my custom page? Or is there something else I should be doing?
Resolution: I'm going to go with running all 5xx errors through the custom page. For reasons that leave me totally baffled, the errors have stopped... I've not changed anything, no one has changed anything on that server - but the errors have stopped. Go figure.
It seems like logging all 5xx errors via your custom page might be a good idea. Your logging seems to be working, but SmarterStats seems like it might be giving you bogus data?
What version of SmarterStats are you using?
My hosting provider has Version 5.2 and I can see a detailed list of which pages have produced a 5xx error by going to Report Items > Server Responses > 5xx - Server Errors in the navigation tree. The report that gets displayed has a table listing the pages that have produced the error codes so you can then go investigate those pages to see what may be causing the error.
Depending on your site the error might be perfectly normal, for example I have some custom HTTP handlers which implement a standardised HTTP protocol which actually specifies that 5xx errors should be thrown under certain circumstances.
Though this shouldn't apply to an internal only site on public websites it is sometimes the case that badly written crawlers will access your pages in such a way as to produce an error
The folk in the QA department use visual studio team test (2008 IIRC) to run load tests against our web application.
The latest set of tests have failed on several pages. The error reported is
Request failed: The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF
Searching for this using google yields quite a few results. it would appear that this error message is generated from the .Net framework WebRequest class (i.e. it is not a visual studio specific message). The most useful result is this one, which details my exact problem and how to suppress the error.
But of course, I want to get to the bottom of why this error occurs in the first place. Here are some more facts: -
This error never used to occur when the tests were run against an older version of the web app. The web app. host OS and web server (Win 2003 and IIS 6) are identical in both cases.
Not all the pages generate this error - only some.
The only significant change to these pages (that I can think of) is that they now use some AJAX whereas before they did not (IIRC)
In order to narrow down the problem, I created the simplest page that I could to replicate the problem. Luckily, that was not too hard. I then inspected the bytes in the header using Fiddler but I could not find an occurrence of a CR (0x0D) that was not followed by a LF (0x0A).
The raw HTTP response (as stored from Fiddler by response saving bytes - so its encoding should not have been altered during the save) is here as text if you don't believe me!
So now I am left thinking that the supposed error might be a false alarm. Does anyone else have experience of this/can help shed light?
This is definitely not a false alarm - I've been getting this error in my app a lot while trying to communicate with Facebook API.
I've just stumbled upon this response from Steven Cheng - http://www.velocityreviews.com/forums/t302174-why-do-i-get-the-server-committed-a-protocol-violation.html - and let me quote him:
From your description, you're using
the HttpWebRequest component to send
some http request to some external web
resource in your ASP.NET web
application. However, you're always
getting the "The server committed a
protocol violation.
Section=ResponseStatusLine" error
unless you set the following section
in the web.config file:
<system.net>
<settings>
<httpWebRequest useUnsafeHeaderParsing="true" />
</settings>
</system.net>
And you're wondering the cause of this behavior, correct?
As for this issue, I've performed some
research on this and found that the
problem is actually caused by the
critical http header
parsing/validating of the
HttpWebRequest component. According to
the Http Specification(http1.1), the
HTTP header keys shoud specifically
not include any spaces in their names.
However, some web servers do not fully
respect standards they're meant to.
Applications running on the Dotnet
framework and making heavy use of http
requests usually use the
httpWebRequest class, which
encapsulates everything a web oriented
developer could dream of. With all the
recently issues related to security,
the "httpWebRequest" class provides a
self protection mechanism preventing
it to accept HTTP answers which not
fully qualify to the specifications.
The common case is having a space in
the "content-length" header key. The
server actually returns a "content
length" key, which, assuming no spaces
are allowed, is considered as an
attack vector (HTTP response split
attack), thus, triggering a "HTTP
protocol violation error" exception.
Will try if this helps right now and post results later