Playframework-2.0 NettyDispatcher 500 Error on Heroku - heroku

I started getting 500 Internal Server Error(ISE) on Heroku, tried enabling debug level logs. The error is not consistent, it occurs for some request, for others it goes thru properly. When there is an ISE there is not even a single log line(even with debug on) in web dynos... I am able to see that the 500 response is given for the request.
From the client side am seeing the following
This exception has been logged with id 6gimbegj7
The above line tells that it should a response from Play! when it is running in production mode.
Attached New Relic monitoring plugin where it says the Exception occurred on NettyDispatcher (Netty IO server), not further info.
Any idea what could be the issue?

We have identified the problem here,
In play! we have a Global.java on the root of the packages, which has the methods like
onRequest(), onRequestRoute()
If there is any error at this place, and if we do not handle the exceptions, then Netty directly throws an 500 response to the request. This can happen from any part of the application. We did catch the exceptions in other part of the application but missed this one.
Mimicking the heoku env with the help of Foreman tool helped us to narrow down on the cause. As such setting the playframework log level to DEBUG did not throw any logs/stacktrace, Not sure why.

Related

open liberty and ConnectionManager

I'm new on Open Liberty and I need to maintain a legacy app
I have on this app logs the following error:
The ConnectionManager was unable to associate Connection com.ibm.ws.rsadapter.jdbc.WSJdbcConnection#3b05f444 with ManagedConnection
WSRdbManagedConnectionImpl#67538473 for resource jdbc/AppDs. Received exception: com.ibm.ws.rsadapter.exceptions.DataStoreAdapterException:
DSRA9420E: Connection cannot be reassociated because child objects are still open.
I'm not sure from to start on this.
I have already searched here
But the code that is shown on logs is based on spring and it couldn't leak connections.
The app uses spring and ehcache.
Please, can someone to provide some guidance?
Looking at the Open Liberty source, that error is raised from a single place in the code, in response to the error condition that a JDBC Connection still has open child JDBC resources (such as Statement, PreparedStatement, ...) associated with it, at a point where it is being asked to reassociate the connection handle across transaction boundaries. Looking at the single caller of that code, it appears that the dissociate method (which closes these same JDBC resources) would have been invoked only a few lines earlier, suggesting that the only way for you to reach the state where the error occurs is if other code were simultaneously using the connection from another thread. To help confirm if this is the case, you could try enabling Liberty trace (via server.xml config) of
<logging traceSpecification="*=info:RRA=all"/>
while reproducing the error and post it somewhere accessible to look at.

Requests getting failed with error 400 when thread count increased in jmeter

.net application starts giving error http 400(bad request) when 200 concurrent users are loaded.
And while running script, when application is browsed manually, could able to see same errors. This basically rules out, its not client issue.
So What is causing this error?
Kindly help
We don't know but it sounds like an application bug in any case because HTTP status codes 4xx are client errors and given it occurs only under the load the application should respond with 5xx server error codes
You need to inspect your application/webserver/loadbalancer logs as most probably the cause can be figured out from them.
Also it would be a good idea to check whether your application has enough headroom to operate in terms of CPU, RAM, etc as the errors could be caused by simple lack of resources. You can use i.e. JMeter PerfMon Plugin for this

stuck thread in createQueueConnection

I am getting the stuck thread error often while trying to send the JMS message to another manager server within the domain in our production environment.
Initially, we felt that it might be due to Load on the server but issue occurred randomly Even at the time of less load and system processing well some high volume time
We are not able to find the reason for the same.
Error Information:
weblogic.jms.client.JMSConnectionFactory.createQueueConnection(JMSConnectionFact
ory.java:199)

licenseNotification API has stopped working

Calling https://www.googleapis.com/appsmarket/v2/licenseNotification/[myAppId] has suddenly started returning 500 Backend Error every time.
500 error is a generic message, given when no more specific message is suitable. There are a number of causes for a 500 Internal Server Error to display in a web browser.
With any error message, particularly one as broad as the 500 Internal Server Error, you will first want to check your error logs for your server. These logs can provide valuable context related to any code failures or other potential causes of a site failure.
As stated in the answer in this SO question.
In general, every Backend error should be handled with an exponential retry, as
there might be service problems.
If the error still persists after let's say 10 hours then you should
contact the support in order to provide you 1:1 help to your problem.

Web API: 500 error after short timeout

I've got my Web API interface working fine but after a fairly short period, like around 5 minutes, it gives a 500 Internal Server Error on the first call and then works fine again until it times out. I'm not getting any additional information on the error message and there's nothing going to the logs. I don't have the issue on my development machine, just on the live server. Any ideas what might be causing this? How can I get additional error information on a live server?
The simple answer to this was set Custom Errors Off within the Web.config file. That gave a very useful error that made it easy to track down the problem.

Resources