I have a top-down EJB Webservice (deployed on Websphere 6.01) that hangs the first time I call it after starting the JVM. If I see the logs, it seems to do everything normal that I expect it to (retrieving data from database) and I see all sysouts printed till the very last statement in EJB method. But client never gets the response back and eventually times out. Something is going on in the container (or in WS router application) after the control goes out of EJB but I don’t know how to troubleshoot it further. If it had something to do with startup, I would see slow down in the beginning of service invocation, not at the end.
From second call onwards it all works normal. Any suggestions would be appreciated.
Two possibile directions:
1). Try stubbing out all the implementation of the EJB, no database calls, no work. See whether you still get the problem. If you don't, then presumably something interesting is happening in your code. If you do, then there's something weird happening in the infrastructure - probably PMR time.
2). WebSphere has plenty of trace you can turn on. Try enabling trace relating to Web container. This may let you see where it's getting stuck.
I would recommend opening a PMR anyway.
Related
I have tomcat application which uses webservice using apache axis. Everything works fine if single user is using application. Multiple user are also working fine except in one condition. I have webservice call which can take 30 sec. When more than one user call a controller which uses this particular method, both thread seems to go into deadlock condition.
Worst part is it hangs whole application. I am able to reproduce this condition in local environment many times, so i am very confident that this particular call block if requested by more than one user at same time. I also tried setting timeout in web service call using axis specific method ((Stub)service).setTimeout(1000*60); but its also not working. Is anyone face this condition before. I have no clue why single thread blocking hang whole tomcat. Tomcat is working but it stopped responding to any other request. I am using spring for my web application.
I have created apps in the past that would have web pages that would call the persistence layer to get some query results or to insert, delete, etc against a db. However, nothing was left running in the background except for the persistence layer. Now I need to develop an app that has an process that is always running in the background, which is waiting for messages to come thru a zeromq messaging system (cannot change this at this point). I am a little lost as to how to setup the object so that it can always be running and yet I can control or query the results from the object.
Is there any tutorial/examples that covers this configuration?
Thanks,
You could use some kind of timer, to start a method every second to look at a specific ressource and process the input taken from that.
If you use Spring than you could have a look at the #Scheduled annotation.
If your input is some kind of java method invokation, than have a look at the java.util.concurrent Package, and concurrent programming at all. -- But be aware of the fact, that there are some restictions one creating own Threads in an EJB environment.
We are running our Junit 4 test suite against Weblogic 9 in front of an Oracle 10 database (using Hudson as a continuous integration server) and occasionally we will get an ORA-12519 crash during script teardown. However, the error is very intermittent:
It usually happens for the same Test class
It doesn't always happen for the same test cases (sometimes they pass)
It doesn't happen for the same number of test cases (anywhere from 3-9)
Sometimes it doesn't happen at all, everything passes
While I can't guarantee this doesn't happen locally (when running against the same database, of course), I have run the same suite of class multiple times with no issues.
Any ideas?
Don't know if this will be everybody's answer, but after some digging, here's what we came up with.
The error is obviously caused by the fact that the listener was not accepting connections, but why would we get that error when other tests could connect fine (we could also connect no problem through sqlplus)? The key to the issue wasn't that we couldn't connect, but that it was intermittent
After some investigation, we found that there was some static data created during the class setup that would keep open connections for the life of the test class, creating new ones as it went. Now, even though all of the resources were properly released when this class went out of scope (via a finally{} block, of course), there were some cases during the run when this class would swallow up all available connections (okay, bad practice alert - this was unit test code that connected directly rather than using a pool, so the same problem could not happen in production).
The fix was to not make that class static and run in the class setup, but instead use it in the per method setUp and tearDown methods.
So if you get this error in your own apps, slap a profiler on that bad boy and see if you might have a connection leak. Hope that helps.
Another solution I have found to a similar error but the same error message is to increase the number of service handlers found. (My instance of this error was caused by too many connections in the Weblogic Portal Connection pools.)
Run SQL*Plus and login as SYSTEM. You should know what password you’ve used during the installation of Oracle DB XE.
Run the command alter system set processes=150 scope=spfile; in SQL*Plus OR any SQL friendly IDE.
VERY IMPORTANT: Restart the database, to make the change effective in the SPFILE.
From here:
http://www.atpeaz.com/index.php/2010/fixing-the-ora-12519-tnsno-appropriate-service-handler-found-error/
I also had the same problem, I searched for the answers many places. I got many similar answers to change the number of process/service handlers. But I thought, what if I forgot to reset it back?
Then I tried using Thread.sleep() method after each of my connection.close();.
I don't know how, but it's working at least for me.
If any one wants to try it out and figure out how it's working then please go ahead. I would also like to know it as I am a beginner in programming world.
I had this problem in a unit test which opened a lot of connections to the DB via a connection pool and then "stopped" the connection pool (ManagedDataSource actually) to release the connections at the end of the each test. I always ran out of connections at some point in the suite of tests.
Added a Thread.sleep(500) in the teardown() of my tests and this resolved the issue. I think that what was happening was that the connection pool stop() releases the active connections in another thread so that if the main thread keeps running tests the cleanup thread(s) got so far behind that the Oracle server ran out of connections. Adding the sleep allows the background threads to release the pooled connections.
This is much less of an issue in the real world because the DB servers are much bigger and there is a healthy mix of operations (not just endless DB connect/disconnect operations).
I had the similar issue. It happened every time when I run a pack of database (Spring JDBC) tests with SpringJUnit4ClassRunner, so I resolved the issue putting #DirtiesContext annotation for each test in order to cleanup the application context and release all resources thus each test could run with a new initalization of the application context.
We have a fairly standard client/server application built using MS RPC. Both client and server are implemented in C++. The client establishes a session to the server, then makes repeated calls to it over a period of time before finally closing the session.
Periodically, however, especially under heavy load conditions, we are seeing an RPC exception show up with code 1754: RPC_S_NOTHING_TO_EXPORT.
It appears that this happens in the middle of a session. The user is logged on for a while, making successful calls, then one of the calls inexplicably returns this error. As far as we can tell, the server receives no indication that anything went wrong - and it definitely doesn't see the call the client made.
The error code appears to have permanent implications, as well. Having the client retry the connection doesn't work, either. However, if the user has multiple user sessions active simultaneously between the same client and server, the other connections are unaffected.
In essence, I have two questions:
Does anyone know what RPC_S_NOTHING_TO_EXPORT means? The MSDN documentation simply says: "No interfaces have been exported." ... Huh? The session was working fine for numerous instances of the same call up until this point...
Does anyone have any ideas as to how to identify the real problem? Note: Capturing network traffic is something we would rather avoid, if possible, as the problem is sporadic enough that we would likely go through multiple gigabytes of traffic before running into an occurrence.
Capturing network traffic would be one of the best ways to tackle this issue. If you can't do that, could you dump the client process and debug with WinDBG or Visual Studio? Perhaps compare a dump when operating normally versus in the error state?
This is my setting: I have written a .NET application for local client machines, which implements a feature that could also be used on a webpage. To keep this example simple, assume that the client installs a software into which he can enter some data and gets some data back.
The idea is to create a webpage that holds a form into which the user enters the same data and gets the same results back as above. Due to the company's available web servers, the first idea was to create a mono webservice, but this was dismissed for reasons unknown. The "service" is not to be run as a webservice, but should be called by a PHP script. This is currently realized by calling the mono application via shell_exec from PHP.
So now I am stuck with a mono port of my application, which works fine, but takes way too long to execute. I have already stripped out all unnecessary dlls, methods etc, but calling the application via the command line - submitting the desired data via commandline parameters - takes approximately 700ms. We expect about 10 hits per second, so this could only work when setting up a lot of servers for this task.
I assume the 700m are related to the cost of starting the application every time, because it does not make much difference in terms of time if I handle the request only once or five hundred times (I take the original input, vary it slighty and do 500 iterations with "new" data every time. Starting from the second iteration, the processing time drops down to approximately 1ms per iteration)
My next idea was to setup the mono application as a remoting server, so that it only has to be started once and can then handle incoming requests. I therefore wrote another mono application that serves as the client. Calling the client, letting the client pass the data to the server and retrieving the result now takes 344ms. This is better, but still way slower than I would expect and want it to be.
I have then implemented a new project from scratch based on this blog post and get stuck with the same performance issues.
The question is: am I missing something related to the mono-projects that could improve the speed of the client/server? Although the idea of creating a webservice for this task was dismissed, would a webservice perform better under these circumstances (as I would not need the client application to call the service), although it is said that remoting is faster than webservices?
I could have made that clearer, but implementing a webservice is currently not an option (and please don't ask why, I didn't write the requirements ;))
Meanwhile I have checked that it's indeed the startup of the client, which takes most of the time in the remoting scenario.
I could imagine accessing the server via pipes from the command line, which would be perfectly suitable in my scenario. I guess this would be done using sockets?
You can try to use AOT to reduce the startup time. On .NET you would use ngen for that purpoise, on mono just do a mono --aot on all assemblies used by your application.
AOT'ed code is slower than JIT'ed code, but has the advantage of reducing startup time.
You can even try to AOT framework assemblies such as mscorlib and System.
I believe that remoting is not an ideal thing to use in this scenario. However your idea of having mono on server instead of starting it every time is indeed solid.
Did you consider using SOAP webservices over HTTP? This would also help you with your 'web page' scenario.
Even if it is a little to slow for you in my experience a custom RESTful services implementation would be easier to work with than remoting.