We recently added MongoCk to our Spring 5 app (using the Spring runner), but are having some issues during our deploys. Our final step in the deploy process is a health check where the deployment server checks a health page every 5s for 5 minutes. Once it gets the correct response the deployment is considered successful and it finishes.
The issue is that MongoCk seems to only start the migration around 30s after the application context loads, resulting in the health check passing and the migration possibly failing after the service was "successfully" launched.
Using a standalone runner might solve this, but we really like the availability of other beans during the changelogs. So is there a way to enforce the changelogs to be processed as part of loading the application context? Or where is this delay coming from, and how can we reduce it?
You don't provide much information, but you are saying that Mongock starts 30 secs after the application context is loaded. That could be happening for two reasons:
The most likely possibility is that you are using runner-type ApplicationRunner(by default). This means that Spring decides when to run it after the entire context is loaded. From what you are saying runner-type InitializingBean is a better fit for you .
Please try this:
mongock:
runner-type: InitializingBean
You have multiple instances fighting for the lock. There is nothing we can do about it, this process is optimised(Although we are improving even more). However, as said, I believe the issue is related with the runner-type
Related
Question:
Is there an option within spring or its embedded servlet container to open ports when spring is ready to handle traffic?
Situation:
In the current setup i use a spring boot application running in google cloud run.
Circumstances:
Cloud run does not support liveness/readyness probes, it considers an open port as "application ready".
Cloud run sends request to the container although spring is not ready to handle requests.
Spring start its servlet container, open its ports while still spinning up its beans.
Problem:
Traffic to an unready application will result in a lot of http 429 status codes.
This affects:
new deployments
scaling capabilities of cloud run
My desire:
Configure spring/servlet container to delay opening ports when application is actually ready
Delaying opening ports to the time the application is ready would ease much pain without interfering too much with the existing code base.
Any alternatives not causing too much pain?
Things i found and considered not viable
Using native-image is not an option as it is considered experimental and consumes more RAM at compile time than our deployment pipeline agents allow to allocate (max 8GB vs needed 13GB)
another answer i found: readiness check for google cloud run - how?
which i don't see how it could satisfy my needs, since spring-boot startup time is still slow. That's why my initial idea was to delay opening ports
I did not have time to test the following, but one thing i stumbled upon is
a blogpost about using multiple processes within a container. Though it is against the recommendation of containers principles, it seems viable for the time until cloud run supports probes of any type.
As you are well aware of the fact that “Cloud Run currently does not have a readiness/liveness check to avoid sending requests to unready applications” I would say there is not much that can be done on Cloud Run’s side except :
Try and optimise the Spring boot app as per the docs.
Make a heavier entrypoint in Cloud Run service that takes care of
more setup tasks. This stackoverflow thread mentions how “A
’heavier’ entrypoint will help post-deploy responsiveness, at the
cost of slower cold-starts” ( this is the most relevant solution
from a Cloud Run perspective and outlines the issue correctly)
Run multiple processes in a container in Cloud Run as you
mentioned.
This question seems more directed at Spring Boot specifically and I found an article with a similar requirement.
However, if you absolutely need the app ready to serve when requests come in, we have another alternative to Cloud Run, Google Kubernetes Engine (GKE) which makes use of readiness/liveness probes.
I'm a newbie in developing with SpringBoot, so maybe this question will sound silly, but I hope someone will be able to show me the error of my ways.
I have a SpringBoot application with 4 Components. One of them processes input files and writes records into database, while the other three read the records, process them according to component configuration and send the result to a Web Service.
My problem is that sometimes there's a lot of files to be processed, so the "reader" Component takes a bit longer to finish. What I've noticed is that, while it's running, none of the other Components are starting. Since the data is relatively time sensitive, it is important for me that the processing Components run periodically and asynchronously.
I have used #EnableAsync in the main Application and I've marked all Components as #Async, yet the blocking problem still occurs. I was under the impression that the scheduled Components will be executed independent of each other. There are no shared resources between Components and even if they were, I would understand that a Component starts and blocks, but the Components (threads) are not started at all (I have a trace entry as a first line of each Component).
What should I look at? Is this how it is supposed to work? If yes, I will then start async threads from the Components or find another way, but I thought I can get by without all of that by using SpringBoot.
Any and all answers will be very much appreciated!
I'm creating a Fitnesse slim test (Decision Table). In order to run the test I need to start my Spring app. context. For that, I'm using a class that extends FixtureWirer. Starting the application context is not a problem, since the test completes successfully. In the page I can see that the test is complete and all the output values are compared. The problem is that the page with the final results never stops loading, but no exception is thrown. And that only happens when I use the FixtureWirer to start the application context, so I'm guessing the problem is related to that, but I haven't been able to figure it out.
Thanks in advance.
Do you mean the test doesn't stop as it is supposed to? If so, can you try to close the context
This may be because your Spring's shut down hook, is not being able to destroy beans which must
be blocked by any resource.
Can you check if there are any live threads after the execution of test that are blocked?
If there are any live threads left then Slim server would not terminate (SlimServer java process will not be killed).
I have created apps in the past that would have web pages that would call the persistence layer to get some query results or to insert, delete, etc against a db. However, nothing was left running in the background except for the persistence layer. Now I need to develop an app that has an process that is always running in the background, which is waiting for messages to come thru a zeromq messaging system (cannot change this at this point). I am a little lost as to how to setup the object so that it can always be running and yet I can control or query the results from the object.
Is there any tutorial/examples that covers this configuration?
Thanks,
You could use some kind of timer, to start a method every second to look at a specific ressource and process the input taken from that.
If you use Spring than you could have a look at the #Scheduled annotation.
If your input is some kind of java method invokation, than have a look at the java.util.concurrent Package, and concurrent programming at all. -- But be aware of the fact, that there are some restictions one creating own Threads in an EJB environment.
We are running our Junit 4 test suite against Weblogic 9 in front of an Oracle 10 database (using Hudson as a continuous integration server) and occasionally we will get an ORA-12519 crash during script teardown. However, the error is very intermittent:
It usually happens for the same Test class
It doesn't always happen for the same test cases (sometimes they pass)
It doesn't happen for the same number of test cases (anywhere from 3-9)
Sometimes it doesn't happen at all, everything passes
While I can't guarantee this doesn't happen locally (when running against the same database, of course), I have run the same suite of class multiple times with no issues.
Any ideas?
Don't know if this will be everybody's answer, but after some digging, here's what we came up with.
The error is obviously caused by the fact that the listener was not accepting connections, but why would we get that error when other tests could connect fine (we could also connect no problem through sqlplus)? The key to the issue wasn't that we couldn't connect, but that it was intermittent
After some investigation, we found that there was some static data created during the class setup that would keep open connections for the life of the test class, creating new ones as it went. Now, even though all of the resources were properly released when this class went out of scope (via a finally{} block, of course), there were some cases during the run when this class would swallow up all available connections (okay, bad practice alert - this was unit test code that connected directly rather than using a pool, so the same problem could not happen in production).
The fix was to not make that class static and run in the class setup, but instead use it in the per method setUp and tearDown methods.
So if you get this error in your own apps, slap a profiler on that bad boy and see if you might have a connection leak. Hope that helps.
Another solution I have found to a similar error but the same error message is to increase the number of service handlers found. (My instance of this error was caused by too many connections in the Weblogic Portal Connection pools.)
Run SQL*Plus and login as SYSTEM. You should know what password you’ve used during the installation of Oracle DB XE.
Run the command alter system set processes=150 scope=spfile; in SQL*Plus OR any SQL friendly IDE.
VERY IMPORTANT: Restart the database, to make the change effective in the SPFILE.
From here:
http://www.atpeaz.com/index.php/2010/fixing-the-ora-12519-tnsno-appropriate-service-handler-found-error/
I also had the same problem, I searched for the answers many places. I got many similar answers to change the number of process/service handlers. But I thought, what if I forgot to reset it back?
Then I tried using Thread.sleep() method after each of my connection.close();.
I don't know how, but it's working at least for me.
If any one wants to try it out and figure out how it's working then please go ahead. I would also like to know it as I am a beginner in programming world.
I had this problem in a unit test which opened a lot of connections to the DB via a connection pool and then "stopped" the connection pool (ManagedDataSource actually) to release the connections at the end of the each test. I always ran out of connections at some point in the suite of tests.
Added a Thread.sleep(500) in the teardown() of my tests and this resolved the issue. I think that what was happening was that the connection pool stop() releases the active connections in another thread so that if the main thread keeps running tests the cleanup thread(s) got so far behind that the Oracle server ran out of connections. Adding the sleep allows the background threads to release the pooled connections.
This is much less of an issue in the real world because the DB servers are much bigger and there is a healthy mix of operations (not just endless DB connect/disconnect operations).
I had the similar issue. It happened every time when I run a pack of database (Spring JDBC) tests with SpringJUnit4ClassRunner, so I resolved the issue putting #DirtiesContext annotation for each test in order to cleanup the application context and release all resources thus each test could run with a new initalization of the application context.