While developing a Grails 1.0.5 app I'm appalled at how slow the grails test-app command is. Even though the actual tests take just ~10 seconds, the whole execution adds up to
real 1m26.953s
user 0m53.955s
sys 0m1.860s
This includes grails bootstrapping, loading plugins, compiling all the code, etc.
Any hints on how to speed up the grails test-app execution would be greatly appreciated.
You can use interactive mode to speed up your test runs.
Just run
grails interactive
Then type
test-app
The first time will be the same as usual but each time after that will be dramatically faster. There are currently some issues with interactive mode (like running out of memory after a few runs) but I still find it worth it.
There aren't any hard and fast rules for speeding it up, and the performance issues that you're seeing might be specific to your app.
If your bootstrapping is taking ~75 seconds, that sounds pretty long. I'd take a close look at whatever you have in your Bootstrap.groovy file to see if that can be slimmed down.
Do you have any extra plugins that you might not need (or that could have a major performance penalty)?
This might not be a possibility for you right now, but the speed improvements in grails 1.1.1/groovy 1.6.3 over grails 1.0.5/groovy 1.5.7 are fairly significant.
Another thing that really helps me when testing, is to specify only integration tests or only unit tests if I'm workiing on one or the other:
grails test-app -unit
grails test-app -integration
You can also specify a particular test class (without the "Tests" prefix), to run a single test which can really help with TDD (ex for "MyServiceTests" integration):
grails test-app -integration MyService
In grails 1.1.1, bootstrapping with 5 plugins and ~40 domain classes takes me less than 20 seconds.
If you're still using Groovy 1.5.x you could probably of shave a few seconds by upgrading to Groovy 1.6
Please see my answer here. A plugin relying on a poorly defined maven artifact can cause grails to go and look every time for a newer version.
Grails very slow to resolve certain dependencies
You can choose to run unit and integration tests in parallel as well - see this article
Increasing the java memory/JVM options can definitely speed things up. The amount of memory you can give depends on your equipment.
If you are running grails from the command line, set the GRAILS_OPTS environment variable. Add something like this to ~/.bash_profile
export GRAILS_OPTS="-Xms3000M -Xmx3000M -XX:PermSize=256m -XX:MaxPermSize=512m"
If you use GGTS(Eclipse) you'll need to add this to the VM arguments of the run configuration.
There are also a few JVM settings that can be modified to increase the speed:
-XX:+UseCodeCacheFlushing
-XX:MaxInlineLevel=15
-noverify (turns off class validation)
grails now comes with http://grails.org/plugin/testing installed. this mocks the domain stuff, so you can do some testing of domain classes as unit tests. they run pretty fast.
Related
Lets say I have 25 spring boot micro-services each of which starts with 1GB JVM in production. At any given time not all are in use and there is no instance when they are using the full 25GB memory at once. In reality many of them will sit idle 90% of the time but any of them might at some point get called and require up to 1GB memory.
In my development environment I would like to run all of them at once but only have 8GB memory. I don't need great performance but I need them all to run at the same time for the entire app to work. I would like to try to run all the applications within a single JVM with 6GB dedicated memory. That should be enough at any given time.
This seems like it would be a common issue as many companies are converting to cloud/microservices. 10 years ago we would have one monolithic app with single JVM (easy to run in dev environment). Now we have dozens of small apps which might not need a ton of memory but they each run in their own JVM so each has a good amount of overhead. This actually makes development more complex rather than simplifying. So Im trying to find a solution for our developers where they can run everything but not kill the memory on their machines.
The spring boot apps need to run without modification aside from
maybe local profiles. Otherwise developers would have to make tons of changes every time they pull the code from git
Each project needs to be able to configure a different port (application-local.properties setting)
for tomcat.
Each project needs its own classpath entries (for instance one might use version 1.0 of a jar and another might use version 2.0 and without separate classpaths one or the other would break)
I have been trying to follow this post but its not 100% what I want. I feel like a proper solution should respect the application.properties / application-local.properties file and use the port set inside the project rather than having to hardcode any configuration outside the project. Essentially his post is starting a separate thread for each microservice and attaching a separate classloader to each thread. Then calling SpringApplication.run and passing in the classname that would normally be used to start the microservice. I think this is maybe ignoring the auto configuration properties.
Any help would be greatly appreciated!
You can manage how much resources your applications are consuming with docker. One spring boot application should be one docker container. You can at runtime change how much resources(in your case memory) container use. Take a look at this
article on how to at runtime change resource allocation in docker. Also, with kubernetes is possible to define minimum and maximum resources that your application needs.
Consider you have a Spring-Boot Application and within this application also a bunch of Integration-Tests, which are annotated with #SpringBootTest and run with the SpringRunner class.
They are invoked by the maven failsafe plugin, which by default does not parallelise tests in any way. The tests all run fine without any issues.
What changes if you use failsafe's feature of forkCount - can you expect the test execution to work out of the box? Do you need to adjust some code? What do you need to look out for that could potentially not allow these integration tests to run in a forked, "parallel" environment via this plugin?
From my understanding, the failsafe plugin will create forkCount-many JVMs and in each some of the integration tests are executed. That sounds like there is nothing to do, you don't need to make anything threadsafe, you don't need to make Singleton-beans into ThreadScoped beans or anything - as the process of having multiple JVM's should already create multiple of these beans.
Sorry if the question appears weird, I tried researching this question but I could not find an answer.
From Maven doc:
The parameter forkCount defines the maximum number of JVM processes
This means that the tests will run in it's own process and therefore you will have separate Spring Boot instances. So you really don't have to care about thread safety.
But you have to care about memory consumption.
I am trying to run JProfiler for grails application. I would really appreciate any suggestions with the following:
1) Since I don't have an explicit class with main() method in grails application, I am assuming Attaching to a running JVM is my only option. Is that true? Is there a way I could attach JProfiler before the grails application starts?
2) After attaching to a running JVM, what does JProfiler need inorder to profile the controller/service/src/domain files. Do I have to execute the test cases. In my case they are rest controllers so do I have to run the requests for all possible scenarios?
3) Is it possible to have the Jprofiler profile the code without me running the test cases, since I may not be able cover all scenario's?
Since I don't have an explicit class with main() method in grails application
A JVM profiler does not depend on a main method that is written by yourself. The only thing you have to be able to do is to pass the -agentpath VM parameter to the JVM. The exact parameter is given by Session->Integration wizard->New Remote Integration in JProfiler and has to be added to the environment variable GRAILS_FORK_OPTS for Grails >= 3.1.5.
The Intellij IDEA integration can profile Grails run configurations directly, so you don't have to do the above.
Using the attach API is also possible, but has a higher overhead when connecting and prevents some profiling capabilities from being enabled.
do I have to run the requests for all possible scenarios
The profiler profiles the entire JVM, so whatever use case you run while profiling will show up in the profiler.
We are working on a new project using Spring, Hibernate and Transaction management using AspectJ (#Transactional annotations) and we are not sure what is the best practices weaving options to go.
We've started with the LTW and placed the tomcat Loader, it worked fine.. but then we thought that it may be less risky to have compile time weaving which instruments and *.classes and not on load time (in memory), so it will done just one time and not when on the tomcat startup. This is done via Maven aspectj-maven-plugin plugin.
Can you please advice ?
What are the pros and cons of using those weaving options ?
Thanks!
Disclaimer: This is not a discussion forum but a Q/A platform, so your question does not have the correct answer, it rather sparks discussion. I am trying to elaborate a bit anyway.
You basically already stated the main facts about LTW versus CTW. I am failing to see why LTW should be more risky than CTW, though. It slows down your server start-up, but the risk is the same as with CTW because the resulting byte code is also the same. Or are you talking about the risk that maybe more code gets (e.g. 3rd party libraries) woven than you intend? In that case yet, CTW keeps you on the safe side and you have more control over what should be woven. It should also speed up your server start-up time in comparison with LTW.
So if you have no compelling reasons to use LTW, such as the wish to dynamically intercept code you are unaware of during development, go ahead and use CTW and you are on the safe side, as long as you are fine with adjusting your build process accordingly. AspectJ Maven plugin is pretty straightforward to use, so that should not be a big deal. You can still switch to LTW as needed and when needed. OTOH, if you are using LTW now and are fine with the server start-up time, maybe there is no need to switch. If unsure, try both approaches and compare the results. ;-)
I have a large spring project, using xml configuration. I'm looking for a quick way to verify changes to the xml configuration.
I can load the whole project locally - the problem is this takes more than 5 minutes, loads a huge amount of data.
My XML editor catches XML formatting errors.
I'm looking for something intermediate - to catch obvious problems like references to beans that aren't defined, or calling constructors with the wrong arguments. Is there a quick way to do this, without having to actually invoke all the constructors and bring up the whole environment?
I'm building with Maven and editing with Eclipse, although my question isn't specific to either.
Since you already use Eclipse, you could try Spring Tool Suite (comes either standalone or as an add-on). It's essentially Eclipse with extra Spring-specific features, like Beans Validator. I'm not sure how thorough the validation is, but it should catch most configuration problems.
It's maintained by SpringSource so its integration with Spring "just works" and it's guaranteed not be more or less in sync with Spring Framework's release cycle.
Beanoh :
http://beanoh.org/overview.html#Verify
this project does exactly what I'm looking for. Verify obvious problems with spring config, but without the overhead of initializing everything.
You can use a Spring testing support to integration test your Spring configuration. However if the loading of the context is taking 5 mins, then the tests will also take the same amount of time. Spring does cache the context so if you have multiple tests using the same set of Spring contexts, then once cached the tests should be very quick.
I can suggest a few ways to more efficiently test your configuration:
Organize your project in modules, with each module being responsible for its own Spring configuration - this way, each module can be independently developed and tested.
If you have a modular structure, the testing can be more localized by mocking out the dependent modules, again this is for speed.