How can I warm up the dependency cache of my maven tests? E.g. mvn test -DskipTests downloads some of the dependencies, but not all, e.g. some of the maven surefire plugin dependencies are only downloaded by mvn test.
I want to create a snapshot of my filesystem to boost up the execution of my tests. Therefore I'd like to have all the dependencies downloaded, but I want to achieve this without executing the tests itself.
Some of the dependencies that are only downloaded during mvn test and not by mvn test -DskipTests:
Downloading: http://repo1.maven.org/maven2/org/apache/maven/surefire/surefire-junit47/2.14/surefire-junit47-2.14.pom
Downloaded: http://repo1.maven.org/maven2/org/apache/maven/surefire/surefire- junit47/2.14/surefire-junit47-2.14.pom (4 KB at 13.9 KB/sec)
Downloading: http://repo1.maven.org/maven2/org/apache/maven/surefire/surefire-providers/2.14/surefire-providers-2.14.pom
Downloaded: http://repo1.maven.org/maven2/org/apache/maven/surefire/surefire-
The dependencies are resolved on a need-by-need basis:
The compile time ones for the application's code are downloaded as a very first step when you start the build.
Any plugins required are downloaded afterwards during the respective phases. This also includes their transitive dependencies.
The dependencies for the tests are downloaded when the tests are to be compiled and executed. These are the dependencies with <scope>test</scope>.
Therefore, at the point where you're at the test phase, you already must have the latest dependencies, unless you have them cached locally and installed and you're in offline mode.
To resolve all your dependencies, you can do:
mvn dependency:go-offline
To resolve all the plugins, you can do:
mvn dependency:resolve-plugins
maven surefire plugin gets downloaded only for mvn test because default Goal execution phase for this plugin is test.
I guess if you want to execute surefire plugin you can specify goal execution phase.
We have the following approach for filling the Maven cache on our Jenkins slaves:
Do a full build
Zip the ~/.m2 folder (which contains the cached instances)
Place it and extract it on the machine where you want to run your "warmed-up" tests
Automate steps above steps to keep it recent and up to date.
Related
Trying to download all maven dependencies for a complex project to speed up subsequent builds. There are some dependencies that are not outlined on pom.xml, such as com.apollographql.apollo:apollo-compiler which are triggered as part of the generate-sources phase (from a dependency I do include apollo-client-maven-plugin)
my original plan of using maven dependency:go-offline did not work because these jars are not included.
i'm currently using the work around maven dependency:go-offline generate-sources which ensure that those extra jars are downloaded, but now i have to wait for the actual generation of the sources...
Is there a way to just force maven to detect dependencies fully?
Alternatively, any way to have incremental mvn --offline, i.e. uses the jar locally without updating repository info, but do download missing ones?
I added to my maven project the a PMD and checkstyle plugins. And when I run them the work perfectly. But when I remove them from the pom.xml I can still run mvn checkstyle:checkstyle or mvn pmd:pmd even though I removed them. Also after removing them I ran mvn clean install. ANy idea of what could happen ?
The commands you execute are plugin goals (plugin:goal) and unlike "mvn install" not a phase.
you can run almost any plugin on a project if maven can find it. The apache maven plugins allow that shortcut notation (pmd:pmd) since maven will try to resolve them in the apache namespace.
Plugins from other sources would need to be run with their full name, for example:
org.codehaus.mojo:versions-maven-plugin:2.5:display-dependency-updates
The plugin itself decides if it can run a goal on its own or if it requires a running reactor and only works within the maven life-cycle (usually because it depends on outputs from other phases)
So in your case: mvn install should not run the pmd plugin anymore if its not in the pom - and install is a phase. mvn pmd:pmd will run it directly with its default config - since pmd:pmd is a plugin goal.
The default plugins per packaging and phase are documented here. These may run if in the pom or not (depending on whats in the project).
I cloned the git repository of Apache ActiveMQ Artemis project (https://github.com/apache/activemq-artemis) and then typed
mvn -Ptests test -pl :integration-tests
I was surprised to see log messages like the following
...
Downloading: http://repository.apache.org/snapshots/org/apache/activemq/artemis-selector/1.4.0-SNAPSHOT/artemis-selector-1.4.0-20160625.030221-11.jar
Downloading: http://repository.apache.org/snapshots/org/apache/activemq/artemis-core-client/1.4.0-SNAPSHOT/artemis-core-client-1.4.0-20160625.030211-11.jar
...
Since e.g. artemis-core-client is contained in the git repository I cloned in the beginning, I'd have expected maven just builds it from there.
That way, when I make changes in the core client source, they get picked up by the integration tests.
Instead, maven is downloading the jar from the repository.
Question: How do I configure maven to always build all modules that are in the git repository and download only "true" dependencies, which I mean things not in the git repository?
You are not executing the Maven build on the main project, on the main pom.xml which indeed defines the artemis-selector and artemis-core-client modules, among others.
You are executing the Maven build on the tests and its pom.xml, where only tests modules are defined. This is a side/test project, which has as parent the previous pom file, but it doesn't play any role in its parent modules definition. Hence, dependencies are not resolved as modules but as Maven dependencies.
You should firstly install (via mvn clean install) the former project, so that libraries will be available in your local Maven cache (hence no downloading would be triggered), then execute the tests project.
Check the Maven docs for a inheritance vs aggregation difference to further clarify it.
From the Stack Overflow, the follow threads could also be interesting:
What is the difference between using maven -pl option and running maven from module level?
Maven multi module project cannot find sibling module
I have a multi-module project with a parent pom.xml and several modules where some of the modules depend on each other. In the project directory I can call
mvn test
to run unittests in each module. No problem here. But if I call
mvn site
one of the modules reports
[ERROR] Failed to execute goal on project myModule_C: Could not resolve dependencies
for project org.myModule_C:jar:0.0.1-SNAPSHOT: The following artifacts could not be
resolved: org.myModule_A:jar:0.0.1-SNAPSHOT, org.myModule_B:jar:0.0.1-SNAPSHOT: Failure
to find org.myModule_A:jar:0.0.1-SNAPSHOT in http://artifactory-server:8081/artifactory/repo
was cached in the local repository, resolution will not be reattempted until the update
interval of server has elapsed or updates are forced -> [Help 1]
I think this should not happen since these dependencies are found during "mvn test". Also, they are not in the artifactory-server but part of the parent project. The goal that is mentioned in the ERROR is the goal site. Why does the mvn test succeed (with respect to dependencies it finds) and mvn site does not? Do I have to build the site in a special way - because this is a reactor build?
You should execute the mvn install as least once. Please see further information at the Maven Build Life Cycle and Maven in 5 Minutes.
Here is the overview
Maven Phases
Although hardly a comprehensive list, these are the most common default lifecycle phases executed.
validate: validate the project is correct and all necessary information is available
compile: compile the source code of the project
test: test the compiled source code using a suitable unit testing framework. These tests should not require the code be packaged or deployed
package: take the compiled code and package it in its distributable format, such as a JAR.
integration-test: process and deploy the package if necessary into an environment where integration tests can be run
verify: run any checks to verify the package is valid and meets quality criteria
install: install the package into the local repository, for use as a dependency in other projects locally
deploy: done in an integration or release environment, copies the final package to the remote repository for sharing with other developers and projects.
There are two other Maven lifecycles of note beyond the default list above
They are:
clean: cleans up artifacts created by prior builds
site: generates site documentation for this project
I hope this may help.
We use the maven dependency plugin (with maven3) to unzip dependencies at runtime for running a set of tests. However, because the number of dependencies and their size is large it can take upto 30 minutes for the download to finish (not all teams are on LAN). I checked with the maven docs and maven only downloads dependencies that belong to seperate groups in parallel. Is there any way that I can force maven or maybe the maven dependency plugin to download these dependencies in parallel?
Have you already tried to use
mvn -T3.0C phase