Forgive me if this is remedial, but I am still new to Maven and it's functionality.
In my project, when it "builds" and gets to the compile phase, it will create a target directory with just compiled libraries and update (or create if not there) the local .m2 directory.
When I get to the "test" phase, I want it to build against the target directory's library files, and not the local .m2 directory.
Any hints, recommendations, or suggests would be greatly appreciated. Thanks!
Maven has this concept of “the reactor”, which is just a fancy term for the list of projects being built. At the start of a Maven build, and at the end, Maven prints out this list of projects (using /project/name if defined or groupId:artifactId otherwise).
For each project in the reactor, Maven maintains a list of artifacts that have been attached. By default, each module's pom.xml is attached, and as each plugin runs, they have the option of attaching additional artifacts. Most plugins do not attach artifacts, here are some plugins that do:
jar:jar creates a .jar and attaches it
war:war creates a .war and attaches it
source:jar creates a .jar of the source Java code and attaches it with a classifier of source
java doc:jar creates a .jar of the JavaDocs ad attaches it with a classifier of javadoc
There is also a default primary artifact (this is the one that gets replaced by jar:jar) which is actually a directory and not a file, as such it will not get installed or deployed to the local repository cache or a remote repository.
So when in the reactor, and a plugin that attaches the primary artifact has not run yet, and another plugin asks for the primary artifact, it will be given the directory ${project.build.outputDirectory}. If after the primary artifact as been attached, then that primary artifact will be provided.
The test phase happens before the package phase, so will use the directory and not the .jar. The integation-test phase happens after, so will always use the .jar.
Things get more complex in a multi-module project (which is where my long intro should help you out)
Maven has to build the test classpath. If one of the dependencies is within the reactor, Maven will use the artifact attached to the reactor. Otherwise it will use the local cache (populating from the remote repositories if necessary).
When you run
mvn test
In a multimdule project from the root, there is no replacement of the default (directory-based) artifact, so intra-module classpath will be to the target/classes directories.
When you run
mvn package
In the same project, however, because each module completes its life cycle sequentially, all the dependent modules will have swapped in their .jar files as their attached artifact.
All of this should show you that Maven is doing the sensible thing. Hope this has helped.
The test phase is going to execute tests on your project. The project won't reference itself via the dependency mechanism. Only dependencies will be referenced via your local repository, i.e. .m2/repository
Also, it's not the compile phase that installs the artifact to the local repository, it's the install phase. And, then, there's a later phase, called deploy, that will deploy the artifact to a remote repository, provided you have a remote repository configured as the deploy target. Note, install and deploy are nearly identical phases except install is a local only thing; thus, it's the common build phase to hit when doing dev environment work. Normally the build server will do the deploy stuff.
Related
I would like to know if there any workaround about having multiple local repositories used by order.
Example :
Default local repository : ~/.m2/repository
Work repository : /var/tmp/m2LocalRepo
When running Maven, to resolve dependencies I would like that at first it looks for the artifact in /var/tmp/m2LocalRepo if it does not find it it looks in the default one.
There is the issue MNG-3655 – Allow multiple local repositories which talks about the same problem but has not been resolved yet.
UPDATE
My use case is as follows:
An application is represented by 2 maven projects projectA and projectB which have the same version.
projectB is a dependency of projectA
when developing a new feature, we create a development branch on each project with the same name. However, we do not modify the version number which remains the same as the main branch.
During a build, on Jenkins for example, I would like to build projectB first but install its artifacts in another local repository (for example, /var/tmp/m2LocalRepo) and not on the default repository so as not to disrupt other builds that also depend on projectB. Then, during the build of projectA, I would like it to get the projectB dependency from /var/tmp/m2LocalRepo and the other dependencies from the local repository.
Re "During a build, on Jenkins for example":
Jenkins Maven projects have an option Build → Advanced... → ☑ Use private Maven repository with its inline help:
Normally, Jenkins uses the local Maven repository as determined by Maven — the exact process seems to be undocumented, but it's ~/.m2/repository and can be overridden by in ~/.m2/settings.xml (see the reference for more details.)
This normally means that all the jobs that are executed on the same node shares a single Maven repository. The upside of this is that you can save the disk space, but the downside of this is that sometimes those builds could interfere with each other. For example, you might end up having builds incorrectly succeed, just because your have all the dependencies in your local repository, despite that fact that none of the repositories in POM might have them.
There are also some reported problems regarding having concurrent Maven processes trying to use the same local repository.
When this option is checked, Jenkins will tell Maven to use $WORKSPACE/.repository as the local Maven repository. This means each job will get its own isolated Maven repository just for itself. It fixes the above problems, at the expense of additional disk space consumption.
When using this option, consider setting up a Maven artifact manager so that you don't have to hit remote Maven repositories too often.
If you'd prefer to activate this mode in all the Maven jobs executed on Jenkins, refer to the technique described here.
and the options:
Default ...
Local to the executor
Local to the workspace
You can copy projectB's artifacts using the Maven Resources Plugin (as described in this answer to Maven, how to copy files?) to projectA's local repo on demand.
Circumstances
It would be nice to split long running build jobs, espescially in multi module projects.
Compile everything mvn clean install or mvn clean package
Execute unit tests on each module mvn surefire:test
Execute integrations tests ant and stuff
Publish artifacts to a remote repository mvn my.own.tools:publish-plugin:publish
Execute post build steps (Tagging, etc) build server stuff
In some build environments, like Atlassian Bamboo, each step will be likely executed on another build agent as the previous one. They can even have their own local repository each. However, it is possible to copy all the files of the working directory to that of a subsequent stage.
Observation
Maven uses the contents of the target folders before it makes a lookup into the local repository or a remote repository. This is true when the contents of the target directories are created by a previous phase during a specific run.
Example: mvn clean install surefire:test
If a test has dependencies, maven will look at first in the target directories which have been created during the compile phase.
If the command is split into two, it seems that maven does not recognize the target folders at all.
Example: mvn clean install; mvn surefire:test
Now maven loads all the dependencies from the local repository or, if they are not there, from a remote repository.
Problem(s)
During step 2 maven ignores the contents inside the target foldera, which have been created during step 1.
Step 2 runs on a different build server, than step 1. However, the whole file structure (including the target folders) from step 1 have been copied into the working directory from step 2. The artifacts which the tests depend on, are not taken out of those directories. Since each build server has it's own local repository, they are not found and on the remote repository they have never been uploaded.
Any idea of making our whole beast more modular and split up the project is unfortunately no possiblity
Question
How can maven be forced to load dependencies out of existing target folders of a multi module project under any circumstances? I understand that maven is not aware of those folders in step 2 since they are not created during that maven run. But how can I force maven to look if those folders exists?
More elaborted problem ;-)
Our "Publish-Plugin" in step 4 seems to work mostly fine and uploads the artifacts out of the target folders. The project iformation is gathered inside the Mojo by usual maven properties. But there are some zip files created during step 1 by the org.apache.maven.plugins:maven-assembly-plugin. Those zip files are not found, although they are residing inside the target folder. Now maven tries to download them, what is funny, because it does that to have them in hand for a upload.
I have a question about how the Maven dependency resolving mechanism is working in a multi module project.
Normally I only use 'mvn clean install' when I build my multi module projects and my assumption was that if any module in the project needs a previous module, dependency will be resolved by going local repository and loading the corresponding 'jar'.
For project internal reason, I have to use 'mvn clean compile,' this command naturally does not create any 'jar' while 'install' is not there. So here I started wondering, how the dependency resolution for a multi module project works, while jar' is not created but project still able to see the changes from the previous builds. Does the target directories used for dependency management?
Or for 'mvn clean compile' target directory used but for 'mvn clean install' the local repository.
Can anybody explain me how the dependency resolution works in a 'multi module' project.
Thx for answers.....
I think you will understand better if you look at https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
There is a life cycle in the process of building the jar. The compile target will compile the code and create a complete classes folder in your target directory. This target will resolve all your dependencies in your poms and download any dependencies to your local repo, not already there.
The install target will create the jar from the classes directory and install it in your local repository.
I really think you will need to run the install target to get anything useful.
Maven is made of separate components.
There a component that deal with a given module and among other things try to get its dependencies. It ALWAYS get the dependencies from the local repository, eventually after having downloaded such dependencies. If the dependencies are not there and can't be dowloaded, it will fail. Eventually the module will create its own artifact that it will publish to the local repo.
Then there a compoment that when you ask it to build several maven modules, for example calling mvn at the root of a project does order the various module use the dependencies to find the best ordering for the build so that if a given module depend on another, it will be build after the module it depend on. It then call the previous compoent I described, building each module in order.
In all cases, a given module dependencies are always taken from the local repo. The expectation is that the previous modules that were built before actually pushed their artifact to the local repo typically with the mvn install but you could force it do it at any step thanks to proper configuration (may not be a good idea).
In all case if the previous component jar was not built and put into the repo, there no way such jar can be added in the classpath for the next module to be compiled.
Doing compile only on multiple projets isn't going to be any useful.
In our company, thousands projects are build on 3 servers, with mvn commands.
A few projects occasionally pack too many jars to its WEB-INF/lib folder, the unwanted jars looks like another projects business code and its dependencies.
This is
the diff in WEB-INF/lib between right one(left) and too many jars one(right)
The jar in red frame looks like another project' jars, project name is "jd-common", and the other green jar on right is another project's dependencies.
This situation always reappeared until I clear local repository.
I guess the another project uses "mvn install" to install jars into local repository on build server, and our project is actually depend on jd-common-cached and jd-common-util only.
How can I avoid this?! Thanks for help.
First of all if the jars are there - you depend on them. You may depend on them implicitly (transitive dependencies). Run mvn dependency:tree to list all the dependencies (including transitive). You may find out that you depend on another project that in turn depends on those red/green jars.
Second, on the Build Server you don't want to share local repo with other projects. That's why, at least in Jenkins, there is an option Use Private Repository - this way all the project are going to be separated. This protects you from the situation when the artifact is not in remote repo anymore but the build is still green since that artifact is in local repo. But this has nothing to do with the problem you described.
It's finally be resolved!
The project A depend on a jar which deployed by another project B, the depend jar is a sub-module in B. (A->B)
Unfortunately：
1. A and B are packaged on the same build server
2. B's sub-module jar has the parent config in its pom.xml.
3. B use "mvn clean install -DskipTests" as the build command, so all the B's modules are installed in local repository.
Maven always package the local installed jar, and use the installed jar's pom file to find the sub dependencies, so when maven is executed, project A found that:"one of my depended jar has a parent, it's B, and all the B's sub-modules are found in local repository because of B installed all of them, I should package them all!".
Main Goal: deploy a project as jar and eclipse-plugin
current state: project builds fine as jar package
Now i want to create a second project which wraps the jar project as eclipse plugin
use tycho-maven-plugin to create eclipse-plugin
add the jar of the original project (with copy-dependency)
add an Activator
export packages from jar
create correct MANIFEST.MF
i tried to copy the jar with copy-dependencies bound to create-resources. This works as long the jar is found in repository, but the local project gets ignored.
This results in a build failure since the jar is not found.
Is it possible to tell copy-dependencies to take the jar from the target directory of the project? Or should i use some other method than using tycho?
Edit:
I solved my problem with 4 projects:
normal project (nothing special here)
the wrapper project using tycho maven and copy-dependencies.
bound copy dependencies to some goal before compile (e.g. generate-resources). Excluded all artefactid which were set as dependency in the MANIFEST.MF.
a prepare project, which calls the normal project and installs it into the repo. This is needed because the tycho-maven-plugin is bound to validate and it is not possible to call the exec plugin beforehand (at least not easy).
a multi module project which calls the prepare project before the wrapper project.
Build your local project (which artifact was missed) with "mvm install". It will be deployed in your local repository ($USER_HOME$/.m2/repositories). After this dependency should be resolved.
Alternatively you can "mvn deploy" if you have local company maven repository like Artifactory or Nexus.