My task is this:
Take a dependency, which is a zip
Unpack the dependency
Run some Java code, add some new files
Assemble the dependency back, including the new files
After assembly, build it as another artifact.
My queries:
The dependency coordinates need to be provided dynamically during build time. Am looking for something like SystemProperty, but not sure how to take them through my pom.
Unpacking is straightforward. unpack-dependencies is all I need.
Running Java code - Here's a hiccup again. This project doesn't produce a jar or a war, it's got to be a pom packaging. SO I cannot compile code in the project. I can use the compiler plugin with Java execution. Not sure it's a good practice.
Not a problem to assemble, if I can get uptill here.
Any suggestions for points 1 and 3?
Thanks for the help.
Alot of the problems with Maven arise from the decision about which build "tasks" translate into discrete Maven builds. The general idea is that each jar, zip, or whatever your building should get it's own mave build ( aka pom, aka module ). Looking at your bullets, I believe you have three maven builds, i.e. three poms.
1) The first pom will build your Java utility, i.e. jar. This jar then becomes a dependency of your other build that needs to execute it as part of the zip assembly process.
2) The second pom will do build the zip. ( I guess you will actually build several different ones; most likely these are then separate poms -- while you can have a single build produce more than one artifact, leveraging the concept of attached artifacts and classifiers, it's an unnecessary complication that should be avoided unless you know you need it. ) These zip assembly poms unzip the primary zip, massage the content via the java util( running the jar via maven-exec plugin), and reassembles the zip.
3) If you need to change the name of an artifact ( your zip ) when it's consumed in the production of another artifact ( your framework ), then you can use the dependency plugins' copy goal. It allows you to specify new names for the dependencies that you are pulling into your build -- but note you aren't actually changing the maven coords, which, I believe, is something to be avoided like the plague. :)
Related
I have a multi-module project, i.e.
parent
module1
module2
In one dev cycle, I added a class mod1.A to module1. Class mod2.B in module2 depends on it.
I do not have the artifacts in my local .m2/repository. Running this:
$ cd prj/module2
$ mvn -o exec:java -Dexec.mainClass=mod2.B
results in an error along the lines of:
The following artifacts could not be resolved: com.example:module1:jar:1.0-SNAPSHOT
After I install the artifacts via mvn install while in the prj folder, it all works as expected.
However, this presents an issue in at least two ways:
I have to go through the slower install phase instead of the faster compile phase
I have two versions of the same project and conflicting modifications in these. I cannot run the same Java class with their respective modifications, only the currently installed modifications, considering they are both the same SNAPSHOT version
There are workaround for both (skip parts of the build for the first, different snapshot versions for the second), but they are far from usable in practice.
Is there a way to make maven use the local modules, instead of using artifacts from local maven repository?
If I understand your question correctly, it seems like you are living a bit outside the norm here: you have two local "copies" of the project with different modifications, that you want to work with alternately when running "exec:java". And Maven is getting in your way: it expects your local .m2 repository area to be in play, but the version strings in each copy are the same, so you end up with the changes interfering among the copies.
To me, it sounds like what you are trying to do is to test your changes. I suggest you just write an actual JUnit or TestNG test in module2 that tests what you want (it can just call mod2.B Main if you want). Then, from your chosen project directory, you can run mvn test -Dtest=MyTestName. It won't "install" anything and it will find the dependencies the way you want it to.
Otherwise, I can see three options.
Change the version string locally in one of the copies (mvn versions:set -DnewVersion=B-SNAPSHOT can do this for you). That way any "installed" jars from your work on that copy will not be considered by the other copy, and vice-versa. You refer to this as being "far from usable" ... I think it should be fine? These are different versions of the project! They should have different version strings! I strongly recommend this option out of the three. (You can do mvn versions:revert when done if you used :set, or you can rely on version control to undo the change.)
Select a different local repository used by Maven when working on one of the projects, with a command-line flag as per https://stackoverflow.com/a/7071791/58549. I don't really think this is a good solution, since you would have to be very careful about using the right flags every time with both projects. Also you'd end up having to re-download Maven plugins and any other dependencies into your new local repository anyway, which is kind of a waste of time.
Try to avoid using any local repository at all. You seem to be trying to make this option work. I don't think this is a great approach either; you're fighting against Maven's expectations, and it limits your flexibility a lot. Maven will indeed find dependencies from the "reactor" (i.e., the executing mvn process) first, but this means all of the required modules must be available in the reactor to be found, which means you can only run mvn at the top level. So if instead you want to just do "mvn exec:java" inside a single module, mvn needs to find that module's dependencies somewhere ... and that's what the local repo is generally used for.
If you're dead set on going with option 3 (instead of option 1), then I suggest you follow the comments on your question and create a profile that runs your exec selectively against module2 and binds it to a lifecycle phase. But this is in practice very close to just wrapping it with a test.
For IntelliJ users:
I solved this problem using IntelliJ's Run configuration. It has the options Resolve workspace artifacts and Add before launch task -> Build. See this picture for clarification:
Run configuration example
The whole point of modules in Maven is to create decoupling between them. You either build each module independently, so that you can work on one module without touching the other, or include both modules as sub-modules in the parent pom and build the parent, which will resolve dependencies between its sub-modules and trigger their builds.
It looks like you have two options here:
Review the structure of your project. Do you really need to split it into two separate modules, if you change code in both of them simultaneously?
Import the project into a Maven-aware IDE (IntelliJ IDEA is very good at working with Maven), and let the IDE handle the compilation. Once finished and stabilized the code-base, build normally with Maven.
I have a project that depends on Jacob. The artifacts on Maven central are quite outdated, so I would like to download them directly from SF. But I don't know how to do that with gradle.
More specifically, what I try to achieve is:
Download the distribution .zip from SF.
Unpack the distribution and extract the .jar (with a rename so that the .jar file name contains the version) and the two .dlls.
Add the extracted .jar to the compile configuration.
The tests have a runtime dependency on the .dlls.
The final project .jar should include both the jacob.jar and the .dlls.
This is a quite open-ended question, and I think I know how to do some of the points on the list. Task 2 is probably just a copy task, and task 5 is just about telling gradle what goes into the jar. Task 1 seems like something that should be easy, but I don't think that the copy task accepts a URL as source.
What I really don't know how to do, is task 3. How can I add a derived resource to a configuration, and make it trigger a task if it is absent? It is like normal dependency resolution, but where the included dependency is derived from another, transformed, dependency.
I realize that the easy way out is to just include the Jacob dependencies in my project, but I really don't like to pollute my SCM repositories with binary artifacts :-)
For step 1, you'll have to write your own download task (should be easy with Groovy). Step 3 can already be done in the configuration phase, provided you know the path to the unzipped Jar beforehand. So you'll just add a regular file dependency. You are right about step 2 and 5. Step 4 will probably configure the test task (e.g. perhaps you'll have to set a system property for the native code location). What remains is to establish the necessary task dependencies.
The entire java project has an ant build; however couple of module(s) have maven build too.
My new module (maven built, say A) has dependency over an existing module(or simply a folder?, say B) which is being built using ant which just packages the src into jar and drops it inside the project.
Maven build for module A fails (unable to locate moduleB files); Options -
1. Package module B using maven, push to m2_repo
I do not want to go with this option.
Please let me know what are the other options available for the same.
If you have control over the source of all your modules, and if you decided that Maven is your way to go forward, then I recommend to to go all-in as soon as possible. If you do not then you will have continuous problems for the two build strategies to play along. And not only during build time but also at deploy time when it is time to collect all your runtime dependencies. Unless you build one uber-JAR as your only deployable artifact.
If your modules will (almost) always release in sync, then consider using a multi-module project setup.
When I say "all-in" then I do not mean that you have to give up Ant completely. You can use the maven-antrun-plugin to kick of your existing ant builds.
You should also consider running your own repository server, e.g. Nexus, to take full advantage of your maven builds.
I have a java application.
I can run the Maven Release task which will do some nice things for me:
Change Version number from 1.0.0-SNAPSHOT to 1.0.0
Increment the version number in my pom to 1.0.1-SNAPSHOT
Tag the release in source control
Upload the resulting package to my maven repository
I'd like to take things a step further. I have some post-build steps that I'm currently doing manually.
Update the launch4j configuration xml file with the appropriate version
Wrap the resulting jar in an executable using launch4j
Copy the resulting EXE into a package directory
Copy several supporting files into the package directory
Zip the package directory up
Email the package to my testers.
Eventually I'm going to have the additional task of building an installer leveraging the package directory.
I don't know that maven or ant are the right tools for automating my remaining 6 tasks, but it looks like either one or a combination of both could potentially accomplish what I need.
I could probably write a batch file or a simple perl script to do these things quicker than figuring out how to do them, but I would prefer to keep things as standard as possible so that I'm not taking on the additional responsibility of supporting a hack of a release process perpetually.
It seems to me that these are tasks that might not be standard part of build/release, but are commonly seen enough that there should be a best/most common practice for accomplishing them.
I would suggest to use the maven-assembly-plugin as well as the maven-launch4j-plugin during your build.
Update the launch4j configuration xml file with the appropriate version
put a placeholder into the configuration xml and let maven replace it during the
the build.
Wrap the resulting jar in an executable using launch4j
use the launch4j-maven-plugin to create the executable.
Copy the resulting EXE into a package directory
I would suggest to put the resulting artifact into a repository manager instead
of a separate folder, cause in Maven all artifacts are stored within
a repository. It might be necessary to setup your own repository manager
(Artifactory, Nexus, Archiva).
Copy several supporting files into the package directory
Using them as resources (src/main/resources) they will be copied
automatically.
Zip the package directory up
Use the maven-assembly-plugin to create a resulting zip file.
Email the package to my testers.
You can use a CI like Jenkins etc. to send the final mail or
you can take a look into maven-changes-plugin which might be solution.
This means all your mentioned steps can be handled by Maven during a usual build. This means in the end you can use the maven-release-plugin to produce a full release which contains all the above steps and produces all the wished artifacts.
If I were you, I would try a combination of the following:
Maven release plugin, unless it is not flexible enough for SCM related processes. If using SVN as SCM, I would use directly SVNKit(inside a custom Maven plugin), if flexibility is a concern.
Maven launch4j plugin.
Maven assembly plugin
Maven Ant Run plugin and/or one or more in-house Maven plugins for the remaining tasks.
I am attempting to create a way in which hermetic builds can be achieved while still relying on SNAPSHOT dependencies in your project.
For the purposes of example, say I have a project which has a dependency structure like this:
┌ other-1.2-SNAPSHOT
mine-1.2.3 ──┤
└ thing-3.1-SNAPSHOT ── gizmo-6.1.3-SNAPSHOT
What I would like to do is resolve all the SNAPSHOT dependencies locally to something which is related to my current version and then deploy those as releases to my Nexus' release repository. Not all of these dependencies are internal so I cannot simply just make a release on each.
So, in this example, other-1.2-SNAPSHOT would become something like other-1.2-mine-1.2.3 and thing-3.1-SNAPSHOT would become thing-3.1-mine-1.2.3. This is relatively trivial in about 60 lines of python.
The problem, however, is in resolving transitive SNAPSHOTs to concrete versions. So I also need to convert gizmo-6.1.3-SNAPSHOT to gizmo-6.1.3-mine.1.2.3 and have thing-3.1-mine-1.2.3 depend on it.
This is only an example of one way in which to achieve what I want. The goal is that in a year or two down the road I can checkout my release branch for version 1.2.3 and be able to run mvn clean package or the like without having to worry about resolving long-since-gone SNAPSHOT dependencies.
It's important that this branch be compilable and not just retain all dependencies using something like the jar-and-dependencies functionality of the assembly plugin. I'd like to potentially be able to modify the source files and make another release build (e.g., applying a hotfix).
So,
Is there anything like this available that will be able to convert SNAPSHOT dependencies in a recursive fashion to be concrete?
Are there any plugins which manage this kind of thing for you? The release plugin had promise with some configuration options on its branch goal but it doesn't resolve external deps to the degree that I want.
Are other techniques available for creating hermetic Maven builds?
This is not a widely used technique, but you can always check your specific SNAPSHOT dependencies into your project as a "project" repository, as described in this blog post: Maven is to Ant as a Nail Gun is to a Hammer
In short, use the Dependencies Plugin to create repository located in your project directory. The below is copied from the linked blog post (which you should read):
1) Run mvn -Dmdep.useRepositoryLayout=true -Dmdep.copyPom=true dependency:copy-dependencies
"This creates /target/dependencies with a repo-like layout of all your projects dependencies"
2) Copy target/dependencies/ to something like libs/
3) Add a repository declaration like the following to your POM:
<repositories>
<repository>
<releases />
<id>snapshots-I-need-forever</id>
<name>snapshots-I-need-forever</name>
<url>file:///${basedir}/libs</url>
</repository>
</repositories>
You make this an automated part of your build/release process: step 1 by configuring the Dependencies plugin to a lifecycle phasephase, and step 2 using AntRun Plugin to move the downloaded dependencies to the right place..
Hope this works for you. I have to go take a shower now...
The maven versions plugin will do most of what you want.
http://mojo.codehaus.org/versions-maven-plugin/
However you will almost certianly need to run it in a pre-build step in which you resolve all the dependencies and update the pom file accordingly. Then re-run maven (which re-reads the pom) to run the real build. You might be able to configure everything within the pom itself triggered with a separate goal thus avoiding a separate script.
This works better if you use particular versions instead of SNAPSHOT dependencies and let the pre-build step upgrade them if necessary. The only real difference for dependency resolution is that maven will always re-download -SNAPSHOT dependencies whereas it will only download normal dependencies if there is a new version available. However many plugins (including the versions plugin) treat -SNAPSHOT dependencies differently causing problems. Since every CI build has a new version number I never use -SNAPSHOT, prefering a different tag like -DEV with more predictable behaviour for things like developer local builds etc.
I've spent a lot of time getting maven to do things similar to this. Most maven projects I know have some kind of pre-build step in order to set version numbers or get around other limitations such as this. Trying to do all this in one step usually fails because maven only reads the pom once, string substitution doesn't work in a few places and the deployed/installed pom doesn't generally doesn't contain the results of string substituion or changes made during the build.