maven-dependency-plugin unpack goal: do not overwrite existing files - maven

I do not ever want maven-dependency-plugin:3.1.2:unpack to overwrite existing files in any circumstances. This is my current pom.xml configuration:
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<version>3.1.2</version>
<executions>
<execution>
<id>unpack-zip-files</id>
<phase>generate-test-resources</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>com.foo</groupId>
<artifactId>bar</artifactId>
<version>${foobar.version}</version>
<type>zip</type>
<classifier>exe-archive</classifier>
<overWrite>false</overWrite>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
<includes>**/*.exe</includes>
</artifactItem>
</artifactItems>
<overWriteIfNewer>false</overWriteIfNewer>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>false</overWriteSnapshots>
</configuration>
</execution>
</executions>
</plugin>
Scenario:
Projects foo and bar
foo has a .exe file as build artifact, inside a ZIP file
bar runs the .exe file during tests
In situation X, bar must use the snapshot version of the mainline development branch of foo. In that case, the directory target/lib shall be empty before mvn install. This is guaranteed by starting from an empty workspace and running mvn clean as a separate build step. This question is not about situation X.
In situation Y, bar must use a custom build of a feature branch of foo. This is done by unpacking the ZIP file with the .exe to the directory target/lib in a separate build step between mvn clean and mvn install.
We are working in situation Y, where the target/lib directory is already pre-filled with the .exe from the correct feature branch.
This is the target/lib directory before mvn is run:
+ ls -al fooBar.exe
-rw-rw-r-- 1 ubuntu ubuntu 18368427 Apr 12 21:27 fooBar.exe
+ md5sum fooBar.exe
03acc8b5c3da700c31efcd6635feb56a fooBar.exe
This is the target/lib directory after mvn is run:
+ ls -al fooBar.exe
-rwxrwxr-x 1 ubuntu ubuntu 18368393 Apr 11 23:10 fooBar.exe
+ md5sum fooBar.exe
ab6dd45c5cc4e41534ad2363c5767601 fooBar.exe
The change in md5sum is hard evidence that the existing fooBar.exe was overwritten by Maven.
Maven command used:
mvn --global-settings /home/jenkins/workspace/bar#tmp/config15592668079584895681tmp \
-Dmaven.repo.local=/home/jenkins/workspace/bar/.repository \
install \
-DgsExec=/usr/bin/gs -DcompareExec=/usr/local/bin/compare \
-Dtest=RunCucumberTest -Dcucumber.options=--plugin json:target/cucumber.json
Expected results
The mvn install command shall not overwrite existing files when overWrite, overWriteIfNewer, overWriteReleases, overWriteSnapshots are all set to false.
The md5sum of fooBar.exe shall be the same before and after running mvn install.
Question
Which magical incantations do I need to add to pom.xml so that existing files are never overwritten in any circumstances?
Documentation referenced
Apache Maven Dependency Plugin – Usage – Overwrite Rules
Apache Maven Dependency Plugin – dependency:unpack

OK, I think I know what might be happening: Like I said in my comment:
I just read your message on the mailing list and tried in one of my own projects, also using plugin version 3.1.2. Actually, just specifying <overWrite>false</overWrite> inside the <artifactItem> was enough to avoid overwriting. I just executed the unpack goal once, then manually modified an unpacked file and it did not get overwritten. I even see my-artifact-1.3.jar already unpacked in the log.
I continued experimenting some more and noticed that even when deleting many unpacked files, they will not be recreated, so the check must be on a more global level, not on a per-file basis.
Even when deleting all files or the whole output directory, the dependency will not be unpacked again. That was a sure indicator that some kind of meta information must be stored somewhere outside the output directory. The first place to look for it was of course the target directory, and obviously enough, in subdirectory target/dependency-maven-plugin-markers there are (empty) marker files like my-dependency-1.3.marker. Deleting one of those files has the effect of the dependency getting unpacked again during the next build, overwriting possibly existing files.
So the way for you to solve this problem is to avoid cleaning the target directory or at least to make sure to keep the corresponding marker file.
Update: You could also create the marker file by yourself if the EXE file you want to protect exists and if for some reason your build workflow needs the clean in between. But the latter would be a bit ugly, you should try to avoid it. With Antrun or some Beanshell or Groovy scripting it would be possible, though.
Somewhat more elegant would be a profile with auto-activation if the EXE file does not exist, then putting the dependency plugin inside the profile, i.e. it would only get active if the EXE does not exist in the first place.

Related

How to best copy files within a maven build?

After updating maven-resources-plugin to version 3.2.0 to get the fix for maintaining file permissions (x-bits), the copy fails when encountering a symbolic link of the form file -> ../file (and possibly other symlinks too):
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.2.0:copy-resources
(copy-jdk) on project preinstall: <target>/jre/lib/amd64/server/libjsig.so -> [Help 1]
ll <source>/jre/lib/amd64/server/
lrwxrwxrwx 1 user users 13 Oct 22 15:09 libjsig.so -> ../libjsig.so
...
The target's jre/lib/amd64 folder received some files, but not yet libjsig.so. <source>/jre/lib/amd64/libjsig.so exists, therefore the symlink's target would be created if the plugin did not abort.
Can anyone tell me if that's a bug or a feature? Any configuration / idea / workaround that allows to keep the source's file permissions and have symbolic links copied?
Here's the relevant pom.xml part:
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<!-- version 3.2.0 stops on symlinks in the jdk, when the symlink target is not yet copied -->
<version>3.1.0</version>
<executions>
<execution>
<id>copy-jdk</id>
<phase>process-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${product.install.folder}/jdk</outputDirectory>
<resources>
<resource>
<directory>${jdk.tmp.folder}/${jdk.directory}</directory>
<filtering>false</filtering>
<include>**/*</include>
</resource>
</resources>
</configuration>
</execution>
...
Edit: BTW, creating a dummy target at <target>/jre/lib/amd64/libjsig.so helps to continue with the copy task, but then, that dummy is not replaced during the copy (probably because it is newer) ...
Edit2: Playing around with the above dummy idea and the <overwrite> option. That option is not usable if the file tree contains read-only files. <overwrite> does not delete and recreate the files but tries to rewrite them, which fails for read-only files.
Therefore I think, the maven-resources-plugin might be the wrong approach to copy files around - here's the evolved question:
What is the best approach to copy a larger set of files (such as a jdk, which includes read-only files, executable files and symbolic links) from within a maven build?

Make maven clean not fail when only folder remains

While developing an application, the command I use most often is mvn clean install. Cleaning probably isn't needed 90% of the time, but it does not hurt and might help to avoid weird issues. There are however times, when I'm working on a console application, when I have trunk open on one terminal, and target on another.
mvn clean in such a case does what I need it to - it deletes every file within the target folder - and then fails due to lock on the folder itself. Is there a way to tell it, that in such a case it should just ignore/skip deleting the folder itself and continue with install?
Yes, you can configure the maven-clean-plugin to ignore errors with the help of the failOnError attribute. It defaults to true, meaning that the plugin will fail on error.
Sample configuration to disable this:
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<failOnError>false</failOnError>
</configuration>
</plugin>
You can also do it directly on the command line, without changing the POM, by setting the maven.clean.failOnError user property:
mvn clean install -Dmaven.clean.failOnError=false
Note that this make the plugin ignore all errors, however it is not currently possible to make it ignore only certain types of errors.

Maven webapp with non-default web resource directory

Maven sets the default webapp directory to src/main/webapp as per http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html.
We use a IDE-configured server for development, which uses the files from this directory to serve. The server doesn't work from another directory and serves directly from the file system. This has the benefit that every change we make to the source files is visible instantly.
However, all the files in the webapp directory are not minified, not concatenated, etc. I have currently setup grunt to take the files from the webapp directory and put the deployment-ready resources in src/main/webapp/dist.
The problem: when building a war, the contents of src/main/webapp are copied into the war, but I want only the deployment-ready files from src/main/webapp/dist to be copied into the war.
I've tried countless google searchs for the topic and I'm feeling stupid. As already stated, I found "http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html", which says "these settings can be overridden via the project descriptor", but it does not show how. I found http://maven.apache.org/pom.html#Build_Element which doesn't show the webapp directory. I've found "http://maven.apache.org/guides/mini/guide-using-one-source-directory.html" which again, doesn't specify how to change the directories.
I know I can just specify src/main/webapp/dist as an additional resource directory and it will be copied into root war directory. But I don't want all the development files available in the production build.
Also, if someone knows of a better way of handling my general approach, I would like to hear it as well.
I found the setting, finally. http://maven.apache.org/plugins/maven-war-plugin/war-mojo.html
Add
<warSourceDirectory>src/main/webapp/dist</warSourceDirectory>
to the maven-war-plugin configuration, like so:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.6</version>
<configuration>
<warSourceDirectory>src/main/webapp/dist</warSourceDirectory>
</configuration>
</plugin>

Maven Thrift Plugin

I am trying to automate the generation of source files from the .thrift files and later the packaging. As far as I know, the maven-thrift-plugin is restrictive in the sense that source and destination directories are fixed. Is there any way I can specify the source and destination directories? I could probably achieve this by using the maven-antrun-plugin but I don't want to pollute my pom unnecessarily if I don't have to.
Thanks.
As far as I can see from the source (https://github.com/dtrott/maven-thrift-plugin/blob/master/src/main/java/org/apache/thrift/maven/ThriftCompileMojo.java) there are configuration properties that control this behaviour.
Try these properties, they should work:
thriftSourceRoot
thriftTestSourceRoot
outputDirectory
These props should be added to the <configuration> section along with <thriftExecutable>, etc:
<plugin>
<groupId>org.apache.thrift.tools</groupId>
<artifactId>maven-thrift-plugin</artifactId>
<version>0.1.10</version>
<configuration>
<thriftExecutable>/usr/local/bin/thrift</thriftExecutable>
<thriftSourceRoot>${basedir}/src/main/my_custom_thrift_root</thriftSourceRoot>
</configuration>
<executions>
...
</plugin>
I also ended up going the maven-antrun-plugin route, here is a functional example: https://github.com/cobbzilla/cobbzilla-wizard/tree/master/wizard-thrift
pom.xml uses maven-antrun-plugin to exec the thrift target in build.xml
build.xml does the thrift compilation and packaging.
Generated sources go back into the source tree; I don't like derived files polluting my upstream source control, so the generated files are under a thrift package, and the package directory is in a .gitignore file. A bit kludgy.
A better way I learned about since writing that code is to compile multiple java source directories in a single maven project, which would be cleaner.

Maven Plugin to mark empty directories

Is there something for Maven that I can use to create "placeholders" (e.g. a .empty or EMPTY) file for empty directories? Mercurial does not include empty directories so I need these directories filled hopefully via some automated way.
In the past, I used a Python script that does exactly this. I was hoping for a more Java-esque or Maven-esque approach.
Thanks
You could always use maven-antrun-plugin or gmaven-plugin to script putting a file in your directories. And then submit those empty files to your source control.
HOWEVER, this would have maven generate source code that is going to be checked in. Which is a bad idea (or at least not what you want to regular build to do).
If your source control does not keep track of empty directories, but they're needed for your build, I would recommend instead having a generate-source hook in your maven lifeycle that creates those. (Ideally, in your target directory to keep it clean, however since your source control won't keep track of them there is little harm in putting them right in your source folders if that makes your life easier).
Something like (consider that pseudo-code, absolutely not tested):
<build>
<plugins>
<plugins>
<artifactId>maven-antrun-plugin</...>
<executions>
<execution>
<phase>generate-source</phase>
<goals><goal>antrun</...
<configuration>
<tasks>
<mkdir dir="my directory1"/>
<mkdir dir="my directory2"/>

Resources