After updating maven-resources-plugin to version 3.2.0 to get the fix for maintaining file permissions (x-bits), the copy fails when encountering a symbolic link of the form file -> ../file (and possibly other symlinks too):
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.2.0:copy-resources
(copy-jdk) on project preinstall: <target>/jre/lib/amd64/server/libjsig.so -> [Help 1]
ll <source>/jre/lib/amd64/server/
lrwxrwxrwx 1 user users 13 Oct 22 15:09 libjsig.so -> ../libjsig.so
...
The target's jre/lib/amd64 folder received some files, but not yet libjsig.so. <source>/jre/lib/amd64/libjsig.so exists, therefore the symlink's target would be created if the plugin did not abort.
Can anyone tell me if that's a bug or a feature? Any configuration / idea / workaround that allows to keep the source's file permissions and have symbolic links copied?
Here's the relevant pom.xml part:
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<!-- version 3.2.0 stops on symlinks in the jdk, when the symlink target is not yet copied -->
<version>3.1.0</version>
<executions>
<execution>
<id>copy-jdk</id>
<phase>process-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${product.install.folder}/jdk</outputDirectory>
<resources>
<resource>
<directory>${jdk.tmp.folder}/${jdk.directory}</directory>
<filtering>false</filtering>
<include>**/*</include>
</resource>
</resources>
</configuration>
</execution>
...
Edit: BTW, creating a dummy target at <target>/jre/lib/amd64/libjsig.so helps to continue with the copy task, but then, that dummy is not replaced during the copy (probably because it is newer) ...
Edit2: Playing around with the above dummy idea and the <overwrite> option. That option is not usable if the file tree contains read-only files. <overwrite> does not delete and recreate the files but tries to rewrite them, which fails for read-only files.
Therefore I think, the maven-resources-plugin might be the wrong approach to copy files around - here's the evolved question:
What is the best approach to copy a larger set of files (such as a jdk, which includes read-only files, executable files and symbolic links) from within a maven build?
Related
I do not ever want maven-dependency-plugin:3.1.2:unpack to overwrite existing files in any circumstances. This is my current pom.xml configuration:
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<version>3.1.2</version>
<executions>
<execution>
<id>unpack-zip-files</id>
<phase>generate-test-resources</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>com.foo</groupId>
<artifactId>bar</artifactId>
<version>${foobar.version}</version>
<type>zip</type>
<classifier>exe-archive</classifier>
<overWrite>false</overWrite>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
<includes>**/*.exe</includes>
</artifactItem>
</artifactItems>
<overWriteIfNewer>false</overWriteIfNewer>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>false</overWriteSnapshots>
</configuration>
</execution>
</executions>
</plugin>
Scenario:
Projects foo and bar
foo has a .exe file as build artifact, inside a ZIP file
bar runs the .exe file during tests
In situation X, bar must use the snapshot version of the mainline development branch of foo. In that case, the directory target/lib shall be empty before mvn install. This is guaranteed by starting from an empty workspace and running mvn clean as a separate build step. This question is not about situation X.
In situation Y, bar must use a custom build of a feature branch of foo. This is done by unpacking the ZIP file with the .exe to the directory target/lib in a separate build step between mvn clean and mvn install.
We are working in situation Y, where the target/lib directory is already pre-filled with the .exe from the correct feature branch.
This is the target/lib directory before mvn is run:
+ ls -al fooBar.exe
-rw-rw-r-- 1 ubuntu ubuntu 18368427 Apr 12 21:27 fooBar.exe
+ md5sum fooBar.exe
03acc8b5c3da700c31efcd6635feb56a fooBar.exe
This is the target/lib directory after mvn is run:
+ ls -al fooBar.exe
-rwxrwxr-x 1 ubuntu ubuntu 18368393 Apr 11 23:10 fooBar.exe
+ md5sum fooBar.exe
ab6dd45c5cc4e41534ad2363c5767601 fooBar.exe
The change in md5sum is hard evidence that the existing fooBar.exe was overwritten by Maven.
Maven command used:
mvn --global-settings /home/jenkins/workspace/bar#tmp/config15592668079584895681tmp \
-Dmaven.repo.local=/home/jenkins/workspace/bar/.repository \
install \
-DgsExec=/usr/bin/gs -DcompareExec=/usr/local/bin/compare \
-Dtest=RunCucumberTest -Dcucumber.options=--plugin json:target/cucumber.json
Expected results
The mvn install command shall not overwrite existing files when overWrite, overWriteIfNewer, overWriteReleases, overWriteSnapshots are all set to false.
The md5sum of fooBar.exe shall be the same before and after running mvn install.
Question
Which magical incantations do I need to add to pom.xml so that existing files are never overwritten in any circumstances?
Documentation referenced
Apache Maven Dependency Plugin – Usage – Overwrite Rules
Apache Maven Dependency Plugin – dependency:unpack
OK, I think I know what might be happening: Like I said in my comment:
I just read your message on the mailing list and tried in one of my own projects, also using plugin version 3.1.2. Actually, just specifying <overWrite>false</overWrite> inside the <artifactItem> was enough to avoid overwriting. I just executed the unpack goal once, then manually modified an unpacked file and it did not get overwritten. I even see my-artifact-1.3.jar already unpacked in the log.
I continued experimenting some more and noticed that even when deleting many unpacked files, they will not be recreated, so the check must be on a more global level, not on a per-file basis.
Even when deleting all files or the whole output directory, the dependency will not be unpacked again. That was a sure indicator that some kind of meta information must be stored somewhere outside the output directory. The first place to look for it was of course the target directory, and obviously enough, in subdirectory target/dependency-maven-plugin-markers there are (empty) marker files like my-dependency-1.3.marker. Deleting one of those files has the effect of the dependency getting unpacked again during the next build, overwriting possibly existing files.
So the way for you to solve this problem is to avoid cleaning the target directory or at least to make sure to keep the corresponding marker file.
Update: You could also create the marker file by yourself if the EXE file you want to protect exists and if for some reason your build workflow needs the clean in between. But the latter would be a bit ugly, you should try to avoid it. With Antrun or some Beanshell or Groovy scripting it would be possible, though.
Somewhat more elegant would be a profile with auto-activation if the EXE file does not exist, then putting the dependency plugin inside the profile, i.e. it would only get active if the EXE does not exist in the first place.
I'm trying to use PigUnit with Maven. In Maven, my pig scripts are located under the module's root directory at <module>/src/main/pig/<scriptName>.pig
All the PigUnit test tutorials either code the absolute url of the pig script (which obviously won't work when the build runs anywhere but my machine) or the relative path and it just "magically" works. But when I put in either the script name directly or src/main/pig/<scriptName>.pig to PigTest, the script can't be found when running mvn test.
Test with line (using scala + scalatest):
val pigTest = new PigTest("src/main/pig/calcProductVectors.pig", args)
Results in:
- Script does something *** FAILED ***
java.io.FileNotFoundException: src/main/pig/calcProductVectors.pig (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.pig.pigunit.PigTest.readFile(PigTest.java:296)
at org.apache.pig.pigunit.PigTest.readFile(PigTest.java:292)
How do I get src/main/pig to be on the path when mvn test runs?
Okay, so I found some tests in our Java projects that just added the pig directory to project.build.testResources in maven:
<testResources>
<testResource>
<!-- Pig is being included in test/resource so that we can access it with PigUnit -->
<directory>src/main/pig</directory>
<filtering>true</filtering>
</testResource>
<testResource>
<directory>src/test/resources</directory>
<filtering>true</filtering>
</testResource>
</testResources>
Then the JUnit tests would pass src/main/pig/<scriptName>.pig to PigTest.
This didn't work in my project, not sure if it's because of scalatest or something else. I could see the contents of the directory getting copied to <module>/target/test-classes, but they still weren't found either with their direct name (which seems like it would make more sense because they were copied without the src/main/pig part) or that whole path which was working for the Java projects.
Eventually from this answer, I found out where the tests were running from on the file system, and it was just the module name, so I adjusted the path like so and it worked:
val pigTest = new PigTest("<module>/src/main/pig/calcProductVectors.pig", args)
I have a Maven RPM plugin mapping thus:
<mapping>
<directory>/etc/myconfig</directory>
<configuration>true</configuration>
<sources>
<source>
<location>${project.build.directory}</location>
<includes>
<include>*.conf</include>
</includes>
</source>
</sources>
</mapping>
However, depending on the packaging process, there may be zero .conf files to put in /etc. When this occurs, RPM plugin says:
[ERROR] Failed to execute goal org.codehaus.mojo:rpm-maven-plugin:2.1.2:rpm (default) on project clients:
Unable to copy files for packaging: You must set at least one file. -> [Help 1]
Is there any way to have a mapping section that is happy with including zero files?
The best I've been able to come up with is omitting the <includes> tag, which takes everything from what's specified in <location>.
location
The file or directory to include. If a directory is specified, all files and subdirectories are also included.
You will need to be as specific as possible in the path for these mappings that don't have include patterns specified. I added confs to the location below to avoid pulling in everything else in project.build.directory.
Even if no files are selected, the <directory> will still be created.
<mapping>
<directory>/etc/myconfig</directory>
<configuration>true</configuration>
<sources>
<source>
<location>${project.build.directory}/confs</location>
</source>
</sources>
</mapping>
Me too faced the same issue and was trying hard to resolve it. The error message is not so intuitive. It's really hard to address such issue.
error :
[ERROR] Failed to execute goal org.codehaus.mojo:rpm-maven-plugin:2.1-alpha-3x:attached-rpm (generate-rpm) on project XXX_assembly: Unable to copy files for packaging: You must set at least one file. -> [Help 1]
I was using below version of plugin
*
<groupId>org.codehaus.mojo</groupId>
<artifactId>rpm-maven-plugin</artifactId>
<version>2.1-alpha-3x</version>
*
Below is my Mapping tag from plugin
What this mapping was trying to is copy batch-test-1.0.war from ../batch/target location to target location at ${app.prefix}/batch
Issue was tag was having name of war file which does not exist and incorrect. i have given wrong name of war file so Maven was complaining that it should have something to copy.i corrected it by giving correct name of my war file as below.
batch-test-1.0.war
In the source mappings, I specify many different directories. All works well.
However if I specify the directory which has the name executable the RPM plugin returns an error code of 1. The files strangely enough are still copied to the RPM buildroot, but the RPM plugin overall fails. I renamed the directory to 'lib' and it builds fine.
Is the executable a reserved word or keyword of some sort??? The mapping below causes the build to fail. I have checked spelling and that is fine. The path exists. So why is it failing???
<mapping>
<directory>/executable</directory> //rpm directory
<filemode>0644</filemode>
<sources>
<source>
<location>/path/to/directory/executable</location> // local dir under target
</source>
</sources>
</mapping>
Is there something for Maven that I can use to create "placeholders" (e.g. a .empty or EMPTY) file for empty directories? Mercurial does not include empty directories so I need these directories filled hopefully via some automated way.
In the past, I used a Python script that does exactly this. I was hoping for a more Java-esque or Maven-esque approach.
Thanks
You could always use maven-antrun-plugin or gmaven-plugin to script putting a file in your directories. And then submit those empty files to your source control.
HOWEVER, this would have maven generate source code that is going to be checked in. Which is a bad idea (or at least not what you want to regular build to do).
If your source control does not keep track of empty directories, but they're needed for your build, I would recommend instead having a generate-source hook in your maven lifeycle that creates those. (Ideally, in your target directory to keep it clean, however since your source control won't keep track of them there is little harm in putting them right in your source folders if that makes your life easier).
Something like (consider that pseudo-code, absolutely not tested):
<build>
<plugins>
<plugins>
<artifactId>maven-antrun-plugin</...>
<executions>
<execution>
<phase>generate-source</phase>
<goals><goal>antrun</...
<configuration>
<tasks>
<mkdir dir="my directory1"/>
<mkdir dir="my directory2"/>