Filtering resources with different encodings - maven

I have a maven project. In /src/main/resources/ I have two files foo.properties and bar.xml. The first is encoded in ISO-8859-1 and the second one in UTF-8.
Is there an easy way to get filtering for both files without breaking encoding?
As far as I see, I cannot specify an encoding in a <resource> block. It seems only possible to specify an encoding for the plugin itself, but this would lead in both files being filtered using the same encoding.
Ideally I would like to specify something like:
*.properties -> filter using ISO-8859-1
*.xml -> filter using UTF-8
*.pdf -> do not filter
anything else -> filter using default encoding
This doesn't feel like a very unusual situation, is there an easy way to configure this?

First i would suggest to put those resources which are iso based into a different directory like src/main/iso-resources and use something like this:
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<name>Resource Test</name>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<configuration>
<nonFilteredFileExtensions>
<nonFilteredFileExtension>pdf</nonFilteredFileExtension>
</nonFilteredFileExtensions>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<id>copy-iso-part</id>
<goals><goal>copy-resources</goal></goals>
<phase>process-resources</phase>
<configuration>
<encoding>ISO-8859-1</encoding>
<resources>
<resource>
<directory>src/main/iso-resources/</directory>
</resource>
</resources>
<outputDirectory>${project.build.outputDirectory}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

If you don't want to keep your resources in separate directories you can also keep them in the default directory src/main/resources and play with exclusions.
The idea is:
file extensions that should not be filtered should be added to the default configuration using nonFilteredFileExtensions
let the default execution of the resources plugin process resources with the default encoding
exclude files that do no use this encoding from the build/resources configuration
for each additional encoding you want to handle, add an execution of goal "copy-resources" with explicit includes that selects only the targeted files
This example only handles .properties but you can easily generalize:
<resources>
<resource>
<directory>src/main/resources</directory>
<excludes>
<exclude>application*.properties</exclude>
</excludes>
<filtering>true</filtering>
</resource>
</resources>
...
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<configuration>
<nonFilteredFileExtensions>
<nonFilteredFileExtension>pdf</nonFilteredFileExtension>
</nonFilteredFileExtensions>
</configuration>
<executions>
<execution>
<id>properties</id>
<phase>process-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<encoding>ISO-8859-1</encoding>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>application*.properties</include>
</includes>
</resource>
</resources>
<outputDirectory>${project.build.outputDirectory}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>

Related

generated file in the wrong place

I'm using the maven resources plugin to generate a plain file via filtering within a war file. I have the template file with variables in a the folder src/main/webapp/app
<build>
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>process-resources</phase>
</execution>
</executions>
</plugin>
</plugins>
<resources>
<resource>
<directory>src/main/webapp/app</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
the problem is that the generated file is in the WEB-INF/classes folder and I need it to be at the app folder at the root of the war.
The war structure should be
*.jsf
app/<generated_file>
WEB-INF/
How can I do it?
You have to add a copy-resources goal if you want to copy something to another structure during your build, with that you can point to a custom output path:
<configuration>
<outputDirectory>${basedir}/target/extra-resources</outputDirectory>
<resources>
<resource>
<directory>src/non-packaged-resources</directory>
<filtering>true</filtering>
</resource>
</resources>
</configuration>
This can be found here in the documentation.
With the comments I could finally get the solution. It was simply to set the outputDirectory parameter to src/main/webapp/app and put the template in another place outside src/main/webapp. In my case, I put it in a folder src/main/jnlp. So finally this was the solution.. hope it helps others
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>process-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${basedir}/src/main/webapp/app</outputDirectory>
<resources>
<resource>
<directory>src/main/jnlp</directory>
<filtering>true</filtering>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>

Maven-resources-plugin won't copy .metadata folder

I'm trying to copy a folder or following structure with maven-resources-plugin:
root
|- .metadata
|- Project
\- .gitignore
Project directory and .gitignore files are copied, but .metadata directory is left out for some reason.
How do I copy all contents of root folder?
Here is configuration I tried:
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.5</version>
<executions>
<execution>
<id>copy-resources</id>
<phase>validate</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/aut-ws</outputDirectory>
<useBuildFilters>false</useBuildFilters>
<nonFilteredFileExtensions>
<nonFilteredFileExtension>metadata</nonFilteredFileExtension>
</nonFilteredFileExtensions>
<resources>
<resource>
<directory>H:\rcptt\workspaces\root</directory>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
You're looking for the configuration parameter addDefaultExcludes. See the documentation page.
So your configuration section should look like the following:
<configuration>
<outputDirectory>${project.build.directory}/aut-ws</outputDirectory>
<addDefaultExcludes>false</addDefaultExcludes>
...
<resources>
<resource>
<directory>H:\rcptt\workspaces\root</directory>
</resource>
</resources>
</configuration>
If this was relating to the maven-assembly-plugin then I had this problem, and had to use the useDefaultExcludes property (recent versions of the plugin only); by default it is true and it needs to be set to false to include directories like .metadata. This doesn't seem to be applicable to maven-resources-plugin though, or it might just not be a documented property.
My first fix attempt would be trying modifying the resources element.
<resources>
<resource>
<directory>H:\rcptt\workspaces\root</directory>
<includes>
<include>**/*</include>
<include>**/.*</include>
</includes>
</resource>
</resources>
Also, if you comment out the <nonFilteredFileExtensions> element, does it work?
Edit to show full plugin configuration that works with Maven 3.2.2, Resources plugin 2.7, on both Windows 7 and RedHat Linux. Command for testing is mvn validate.
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<phase>validate</phase>
<goals><goal>copy-resources</goal></goals>
<configuration>
<outputDirectory>${project.build.directory}/testing123</outputDirectory>
<resources>
<resource>
<directory>${project.basedir}/src/main/resources</directory>
<includes>
<include>**/*</include>
<include>**/.*</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
Just add addDefaultExcludes as #Matthew Wise said.
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<addDefaultExcludes>false</addDefaultExcludes>
</configuration>
</plugin>
By default files like .gitignore, .cvsignore etc. are excluded which
means they will not being copied. If you need them for a particular
reason you can do that by settings this to false.
Documentation: https://maven.apache.org/plugins/maven-resources-plugin/copy-resources-mojo.html#addDefaultExcludes

How to loop the files in a directory, and do some tasks based on each filename?

There is a properties directory in my project:
properties
- project_dev.properties
- project_test.properties
- project_prod.properties
- project_other.properties
Which defines some different values for different enviroments. And there is also a template directory, which contains some configuration file templates with placeholders.
What I want to do is to loop all the files in properties, and combine each one to the template directory to generate the final configuration files, which are under different directories based the file name.
So it will be:
target
- configurations
- dev
- ... some files
- test
- ... some files
- prod
- ... some files
- other
- ... some files
What I'm doing now is to use the maven-resources-plugin, and define several executions for each of the files, and hard-coded the names.
It looks like:
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.6</version>
<executions>
<!--- for dev ---->
<execution>
<id>dev-filter</id>
<phase>validate</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<encoding>UTF-8</encoding>
<outputDirectory>${basedir}/target/conf/dev</outputDirectory>
<useDefaultDelimiters>false</useDefaultDelimiters>
<resources>
<resource>
<directory>${basedir}/conf/template</directory>
<filtering>true</filtering>
<includes>
<include>channel/*</include>
<include>rule-config/*</include>
<include>server/*.properties</include>
</includes>
</resource>
</resources>
<delimiters>
<delimiter>${*}</delimiter>
</delimiters>
<filters>
<filter>${basedir}/properties/project_dev.properties</filter>
</filters>
</configuration>
</execution>
<!--- for prod ---->
<execution>
<id>prod-filter</id>
<phase>validate</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<encoding>UTF-8</encoding>
<outputDirectory>${basedir}/target/conf/prod</outputDirectory>
<useDefaultDelimiters>false</useDefaultDelimiters>
<resources>
<resource>
<directory>${basedir}/conf/template</directory>
<filtering>true</filtering>
<includes>
<include>channel/*</include>
<include>rule-config/*</include>
<include>server/*.properties</include>
</includes>
</resource>
</resources>
<delimiters>
<delimiter>${*}</delimiter>
</delimiters>
<filters>
<filter>${basedir}/properties/project_prod.properties</filter>
</filters>
</configuration>
</execution>
<!--- for test --->
...
You can see I'm copying the execution block for each file, so there will be many duplicated code with small differences.
I wonder is there any solution to make things simpler? I tried to find a way to loop the files under properties and do the filter task for each of them, but not sure how to do it.
A simple solution for such purposes is the Iterator-maven-plugin
<plugin>
<groupId>com.soebes.maven.plugins</groupId>
<artifactId>iterator-maven-plugin</artifactId>
<version>0.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>executor</goal>
</goals>
<configuration>
<items>
<item>dev</item>
<item>test</item>
<item>prod</item>
</items>
<pluginExecutors>
<pluginExecutor>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>2.6</version>
</plugin>
<goal>copy-resources</goal>
<configuration>
<outputDirectory>${basedir}/target/conf/#item#</outputDirectory>
<useDefaultDelimiters>false</useDefaultDelimiters>
<resources>
<resource>
<directory>${basedir}/conf/template</directory>
<filtering>true</filtering>
<includes>
<include>channel/*</include>
<include>rule-config/*</include>
<include>server/*.properties</include>
</includes>
</resource>
</resources>
<delimiters>
<delimiter>${*}</delimiter>
</delimiters>
<filters>
<filter>${basedir}/properties/project_#item#.properties</filter>
</filters>
</configuration>
</pluginExecutor>
</pluginExecutors>
</configuration>
</execution>
</executions>
</plugin>
The documentation contains also an example which is very similar to your needs.
Apart from that i would suggest to use the following properties in your pom file:
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
than you don't need to define the encoding separately for the maven-resources-plugin.

How to use <includes> / <excludes> in maven-remote-resources-plugin

I am trying to use the maven-remote-resources-plugin as per this example to selectively share common resources between multiple maven modules and I'm having a lot of difficulty getting the selective import of the resources to work.
I am trying to use <includes> and <excludes> elements as per below. I haven't seen these mentioned in doco for the plugin anywhere but eclipse provides them as valid options in the command completion and I don't get any errors when I run the pom. So far I haven't been able to get <includes> or <excludes> to have any effect at all on the imported resources
The relevant sections of my pom are;
Shared resources
<build>
<plugins>
<plugin>
<artifactId>maven-remote-resources-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>bundle</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/*</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-remote-resources-plugin</artifactId>
<version>1.3</version>
</dependency>
</dependencies>
Resource consumer
<build>
...
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-remote-resources-plugin</artifactId>
<version>1.3</version>
<configuration>
<resourceBunldes>
<resourceBundle>myApp:myApp_sharedresources:${project.version}</resourceBundle>
</resourceBundles>
<outputDirectory>${project.build.outputDirectory}</outputDirectory>
<includes>
<include>theOnlyResourceIWant.properties</include>
</includes>
</configuration>
<executions>
<execution>
<goals>
<goal>process</goal>
</goals>
<phase>generate-resources</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>myApp</groupId>
<artifactId>myApp_sharedresources</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
I've tried many combinations of <includes> and <excludes> but all so far have had no impact.
So, are
<includes></includes>
and
<excludes></excludes>
valid elements for a maven-remote-resources-plugin configuration, and how do I use them?
I can reasonably seperate the resources out into seperate maven modules, but that could create a large number of single file maven modules and add a lot of extra xml so I'd like to avoid it if possible.
I'd really rather not start pawing through the plugin source code, but that is the next step.
I use a temp directory for the shared resources I'm importing and filter that.
Remote resource plugin configuration below. This copies all the shared resources into a temp directory in your project. Setting attached to false means they are not included in your final project artifact, which gives you the opportunity to select the ones you want to include using Maven's normal resource processing.
<plugin>
<artifactId>maven-remote-resources-plugin</artifactId>
<configuration>
<resourceBundles>
<resourceBundle>myApp:myApp_sharedresources:${project.version}</resourceBundle>
</resourceBundles>
<attached>false</attached>
<outputDirectory>${project.build.directory}/shared-resources</outputDirectory>
</configuration>
<executions>
<execution>
<goals>
<goal>process</goal>
</goals>
<phase>generate-resources</phase>
</execution>
</executions>
</plugin>
Resource definition. When the maven-resource-plugin runs (it is bound to the lifecycle by default for jars/wars/ears), it will use the shared resource directory as well as the normal src/main/resources dir. You need to define both. (You may also enable resource filtering if you want.)
<resources>
<resource>
<directory>${project.build.directory}/shared-resources</directory>
<includes>
<include>theOnlyResourceIWant.properties</include>
</includes>
</resource>
<resource>
<directory>${basedir}/src/main/resources</directory>
</resource>
</resources>
I recommend making the shared directory be a subdirectory of ${project.build.directory}, so the clean lifecycle works without changes.
To enable the filter delimiters for the format '#{expr}' (Ruby-style), add the following to your plugin configuration:
<plugin>
<artifactId>maven-remote-resources-plugin</artifactId>
<version>1.3</version>
<executions>
<execution>
<id>process-remote-resources</id>
<goals>
<goal>process</goal>
</goals>
<configuration>
<filterDelimiters>
<filterDelimiter>#{*}</filterDelimiter>
</filterDelimiters>
[...]
</configuration>
</execution>
</executions>
</plugin>
Check this link for reference

Configure encoding for different filetypes in maven?

I use the maven-resource-plugin to filter some resources in my maven project. In my parent project I have:
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
In a sub project I have a test.properties file which is a plain java properties file with default encoding=ISO-8859-1. This file contains:
aboutText=Version ${project.version} © 2012 blabla
To make sure this file filters correctly I have split the maven-resource-plugin into separate executions each with its encoding:
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<configuration>
<nonFilteredFileExtensions>
<nonFilteredFileExtension>ico</nonFilteredFileExtension>
<nonFilteredFileExtension>jar</nonFilteredFileExtension>
</nonFilteredFileExtensions>
</configuration>
<executions>
<execution>
<id>filter-properties-files</id>
<phase>generate-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<!-- java properties files are encoded in ISO-8859-1 so when
filtering those files we stick with that encoding. -->
<encoding>ISO-8859-1</encoding>
<outputDirectory>${basedir}/after</outputDirectory>
<resources>
<resource>
<filtering>true</filtering>
<directory>${basedir}/before</directory>
<includes>
<include>**/*.properties</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
<execution>
<id>filter-non-properties-files</id>
<phase>generate-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<encoding>${project.build.sourceEncoding}</encoding>
<outputDirectory>${basedir}/after</outputDirectory>
<resources>
<resource>
<filtering>true</filtering>
<directory>${basedir}/before</directory>
<includes>
<include>**/*.product</include>
<include>**/*.inf</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
This seems overkill and I have a feeling that I am either not using the plugin correctly or that this problem should be handled in another way. Maybe stick to encoding special characters in properties files directly:
aboutText=Version ${project.version} \u00a9 2012 blabla
?
The question is an invaluable answer in itself, as apparently the complicated procedure the author provides is the only way to configure different encodings for various filtered file types. The example given, however, is specific to the author's non-standard use-case, and glosses over a few important details, without which actual use of the example is fraught with gotchas:
It isn't obvious, but in the author's example the default resource copying goal resources is still enabled and runs in addition to the two defined goals!
You'll notice that the author used the lifecycle phase generate-resources instead of the default process-resources. This is a trick to get around the first point above; by making the two copy-resources goals occur in an earlier lifecycle phase, the resources are copied according to the given rules, and then when the default-resources goal comes along the original resource copying is left intact, apparently because its overwrite setting defaults false. But it would be better to disable altogether the default-resources execution.
The author provides an outputDirectory declaration. It would be natural to think that the author only provided this because a custom output directory was desired; after all, the resources goal provides a default value for this setting. Strangely, though, for the copy-resources goal this setting is actually required! There is a standard Maven variable ${project.build.outputDirectory} which can be used as the value, however.
Building on the author's own example in the question, here is a cut-and-paste way to filter properties files using ISO-8859-1, copy other files without filtering, and prevent the default resource copying from occurring; all using the standard source and target directories:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>2.7</version>
<executions>
<execution>
<!-- Turn off default resource copying. -->
<id>default-resources</id>
<phase />
</execution>
<execution>
<!-- Filter resources in properties files. -->
<id>filter-properties-files</id>
<phase>process-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<encoding>ISO-8859-1</encoding>
<outputDirectory>${project.build.outputDirectory}</outputDirectory>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>**/*.properties</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
<execution>
<!-- Do not do property substitution in files that are not properties files, such as binary files. -->
<id>copy-other-resources</id>
<phase>process-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${project.build.outputDirectory}</outputDirectory>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>false</filtering>
<excludes>
<exclude>**/*.properties</exclude>
</excludes>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
Update: After more research, I believe that instead of disabling the default execution, one could simply modify the default execution to ignore properties files, and add filtering of properties files as an additional execution. Moreover if the resources goal were used instead of copy-resources, there would be no need to indicate an output directory or a phase, as resources:resources automatically binds to the process-resources phase and outputs to ${project.build.outputDirectory}. Note that I've updated to version 3.1.0 of the plugin, so perhaps some of these options were not available in the version I used above. I have not tested this new, shorter configuration.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<!-- Ignore properties files by default. -->
<id>default-resources</id>
<configuration>
<resources>
<resource>
<directory>${project.basedir}/src/main/resources</directory>
<excludes>
<exclude>**/*.properties</exclude>
</excludes>
</resource>
</resources>
</configuration>
</execution>
<execution>
<!-- Filter resources in properties files using ISO-8859-1. -->
<id>filter-properties-files</id>
<goals>
<goal>resources</goal>
</goals>
<configuration>
<encoding>ISO-8859-1</encoding>
<resources>
<resource>
<directory>${project.basedir}/src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>**/*.properties</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
Maybe someone could test this new configuration and let me know if it works the same as the original one I gave above.
You are right that you have to configure executions... You could get away with two executions rather that the three you seem to be using if you were following the standard directory layout but you seem to be putting files from ${basedir}/before into ${basedir}/after so the default execution of resources:resources coupled with adding multiple resources to the build section will not work for you.
The encoding of filtered files was an oversight in the original pom model specification and will likely be fixed in some as yet unspecified version of Maven

Resources