Use maven filtering in server.xml without breaking mvn liberty:dev - websphere-liberty

I would like to use maven filtering, in my src/main/liberty/config/server.xml without breaking the use of liberty:dev from the maven-liberty-plugin. My biggest problem seems to be that the liberty-maven-plugin does not honor filtering.
For example, consider this webApplication element:
<webApplication id="${project.artifactId}" contextRoot="/"
location="${server.config.dir}/apps/${project.artifactId}.war">
</webApplication>
Without any other guidance, this file is copied to target/liberty/wlp/usr/servers/defaultServer/server.xml without any filtering, so the runtime cannot find the WAR file.
Let's say I manually turn on the filtering using maven-resources-plugin:
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>3.2.0</version>
<executions>
<execution>
<id>01-liberty-config</id>
<phase>package</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/liberty/wlp/usr/servers/defaultServer</outputDirectory>
<resources>
<resource>
<directory>src/main/liberty/config</directory>
<filtering>true</filtering>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
Now the filtering works and the file is in the correct location. Unfortunately, I observe that when I run mvn liberty:dev, this gets overwritten by the unfiltered server.xml from src/main/liberty/config/server.xml.
Is it possible to use maven filtering in a server.xml?

BACKGROUND
This is essentially not supported today. The liberty-maven-plugin doesn't let you do this, and the way in which the liberty-maven-plugin manages and controls the Liberty config also doesn't make it easy for you to use standard Maven plugins like 'dependency' or 'resources' plugin either.
Since this issue was raised before I shared a sample approach which you might find useful, though it has the feel of a workaround.
SOLUTION OVERVIEW
Basically, although although we can't substitute into server.xml itself via filters we can substitute into a config snippet that gets included by server.xml, and copy this into place using the resources plugin, rather than liberty-maven-plugin.
SOLUTION DETAIL
Say I wanted to use a "filter"-style Maven variable substitution ${tidal.url} for a URL in Liberty server config.
1. src/main/filtered-config/environment.xml
First define a config snippet, which we are going to apply the filter to.
<server description="environment">
<!-- Expect to come from filter -->
<variable name="tidal.url" value="${tidal.url}"/>
</server>
2. pom.xml
Configure an execution of resources:copy-resources copying the "environment.xml" snippet above to the shared config dir location, target/liberty/wlp/usr/shared/config, with filtering enabled:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>3.2.0</version>
<executions>
<execution>
<id>default-cli</id>
<phase>none</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<overwrite>true</overwrite>
<!-- This location can persist across a server recreate, where the refresh can annoyingly wipe out your earlier copy -->
<outputDirectory>target/liberty/wlp/usr/shared/config</outputDirectory>
<resources>
<resource>
<directory>src/main/filtered-config</directory>
<filtering>true</filtering>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
3. server.xml
In your main server.xml config file, add an <include> of the config snippet you copied into the shared config dir via "copy-resources".
Also shown is how we finally use or "consume" the value applied via the filter, here in this <jndiEntry>, in this sample:
<include location="${shared.config.dir}/environment.xml"/>
<!-- This is how I'm ultimately going to "consume" the filtered value -->
<jndiEntry jndiName="url/tidal-api" value="${tidal.url}" id="TidalJNDI" />
4. Run dev mode, invoking the extra goal first, and activating your filter somehow
E.g.:
mvn resources:copy-resources liberty:dev
As far as activating your filter, maybe you have a filter defined in your build (via build.filters.filter like in my sample repo) or maybe you're just using -Dtidal.url=<value>.
FOLLOW-UP
Besides being complicated a significant limitation of the above is that you only get a single chance to apply the filter. You cannot iterate through different values in a single dev mode "session".
Feel free to give feedback on the issue: https://github.com/OpenLiberty/ci.maven/issues/587
Also I will note we are considering enhancing filter support for general resources and web resources here.
ONE MORE THOUGHT
If all you need is a dynamic way to select, at build time, one set of Liberty config values vs. another you don't necessarily need to use filtering.
You could instead use the support which maps Maven properties to Liberty config.
E.g. for this example you could have one profile which defines
<properties>
<liberty.var.tidal.url>URL1</liberty.var.tidal.url>
</properties>
and another profile defining the same property with a different value.
This would parameterize my sample:
<jndiEntry jndiName="url/tidal-api" value="${tidal.url}" id="TidalJNDI" />
just fine.
The problem though is if you wanted to use the same sets of properties in other contexts with other plugins that did fully support filtering. Then, you want standard Maven filtering.

Related

More elegant way to define a flexible application configuration in spring boot

I want a spring boot app that has a flexibility to use the default embedded configuration files; but also has a flexibility to be able to override it with an external configuration files.
During debugging, I wish it to uses the embedded property file inside the jar file. But during deployment, I want it to use the external application.yml so I can customize values on production.
I tried a lot of forum but there seems to be no other way out of this. Does this give priority to the embedded one and overwrite it
#PropertySources({
#PropertySource("classpath:application.yml"),
#PropertySource(value = "file:application.yml", ignoreResourceNotFound = true)
})
I manually moved the yml to be in target directory.
<!-- Copy the config to target -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>${maven.resource.plugin.version}</version>
<executions>
<execution>
<id>copy-resources</id>
<phase>validate</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${basedir}/target/</outputDirectory>
<resources>
<resource>
<directory>${basedir}/src/main/resources</directory>
<filtering>*.yml</filtering>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
The above seems able to work. But I am not sure if there is a more elegant way to do this. I have my doubts since the property source array did not seem to specify the priority, it may just work for now. Do we have another way to do this aside from how I did it?
Spring has profiles for this.
Spring boot can read from src/main/resources/application.properties (and config folder works as well) by default, however if you put property like this:
--spring.profiles.active=dev
It will also read stuff from /src/main/resources/application-dev.properties (suffix must match the profile). Yaml works like this as well.
So, this should cover the "development" mode.
Now, in a real environment you can activate some other profile + use another option that spring boot offers for free: --spring.config.locations=.... This one allows specifying an external source of configuration.
So, you don't need to copy resources, fiddle with property source and so forth, instead, you can rely on spring boot regular mechanisms of configurations.
All-in-all you can read the relevant chapter in documentation but this should cover a deployment case.
Last but not least, there are precedences here, so some configuration ways can override the others (I mean if there is a clash between property values with the same name).

Maven : exclude target/generated-sources from compilation

This question, just to be sure my interpretation is correct :
I'm using Mojohaus jaxb2-maven-plugin to generate java classes from .xsd files, and by default it puts them in target/generated-sources
Now, I want to get track of these classes in source control (target is of course excluded), and I may one day slightly customize one with an annotation or a line of code, and I may even change my class generation plugin, so what do is I copy these classes and packages in src/main/java
This upsets Maven when I try to compile because he considers "target/generated-sources" as a source directory and he finds all clases twice. For what I understand, I can exclude classes inside a source directory, but I can't remove a source directory from Maven build, am I right ?
So the only solution would be to configure my jaxb2 plugin to generate the classes elsewhere, right ?
UPDATE :
Ok, this doesn't work as I thought, if I change the outputDirectory of my jaxb plugin, it's still included as a source directory by Maven, and I have no clue why.
<configuration>
<outputDirectory>${project.build.directory}/tatata/jaxb</outputDirectory>
</configuration>
UPDATE 2 : The explanation is the plugin is adding the outputDirectory as a maven source directory during the generate-sources phase of the build, and it's not optionnal or customizable.
First things first, do not add generation code to source control. Do not modify it manually. You will get into trouble. Believe me, I've seen it too many times. A new version of the schema and you're lost.
Ok, now to your question.
With maven-jaxb2-plugin you could turn off adding generation directory as a compile source root with:
<configuration>
<addCompileSourceRoot>false</addCompileSourceRoot>
</configuration>
Disclaimer: I'm the author of maven-jaxb2-plugin.
The answer from Lexicore is an interesting lead but my question was about the plugin I'm currently using, not how to do it with an other plugin.
So here is the workaround for the Mojohaus plugin : you can just skip the generate-sources by default (no need to do this task at every build when your model changes once in a week, then once in a year), and trigger it only when needed using a dedicated maven profile : How to skip generate-sources in Maven
you can always specify the target directory(generateDirectory) in pom config file as below. Hope it helps
`
<plugin>
<groupId>org.jvnet.jaxb2.maven2</groupId>
<artifactId>maven-jaxb2-plugin</artifactId>
<version>0.12.3</version>
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
</execution>
</executions>
<configuration>
<schemaLanguage>WSDL</schemaLanguage>
<generateDirectory>${basedir}/src/main/java</generateDirectory>
<generatePackage>com.myproj.proxy</generatePackage>
<schemas>
<schema>
<!-- <url>${project.basedir}/src/main/resources/wsdl/test.wsdl</url> -->
<fileset>
<!-- Defaults to schemaDirectory. -->
<directory>${basedir}/src/main/resources/wsdl</directory>
<!-- Defaults to schemaIncludes. -->
<includes>
<include>*.wsdl</include>
</includes>
</fileset>
</schema>
</schemas>
</configuration>
</plugin>
`

Maven : How to filter one file with own plugin

I'm modifying some maven3 plugin and I just need to filter with file using some properties already defined in the pom starting the whole process.
I haven't found much doc/examples on the topic and all my attempts failed.
First one :
/**
* #component
* #required
* #readonly
*/
private MavenFileFilter mavenFileFilter;
and then in my code :
mavenFileFilter.copyFile(tempFile, resultingFile, true, getProject(), Collections.<String> emptyList(), true, "utf-8", session);
However nothing happens : the replacement isn't done.
Then I saw various examples evolving around
MavenResourcesExecution mavenResourcesExecution = new MavenResourcesExecution(resources, outputDirectory, getProject(), "utf-8", null, nonFilteredFileExtensions, session);
However I don't get how to put in there the file I want... Resource has no constructor with actual content...
Any help welcome!
Best
EDIT (more context) :
The context is the following : we built a web app deployed at many customers' servers.
On one hand, the folder hierarchy depends from customer to customer.
On the other hand, we have some default logback.xml config file, which needs to be filtered (to use the correct folder hierarchy of the given customer server). This file sits in some common project. We would also like to be able to specialize this logback.xml, when we wish so, on a per customer basis. We've put these files in the source folders of the respective common project/customer project.
As such, the plugin doing the packaging now looks into each artifact, by order of dependencies, and pick up the first logback.xml and put it where needed. It also needs to do the filtering to put the right folders.
It all works apart the last bit...
Does it make sense ? Any better way ?
Thanks again
I would suggest to take a look at this:
<project>
...
<build>
<plugins>
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<id>copy-resources</id>
<!-- here the phase you need -->
<phase>validate</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${basedir}/target/extra-resources</outputDirectory>
<resources>
<resource>
<directory>src/non-packaged-resources</directory>
<filtering>true</filtering>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
...
</build>
...
</project>
Why do you need to implement a plugin which only copies resources within an other life cyclce phase.

How to access maven.build.timestamp for resource filtering

I am using maven 3.0.4 and would like to make the build timestamp accessible to my application. For this, I'm putting a placeholder in a .properties file and let maven filter on build. While this is working fine for ${project.version}, ${maven.build.timestamp} is not substituted on filtering.
The property seems to be available on build - I can use it to modify the artifact name:
<finalName>${project.artifactId}-${maven.build.timestamp}</finalName>
So why is it not available for resource filtering? And, more importantly, how do I make it accessible?
I have discovered this article, explaining that due to a bug in maven, the build timestamp does not get propagated to the filtering. The workaround is to wrap the timestamp in another property:
<properties>
<timestamp>${maven.build.timestamp}</timestamp>
<maven.build.timestamp.format>yyyy-MM-dd HH:mm</maven.build.timestamp.format>
</properties>
Filtering then works as expected for
buildTimestamp=${timestamp}
I can confirm as of Maven 3.x {maven.build.timestamp} is "working" now. They work arounded the problem, apparently. No additional properties workaround needed anymore.
However, be careful your "filtering" plugin (maven-resources-plugin) is up to date. It needs to be relatively new, so if mvn help:effective-pom shows an old version (ex: 2.6), bump it to something newer, fixed it for me, 3.x ex:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>3.1.0</version>
</plugin>
<properties><timestamp>... workaround is no longer required...
This also cleared up, kind of, why it was working in IntelliJ but not the command line. IntelliJ probably uses their own "modified/internal" maven constants, so it was working there, but not from maven command line.
Also note if you add a filtering resource directory to you pom, you may need to also "re-add" the default directory, it gets lost, ex:
<resource>
<directory>src/main/resources-filtered</directory> <!-- to get "maven.build.timestamp" into resource properties file -->
<filtering>true</filtering>
</resource>
<resource>
<directory>src/main/resources</directory> <!-- apparently have to add this is you have the other... -->
</resource>
NB if you're using spring boot as your parent, you have to use #maven.build.timestamp# instead. Also note if you're using spring boot there's a file META-INF/build-info.properties that is optionally created by the spring-boot-maven-plugin that you can read (spring provides a BuildProperties bean for convenience reading it).
In order to enrich the Stackoverflow content for others, that like me, found this post as a way to solve the "problem" of ${maven.build.timestamp}. This is not a maven bug, but an expected behavior of m2e, as can be seen in this post.
Therefore, I believe that we can not expect the solution to be "corrected", since, from what I understand, the correction involves conceptual issues.
In my case, what I did was use the plugin (buildnumber-maven-plugin) as described in this other post.
Adding Maven properties at the pom project level doesn't take into account correct local Timezone, so timestamp may appear wrong :
<properties><timestamp>${maven.build.timestamp}</timestamp></properties>
Using the build-helper-maven-plugin applies the correct timezone and current daylight saving to the timestamp :
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>timestamp-property</id>
<goals>
<goal>timestamp-property</goal>
</goals>
<configuration>
<name>timestamp</name>
<pattern>yyyy-MM-dd HH:mm:ss</pattern>
<timeZone>Europe/Zurich</timeZone>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
When packaging, Maven will replace any token timestamp in /resources folder, e.g. resources/version.properties :
build.timestamp=${timestamp}
You can then load this properties file in your Application.

maven 3 javadoc plugin doesn't take the excludepackagename config

I'm trying to exclude a bunch of packages from a javadoc site.
Unfortunately this plugin seems to live its own life and when it was configured as a report plugin it failed with access denied when moving files, so it was changed to be a normal plugin and then configured to run with the site goal (aggregated). By doing that we have the javadoc generated and it's published under the site as it should be.
But it seems that the configuration parameters for the plugin doesn't take effect at all. I've tried to move the <excludePackageNames> element around - both being a general config and to be a specific config for the aggregate goal - and I even added an exclusion for our entire code base and all files was still generated.
What I'm trying to do is to simply remove a couple of packages that shouldn't be in the javadoc. Anyone who got this plugin and the config to play nicely, to exclude packages?
This is the config I use right now, the javadoc is created but all packages, including the excluded, is generated.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.8</version>
<configuration>
<excludePackageNames>my.company.packages.*</excludePackageNames>
</configuration>
<executions>
<!-- Hook up the Javadoc generation on the site phase -->
<execution>
<id>aggregate</id>
<goals>
<goal>aggregate</goal>
</goals>
<phase>site</phase>
</execution>
</executions>
</plugin>
Any ideas, pretty please?
I solved identical problem by adding the sourcepath parameter to the configuration:
<configuration>
<sourcepath>${project.basedir}/src/main/java</sourcepath>
<excludePackageNames>my.company.packages.*</excludePackageNames>
</configuration>
The configuration above will exclude all packages below my.company.packages but not my.company.packages itself. To exclude also my.company.packages use <excludePackageNames>my.company.packages</excludePackageNames> instead.

Resources