How to stop Maven from expanding env. variable w/Jenkins? - maven

I'm familiar with Jenkins, but new to Maven, and I'm trying to figure out how to stop our Maven jobs in Jenkins from expanding an environment variable inside of the ApplicationContext.xml file.
Inside our ApplicationContext.xml file we reference a ${DeployMode} environment variable (that we created) which Tomcat expands at load/run time:
<!-- SPRING CONTEXT static accessor -->
<beans:bean id="contextApplicationContextProvider"
class="com.dartneuroscience.compserv.rest.appcontext.AppContextProvider">
<beans:constructor-arg>
<beans:value>${DeployMode}</beans:value>
</beans:constructor-arg>
</beans:bean>
<beans:bean id="placeholderConfig"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<beans:property name="location"
value="/WEB-INF/App_${DeployMode}.properties" />
</beans:bean>
The problem is that in a Maven build run by Jenkins, the built WEB-INF/ApplicationContext.xml looks like this if there's an environment variable set on the build machine (let's say to 'Prod' in this example):
<!-- SPRING CONTEXT static accessor -->
<beans:bean id="contextApplicationContextProvider"
class="com.dartneuroscience.compserv.rest.appcontext.AppContextProvider">
<beans:constructor-arg>
<beans:value>Prod</beans:value> <!-- Expanded env. var -->
</beans:constructor-arg>
</beans:bean>
<beans:bean id="placeholderConfig"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<beans:property name="location"
value="/WEB-INF/App_Prod.properties" /> <!-- Expanded env. var -->
</beans:bean>
And so the value is now "hard coded" into the WAR, and will always act like "Prod" even if the target web server has its DeployMode environment variable set to something else, like 'Staging'.
This does not happen when I run Maven manually on the same build server--it only happens when Jenkins builds the job.
Is there a setting I can pass to Jenkins to stop this behavior?
I've looked at options like the EnvInject Plugin to UNSET all environment variables as a job is run, but I am really puzzled by this behavior and would like to get to the bottom of it.
Thank you.
Update 1
I found the following resource filtering block in the top-level POM and added the <excludes/> block to skip our AppContext.xml file:
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.1.1</version>
<configuration>
<webResources>
<webResource>
<directory>${basedir}/src/main/webapp/WEB-INF</directory>
<includes>
<include>*.xml</include>
</includes>
<excludes>
<exclude>*AppContext.xml</exclude>
</excludes>
<targetPath>WEB-INF</targetPath>
<filtering>true</filtering>
</webResource>
</webResources>
But this still doesn't answer why the environment variables are added to the properties list when run by Jenkins, and ignored when run from the command line; though I did find this Hudson issue from a couple of years ago: Resource filtering fails when run from Hudson
Also, EnvInjectPlugin does what it advertises, but in removing at least the PATH var, the build broke because maven could not find the ls command.
Update 2
Changing the job in Jenkins from a "Maven 2/3 project" to a "Free-style software project" and using the Invoke top-level Maven targets build step produces the desired result without having to modify the POM.

Jenkins may be enabling resource filtering somehow. You have lots of choices for configuring filtering. You may disable it entirely, limit filtering to files with certain extensions, tell it not to include build properties in the filters, or choose different delimiters to be filtered entirely. See the resources:resources docs or the description of resource filtering concepts.

Related

Setting Spring applicationContext properties from POM file

I have the following configurations in my applicationContext.xml file:
<bean class="org.springframework.context.support.PropertySourcesPlaceholderConfigurer">
<property name="locations">
<list>
<value>classpath:application.properties</value>
<value>classpath:database.properties</value>
</list>
</property>
</bean>
<bean id="javaMailSender" class="org.springframework.mail.javamail.JavaMailSenderImpl">
<property name="host" value="${smtp.host}" />
</bean>
with smtp.host being set in my POM file like so:
<build>
<defaultGoal>install</defaultGoal>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
...
</build>
<profile>
<id>local</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<application.env>local</application.env>
<profile.scope>compile</profile.scope>
<skip.test>true</skip.test>
<smtp.host>my.smtp.server</smtp.host>
</properties>
</profile>
Upon deploying my application, I ran into an error message saying that Spring was not able to resolve smtp.host. I added the following mapping to my application.properties file:
smtp.host=${smtp.host}
But Spring started to complain that I had a circular placeholder reference on the property. Is there anything I am missing?
Thanks!
You mix here two things up. The pom.xml is for building the application. Properties you define there has normally nothing to do with your application properties. And Maven profiles has nothing to do with Spring profiles. They are only named equaly.
You should configure your Spring Application as you can read here. You could -- what I would not suggest -- use your pom as property source.
The normal way would be to read it from a externalized configuration. As I do not know if you use Spring Boot, you can have a look at the Spring Boot Way and adapt it, if you use Spring without Boot.
So add a apllication.properties file into src/mein/resources like
smtp.host=my.smtp.server
If you use boot, you are done, else you have to add a
#PropertySource("classpath:/application.properties")
to your #Configuration
Here you are mixing the build and runtime phase of application which are mutually exclusive.
Mavens' role end once the build is complete thus any properties used perishes with it. Moreover application start up is agnostic to the tool / process used to build it and thus there isn't any information shared between them. Thus the idea to use properties specified in pom.xml is not feasible.
Regarding the circular reference the statement smtp.host=${smtp.host} is loosely similar to java code int i = i; which essentially has no effect because i is defined and assigned to itself.

Freemarker template location and Spring Batch Admin

I am using Spring Batch Admin as a web frontend for my Spring Batch project together with Spring Boot.
Batch Admin provides some templates using Freemarker to set up the layout. I have added some more templates which are stored in src/main/webapp/web/layouts/html and the ressources are included in the packaging process into the .jar file.
When I start the app, my own layouts are not found ("layouts/html/myOwn.ftl not found" is the error message).
I can solve this by adding a FreeMarkerConfigurer like this:
<bean id="freemarkerConfig" class="org.springframework.web.servlet.view.freemarker.FreeMarkerConfigurer">
<property name="templateLoaderPath"><value>classpath:/WEB-INF/</value></property>
</bean>
However, when I do this, my own templates are found but the standard templates are gone (like layouts/html/home.ftl).
Is there a way to provide two paths or two template loaders such that the default template loader of Spring Batch Admin is not overwritten but used as a fallback?
Or is there any other solution like having the ressources in a specific place?
Thanks to #ddekany I came up with the following solution.
Necessary configuration for Freemarker:
<bean id="freemarkerConfig" class="org.springframework.batch.admin.web.freemarker.HippyFreeMarkerConfigurer">
<property
name="templateLoaderPaths"
value="classpath:/WEB-INF/web,classpath:/org/springframework/batch/admin/web"
/>
<property name="preferFileSystemAccess" value="false" />
<property name="freemarkerVariables">
<map>
<entry key="menuManager" value-ref="menuManager" />
</map>
</property>
<property name="freemarkerSettings">
<props>
<prop key="default_encoding">UTF-8</prop>
<prop key="output_encoding">UTF-8</prop>
</props>
</property>
</bean>
The first property templateLoaderPaths (observe the additional s) allows to specify multiple paths separated by commas. The two paths are my own path classpath:/WEB-INF/web and the path to the default Spring Boot Admin files classpath:/org/springframework/batch/admin/web.
The additional configuration for the menuManager is necessary as otherwise the menu entries from the navigation disappear.
The custom freemarker layout files are stored in the default location src/main/webapp/WEB-INF/web/layouts/html/ and to be visible by the template loader have to be included in the jar build via
<resources>
<!-- copy the Freemarker templates -->
<resource>
<targetPath>WEB-INF</targetPath>
<filtering>false</filtering>
<directory>${basedir}/src/main/webapp/WEB-INF</directory>
</resource>
</resources>
in the project's pom.xml.

Jenkins is updating my applicationContext.xml file with it's system properties?

This is pretty crazy...
I have an applicationContext.xml, that configures a bean using system properties.
In my case I am configuring a bean to inject values from a file or if not there look into the system.properties(I expect this only to happen at run time!)
i have:
<bean id="myBean" class="foo.foo.fooBarImpl">
<property name="keyStoreFile" value="${javax.net.ssl.KeyStore}"/>
...
...
</bean>
So when my java application that uses this applicationContext.xml(resides inside the jar on the classpath) starts it will pull the ${javax.net.ssl.KeyStore} from a properties file or if the properties file is not there, attempt to get it from the system properties.
What is happening that I cannot explain is..when jenkins pulls from the repository, and builds..
it is modifying my applicationContext.xml! and actually writing in what is present in the system properties..and saving it before buildling the .jar! my jar now as hardcoded values in it of SSL information(like the password...)
<bean id="myBean" class="foo.foo.fooBarImpl">
<property name="keyStoreFile" value="/mympath/keystore.jks"/>
...
...
</bean>
this above modified applicationContext.xml is now in my .jar!?
is there a setting in Spring, or Jenkins(maybe) to prevent my applicationContext.xml to be modified and resaved into the .jar?
Are you using Maven on your build?
Could be that maven resource filtering is taking in place.
Please try on pom.xml something like
<project>
...
<build>
...
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>false</filtering>
<includes>
<include>**/applicationContext.xml</include>
</includes>
</resource>
...
</resources>
...
</build>
...
</project>

Using Maven settings.xml properties inside Spring context

I've got a Maven settings.xml file in my ~/.m2 directory; it looks like this:
<settings>
<profiles>
<profile>
<id>mike</id>
<properties>
<db.driver>org.postgresql.Driver</db.driver>
<db.type>postgresql</db.type>
<db.host>localhost</db.host>
<db.port>5432</db.port>
<db.url>jdbc:${db.type}://${db.host}:${db.port}/dbname</db.url>
</properties>
</profile>
</profiles>
<activeProfiles>
<activeProfile>mike</activeProfile>
</activeProfiles>
<servers>
<server>
<id>server_id</id>
<username>mike</username>
<password>{some_encrypted_password}</password>
</server>
</servers>
</settings>
I'd like to use these properties twice
Once inside Maven's integration-test phase to set up and tear down my database. Using Maven filtering, this is working perfectly.
A second time when running my Spring application, which means I need to substitute these properties into my servlet-context.xml file during Maven's resources:resources phase. For properties in the upper section of settings.xml, such as ${db.url}, this works fine. I cannot figure out how to substitute my database username and (decrypted) password into the Spring servlet-context.xml file.
The pertinent part of my servlet-context.xml file looks like:
<bean id="myDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName"><value>${db.driver}</value></property>
<property name="url"><value>${db.url}</value></property>
<property name="username"><value>${username}</value></property>
<property name="password"><value>${password}</value></property>
</bean>
The end goal here is for each developer to have their own Maven settings (and database on their own machine for integration testing)...And a similar setup on the Jenkins server. We do not want to share a common username/password/etc.
There is a way of filtering web resources by configuration of Maven War Plugin. Look at this for a snippet from official plugin's docs.
And by the way, I strongly recommend reconsidering this filtering-based way for providing de facto run-time configuration at build-time. Just notice that you have to rebuild the same code to just prepare package for another environment (or alternatively edit package contents). You can use application server's specific stuff for this (at least JBoss has one) or use Spring that AFAIR also can be configured like this.
I recommend you to use a property file in the middle. I mean: Spring application would load properties values form the property file using context:property-placeholder and Maven would be the one who replace ${...} variables using values from settings.xml using filtering.
Your property file:
db.driver=${db.driver}
db.url=${db.url}
username=${username}
password=${password}
Your servlet-context.xml file
<context:property-placeholder location="classpath:your-property-file.properties" />
<bean id="myDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName"><value>${db.driver}</value></property>
<property name="url"><value>${db.url}</value></property>
<property name="username"><value>${username}</value></property>
<property name="password"><value>${password}</value></property>
</bean>
In your pom.xml
<resources>
...
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
...
</resources>
I haven't tried it, but as per this maven wiki page, you should be able to refer to properties in settings.xml using settings. prefix. So ${settings.servers.server.username} should ideally return the username in settings.xml.

Maven: include resource file based on profile

I'm converting an Ant webapp project over to Maven. I have most of it working, but I'm stuck trying to figure out how to copy some resource files from different sources based on the profile.
I have src/main/resources/persistence-{dev, prod}.xml. One of these needs to be included in the war file as WEB-INF/classes/META-INF/persistence.xml.
I would like the dev version to be copied when the dev profile is active, and the prod version when prod is active.
Just use the maven resources plugin like so http://maven.apache.org/plugins/maven-resources-plugin/examples/include-exclude.html and have a property for the file name or extension set in a profile.
If you are not wedded to the paradigm of having 3 separate persistence.xml files and copying one or the other selectively, you can use maven profiles with filtering like this (just implemented this the other day and today came across your post):
In persistence.xml:
<property name="hibernate.show_sql" value="${hibernate.debug}" />
<property name="hibernate.format_sql" value="${hibernate.debug}" />
In pom.xml create a profile and define the variable:
<profiles>
<profile>
<id>hib-debug</id>
<properties>
<hibernate.debug>true</hibernate.debug>
</properties>
</profile>
</profiles>
define a default for when you build without specifying a profile:
<properties>
<hibernate.debug>false</hibernate.debug>
</properties>
and.... make sure you turn on resource filtering:
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
Then you build with mvn -Phib-debug and voila! Substitution is done.

Resources