TeamCity provides several environment variables that are made available in Maven's pom.xml. For example, my build agent has a property env.JDK_18 pointing to a JRE installation on the server. I use this property in my configuration of the surefire plugin in Maven:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<jvm>${env.JDK_18}/bin/java</jvm>
<!-- ... -->
</configuration>
</plugin>
Unfortunately, the environment variable is not available on my local machine. Is there any Maven option to define the environment variable locally (in my pom.xml file) and have TeamCity override it, when unit tests are run on the build server?
Related
I have a maven multi-module project on TeamCity. I'm using TeamCity's built in maven 3.5 tooling.
In one of the child projects, in the section of its pom.xml I set "<target.env>dev</target.env>".
Later in the pom I use the properties-maven-plugin to load a file with the name "${target.env}.env.properties"
Locally if I run "mvn package -Dtarget.env=prod" in the parent project, the child project loads prod.env.properties as expected.
If I configure my teamcity build with param("system.target.env", "prod"), I can see "-Dtarget.env=prod" passed to the maven execution in the build log (where teamcity invokes the plexus-classworlds launcher to do so), the child project loads dev.env.properties, breaking the build.
Here's my questions:
Why does the behavior differ?
How do I reconcile this?
Update including some of the information #khmarbaise asked for:
The properties-maven-plugin is being used to load an environment specific set of properties based on which environment the application will run in.
It is set up to choose which file to load based on a system property, and a default value is set in the properties block to avoid having to constantly
add -Dtarget.env=dev during development. The properties-maven-plugin configuration for the child project is as follows:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<executions>
<execution>
<id>load-environment-properties</id>
<phase>validate</phase> <!-- Bound to validate phase to ensure it comes before loading of local.build.properties --> # No local.build.properties on TeamCity, so nothing clobbers this in practice
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>src/main/targetEnvironment/default.env.properties</file>
<file>src/main/targetEnvironment/${target.env}.env.properties</file>
</files>
</configuration>
</execution>
<execution>
<id>write-properties</id>
<phase>generate-resources</phase>
<goals>
<goal>write-project-properties</goal>
</goals>
<configuration>
<outputFile>${project.build.directory}/effective.build.properties</outputFile>
</configuration>
</execution>
</executions>
</plugin>
Version info:
Teamcity version: 2020.2.1 (build 85633)
Maven version: 3.3.9
Java version:
openjdk version "1.8.0_272"
OpenJDK Runtime Environment Corretto-8.272.10.3 (build 1.8.0_272-b10)
OpenJDK 64-Bit Server VM Corretto-8.272.10.3 (build 25.272-b10, mixed mode)
This is the line TeamCity is using to invoke maven, I've only included property definitions that seemed relevant since there were so many that are definitely irrelevant (build numbers, names, timestamps, and other TC specifc, maven agnostic, config)
/usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64/bin/java
-Dclassworlds.conf=/home/ec2-user/BuildAgent/temp/buildTmp/teamcity.m2.conf
-Dmaven.home=/home/ec2-user/BuildAgent/tools/maven3_3
-DskipTests=true
-Dteamcity.build.properties.file=/home/ec2-user/BuildAgent/temp/buildTmp/teamcity.build4380238471360533686.properties
-Dtarget.env=prod
-classpath /home/ec2-user/BuildAgent/tools/maven3_3/boot/plexus-classworlds-2.5.2.jar: org.codehaus.plexus.classworlds.launcher.Launcher -f /home/ec2-user/BuildAgent/work/4508a7116faa21f3/pom.xml -B clean package
The contents of teamcity.m2.conf is as follows:
main is org.apache.maven.cli.MavenCli from plexus.core
set maven.home default ${user.home}/m2
[plexus.core]
load ${teamcity.maven.watcher.home}/*.jar
optionally ${maven.home}/lib/ext/*.jar
load ${maven.home}/lib/*.jar
load ${maven.home}/conf/logging
teamcity.build4380238471360533686.properties contains many properties, the value of target.env within that file is 'prod' as expected
This ended up being a known bug in TeamCity
The underlying issue seems to be that TeamCity uses the MAVEN_OPTS environment variable to pass system properties into maven by default, but properties in MAVEN_OPTS are treated differently from properties passed as arguments to the maven command itself.
The workaround is that for any property "foo" that gets set in a POM section, that you want to override in a TeamCity build, you have to specify it in the "Additional Maven command line parameters" with -Dfoo=value, or, if you're setting the value in a system or build property within the TeamCity build -Dfoo=%system.foo%.
I have a multi-module maven project. In the child module, failsafe plugin is used for the integration tests run. Some argLines are defined accordingly :
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<skipAfterFailureCount>1</skipAfterFailureCount>
<argLine>-Xmx2048M -Xss512M -XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC</argLine>
</configuration>
</plugin>
The problem is when I run the tests as mvn test or mvn integration-test, the arglines are not applied for the tests from neither the parent pom directory nor the child pom directory, but if I run the tests as mvn failsafe:integration-test from both of the directories, the arglines param are applied.
What is the reason behind this ? Is there any way to apply those params when I run the tests with mvn test command ? I tried to pass the parameters via command line as mvn test -Dchild.argline="-Xmx2048M -Xss512M -XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC"
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<skipAfterFailureCount>1</skipAfterFailureCount>
<argLine>${child.argline}</argLine>
</configuration>
</plugin>
But it didn't work.
Also I tried to bind test and integration-test phases to failsafe integration-test and defined configuration params there, but it didn't work as well..
I set the MAVEN_OPTS accordingly but it didn't help...
The failsafe plugin goals are not included in the lifecycle by default. The POM configuration must include it. See this SO answer for an example. Also ensure that the plugin definition is in the <plugins> section, not inside a <pluginManagement> element.
As for how to define options in the POM - use the names supplied in the documentation. So, to specify argLine, add
<argLine>...</argLine>
in the plugin config. To specify on the command line, note that "User property" for the value you want to set. For the failsafe plugin's argLine, the user property is also argLine, so on the command line specify
-DargLine=...
Maven knows nothing about child.argline so silently ignores it. Also note, attribute and user property names are case sensitive.
In a Java project, I depend on a third-party native library that in turn loads dependency dylibs via DYLD_LIBRARY_PATH. I've successfully run tests via Tycho's surefire plugin by setting this with the environmentVariables property of the plugin config, but a similar setup in a non-OSGi project leaves the DYLD_LIBRARY_PATH variable unset.
Here's a snippet of my functioning Tycho configuration:
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>tycho-surefire-plugin</artifactId>
<version>0.25.0</version>
<configuration>
<argLine>-Dfile.encoding=UTF-8 -Djava.library.path="${dylib-program}"</argLine>
<environmentVariables>
<DYLD_LIBRARY_PATH>${dylib-program}</DYLD_LIBRARY_PATH>
</environmentVariables>
</configuration>
</plugin>
With that, the tests run correctly, and outputting System.getenv("DYLD_LIBRARY_PATH") shows the set path.
Here's the equivalent snippet from my non-Tycho config:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Dfile.encoding=UTF-8 -Djava.library.path="${dylib-program}"</argLine>
<environmentVariables>
<DYLD_LIBRARY_PATH>${dylib-program}</DYLD_LIBRARY_PATH>
<OtherVar>bar</OtherVar>
</environmentVariables>
</configuration>
</plugin>
When I run this, however, the dependency library doesn't load properly and System.getenv("DYLD_LIBRARY_PATH") returns null. System.getenv("OtherVar"), however, returns "bar", so setting environment variables generally seems to work. That makes me suspect that there's something peculiar about DYLD_LIBRARY_PATH (and the same happened with LD_LIBRARY_PATH, but not PATH).
The behavior is the same when run in Eclipse (either as-is or with the path also set in the Run Configuration environment) and via the command line (again either as-is or with the environment variable explicitly exported before the run). The Tycho and non-Tycho projects are run on the same machine, with the same tools (other than the test plugins). I'm using macOS 10.12.3, Java 1.8.0_111, and Maven 3.3.9.
Is there a general limitation about setting this property, at least on a Mac, or is there a way I can work around this?
I am facing an issue with Maven and the Surefire plugin.
I have two tests: testDatePos.java and testDateNeg.java and for each test an environment variable must be set. It is the same environment variable (DATE_SHIFT) but not the same value (-1 and 1).
Is it possible to configure the section surefire-plugin in the pom.xml of maven to have those tests running?
Here is my pom.xml that exclude the testDatePos.java to have the mvn test running OK (I know this is NOT a solution):
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/testDatePos.java</exclude>
<!-- this test needs the env variable DATE_SHIFT=1 but
the test testDateNeg.java needs it at -1 -->
</excludes>
<environmentVariables>
<DATE_SHIFT>-1</DATE_SHIFT>
</environmentVariables>
</configuration>
</plugin>
You can do this by specifying two different executions of surefire in your pom, and forking each execution.
However, this means that these tests will only work when you're running them from maven, or at least you have to change the configuration everywhere you run them from. So, for the tests which require a environment variable, I would add this to the #Before/#After (or #BeforeClass/#AfterClass) of those specific tests.
#Before public void before() {
System.setProperty("DATE_SHIFT", "-1");
}
This way, you don't need to execute the tests in maven for them to work. You'll probably want to store the original value of DATE_SHIFT and restore it at the end of the test.
We are using JBehave and while running test we need to add a local folder and jars to the classpath while running the tests.
The problem is the local folder might vary from system to system. We want the tests to run by looking at the jars installed on that system and the resources defined on that system.
How to add a dependency to maven that could change from system to system?
You can use environment variables in your pom.xml using ${env.VARIABLE_NAME}. If you have the path to your local folder in the pom, you could replace it by a variable. If you do so you have to set that variable on every system you execute the maven job on. I have found some guides for linux and windows on how to do that. Hope this fits your problem.
The test are executed by the maven-surefire-plugin. The plugin has only one goal surefire:test and this goal supports the configuration of additionalClasspathElements.
You can find a usage example here. The example configuration on this page looks like this:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.2</version>
<configuration>
<additionalClasspathElements>
<additionalClasspathElement>path/to/additional/resources</additionalClasspathElement>
<additionalClasspathElement>path/to/additional/jar</additionalClasspathElement>
</additionalClasspathElements>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
I would go this way. To change the local folder location for each system you can use environment variables or different maven profiles.