I'm trying to use PigUnit with Maven. In Maven, my pig scripts are located under the module's root directory at <module>/src/main/pig/<scriptName>.pig
All the PigUnit test tutorials either code the absolute url of the pig script (which obviously won't work when the build runs anywhere but my machine) or the relative path and it just "magically" works. But when I put in either the script name directly or src/main/pig/<scriptName>.pig to PigTest, the script can't be found when running mvn test.
Test with line (using scala + scalatest):
val pigTest = new PigTest("src/main/pig/calcProductVectors.pig", args)
Results in:
- Script does something *** FAILED ***
java.io.FileNotFoundException: src/main/pig/calcProductVectors.pig (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.pig.pigunit.PigTest.readFile(PigTest.java:296)
at org.apache.pig.pigunit.PigTest.readFile(PigTest.java:292)
How do I get src/main/pig to be on the path when mvn test runs?
Okay, so I found some tests in our Java projects that just added the pig directory to project.build.testResources in maven:
<testResources>
<testResource>
<!-- Pig is being included in test/resource so that we can access it with PigUnit -->
<directory>src/main/pig</directory>
<filtering>true</filtering>
</testResource>
<testResource>
<directory>src/test/resources</directory>
<filtering>true</filtering>
</testResource>
</testResources>
Then the JUnit tests would pass src/main/pig/<scriptName>.pig to PigTest.
This didn't work in my project, not sure if it's because of scalatest or something else. I could see the contents of the directory getting copied to <module>/target/test-classes, but they still weren't found either with their direct name (which seems like it would make more sense because they were copied without the src/main/pig part) or that whole path which was working for the Java projects.
Eventually from this answer, I found out where the tests were running from on the file system, and it was just the module name, so I adjusted the path like so and it worked:
val pigTest = new PigTest("<module>/src/main/pig/calcProductVectors.pig", args)
Related
I have a maven project in which I have a property. I want to access it from Jenkins.
POM snippet:
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<timestamp>${build.time}</timestamp>
<outputFolder>C:/AutomationTestReports/${project.name}/ExecutionReport${timestamp}</outputFolder>
</properties>
This output folder property gets evaluated at runtime. I want to access this property in Jenkins to navigate to the output folder which gets generated as per the time stamp. For this to happen I need to capture this output folder property in Jenkins, which I am not able to do.
Using this output folder only I will be able to navigate to the folder to access the result file.
Question: How to access Maven properties from Jenkins job?
In the current version of Jenkins there is a nice and simple way to achieve this using "readMavenPom()".
For example, I want to read the version of the pom.xml file. But if it is using the newer maven "revision" practice, I need to read that "revision" value from the properties defined in the pom.xml.
def pomVersion = readMavenPom().version // read version
if (pomVersion.equals("\${revision}")) // using revision strategy, read revision property
{
pomVersion = readMavenPom().properties['revision']
}
So the best way is to use
readMavenPom().properties['PROPETY_NAME']
Jenkins Maven jobs provide by default some maven properties to be used during job configuration, however it doesn't provide any easy access to further properties defined into the pom file.
Hence, a custom approach is required. Here is a working solution which makes use of a groovy script and could be used to transform any defined Maven property into a new Jenkins variable at build time, to be then further used in additional post build steps.
Pre-requirements to this solution are:
Groovy is installed in the Jenkins server (easy step, just download it, unzip, set it to the path
The Jenkins Groovy Plugin is installed in Jenkins (easy step, Manage Jenkins > Manage Plugins, Available Plugins, install it, then configure it under Manage Jenkins > Configure System > Groovy, to make it point to the installation of step above
You can then add to the project pom the following to is build/plugins section:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0.0</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>write-project-properties</goal>
</goals>
<configuration>
<outputFile>${project.build.directory}/build.properties</outputFile>
</configuration>
</execution>
</executions>
</plugin>
What are we doing? Simply using the properties-maven-plugin and its write-project-properties goal to write the project properties into a new file under the target folder (target\build.properties). Note: properties in this file will already be interpolated.
This new file will be harmless for your build and project, ignored by any other build step, removed as part of a mvn clean invocation, but yet really helpful for Jenkins.
Then, in the concerned Jenkins job you can add a new Post Steps > Execute system Groovy script (and not an Execute Groovy script, bear the difference) pointing to a new groovy file you can store where desired or type into the console the following groovy code:
import hudson.model.*;
import hudson.util.*;
def thr = Thread.currentThread();
def currentBuild = thr?.executable;
def props = new Properties();
new File(currentBuild.workspace.toString()+"\\target\\build.properties").withInputStream {
stream -> props.load(stream);
}
println "detected from properties: "+props.get("outputFolder");
def newParamAction = new ParametersAction(new StringParameterValue("outputFolder", props.get("outputFolder")));
currentBuild.addAction(newParamAction);
What is this script doing? Simply reading the property file, printing to the console the outputFolder property as a log (to be removed, optionally) and then set it as a new Jenkins job variable with the same name (to be changed if required).
You can then use in further post build steps the ${outputFolder} new variable (or the %outputFolder% in Windows commands) and it will be correctly present.
As an example, you can debug it via a new Post Steps > Execute Windows Batch command and simply echo it. Here is a screenshot:
As output of a sample Jenkins job you would then have:
detected from properties: C:/AutomationTestReports/sample-project/Execution_(2016-04-24_12-11-13UTC)
[workspace] $ cmd /c call C:\dev\tomcat-7\temp\hudson6024790907972070905.bat
C:\Users\user-name\.jenkins\jobs\sample-maven\workspace>echo C:/AutomationTestReports/sample-project/Execution_(2016-04-24_12-11-13UTC)
C:/AutomationTestReports/sample-project/Execution_(2016-04-24_12-11-13UTC)
C:\Users\user-name\.jenkins\jobs\sample-maven\workspace>exit 0
Finished: SUCCESS
A quick and dirty solution.
Add an 'Execute Shell' step, and assign the property's value into a variable:
outputFolderVar=$(awk -F '[<>]' '/outputFolder/{print $3}' pom.xml)
environment {
OUTPUT_FOLDER = "${sh(script: 'mvn help:evaluate -Dexpression=outputFolder -q -DforceStdout', returnStdout: true)}"
}
pipeline-utility-steps
I have a set of files I'd like to include in the .jar generated by mvn compile. Unfortunately, I would like them to be placed in a specific path inside the .jar. For example, I want shaders/main.glsl to be in the .jar file as com/purplefrog/expglsl/castle/main.glsl
How do I specify this mapping in the pom.xml ? I can not mess with the directory heirarchy of the source project without throwing a wrench into other coders' workflows.
During the process-resources phase non-compilable files can be moved (by the maven-resources-plugin). What you should do is add a resource-block to your pom. Here you need to specify the directory. You can also add a targetPath. All together it would look like
<resource>
<directory>shaders</directory>
<!-- include all ore just a couple of files? -- >
<includes>
<include>main.glsl</include>
</includes>
<targetPath>com/purplefrog/expglsl/castle</targetPath>
</resource>
Now these files are copied to the target/classes and during the package phase they'll become part of the jar.
Take a look at the Maven Resources Plugin and this question.
Sounds like that should handle what you're looking to do if modifying the project structure up front isn't an option.
I use jmeter-maven-plugin (version 1.10.0) to run JMeter test - first I run it from IntelliJ, then from TeamCity (for both - command: mvn jmeter-graph:create-graph)
When I want to use the following configuration for jmeter-graph-maven-plugin:
<plugin>
<groupId>de.codecentric</groupId>
<artifactId>jmeter-graph-maven-plugin</artifactId>
<version>0.1.0</version>
<configuration>
<inputFile>${project.build.directory}/jmeter/results/*.jtl</inputFile>
<graphs>
<graph>
<pluginType>TransactionsPerSecond</pluginType>
<outputFile>${project.build.directory}/jmeter/results/TPS.png</outputFile>
</graph>
</graphs>
</configuration>
</plugin>
it works from IntelliJ, but in TeamCity I get:
ERROR: java.lang.IllegalArgumentException: Cannot find specified JTL file: /project/XX/opt/team-city-8.0.5/buildAgent/work/xxxxx/JMeter/target/jmeter/results/*.jtl
Result file exists (and it is previous used in xml-maven-plugin - even configuration is *.jtl - xml plugin works correctly in TeamCity).
When I use specific file name (so e.g. 20150317test-result.jtl instead of *.jtl) it works also from TeamCity.
How can I use general file name? Or maybe there is an option in jmeter-maven-plugin to define some fixed jtl file name (and then use it in jmeter-graph-maven-plugin)?
I did workaround for this issue.
I changed jmeter-graph-maven-plugin configuration to:
<inputFile>${project.build.directory}/jmeter/results/${fileName}.jtl</inputFile>
and now I run it using mvn jmeter-graph:create-graph -DfileName=%profile% (where profile name is the same as jmx test file).
I use cxf-codegen-plugin to generate a series of WS clients using Maven, on construction time. These WSDL reference some XSD schema definitions using a relative path like so: ../someService/schema.xsd
Now when I trigger a construction from Eclipse this works properly since my XSD files are placed in the right path.
But when I launch a construction job from Jenkins, it fails because it seems it's using Jenkins workspace as the root of the construction.
I don't even know if you can change this behavior of Jenkins, but since I have no control over my Jenkins instance, what I would like to know is for cxf-codegen-plugin to exclude XSD processing altogether, and then generate those classes explicitly using a different execution phase with a different plugin.
I've read you can do it like this:
<defaultOptions>
<extraargs>
<extraarg>-nexclude</extraarg>
<extraarg>http://*.ws.cntxes.emprego.xunta.es</extraarg>
</extraargs>
</defaultOptions>
But this assumes I know those namespaces prior to constructing, which I don't (WSDL files are taken from external dependencies using maven dependency plugin).
I also tried:
<wsdlRoot>${basedir}/src/main/resources/wsdl</wsdlRoot>
<includes>
<include>
**/*.wsdl
</include>
</includes>
<excludes>
<exclude>
*.xsd
</exclude>
</excludes>
But this does not work, the plugin just keeps parsing the XSD files and generating the related classes.
Is there any other way I'm missing to prevent the parsing of XSD files and just process the WSDL definitions?
EDIT: this is the error Jenkins is giving me:
[ERROR] Failed to execute goal org.apache.cxf:cxf-codegen-plugin:2.7.3:wsdl2java (generate-sources-wsclient-cxf) on project my-project: Execution generate-sources-wsclient-cxf of goal org.apache.cxf:cxf-codegen-plugin:2.7.3:wsdl2java failed: org.apache.cxf.wsdl11.WSDLRuntimeException: Fail to create wsdl definition from : file:/var/lib/jenkins/workspace/MYPROJECT/myproject-webservice/src/main/resources/wsdl/Descriptor/serviceDescriptor.wsdl
[ERROR] Caused by : WSDLException (at /definitions/types/xsd:schema): faultCode=PARSER_ERROR: Problem parsing '../xsd/schema.xsd'.: java.io.FileNotFoundException: /var/lib/jenkins/workspace/xsd/actividadFormativa.xsd (No such file or directory)
It is looking on the root of jenkins' workspace instead of /var/lib/jenkins/workspace/MYPROJECT/myproject-webservice/src/main/resources/wsdl/xsd/schema.xsd
I had the same problem (only with the wsdl files). After long research I figured out that the problem was a case sensitivity issue - windows (local CLI and eclipse builds) and the linux/unix hudson/jenkins build environment:
The problematic wsdl had a big letter S
<wsdlOption>
<wsdl>${basedir}/src/main/resources/Some.wsdl</wsdl>
</wsdlOption>
But on the filesystem the file was some.wsdl
So it was not the path issue (.../workspace/...) as I also initially expected...
I am trying to automate the generation of source files from the .thrift files and later the packaging. As far as I know, the maven-thrift-plugin is restrictive in the sense that source and destination directories are fixed. Is there any way I can specify the source and destination directories? I could probably achieve this by using the maven-antrun-plugin but I don't want to pollute my pom unnecessarily if I don't have to.
Thanks.
As far as I can see from the source (https://github.com/dtrott/maven-thrift-plugin/blob/master/src/main/java/org/apache/thrift/maven/ThriftCompileMojo.java) there are configuration properties that control this behaviour.
Try these properties, they should work:
thriftSourceRoot
thriftTestSourceRoot
outputDirectory
These props should be added to the <configuration> section along with <thriftExecutable>, etc:
<plugin>
<groupId>org.apache.thrift.tools</groupId>
<artifactId>maven-thrift-plugin</artifactId>
<version>0.1.10</version>
<configuration>
<thriftExecutable>/usr/local/bin/thrift</thriftExecutable>
<thriftSourceRoot>${basedir}/src/main/my_custom_thrift_root</thriftSourceRoot>
</configuration>
<executions>
...
</plugin>
I also ended up going the maven-antrun-plugin route, here is a functional example: https://github.com/cobbzilla/cobbzilla-wizard/tree/master/wizard-thrift
pom.xml uses maven-antrun-plugin to exec the thrift target in build.xml
build.xml does the thrift compilation and packaging.
Generated sources go back into the source tree; I don't like derived files polluting my upstream source control, so the generated files are under a thrift package, and the package directory is in a .gitignore file. A bit kludgy.
A better way I learned about since writing that code is to compile multiple java source directories in a single maven project, which would be cleaner.