I have code that reads in a pom.xml file then attempts to re-serialize and write it back out:
// Get the file raw text
def pomXMLText = readFile(pomFile)
// Parse the pom.xml file
def project = new XmlSlurper(false, false).parseText(pomXMLText)
... do some useful stuff ...
def pomFileOut = "$WORKSPACE/pomtest.xml"
def pomXMLTextOut = groovy.xml.XmlUtil.serialize(project)
println "pomXMLTextOut = $pomXMLTextOut" // <-- This line prints to updated XML
writeFile file: pomFileOut, text: pomXMLTextOut // <-- This line crashes with the error listed in the posting title: java.io.NotSerializableException: groovy.util.slurpersupport.NodeChild
I've tried casting the pomXMLTextOut variable to a String. I tried applying the .text() method, which gets a jenkins sandbox security error. Has anyone else been able to successfully write an XML file from a groovy script running in a Jenkins pipeline?
BTW, I've also tried using a File object, but that isn't remotable across jenkins nodes. It works as long as the job always runs on master.
You could try a #NonCPS annotation and close those non-serializable objects in a funcation like this
#NonCPS
def writeToFile(String text) {
...
}
Here's the explanation from Pipeline groovy plugin
#NonCPS methods may safely use non-Serializable objects as local
variables
Related
I have a groovy file created under "vars" in a Jenkins-shared-lib.
Few variables are defined inside call().
I want to mock the variables in a groovy test file.
Variables are defined in sonarGradleProject.groovy in vars:
#!/usr/bin/env groovy
import static com.example.jenkins.Constants.*
def call(Map params = [:]) {
def sonarCredId = params.sonarCredentialId ?: SONAR_CREDENTIAL_ID_DEFAULT;
def sonarUrl = params.sonarUrl ?: SONAR_URL;
def profile = params.profile ?: 'example';
withCredentials([string(credentialsId: sonarCredId, variable: 'SONAR_TOKEN')]) {
sh "./gradlew --no-daemon jacocoTestReport sonarqube -P${profile} -Dsonar.host.url=${sonarUrl} -Dsonar.login=$SONAR_TOKEN -Dsonar.verbose=true"
}
}
Test file looks like this:
import com.example.jenkins.testing.JenkinsPipelineSpecification
class SonarGradleProjectSpec extends JenkinsPipelineSpecification {
def "expected minor value"() {
setup:
def sonarGradleProject = loadPipelineScriptForTest("vars/sonarGradleProject.groovy")
explicitlyMockPipelineStep('withCredentials')
when:
def verType = sonarGradleProject(profile: 'foo')
then:
1 * getPipelineMock("sh")("./gradlew --no-daemon jacocoTestReport sonarqube -P${profile} -Dsonar.host.url=${sonarUrl} -Dsonar.login=$SONAR_TOKEN -Dsonar.verbose=true")
expect:
'foo' == profile
}
}
On executing the test case, I get this error:
java.lang.IllegalStateException: There is no pipeline variable mock for [profile].
1. Is the name correct?
2. Is it a GlobalVariable extension point? If so, does the getName() method return [profile]?
3. Is that variable normally defined by Jenkins? If so, you may need to define it by hand in your Spec.
4. Does that variable come from a plugin? If so, is that plugin listed as a dependency in your pom.xml?
5. If not, you may need to call explicitlyMockPipelineVariable("profile") during your test setup.
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at com.example.jenkins.testing.JenkinsPipelineSpecification.addPipelineMocksToObjects_closure1$_closure15(JenkinsPipelineSpecification.groovy:755)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at SonarGradleProjectSpec.expected minor value(sonarGradleProjectSpec.groovy:12)
Caused by: groovy.lang.MissingPropertyException: No such property: (intercepted on instance [SonarGradleProjectSpec#3dfb1626] during test [SonarGradleProjectSpec#3dfb1626]) profile for class: SonarGradleProjectSpec
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at com.example.jenkins.testing.JenkinsPipelineSpecification.addPipelineMocksToObjects_closure1$_closure15(JenkinsPipelineSpecification.groovy:754)
... 3 more
I want to use the variables in test.
How do I mock them?
I've run into similar issues before. The issue for me is that I was trying to verify the interactions of a mock that uses string interpolation. So for the one entry in the then section...
1 * getPipelineMock("sh")("./gradlew --no-daemon jacocoTestReport sonarqube -P${profile} -Dsonar.host.url=${sonarUrl} -Dsonar.login=$SONAR_TOKEN -Dsonar.verbose=true")
I would define the profile and sonarUrl variables in the setup section (i.e., def profile = 'test123'), referenced to verify the mocked sh step interaction. I believe you would also want to adjust the assignment of default values to use the null-safe ?. operator to access key-value pairs in the params map. So def sonarUrl = params.sonarUrl ?: SONAR_URL; would become def sonarUrl = params?.sonarUrl ?: SONAR_URL;
You may have already addressed this and omitted it from the question, but I figured I'd share this information too. Please ignore the following information if you've already addressed it. :)
The three environment variables defined in the script - SONAR_CREDENTIAL_ID_DEFAULT, SONAR_URL, and SONAR_TOKEN - will need to be injected into the script before calling sonarGradleProject. That should be sufficient to test default values if key-value pairs are not provided in the params map.
setup:
def sonarGradleProject = loadPipelineScriptForTest("vars/sonarGradleProject.groovy")
explicitlyMockPipelineStep('withCredentials')
// Inject environment variables.
// First argument is the variable name and second argument is the variable value.
sonarGradleProject.getBinding().setVariable('SONAR_CREDENTIAL_ID_DEFAULT', 'test123')
sonarGradleProject.getBinding().setVariable('SONAR_URL', 'test123')
sonarGradleProject.getBinding().setVariable('SONAR_TOKEN', 'test123')
We have a large project that has multiple separate declarative pipeline file definitions. This is used to build different apps and installers from the single code base.
Right now, all of these files contain a large block of "code" used to generate the email body and JIRA update messages. examples:
// Get a JIRA's to add Comments to
// Return map of JIRA id to comment text from all commits for that JIRA
#NonCPS
def getJiraMap() {
a bunch of stuff
return jiraset
}
// Get the body text for the emails
def getMailBody1() {
return "See: ${BUILD_URL}\n\nChanges:\n" + getChangeString() + "\n" + testStatuses()
}
etc...
What I would like to do is have all these common methods in a separate file that all the other pipeline files can include. This seems like it SHOULD be easy, but all examples I've found appear to be rather complex involving a separate SCM - which is NOT what I want.
Updates:
Going through the various suggestions given in that link, I make the following file - BuildTools.groovy: Note that this file is in the same directory as the jenkins pipeline file that uses it.
import hudson.tasks.test.AbstractTestResultAction
import hudson.model.Actionable
Class BuildTools {
// Get a JIRA's to add Comments to
// Return map of JIRA id to comment text from all commits for that JIRA
#NonCPS
def getJiraMap() {
def jiraset = [:]
.. whole bunch of stuff ..
Here are the various things I've tried, and the results.
File sourceFile = new File("./AutomatedBuild/BuildTools.groovy");
Class gcl = new GroovyClassLoader(getClass().getClassLoader()).parseClass(sourceFile);
GroovyObject bt = (GroovyObject) gcl.newInstance();
Fails with:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use method java.lang.Class getClassLoader
evaluate(new File("./AutomatedBuild/BuildTools.groovy"))
def bt = new BuildTools()
Fails with:
15:29:07 WorkflowScript: 8: unable to resolve class BuildTools
15:29:07 # line 8, column 10.
15:29:07 def bt = new BuildTools()
15:29:07 ^
import BuildTools
def bt = new BuildTools()
Fails with:
15:35:58 WorkflowScript: 16: unable to resolve class BuildTools (note that BuildTools.groovy is in the same folder as this script)
15:35:58 # line 16, column 1.
15:35:58 import BuildTools
15:35:58 ^
GroovyShell shell = new GroovyShell()
def bt = shell.parse(new File("./AutomatedBuild/BuildTools.groovy"))
Fails with:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new groovy.lang.GroovyShell
I've got a Java project build with Gradle and a property file that contains custom configuration for my testing framework (amount of thread to use, test environment url, custom username & password for those environments, etc...).
I'm facing an issue related to using properties from that file that I can't figure out:
if my Test task include '**/*Test.class', all tests are running as expected.
if my Test task include '**/MyTest.class', only that test is running as expected.
if my Test task include readProperty(), the task is skipped as NO-SOURCE. <- this is the part I can't understand - as the readProperty return the correct value.
Let's get into details:
This is how the property is defined in a my.property file:
testng.class.includes='**/MyTest.class'
This is what the build.gradle file looks like:
Properties props = new Properties()
props.load(new FileInputStream(projectDir.toString() + '/common.properties'))
def testsToRunWorking(p) {
String t = 'MyTest.class'
println "Tests = $t"
return t ? t : '**/*Test.class'
}
def testsToRunNotWorking(p) {
String t = getProperty(p, "testng.class.includes")
println "Tests = $t"
return t ? t : '**/*Test.class'
}
task testCustom(type: Test) {
outputs.upToDateWhen { false }
testLogging.showStandardStreams = true
classpath = configurations.customTest + sourceSets.customTest.output
include testsToRunNotWorking(props) ///< Does not work!
// include testsToRunWorking(props) ///< Works!
useTestNG()
}
In terms of debugging:
The println properly return the value I expect, even when the testCustom task doesn't do what I would expect.
I tried adding a dependsOn task just to print the content of testCustom.configure { println $includes } which looks correct as well.
--info
Tests = '**/MyTest.class'
:clean
:compileCustomTestJava - is not incremental (e.g. outputs have changed, no previous execution, etc.).
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:processCustomTestResources
:customTestClasses
:testCustom NO-SOURCE
The core of the issue seems to be coming from the fact that I'm reading that value from property. I hard coded inside the build.gradle everything works as expected. If read from a property file - build stops with a NO-SOURCE statement.
Any idea?
Thanks!
You are using quotation marks in the values of your property files. Everything that comes after the assignment sign in a property file is used as value, so the quotation marks remain in the string. They are printed in your output Tests = '**/MyTest.class'. On the other hand, if you define a string in your (Groovy) code with quotation marks, they are not included in the string. Therefor, the passed strings are not the same.
Remove the quotation marks from your property file(s) and everything should work, since the class files will match your string without the quotation marks.
I have created a complex pipeline. In each stage I have called a job. I want to see the console output for each job in a stage in Jenkins. How to get it?
The object returned from a build step can be used to query the log like this:
pipeline {
agent any
stages {
stage('test') {
steps {
echo 'Building anotherJob and getting the log'
script {
def bRun = build 'anotherJob'
echo 'last 100 lines of BuildB'
for(String line : bRun.getRawBuild().getLog(100)){
echo line
}
}
}
}
}
}
The object returned from the build step is a RunWrapper class object. The getRawBuild() call is returning a Run object - there may be other options than reading the log line-by-line from the looks of this class. For this to work you need to either disable the pipeline sandbox or get script approvals for these methods:
method hudson.model.Run getLog int
method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild
If you are doing this for many builds, it would be worth putting some code in a pipeline shared library to do what you need or define a function in the pipeline.
I am currently modularizing our gradle build in order to have a libs/commons.gradle file containing a lot of global stuff. I need this because of various branches of the software beeing developed in parallel and we'd like to avoid to spread every scriptfile change among all branches.
So I created that lib file and used "apply from" to load it :
apply from: 'gradle/glib/commons.gradle'
Inside commons.gradle I define the svnrevision function :
...
def svnRevision = {
ISVNOptions options = SVNWCUtil.createDefaultOptions(true);
SVNClientManager clientManager = SVNClientManager.newInstance(options);
SVNStatusClient statusClient = clientManager.getStatusClient();
SVNStatus status = statusClient.doStatus(projectDir, false);
SVNRevision revision = status.getCommittedRevision();
return revision.getNumber().toString();
}
...
I am calling the function from my including build.gradle:
...
task writeVersionProperties {
File f = new File(project.webAppDirName+'/WEB-INF/version.properties');
if (f.exists()) { f.delete(); }
f = new File(project.webAppDirName+'/WEB-INF/version.properties');
FileOutputStream os = new FileOutputStream(f);
os.write(("version="+svnRevision()).getBytes());
os.flush();
os.close();
}
...
But I end up in :
...
FAILURE: Build failed with an exception.
* Where:
Build $PATH_TO/build20.gradle
* What went wrong:
A problem occurred evaluating root project 'DEV_7.X.X_GRADLEZATION'.
> Could not find method svnRevision() for arguments [] on root project 'DEV_7.X.X_GRADLEZATION'.
...
So my queston is : How can I call a subfunction in gradle, which is defined in an included script?
Any help appreciated!
From http://www.gradle.org/docs/current/userguide/writing_build_scripts.html:
13.4.1. Local variables
Local variables are declared with the def keyword. They are only
visible in the scope where they have been declared. Local variables
are a feature of the underlying Groovy language.
13.4.2. Extra properties
All enhanced objects in Gradle's domain model can hold extra
user-defined properties. This includes, but is not limited to,
projects, tasks, and source sets. Extra properties can be added, read
and set via the owning object's ext property. Alternatively, an ext
block can be used to add multiple properties at once.
If you declare it as:
ext.svnRevision = {
...
}
and don't change the call, I expect it will work.