I am trying to run Testim tests with Jenkins. For which i have created a Jenkins Pipeline job. I am able to run it successfully but not able to publish a report. I am trying to publish junit report, for that i am using following build command
node {
if(Names){
def nameFlags = Names.split(',').collect { " --name \"${it.trim()}\" "}
def browserFlags = Browser_Configuration.split(',').collect { " --test-config \"${it.trim()}\" " }
bat "testim --host localhost --port 4444 --mode selenium --token \"xxxxx\" --project \"xxxx\" --params-file \".\\testdata.js\" ${nameFlags.join(' ')} ${browserFlags.join(' ')} --report-file test-results/testim-tests-$BUILD_NUMBER-report.xml"
}else if(Suits){
def suitFlags = Suits.split(',').collect { " --suite \"${it.trim()}\" " }
def browserFlags = Browser_Configuration.split(',').collect { " --test-config \"${it.trim()}\" " }
bat "testim --host localhost --port 4444 --mode selenium --token \"xxx\" --project \"xxxx\" --params-file \".\\testdata.js\" ${suitFlags.join(' ')} ${browserFlags.join(' ')} --report-file test-results/testim-tests-$BUILD_NUMBER-report.xml"
}
junit checksName: 'Reports', testResults: 'test-results/*.xml'
}
By using this code, i am able to see the generated xml file in the workspace folder. But it is not publishing in Jenkins UI. How can i publish the report in Jenkins UI for this Testim's test.
Please use post pipeline section in order to publish testim results to Jenkins UI. E.g.:
post{
always {
junit 'test-results/*.xml'
}
Related
I am currently having issues with using tagging with the cypress-cucumber-preprocessor package. I know that the cypress-tags has been removed and made redundant so I'm trying to set up tagging using the new syntax but to no avail.
Here is my feature:
Feature: duckduckgo.com
Rule: I am on a desktop
Scenario: visiting the frontpage
When I visit <site>
Then I should see a search bar
#google
Examples:
| site |
| google.com |
#duckduckgo
Examples:
| site |
| duckduckgo.com |
And my step definitions:
import { When, Then } from "#badeball/cypress-cucumber-preprocessor";
When(`I visit` + url, () => {
if(url === 'duckduckgo.com') return cy.visit("https://www.duckduckgo.com");
if(url === 'google.com') return cy.visit("https://www.google.com");
});
Then("I should see a search bar", () => {
cy.get("input").should(
"have.attr",
"placeholder",
"Search the web without being tracked"
);
});
When I try to run my tests with npx cypress run --env tags="#google", it gives me an error saying url in my steps definitions isn't defined. What am I doing wrong?
Try to add script with this command in package.json file this way:
"scripts": {
"open:test": "npm run clean:report && cypress open --env configFile=test,TAGS=#test" (or any tag you need)
}
And then use it as:
npn run open:test
The main difference besides rapping it into script is not using quotes, maybe this will help you
I have a Jenkins pipeline where I want to first build my project (Stage A) and trigger an asynchronous long running external test process with the built artifacts. The external test process then resumes the Job using a callback. Afterwards (Stage B) performs some validations of the test results and attaches them to the job. I don't want to block an executor while the external test process is running so I came up with the following Jenkinsfile which mostly suites my needs:
#!/usr/bin/env groovy
pipeline {
agent none
stages {
stage('Stage A') {
agent { docker { image 'my-maven:0.0.17' } }
steps {
script {
sh "rm testfile.txt"
sh "echo ABCD > testfile.txt"
sh "cat testfile.txt"
}
}
}
stage('ContinueJob') {
agent none
input { message "The job will continue once the asynchronous operation has finished" }
steps { echo "Job has been continued" }
}
stage('Stage B') {
agent { docker { image 'my-maven:0.0.17' } }
steps {
script {
sh "cat testfile.txt"
def data = readFile(file: 'testfile.txt')
if (!data.contains("ABCD")) {
error("ABCD not found in testfile.txt")
}
}
}
}
}
}
However, depending on the load of the Jenkins or the time passed or some unknown other conditions, sometimes the files that I create in "Stage A" are no longer available in "Stage B". It seems that Jenkins switches to a different Docker node which causes the loss of workspace data, e.g. in the logs I can see:
[Pipeline] { (Stage A)
[Pipeline] node
Running on Docker3 in /var/opt/jenkins/workspace/TestJob
.....
[Pipeline] stage
[Pipeline] { (Stage B)
[Pipeline] node
Running on Docker2 in /var/opt/jenkins/workspace/TestJob
Whereas with a successful run, it keeps using e.g. node "Docker2" for both stages.
Note that I have also tried reuseNode true within the two docker sections but that didn't help either.
How can I tell Jenkins to keep my workspace files available?
As pointed out by the comment from #Patrice M. if the files are not that big (which is the case for me) stash/unstash are very useful to solve this problem. I have used this combination now since a couple of months and it has solved my issue.
Since the Gradle integrated jacoco plugin cannot be used, there is a task skipped problem. I try to use Jacoco Command Line Interface and Exec - Gradle DSL Version 7.0.
def reportTask = tasks.create(reportTaskName, Exec.class) {
group = 'Reporting'
description = "Generate End To End Android Test Jacoco coverage reports on the ${buildVariant.capitalize()} build by Command Line Interface."
workingDir "${project.buildDir}"
commandLine "java", "-jar", "${jacocoCLI}", "report", "${project.buildDir}/outputs/code_coverage/${buildVariant}AndroidTest/connected/*.ec",
"--classfiles", "${project.buildDir}/intermediates/javac/${buildVariant}/classes/",
"--classfiles", "${project.buildDir}/tmp/kotlin-classes/${buildVariant}/",
"--html", "${project.buildDir}/jacocoReport/jacocoHtml/",
"--sourcefiles", "${project.projectDir}/src/main/java",
"--xml", "${project.buildDir}/jacocoReport/jacocoXml.xml"
}
reportTask
When I run this task, I get:
> Task :qtx:jacocoEtoeAndroidTestReportByCLI FAILED
[INFO] Loading execution data file <xxx>/build/outputs/code_coverage/etoeDebugAndroidTest/connected/*.ec.
Exception in thread "main" java.io.FileNotFoundException: <<xxx>>/build/outputs/code_coverage/etoeDebugAndroidTest/connected/*.ec (No such file or directory)
However, if I execute it in the terminal, as shown in the figure below, the task will succeed.
What's the difference between these two usages and how can I use Regular Expressions in Gradle Exec?
UPDATE
def _createAndroidTestCoverageReportByCLI(productFlavorName = "") {
def buildTypeName = "debug"
def buildVariant
buildVariant = "${productFlavorName}${buildTypeName.capitalize()}"
def getExecDataTaskName = "${ANDROID_TEST_REPORT_TASK_PREFIX}${productFlavorName.capitalize()}${ANDROID_TEST_REPORT_TASK_SUFFIX}ExecDataGetter"
def reportTaskName = "${ANDROID_TEST_REPORT_TASK_PREFIX}${productFlavorName.capitalize()}${ANDROID_TEST_REPORT_TASK_SUFFIX}ByCLI"
def jacocoCLI
String execData
project.rootProject.allprojects.each { project ->
if (project.name == "testlib") {
jacocoCLI = "${project.projectDir}/lib/jacococli.jar"
}
}
def getExecDataTask = tasks.create(getExecDataTaskName, JacocoReport.class) {
def listEc = []
def dir = new File("${project.buildDir}/outputs/code_coverage/${buildVariant}AndroidTest/connected/")
dir.eachFileRecurse() { file ->
listEc << file
}
execData = listEc.join(" ")
}
def reportTask = tasks.create(reportTaskName, Exec.class) {
group = 'Reporting'
description = "Generate End To End Android Test Jacoco coverage reports on the ${buildVariant.capitalize()} build by Command Line Interface."
workingDir "${project.buildDir}"
// commandLine "java", "-jar", "${jacocoCLI}", "report", "${execData}",
// "--classfiles", "${project.buildDir}/intermediates/javac/${buildVariant}/classes/",
// "--classfiles", "${project.buildDir}/tmp/kotlin-classes/${buildVariant}/",
// "--html", "${project.buildDir}/jacocoReport/jacocoHtml/",
// "--sourcefiles", "${project.projectDir}/src/main/java",
// "--xml", "${project.buildDir}/jacocoReport/jacocoXml.xml"
commandLine "java -jar ${jacocoCLI} report ${execData} --classfiles ${project.buildDir}/intermediates/javac/${buildVariant}/classes/ --classfiles ${project.buildDir}/tmp/kotlin-classes/${buildVariant}/ --html ${project.buildDir}/jacocoReport/jacocoHtml/ --sourcefiles ${project.projectDir}/src/main/java --xml ${project.buildDir}/jacocoReport/jacocoXml.xml"
}
reportTask.dependsOn getExecDataTask
}
Error
> Task :qtx:jacocoEtoeAndroidTestReportByCLI FAILED
Execution failed for task ':qtx:jacocoEtoeAndroidTestReportByCLI'.
> A problem occurred starting process 'command 'java -jar <RepoPath>/testlib/lib/jacococli.jar report <RepoPath>/QTX/build/outputs/code_coverage/etoeDebugAndroidTest/connected/spoon1619160691046.ec <RepoPath>/QTX/build/outputs/code_coverage/etoeDebugAndroidTest/connected/spoon1619161047292.ec <RepoPath>/QTX/build/outputs/code_coverage/etoeDebugAndroidTest/connected/spoon1619160787358.ec --classfiles <RepoPath>/QTX/build/intermediates/javac/etoeDebug/classes/ --classfiles <RepoPath>/QTX/build/tmp/kotlin-classes/etoeDebug/ --html <RepoPath>/QTX/build/jacocoReport/jacocoHtml/ --sourcefiles <RepoPath>/QTX/src/main/java --xml <RepoPath>/QTX/build/jacocoReport/jacocoXml.xml''
I tried to pass in execfiles through list, the task still failed. When I copied the failed command to the terminal for execution, it could be executed successfully.
Why? How can it be executed in Gradle?
I have wrote a Jenkins Pipeline Groovy for executing multiple project maven sonar analysis. The code is working fine but the issue is that sometimes build fails for some projects which I need to track it properly. My executeMavenSonarBuild function is given as below
def executeMavenSonarBuild(projectName) {
stage ('Execute Maven Build for '+projectName)
{
sh """ {
cd ${projectName}/
mvn clean install verify sonar:sonar
} || {
echo 'Build Failed'
}
"""
}
return true;
}
If build fails it prints echo 'Build Failed' but how we can return a false Boolean as the return to the function.
You have to get the status from the mvn call itself..which should look like this:
def result = sh ( script: 'mvn ...', returnStatus: true)
I am using a Phing build script with Jenkins and would like to run it end to end on a job and capture all the reports. The problem is it stop building on a failed build step. Is there a way or a plugin that would continue the job even on failures?
Thanks
I don't know a lot about Phing but, since it's based on Ant, if the build step you are executing has a "failonerror" attribute you should be able to set it to false so that the entire build doesn't fail if the step returns an error.
Yes, use try, catch block in you pipeline scripts
example:
try {
// do some stuff that potentially fails
} catch (error) {
// do stuff if try fails
} finally {
// when you need some clean up to do
}
Or alternatively if you use sh commands to run these tests, consider running your sh scripts with the "|| true" suffix, this tells the linux sh script to exit with a result code of 0, even if your real command exited with an exit code.
example:
stage('Test') {
def testScript = ""
def testProjects = findFiles(glob: 'test/**/project.json')
if (!fileExists('reports/xml')) {
if (!fileExists('reports')) {
sh "mkdir reports"
}
sh "mkdir reports/xml"
}
for(prj in testProjects) {
println "Test project located, running tests: " + prj.path
def matcher = prj.path =~ 'test\\/(.+)\\/project.json'
testScript += "dotnet test --no-build '${prj.path}' -xml 'reports/xml/${matcher[0][1]}.Results.xml' || true\n"
}
sh testScript