I am trying to get my test results visible in Jira using Allure 2.13.8. The test results are visible but are not displayed as links back to the report, so called backlinks.
Here is my situation.
When I generate an Allure report with maven using allure-maven:
mvn clean test allure:report
an executor.json is created with maven build info inside allure-results dir.
environment {
ALLURE_JIRA_LAUNCH_ISSUES = 'xxx-9'
ALLURE_JIRA_ENABLED = 'true'
ALLURE_JIRA_ENDPOINT = 'http://localhost:8080/rest/'
ALLURE_JIRA_USERNAME = 'xxx'
ALLURE_JIRA_PASSWORD = 'xxxx'
}
stage('Test Execution') {
steps {
withMaven(options: [artifactsPublisher(disabled: true)],
maven: 'MAVEN_HOME') {
sh "mvn clean test -Dtest=ApplyForCreditCardTestsDE,_FailedTests* -Denv=qa-de"
}
}
}
The above env variables trigger the jira-plugin within Allure to export test results and launch to Jira. This works, but this executor.json do not contain links needed to create these backlinks. Just some limited info such as project name, that's it.
When I generate an Allure report with Jenkins Allure plugin:
stage('Generate Test Report') {
steps {
script {
allure([
includeProperties: true,
jdk: '',
properties: [
[key: 'ALLURE_JIRA_LAUNCH_ISSUES',value: 'xxx-9'],
[key: 'ALLURE_JIRA_ENABLED',value: 'true'],
[key: 'ALLURE_JIRA_ENDPOINT',value: 'http://localhost:8080/rest/'],
[key: 'ALLURE_JIRA_USERNAME',value: 'xxx'],
[key: 'ALLURE_JIRA_PASSWORD',value: 'xxxx']
],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'target/allure-results']]
])
}
}
}
an executor.json is created with jenkins build info inside allure-results dir.
{"buildName":"test #35","buildOrder":"35","reportName":"AllureReport","name":"Jenkins","buildUrl":"http://localhost:8085/job/test/35/","reportUrl":"http://localhost:8085/job/test/35/allure","type":"jenkins","url":"http://localhost:8085/"}
This is what I need to be pushed to Jira.
For some reason I cannot figure out how the jira-plugin can be triggered from this plugin? Or do I need to use Allure Command Line Interface to do this? I'm lost.
If you can help me out that would be much appreciated.
Related
I have a Nightwatch.js test suite running. When it completes, I've configured the output directory using the output_folder setting. It produces JUnit XML files in that directory correctly. I have an existing automation tool which scans the directory for JUnit test XML files and reports on them. Unfortunately it only matches files in the directory with a naming scheme: TEST-.xml. Let's assume that I can't change the matching rules on my automation tool. I'm looking for a way to add "TEST-" as a prefix to my tests. Ideally I can do this by configuring Nightwatch. Does Nightwatch support this configuration? I can't find any such options.
I ended up changing my test scripts in package.json so that they did a rename after the test was run. Here is what they were before:
{
// ...
"scripts": {
"integ-tests": "<some nightwatch command>"
}
// ...
}
Here is what they were after:
{
// ...
"scripts": {
"rename-integ-tests": "node -e \"require('fs').readdir('<my test directory>', (err, files) => { files.forEach(file => { if(file.endsWith('.xml') && ! file.startsWith('TEST-')) { fs.rename('<my test directory>' + file, '<my test directory>/TEST-' + file, function(err) { if (err) console.log(err); console.log('Renamed Smoke Test: ' + file + ' to TEST-' + file) }) } }); });\"",
"private-integ-tests": "<some nightwatch command>",
"integ-tests": "npm run private-integ-tests && npm run rename-integ-tests"
}
// ...
}
Usecase: I want to send jenkins job console log to elasticsearch, from there to kibana so that i can visualise the data.
I am using logstash plugin to achieve this. For freestyle job logstash plugin configuration is working fine but for jenkins pipeline jobs I am getting all required data like build number, job name, build duration and all but it is not showing the build result i.e., success or failure it is not showing.
I tried in two ways:
1.
stage('send to ES') {
logstashSend failBuild: true, maxLines: -1
}
2.
timestamps {
logstash {
node() {
sh'''
echo 'Hello, World!'
'''
try {
stage('GitSCM')
{
git url: 'github repo.git'
}
stage('Initialize')
{
jdk = tool name: 'jdk'
env.JAVA_HOME = "${jdk}"
echo "jdk installation path is: ${jdk}"
sh "${jdk}/bin/java -version"
sh '$JAVA_HOME/bin/java -version'
def mvnHome = tool 'mvn'
}
stage('Build Stage')
{
def mvnHome = tool 'mvn'
sh "${mvnHome}/bin/mvn -B verify"
}
currentBuild.result = 'SUCCESS'
} catch (Exception err) {
currentBuild.result = 'FAILURE'
}
}
}
}
But in both ways I am not getting build result i.e., success or failure in my elasticsearch or kibana.
Can someone help.
I didn't find a clear way to do that, my solution was add those lines at the end of the Jenkinsfile:
echo "Current result: ${currentBuild.currentResult}"
logstashSend failBuild: true, maxLines: 3
In my case, I dont need it to send all console logs, only one log with the result per job.
I'm trying to add test step information into an allure report, i use cypress and generate report with allure, report are correctly generate but no details appears in testcases.
I try to use the on('task') without success...
my plugins/index.js contain:
...
on('task', {
allureTestStep () {
const testStep = allure.createStep("Initial", () => {
console.log("First Test")
});
testStep()
return null
}
})
...
and i call it with:
cy.task('allureTestStep')
in my test.
No log in console only two error:
allure-js-commons: Unexpected startStep() of initial. There is no parent step
First Test
allure-js-commons: Unexpected endStep(). There are no any steps running
and in the report nothing is displayed(no error, no step detail).
Any help is welcome :)
I'm using the cypress-allure plugin and you can set it to show cypress commands in the test results. If you call cy.log those also get placed there. It's pretty nice.
it(`Should have title of ${treeListTitle}`, () => {
cy.log('title should be set');
...
}
Notice in the allure results where that is printed out.
Automated the e2e tests in CI enabled framework with protractor and jasmine in VSTS. Used the jasmine custom reporter to load the e2e test results under sauce labs tab in VSTS build definition. But, it is not loading all the tests. It is displaying only the last e2e test ran in the build. Console log is getting printed for all the e2e tests. Please see the code below.
let sauceLabsReporter: jasmine.CustomReporter = {
specDone: (result: jasmine.CustomReporterResult): void => {
Util.log('*** sauceLabsReporter: result.fullName:', result.fullName);
Util.log('*** sauceLabsReporter: result.status:', result.status);
Util.log('*** sauceLabsReporter: result.testCaseId:', result.testCaseId);
if (result.testCaseId) {
result.fullName = `(Testcase ID: ${result.testCaseId}): ${result.fullName}`;
Util.log('*** sauceLabsReporter: UPDATED result.fullName:', result.fullName);
}
Util.updateSauceLabsJobTitle(result.fullName);
if (result.status) {
Util.updateSauceLabsTestState(result.status);
}
}
};
export function updateSauceLabsJobTitle(title: string): promise.Promise<void> {
let fullTitle: string = `${title} | (${getHostname()})`;
return browser.executeScript(`sauce:job-name=${fullTitle}`)
.then(() => browser.getSession())
.then((session: Session) => {
if (isThisVSTSBuildAgent()) {
//The VSTS Sauce Labs add-on gets information by parsing the console log.
//tslint:disable-next-line:no-console
console.log(`SauceOnDemandSessionID=${session.getId()} job-name=${fullTitle}`);
}
});
}
export function updateSauceLabsTestState(state: string): promise.Promise<{}> {
return browser.executeScript(`sauce:job-result=${state}`);
}
I work on the same project as Padma. It had to do with restartBrowserBetweenTests being set to false which conflates all tests into a single job. After setting it to true, each test became its own job.
I'm using Nightwatch JS to run my e2e tests with the Mocha runner.
I want to integrate an HTML reporter that with the suite.
I'm trying to use the nightwatch-html-reporter package. But as far as I understand there is a problem with the CLI commands (it's written in the Nightwatch docs that --reporter will not work when using mocha).
I also copied the code sample from nightwatch-html-reporter to my globals.js but it doesn't seem to work either.
The tests run but there is no output anywhere.
Here is my folder structure:
project
src
spec
e2e
globals
globals.js
tests
smoke
testFile.js
nightwatch.conf.js
Here is my conf file:
const seleniumServer = require('selenium-server-standalone-jar');
const chromeDriver = require('chromedriver');
module.exports = {
src_folders: ['src/spec/e2e/tests'],
output_folder: 'report',
page_objects_path: [
'src/spec/e2e/pageObjects'
],
globals_path: 'src/spec/e2e/globals/globals.js',
custom_commands_path: 'src/spec/e2e/customCommands',
selenium: {
start_process: true,
server_path: seleniumServer.path,
host: '127.0.0.1',
port: 4444,
cli_args: {
'webdriver.chrome.driver': chromeDriver.path
}
},
test_runner: {
type: 'mocha',
options: {
ui: 'bdd',
reporter: 'list'
}
},
test_settings: {
default: {
launch_url: 'http://URL',
silent: true,
desiredCapabilities: {
browserName: 'chrome',
javascriptEnabled: true,
acceptSslCerts: true,
chromeOptions: {
args: [
"--no-sandbox",
"start-fullscreen"
]
}
}
}
}
};
And here is my global.js file:
var HtmlReporter = require('nightwatch-html-reporter');
var reporter = new HtmlReporter({
openBrowser: true,
reportsDirectory: __dirname + '/reports'
});
module.exports = {
reporter: reporter.fn
};
I don't think it will work with nightwatch-html-reporter as it is probably not a mocha reporter (but correct me if I'm wrong).
You want to use built in or custom mocha reporters when using nightwatch with mocha.
You can use a custom mocha html reporter like mochawesome but you'll have to hack around a bit and I offer no guarantees as I only tested those hacks lightly.
Here are the instructions to use mochawesome
(tested with
"mocha": "^5.2.0",
"mochawesome": "^3.1.1",
"nightwatch": "^0.9.21")
npm install mochawesome
Modify mochawesome node_modules\mochawesome *.js files to require mocha-nightwatch instead of mocha. (See instructions/explanations towards the end of the answer)
Presuming you're using a nightwatch.conf.js, configure your test runner to the equivalent of
test_runner : {
type : "mocha",
options : {
ui : "bdd",
reporter : require("mochawesome") // Please observe that you can pass a custom report constructor function here, not just reporter names
}
}
Run tests and observe that you still see the default console reporter (spec) but that at the end of the run you also see an output like:
[mochawesome] Report HTML saved to C:\projects\myWebApp\mochawesome-report\mochawesome.html
Open the html report.
This solution is hackish and fragile because nightwatch comes with it's own version of mocha.
When you install nightwatch you will see in your node_modules a mocha-nightwatch folder. This is the mocha that is being used by nightwatch.
However mochawesome doesn't use mocha-nightwatch. If you look at node_modules\mochawsome\dist\mochawesome.js you will see lines of code like:
var Base = require('mocha/lib/reporters/base');
var Spec = require('mocha/lib/reporters/spec');
This means is requires mocha, not mocha-nightwatch.
Those lines should ideally be: require('mocha-nightwatch/...).
So please change them in all *.js files that need fixing.
You could also fork mochawesome and make them like that ;)
Debugging notes:
Try putting some additional console.logs in node_modules\mocha-nightwatch\lib\mocha.js in the Mocha.prototype.reporter function. That's how I figured out what's going on.
If you use Mocha you can always go with mochawsome: https://www.npmjs.com/package/mochawesome
I haven't tried it myself but it looks pretty neat.