i have some set of jbehave tests and regular jenkins job with xunit plugin that create report from xml files. So far so good. Now, unfortunately i need to add some actions and it is easier to add them in pipeline. But i don't get how to generate jbehave report in pipeline. Been trying to use 'pipeline-syntax' help but this page only generate xunit([<object of type org.jbehave.jenkins.JBehavePluginType>])
Related
we are using jenkins for running the selenium maven testng tests, we are using jenkins dashboard plugin to see the no. of tests passed and failed.
We have recently added the retry analyzer to execute the failed test twice, and the retried tests are correctly reported by testng as the skipped tests.
But after the implemntation of retry analyzer number of tests available on jenkins dashborad are going up and down because now we have alot of skipped tests on jenkins and that started causing issues with the other charts.
So we implemnted the logic to update the testng results file to autometically remove the skipped tests. This is giving te correct results.
But jenkin maven plugin does not consider the testng results, it is still reading the junit results file and producing the results with the skipped tests on the jenkins dashboard.
We have added another step to archive the testng results on jenkins job, but after that we are seeing multiple graphs inside the job execution results page and the dashboard page is still considering the junit results.
Now I have two questions
1) Is there a way to disable the generation of JUnit results inside maven testng tests.
2) Is there a way to force jenkins to use the testng results.
or if you have any other suggestions for resolving this issue, then please let me know.
Below is the jenkins job snapshot
Thanks in advance
In Jenkins I am not getting the Publish Junit test result report option in post build actions for my maven projects.
However, for freestyle projects I am getting the option, and am able to configure it properly.
Question: Is there any different approach that I need to follow for Maven projects? How can I configure JUnit for maven projects?
This is a known issue:
JENKINS-9980: Add the "Publish JUnit test result report" option to maven job configuration page
The main idea around this is that a Jenkins Maven job should publish the JUnit report automatically (so no need for this option), while in a freestyle job you may turn it to be a maven job yet having the flexibility of a freestyle/open configuration.
From comments to the JIRA ticket above, you could check suggestion on this old SO thread.
Basically, you are left with two options:
Use a freestyle Jenkins job to have full control, or
Use a Maven Jenkins job and review your pom.xml configuration or the way you are getting JUnit tests executed if the report isn't provided automatically (which could already be the case, hence end of the story).
Our project uses Maven as the build tool and we are using Sonar to track quality. JUnit tests are executed by SureFire and the results are displayed in Sonar. We've added some JavaScript tests which are run by the jasmine-maven-plugin and want to include these results in the Sonar project.
The plugin generates a JUnit style XML report. How should we go about including the XML report in Sonar? Do we want to merge the XML reports as part of the build maybe?
You need to be sure the format generated by Jasmine is compliant with the JUnit XML format expected by the JavaScript Plugin : http://docs.sonarqube.org/display/PLUG/JavaScript+Unit+Tests+Execution+Reports+Import
You can also have a look at the format supported by the Generic Test Coverage Plugin : http://docs.sonarqube.org/display/SONAR/Generic+Test+Coverage
You don't need to merge the XML reports, there are different sonar.* properties to feed depending on the way you want to load your data: thru the JavaScript Plugin or the Generic Test Coverage. As a consequence, you don't need to run SonarQube analysis twice.
I currently have a maven project with ~500 tests. These are webdriver test with the Thucydides framework. There are several types of tests such as swiping tests, article link tests, sharing tests, etc.
Details of the current setup:
* Selenium
* Thucydides
* Jenkins
I would like to run parallel tests in Jenkins but I want each jenkins job to only run a specific type of test. Is there anyway of using the "Goals and options" in the Build section in jenkins job configuration page to only run a specific type of test i.e. run only the swiping tests? or are there other options available?
Ideally I Would like to run separate jobs for each type of tests and run in parallel all those jobs together.
Not sure how to achieve it purely in Jenkins, but the first option coming to my mind is that you could create separate maven profiles for each test type and create one jenkins job per profile.
For more info on maven profiles, see official docs: http://maven.apache.org/guides/introduction/introduction-to-profiles.html
I'm trying to decide how to create a set of Acceptance Tests for a Java-EE web application.
Here's the setup: Maven is used to generate a WAR file and deploy it into Glassfish. On deployment, the MySQL database schema is automatically updated from model classes using Hibernate ("hbm2ddl=auto" option).
The Acceptance Tests need to test the deployed code by invoking various methods and checking the results are as expected(*). We wrote an additional set of packages to hook into an existing system so the Acceptance Tests should show how these can be integrated into the existing codebase.
(*) This may sound more like Unit/Integration Testing but they are Acceptance Tests in the sense that they should prove what we did works and they need to be run after deployment so there is a database in place.
From the above, my current thinking is to use JUnit to check expected values etc. The bit I'm struggling with is how to invoke these tests after deployment. "deploy" is Maven's last phase so not sure if this is possible?
Just because that phase is called deploy doesn't mean that you have to use it for deploying your application for testing. In fact, it should only be used for "deploying" the artifact to a maven repository. Read through the description of the Maven lifecycle phases and you'll see that there are some phases dedicated to your use case:
pre-integration-test
integration-test
post-integration-test
Have a look at the Cargo Maven plugin. It's made to deploy your WAR file to various containers for testing. They definitely show demos of use cases like the one you describe on your site. I would expect that ultimately, you can be using Cargo to deploy to your container ( from one of the earlier phases like pre-integration-test )
Note, Jenkins also has a plugin that is a wrapper around the Cargo plugin. So you might do what you need via Jenkins. Also note, you don't need to run your Jenkins build job as mvn clean deploy. You could have one build job that just runs the integration tests, and fires another "deploy" job only when it succeeds.
If you really need to do stuff after deployment, then you can either run failsafe, and by implication JUnit) as part of the deploy phase.
What I usually do, if to have seperate module. So, you can have one maven project, which contains your project and a separate 'deployment test' project. Then, building the parent project will build and run your war and then run the deployment tests. You can use junit as normal.
The second fits better into jenkins because you'll still have a single project as well.