I'm looking at using Sonar to reduced the number of plugins I've had to install in Jenkins to get some decent code analysis (and sonar seems to do more and present it better). However when I kick of the sonar job the JUnit / Concordion tests are executed. I don't want these tests run as Jenkins is already executing the tests.
How do I stop the tests executing and just perform code analysis?
I've installed sonar 3.7.3 and executing using the Gradle sonar-runner plugin and specifying the :sonarRunner task.
I'm just facing the same issue and it seems that you can run :sonarqube -x test to avoid having tests run before.
You can set the sonar.dynamicAnalysis property to false. This will skip tests and only perform the static analysis part.
See http://docs.codehaus.org/display/SONAR/Analysis+Parameters#AnalysisParameters-OptionalParameters
Related
I am encountering an issue with a Maven build job on Jenkins -
when it runs on "agent2" it succeeds!
when the same job runs on "agent2" it fails a specific test every time...
the code is the same in both runs.
we use Maven Integration plugin.
artifact resolve from artifactory.
edit with more info:
java 1.8, a junit test that uses assert - on one agent gets an expected value - while on the other not. there are no clear-to-the-eye differences between the agents - trying to find out what this difference may be. the code is pulled from bitbucket and then built with maven(clean install)
any ideas?
Thanks!
I am using Maven and the failsafe plugin to execute a set of runners that run integration tests.
One of these runners execute tests that involve altering my configuration. For this reason, I would like this runner to be executed only when the other tests have finished executing. Otherwise, there is the possibility of unrepresentative tests.
How can I go about this?
To expand on #khmarbaise comment, in Testng you have annotations that can help you set and reset config files
http://testng.org/doc/documentation-main.html#annotations
So I have Jenkins with the Cobertura plugin installed. I have Cobertura and findbugs in the POM and my tests are running twice...
I assume that this is because Cobertura instruments the bytecode and this causes the tests to re-run, which isn't a bad thing I guess, since the instrumented isn't the same as non-instrumented code...but I would really like the tests to be run only once.
I have tried running them locally on commandline using these commands:
mvn cobertura:cobertura -Dcobertura.report.format=xml
mvn findbugs:findbugs -Dfindbugs.onlyAnalyze=true
mvn cobertura:cobertura -Dcobertura.report.format=xml findbugs:findbugs -Dfindbugs.onlyAnalyze=true
but I can't get the tests to run twice locally, where as on Jenkins the are running twice. I am not sure why this is happening and whether I could make it stop.
I am using Cobertura to generate reports for me...I assume that to generate them it needs to re-run the tests? But it doesn't make sense since they are already being run once.
We have faced the same behavior. It seems to be default behavior of Cobertura to rerun test cases for calculating coverage.
We switched to JaCoCo tool, which proved to be better . It does not re-run the test cases for coverage report.
Indeed you have to run tests twice with cobertura-maven-plugin (or use different profiles). This behavior is due to the fact that it runs Cobertura instrumentation in its own lifecycle and uses Cobertura executable instead of Cobertura API.
If you want to generate a Cobertura report while only running your tests once, you can give a try to the qualinsight-mojo-cobertura-core plugin. This plugin uses a different approach to run Cobertura instrumentation.
You'll find a documentation on the project's page: https://github.com/QualInsight/qualinsight-mojo-cobertura .
Note that this plugin has still some limitations, but it may be a viable alternative in your context.
Hope it helps !
Regards.
I am running tests via Jenkins and Maven using surefire-report:report goal. Even when some tests fail, Jenkins displays the Job as successful due to correct completion of the job. The Surefire Reports are correct corresponding to the test results. Are there any ideas how to configure Jenkins to run and report the tests properly?
Thanks in advance!
I'd like to configure jasmine-maven-plugin to make jenkins unstable if a test fails but the only options appear to be:
set haltOnFailure true and have failures break the build
set haltOnFailure false and have failures reported in the logs but the build succeeds.
Is there a way to check the logs post-test and mark the build unstable?
Sam Hasler's answer only works for freestyle Jenkins jobs. We use Maven jobs and this configuration option of the JUnit Jenkins plugin is unavailable for Maven jobs. So I was looking for a more universal solution.
What we did in order to get it working is to reconfigure the Jasmine Maven plugin so as to
no longer halt the build upon test failures and
write the Jasmine test reports to target/surefire-reports where Jenkins expects to find them. This has the additional advantage that we now also see the failed Jasmine test in the build job alongside our Java tests.
<haltOnFailure>false</haltOnFailure>
<jasmineTargetDir>${project.build.directory}/surefire-reports</jasmineTargetDir>
Now our build jobs are yellow (unstable) as expected, no longer red (failed).
Found the answer myself!
I had to configure jenkins to also look at the jasmine junit report:
under Publish JUnit test result report add **/TEST-jasmine.xml to Test report XMLs, comma separated if there is something there already:
**/TESTS-TestSuites.xml,**/TEST-jasmine.xml