My company has recently switched to maven, and the transition was very smooth, but there is one this that annoys me.
So earlier, in pre-maven era you could see which test is current class is being run (f.e if you had 40 unit tests in a class, and 2nd test failed you would see it).
Now it looks this way: it displays the name of tested class and thats it. You have to wait till all the tests are done in the class to actually see the results (you can stop the test and it will show the progress to the point you stopped, but you don't see anything before actually stopping them). This is really annoying and time consuming in some integration tests.
If anyone knows a solution for this I'd be grateful. Thanks in advance.
You can configure a listener that will print the currently run test on the console. See the sections Using custom listeners and reporters of the junit maven-surefire-plugin documentation or the testng maven-surefire-plugin documentation
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<properties>
<property>
<name>listener</name>
<value>com.mycompany.MyResultListener,com.mycompany.MyResultListener2</value>
</property>
</properties>
</configuration>
</plugin>
If you use JUnit, you could use a #Rule for this.
If you only want it to apply to when Maven runs the tests, use a System property to en-/disable the output.
Related
After running a test case with Karate, some html reports are published with surefire plugin. In particular, I've found that there is an html report for each feature file. This is inconvenient when tests are run from an automated pipeline, like in my case, where I use htmlpublish Jenkins plugin to get a public link to access reports and spread them across slack channels or emails.
I've tried to add this snippet in my pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<aggregate>true</aggregate>
<!--also set this to link to generated source reports-->
<linkXRef>true</linkXRef>
</configuration>
</plugin>
But it had not the desired effect.
I'm trying to achieve a single index.html into the target/surefire-reports directory so i can publish and browse all test reports
Any suggestion?
Thanks
Are you using the parallel runner ? If not, please read up about it: https://github.com/intuit/karate#parallel-execution
Since we emit the cucumber-compatible JSON report in addition to the industry-standard JUnit XML format, you have the choice of any reporting solution that fits your needs. I think the maven-cucumber-reporting library should work for you - but you can decide: https://github.com/intuit/karate/tree/master/karate-demo#example-report
EDIT: For any other advanced needs, please consider writing your own report: https://stackoverflow.com/a/66773839/143475
I am fairly new to writing Maven pom files and JUnit tests. I have following in my pom and it is calling my test scripts as expected.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
</plugin>
It seems JUnit is more popular than sure fire plugin.
1) How is JUnit similar/different from maven sure fire plugin's default behavior (that is working for me from above plugin configuration). I can imagine JUnit having additional API/library; but what do they give me in addition to sure fire plugin
2)what is the easiest way to change my current tests that are running with sure fire plugin to JUnit. I came across following link which sort of implies that adding few lines to pom would be sufficient (?)
http://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html
3)about previous bullet, what benefits would I have if I convert sure fire plugin tests to JUnit.
Hopefully, I am unambiguous (with my intro background to maven and JUnit)
maven-surefire-plugin is not itself a test framework: it's a Maven plugin that will run tests written with a test framework, either JUnit or TestNG.
I have following in my pom and it is calling my test scripts as expected.
If this is already running your tests then, as Surefire only knows about those two test frameworks, it means you're already using either JUnit or TestNG. You should be able to tell which from the classes you're importing to write your test classes.
(that is working for me from above plugin configuration)
Unless you have a particular requirement there's little reason to move away from the framework you're already using; it doesn't sound like you need to change anything.
As it says right here:
To get started with JUnit, you need to add the required version of JUnit to your project ... This is the only step that is required to get started - you can now create tests in your test source directory (eg, src/test/java).
Your question is confusing and suggests you haven't done any preliminary research yet. When you say "surefire tests" you may mean Pojo tests. If you know what a JUnit test is, it's pretty common sense thing to convert the Pojo tests to JUnit tests. Just put #Test before the Pojo test methods. You may also want to convert assert into the appropriate JUnit assert methods. In summary, just read a JUnit tutorial and the rest will be straight forward.
It appears that surefire and failsafe plugins execute test classes in order while tests defined within a class execute in undetermined order.
To discover tests which rely on order (what we consider bad tests) we want to force the order to be different for each run. Ideally, we'd have a mechanism to disable randomization or a seed number that would repeat the order (must like the old palm OS emulator had a seed number that drove a sequence of random tests).
Let me know if you know a way to do this? If not, I guess I can work one into a local fork and then submit it.
Thanks
Peter
Some of the other answers link to the surefire maven documentation page, but like most maven documentation it provides no examples of how to actually specify the settings in the maven XML morass. Here is how to do it with the surefire plugin:
<properties>
<surefire.plugin.version>2.16</surefire.plugin.version>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.plugin.version}</version>
<configuration>
<runOrder>random</runOrder>
</configuration>
</plugin>
</plugins>
</build>
Specify the runOrder of Surefire to "random" http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder
I think this is more the responsibility of your unit test framework, not the Surefire/Failsafe plugins, which are just responsible for bootstrapping the test framework.
There is already a Stackoverflow question about how to make Junit tests run in random order (the answer is to use a custom ClassRunner):
How can I make my JUnit tests run in random order?
This library supplies an implementation if you don't want to write your own: http://randomjunit.sourceforge.net/
First it seemed to me that you are mixing things. Maven-Surefire-PLugin is responsible to run unit tests where it is the case to be independant of the order of execution.
Maven-Failsafe-plugin is responsible for exectution of integration tests which is different, cause integration tests could be dependant on the order which os no problem. Apart from that maven-surefire-plugin has some possibilities to influcence of the order of execution order:
http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder
This will be of course influenced by the testing framework which you use. In JUnit you can influence the order only in limited way. In TestNG it is a complete different story, cause TestNG has the ability to define dependencies etc.
Maven-Failsafe-Plugin has the same capabilities to influence the order of executions.
http://maven.apache.org/surefire/maven-failsafe-plugin/integration-test-mojo.html#runOrder
I'm working with a Java RPC service to communicate between the server and client side in my GWT application. I don't want to remove the asynchronous interface from my version control, because it's required in order for the project to compile and run.
However, the Async interface gets generated automatically (along with the Messages class) in the target/generated-sources folder by a Maven plugin, and I get a duplicate class error when the application tries to run.
Does anybody know how to disable the auto-generation of the Asynchronous RPC interface?
I found the answer by following this question:
How to configure IntelliJ IDEA and/or Maven to automatically add directories with Java source code generated using jaxb2-maven-plugin?
And then looking at the Maven GWT plugin at Mojo's website:
http://mojo.codehaus.org/gwt-maven-plugin/user-guide/async.html
... so it wasn't IDEA that was generating the code, but rather it was the GWT Maven plugin. I disabled the auto-generation of the Async RPC and i18n Messages interfaces by commenting out the following two lines in my pom:
<goal>i18n</goal>
<goal>generateAsync</goal>
so that the plugin definition in pom.xml now looks like:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>gwt-maven-plugin</artifactId>
<version>2.5.0</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>test</goal>
<!--<goal>i18n</goal>-->
<!--<goal>generateAsync</goal>-->
</goals>
. . .
And the problem is solved.
Actually commenting your POM is not the best way to do it.
I don't know about IDEA but with Eclipse "run configurations" you can specify the goals you want to execute. I guess IDEA must have something similar (see link).
For instance I would have two distinct runners. One with the goals
gwt:generateAsync gwt:i18n gwt:css
and one with no specific goals, which will execute a full build.
clean install
In your case you would want to have
gwt:compile gwt:test
If you NEVER want to execute the gwt:generateAsync goal though, removing it might be considered as a good option.
Imagine that I have two profiles, one is for production, the other one is for test environment. The thing is, I just want to execute some tests only for development environment, not production.
I tried to put #ignore, but I cannot tell just ignore it ONLY during prod build. I tried to use sure-fire plugin of maven, but again I couldn't get it worked with different behaviour per profile.
(PS: It is a spring-hibernate project featured by such different tech.s like hazelcast, jms (activeMq), scheduler (quartz), else..)
Any help would be appreciated.
We have a system where we have three test sizes (Small, Medium and Large). We have created an annotation that allows us to mark tests with these sizes. We also created a Rule that will check a particular env variable and will skip tests based on size. Sorry, but I don't have the ability to post the code, but that is how we did it.
Here are some posts on how Google does it (we based our stuff on Google)
http://blog.utest.com/how-google-tests-software-small-medium-and-large/2011/04/
You can ignore junit test using maven.test.skip parameter as it is described in the documentation. You can also set the parameter as part of as profile.
<profile>
<id>noTest</id>
...
<properties>
<maven.test.skip>true</maven.test.skip>
...
</properties>
...
</profile>
Maven's Surefire Plugin: Inclusions and Exclusions of Tests