After running a test case with Karate, some html reports are published with surefire plugin. In particular, I've found that there is an html report for each feature file. This is inconvenient when tests are run from an automated pipeline, like in my case, where I use htmlpublish Jenkins plugin to get a public link to access reports and spread them across slack channels or emails.
I've tried to add this snippet in my pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<aggregate>true</aggregate>
<!--also set this to link to generated source reports-->
<linkXRef>true</linkXRef>
</configuration>
</plugin>
But it had not the desired effect.
I'm trying to achieve a single index.html into the target/surefire-reports directory so i can publish and browse all test reports
Any suggestion?
Thanks
Are you using the parallel runner ? If not, please read up about it: https://github.com/intuit/karate#parallel-execution
Since we emit the cucumber-compatible JSON report in addition to the industry-standard JUnit XML format, you have the choice of any reporting solution that fits your needs. I think the maven-cucumber-reporting library should work for you - but you can decide: https://github.com/intuit/karate/tree/master/karate-demo#example-report
EDIT: For any other advanced needs, please consider writing your own report: https://stackoverflow.com/a/66773839/143475
Related
I'd like to to generate two separate Maven HTML CheckStyle reports with different configurations:
the first one checks the main classes (src/main/java/**) with configuration #1 (<configLocation>)
the second one checks the test classes (src/test/java/) with configuration #2 (<configLocation>) which is somewhat relaxed regarding of configuration #1
I played a lot with multiple executions, configurations, and reportSets, etc. but I've never been able to make it work. The generated report only take into account the last execution of the plugin. I'm now wondering if it's even possible...
Edited:
Added HTML report.
You need to specify outputFile property in your config to set different file for report. And create 2 executions for main code and test code with different output files.
For example, config below will generate result report in file your_file.xml in base directory of your project.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
<version>${checkstyle.plugin.version}</version>
<configuration>
<outputFile>${project.basedir}/your_file.xml</outputFile>
</configuration>
</plugin>
Full description of property is here
Currently I am developing an application using quarkus. I have an openapi yaml file that describes everything. I wanted to know if there are quarkus extensions or tool with which I can generate the rest end points that would produce and consume same file format.
As long as you dont need any Quarkus specific features(e.g reactive, #Cached,..), then the REST endpoints are standard JAXRS endpoints and you can use the openapi generator to generate the interfaces and model POJOs for you. After that you simply implement the generated interface:
openapi-cli generate -i url_or_path_to_your_api_schema.yml -g jaxrs-spec --skip-validate-spec --additional-properties=interfaceOnly=true,useSwaggerAnnotations=false
The above command will generate only JAXRS endpoint interface any POJO classes for any models defined in API. You can copy them to your project and implement the iface.
Instead of manual cli invocation, you could integrate this in your maven build, using the openapi generator plugin:
<plugin>
<groupId>org.openapitools</groupId>
<artifactId>openapi-generator-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<inputSpec>${project.basedir}/src/main/resources/yourapi.yaml</inputSpec>
<generatorName>jaxrs-spec</generatorName>
<additionalProperties>interfaceOnly=true,useSwaggerAnnotations=false</additionalProperties>
<configOptions>
<sourceFolder>src/gen/java/main</sourceFolder>
</configOptions>
</configuration>
</execution>
</executions>
</plugin>
There is also a complete project stub generator for Quarkus if thats something you fancy: (as --library=quarkus option for jaxrs-spec generator)
There are many more options you may wish to configure described in the docs of OpenApi generators. You can also customize the code templates(moustache) in case the built in ones miss sth you need, though at that point you might look into creating your own generator.
If you need a client and you are ok with the standard Microprofile REST client approach, then the Openapi generator will also work fine:
openapi-cli generate -i path_to_your_openapi_schema.yml -g java --library microprofile
The OpenApi generator is a complex project and not without issues, so it would be best if Quarkus community adopted a custom optimized tooling for server and client code generators, to fully utilise all features e.g. reactive client...
We are currently working on an official OpenAPI Generator extension: https://github.com/quarkiverse/quarkus-openapi-generator#readme
Please take a look if you're interested. We'd love to hear feedback from the community.
Currently, there is no offical extension / tools inside Quarkus to generate JAX-RS stubs from an OpenAPI file.
However, this has been discussed on the Quarkus dev mailing list, you can have a look and provide feedback on it: https://groups.google.com/g/quarkus-dev/c/f8hJRm8oWbQ/m/UW_kQ7UEAwAJ
Note that there is an OpenAPI generator for JAX-RS, you may be able to use it to generate a JAX-RS compliant endpoint that would works with only small modifications in Quarkus: https://openapi-generator.tech/docs/generators/jaxrs-spec
It appears that surefire and failsafe plugins execute test classes in order while tests defined within a class execute in undetermined order.
To discover tests which rely on order (what we consider bad tests) we want to force the order to be different for each run. Ideally, we'd have a mechanism to disable randomization or a seed number that would repeat the order (must like the old palm OS emulator had a seed number that drove a sequence of random tests).
Let me know if you know a way to do this? If not, I guess I can work one into a local fork and then submit it.
Thanks
Peter
Some of the other answers link to the surefire maven documentation page, but like most maven documentation it provides no examples of how to actually specify the settings in the maven XML morass. Here is how to do it with the surefire plugin:
<properties>
<surefire.plugin.version>2.16</surefire.plugin.version>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.plugin.version}</version>
<configuration>
<runOrder>random</runOrder>
</configuration>
</plugin>
</plugins>
</build>
Specify the runOrder of Surefire to "random" http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder
I think this is more the responsibility of your unit test framework, not the Surefire/Failsafe plugins, which are just responsible for bootstrapping the test framework.
There is already a Stackoverflow question about how to make Junit tests run in random order (the answer is to use a custom ClassRunner):
How can I make my JUnit tests run in random order?
This library supplies an implementation if you don't want to write your own: http://randomjunit.sourceforge.net/
First it seemed to me that you are mixing things. Maven-Surefire-PLugin is responsible to run unit tests where it is the case to be independant of the order of execution.
Maven-Failsafe-plugin is responsible for exectution of integration tests which is different, cause integration tests could be dependant on the order which os no problem. Apart from that maven-surefire-plugin has some possibilities to influcence of the order of execution order:
http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder
This will be of course influenced by the testing framework which you use. In JUnit you can influence the order only in limited way. In TestNG it is a complete different story, cause TestNG has the ability to define dependencies etc.
Maven-Failsafe-Plugin has the same capabilities to influence the order of executions.
http://maven.apache.org/surefire/maven-failsafe-plugin/integration-test-mojo.html#runOrder
I'm working with a Java RPC service to communicate between the server and client side in my GWT application. I don't want to remove the asynchronous interface from my version control, because it's required in order for the project to compile and run.
However, the Async interface gets generated automatically (along with the Messages class) in the target/generated-sources folder by a Maven plugin, and I get a duplicate class error when the application tries to run.
Does anybody know how to disable the auto-generation of the Asynchronous RPC interface?
I found the answer by following this question:
How to configure IntelliJ IDEA and/or Maven to automatically add directories with Java source code generated using jaxb2-maven-plugin?
And then looking at the Maven GWT plugin at Mojo's website:
http://mojo.codehaus.org/gwt-maven-plugin/user-guide/async.html
... so it wasn't IDEA that was generating the code, but rather it was the GWT Maven plugin. I disabled the auto-generation of the Async RPC and i18n Messages interfaces by commenting out the following two lines in my pom:
<goal>i18n</goal>
<goal>generateAsync</goal>
so that the plugin definition in pom.xml now looks like:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>gwt-maven-plugin</artifactId>
<version>2.5.0</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>test</goal>
<!--<goal>i18n</goal>-->
<!--<goal>generateAsync</goal>-->
</goals>
. . .
And the problem is solved.
Actually commenting your POM is not the best way to do it.
I don't know about IDEA but with Eclipse "run configurations" you can specify the goals you want to execute. I guess IDEA must have something similar (see link).
For instance I would have two distinct runners. One with the goals
gwt:generateAsync gwt:i18n gwt:css
and one with no specific goals, which will execute a full build.
clean install
In your case you would want to have
gwt:compile gwt:test
If you NEVER want to execute the gwt:generateAsync goal though, removing it might be considered as a good option.
My company has recently switched to maven, and the transition was very smooth, but there is one this that annoys me.
So earlier, in pre-maven era you could see which test is current class is being run (f.e if you had 40 unit tests in a class, and 2nd test failed you would see it).
Now it looks this way: it displays the name of tested class and thats it. You have to wait till all the tests are done in the class to actually see the results (you can stop the test and it will show the progress to the point you stopped, but you don't see anything before actually stopping them). This is really annoying and time consuming in some integration tests.
If anyone knows a solution for this I'd be grateful. Thanks in advance.
You can configure a listener that will print the currently run test on the console. See the sections Using custom listeners and reporters of the junit maven-surefire-plugin documentation or the testng maven-surefire-plugin documentation
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<properties>
<property>
<name>listener</name>
<value>com.mycompany.MyResultListener,com.mycompany.MyResultListener2</value>
</property>
</properties>
</configuration>
</plugin>
If you use JUnit, you could use a #Rule for this.
If you only want it to apply to when Maven runs the tests, use a System property to en-/disable the output.