Well, I would like to have a maven goal execute-custom-tests inside my custom-maven-plugin that consists of running test methods (This tests are not unit tests). Something similar to test goal of soapui-pro-maven-plugin, for example.
Why? Basically the main objectives of the plugin are testing stuff (not unit testing) and the tests in src/test are for unit testing, right?
Being more specific I was thinking about something like this:
#Mojo (name = "run-custom-tests", LifecyclePhase.TEST)
public class TesterMojo extends AbstractMojo {
#Parameter(property = "someParameter")
private String someParameter;
// [...] parameters for test configuration
#Override
public void execute() throws MojoExecutionException, MojoFailureException {
// Piece of code that executes a set of custom tests which procedure I specified.
}
}
When test fail, I would like them to be marked as failed tests not as failed executions. What's the right thing to do here? Show me the light, please.
Maven conventions support two types of testing out of the box: unit tests (via maven-surefire-plugin) and integration tests (via maven-failsafe-plugin).
By default, maven-surefire-plugin only looks for the following files with unit tests:
**/Test*.java
**/*Test.java
**/*TestCase.java
Similarly, default includes for integration tests run by maven-failsafe-plugin are the following:
**/IT*.java
**/*IT.java
**/*ITCase.java
So, as you can see, Maven lets each plugin figure out which tests it should care about. So it's perfectly fine for src/test/java to contain different types of tests, not just unit tests.
Different folder
You can put tests in a different folder too. One example would be if you have non-Java tests, since then src/test/java location doesn't make sense. Standard Maven plugins get project model from Maven to figure out the src/test/java location and some 3rd party plugins use the same mechanism. Depending on the plugin you use, you might want to check out its configuration or use maven-build-helper-plugin to add-test-source in order for some plugins to pick up another test folder automatically.
Different tests on demand
From the Maven perspective the core difference between unit tests and integration tests is the additional requirements for the later: they often need to have your project already packaged and they often need additional setup or teardown. But you yourself can set up multiple test goals during both test and integration-test phases. All major test frameworks support specifying which test suite should be run when (e.g., via groups). If your framework doesn't, you can still use plugin includes/excludes. It is a standard practice to combine this with Maven profiles in order to only run smoke tests by default (during development) and to run full tests on CI environment. You can use the same approach to enable anyone (a tester?) to run extra tests on demand, e.g., to run extra heavy tests when certain important part of the code has changed.
Related
I am working on a large project and need to offer users the ability to optionally enable or disable local integration test cases ( For pipeline, test cases must be enforced).
First of all, welcome to the community.
Next, you can modify the test task inside the build.gradle file or maybe add a new task called integrationTest and implement your custom logic there.
As an instance, you can check this gist on Github: Separating tests from integration tests with Gradle
You can also use #Profile annotation to your integration test classes and run your tests with different profiles. You can read more about profiles using the following link: Spring Profiles
Is there any way to specify the order of the unit tests classes run by a Gradle Test task?
I'd like to get some known longer-running tests at either the front or the back of the list, but don't know if it's possible without splitting my tests' execution between multiple tasks.
Running JUnit 4.12 and Gradle 4.5.
Gradle simply delegates execution to the JUnit runner.
So if you want specific test class ordering, you will need to create a Suite and specify the test classes in the order you want, see the JUnit documentation for this.
Now, given the flexibility of Gradle in terms of having different source roots, I would strongly recommend doing the separation at the Gradle level, by create extra test source roots, test task and the like. This will allow you to effectively control when these long running tests are run in a standard build execution, but also to skip them or run these only when desired. The ordering at the JUnit level will not give that flexibility without much more tweaking. See the Gradle documentation on adding source sets.
I'm using Junit toolbox to run my unit and integration tests. It's working fine. Now, I have created another layer of tests that I would also like to run with junit toolbox but with a different suffix, not *IT or *Test.
#RunWith(WildcardPatternSuite.class)
#SuiteClasses("**/*BCT.class")
public class AllBCTTests {
}
But these tests are not running in my Maven build. What do I have to do to get these tests with a different suffix to run?
The above piece of code works. Different suffixes can be used with WildcardPatternSuite.
I am fairly new to writing Maven pom files and JUnit tests. I have following in my pom and it is calling my test scripts as expected.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
</plugin>
It seems JUnit is more popular than sure fire plugin.
1) How is JUnit similar/different from maven sure fire plugin's default behavior (that is working for me from above plugin configuration). I can imagine JUnit having additional API/library; but what do they give me in addition to sure fire plugin
2)what is the easiest way to change my current tests that are running with sure fire plugin to JUnit. I came across following link which sort of implies that adding few lines to pom would be sufficient (?)
http://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html
3)about previous bullet, what benefits would I have if I convert sure fire plugin tests to JUnit.
Hopefully, I am unambiguous (with my intro background to maven and JUnit)
maven-surefire-plugin is not itself a test framework: it's a Maven plugin that will run tests written with a test framework, either JUnit or TestNG.
I have following in my pom and it is calling my test scripts as expected.
If this is already running your tests then, as Surefire only knows about those two test frameworks, it means you're already using either JUnit or TestNG. You should be able to tell which from the classes you're importing to write your test classes.
(that is working for me from above plugin configuration)
Unless you have a particular requirement there's little reason to move away from the framework you're already using; it doesn't sound like you need to change anything.
As it says right here:
To get started with JUnit, you need to add the required version of JUnit to your project ... This is the only step that is required to get started - you can now create tests in your test source directory (eg, src/test/java).
Your question is confusing and suggests you haven't done any preliminary research yet. When you say "surefire tests" you may mean Pojo tests. If you know what a JUnit test is, it's pretty common sense thing to convert the Pojo tests to JUnit tests. Just put #Test before the Pojo test methods. You may also want to convert assert into the appropriate JUnit assert methods. In summary, just read a JUnit tutorial and the rest will be straight forward.
Let's say we have a project that consists of some Eclipse-projects and Spring 3.1, the final result is a WAR-file. We use WTP for development. All the unit tests and integration tests are working (our Maven does this automatically). The project runs in WTP with a local configuration. In other words everything looks as if it is ready to roll.
Now we want to test run that WAR-file with different sets of configuration files for different platforms. The test should only start the context and see if that causes any issues (missing/misspelt property in a property file, too many beans for auto-wiring, ...). AFAIK it isn't necessary to have access to (or it accessible to) the outside world. Basically it should only start the context, close it and continue with the next configuration. If one context fails, the build should break.
How should we do this? Can we do this with Maven? Something external?
EDIT: Forgot to say: We will run our stuff with Tomcat 6.
Sounds like you are talking about integration test.
You should look at the failsafe plug for this:
http://maven.apache.org/surefire/maven-failsafe-plugin/usage.html
Using jetty and maven-failsafe-plugin You need to bind one of
jetty:run, jetty:run-exploded or jetty:run-war to the
pre-integration-test phase with deamon set to true, bind
failsafe:integration-test to the integration-test phase, bind
jetty:stop to the post-integration-test phase and finally bind
failsafe:verify to the verify phase. Here is an example:
Another possibility is a selenium test. Selenium tests require the war to be deployed and running before the tests are run. So there are plugins that do all this.
Then you would have a very simple selenium test case that just made a simple http request to the app to see if it was running.
You would use a different profile for each different configuration you wanted to test.