Problem statement
How to invoke tests within a given test server(i.e. within the process)?
Explanation
We know that there is surefire plugin, but this uses surefire:test goal which runs unit tests in separate process. In my case, i need to run the tests within the existing process (pre-configured test server and required environment) to run test. In surefire plugin, there is configuration that has forkCount and reuseForks. If i use it with forkCount as 0, reuseForks true, still creates separate server and hence process. I simply want to avoid setup part for test.
Update after Joe's comment saying it as duplicate As my application usage SpringBoot, running mvn clean install build the project and execute tests. While executing tests, it creates server for each running test. Even though i used configuration given below, in console it shows same process id, but starting server every time. In my case i don't want to make server up for every test case, rather used pre-existing test server and the test should use that server.
Configuration used
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<forkCount>0</forkCount>
<reuseForks>true</reuseForks>
</configuration>
</plugin>
Related
I have 2 testng.xml files called testng_smoke.xml and testng_regression.xml. I've configured the pom.xml as below to run these testng files.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>src/test/resources/testng_smoke.xml</suiteXmlFile>
<suiteXmlFile>src/test/resources/testng_regression.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
In here it executes both testng files and give the status of the build (Success or Fail) at the end.
But my requirement is to run the smoke test suite (testng_smoke.xml) first and fail the build without moving to regression test suite (testng_regression.xml) if there any test failure in the smoke test suite. Is there any possibility to achieve this requirement.
AFAIK, there is no out of the box way in testng to do this.
If you are using Jenkins or some CI, then it is better to create two jobs or pipeline so that the second task is dependent on the results of the first one.
If not and you want to use single file then one of the possible solutions may be to maintain a variable either as a file or an environment variable who's value can be set in the suiteListener afterSuite method. The beforeSuite method can then throw a SkipException to ignore the suite based on the variable value.
I am working on a simple spring boot application (created from spring initializr contains web mega-dependency).
I added also Cucumber dependencies for acceptance test.
When I run mvn clean install it runs directly the cucumber test, but me I want to run the app before tests (And why not stop the app after the tests terminate).
I tried maven exec plugin but it does not work:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.1</version>
<executions><execution>
<goals><goal>java</goal></goals>
</execution></executions>
<configuration>
<mainClass>org.dhappy.test.NeoTraverse</mainClass>
</configuration>
</plugin>
Skipping tests
If there is a need to skip tests and just compile and package the application, you can set the skipTests property as follows:
mvn install -DskipTests
Application execution
If you want to run the application, you can use the exec plugin. There are two flavours ("goals" really): one runs the package application and the other runs a Java class. If you know the class name, you can run that plugin as follows:
mvn exec:java -Dexec.mainClass="org.dhappy.test.NeoTraverse"
Integration testing
Finally, if you want to run the application first before you can run the tests, you are actually not doing unit testing. Your tests in this case are perhaps integration and not unit tests. The actual details would vary depending on whether your application has complex needs such as a database and a web server.
Integration testing happens at a different "phase" in maven lifecycle.
In short, you need to rename your Java test classes something other than *Test.java so that maven surefire plugin ignores your tests. You then need to include maven-failsafe-plugin so that integration testing happens.
You want to perform these steps:
Start the application
Run a set of tests
Stop the application
Starting and stopping the application is one thing and could be done in the <phase>pre-integration-test</phase> and <phase>post-integration-test</phase>. Integration testing is done in the <phase>integration-test</phase>
You want to read and understand https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html. Specifically, look at the Using jetty and maven-failsafe-plugin example.
I have complex web application using Spring,Hibernate, Maven joint build(Java+Groovy) with test in Spock and different Maven profiles. All source files are in Java. I have Maven configuration(part of it) for local testing:
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<version>2.0</version>
<configuration>
<path>/services</path>
<port>8062</port>
<contextReloadable>true</contextReloadable>
<backgroundProcessorDelay>2</backgroundProcessorDelay>
<contextFile>../context.xml</contextFile>
</configuration>
</plugin>
So when I have changes in the bytecode maven contextReloadable triggers redeploy of the app. That is the desire state. When I run some test with maven configuration usingin InteliJ, in my system I see that redeploy is triggered.
I have no changes in the files or in the tests. But redeploy is triggered and everything is OK. But there is a problem for the same test when I click right button and click run on UploadTest using InteliJ 14.0.3 Community Edition
Redeploy happens but the profiles are not taken under consideration and the environment variables are not set. If I add them
org.apache.catalina.core.ContainerBase backgroundProcess WARNING: Exception processing loader WebappLoader[/services] background process java.util.ConcurrentModificationExceptio
And my backendSystem crashes. So my questions are:
Why InteliJ triggers redeploy when run Spock test without changes in the code/tests?
Why starting a single Spock test using InteliJ, cause my backednSystem brokes(I need to restart it)(what can be the source of the problems?? InteliJ runs the test with diferent parameters, profiles are not taken or ...).
Does inteliJ start compilation of the test with different eviroment variables set? (I see that inteliJ start its own launcher for the test).
I have the next pom.xml
<project>
...
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<argLine>${failsafeArgLine}</argLine>
<includes>
<include>**/dmg/*IT.java</include>
</includes>
<skipTests>${skipTests}</skipTests>
</configuration>
</execution>
</executions>
</plugin>
...
</project>
The problem is that when I'm taking off verify goal then the build is every time successful even if there was test failures.
And when I'm Taking off integration-test goal the integration tests simply do not run
Why failsafe plugin requires both integration-test and verify goals?
In Maven Failsafe plugin reference you can find simple answer why build is always successful
failsafe:integration-test runs the integration tests of an application.
failsafe:verify verifies that the integration tests of an application passed.
Without verify goal test results are not checked at all(but they are executed), so failsafe plugin requires integration-test goal to run tests, and verify to "verify" their results.
In Maven there are two types of test runner plug-ins, Surefire and Failsafe and they both serve different purposes. (I mention Surefire here because I feel it helps explain and contrast different types of failure in builds.)
Surefire
Surefire is a plug-in designed for your pre-deploy tests, such as Unit and Component tests. Surefire is guaranteed to return a failure (and therefore break your build) if any of the tests fail.
You want this behaviour in pre-deploy tests because you want to fail the build early if your tests fail, and as Unit and Component tests don't (or shouldn't) have any external dependencies it's safe to fail a build there and then.
Failsafe
Failsafe is a plug-in designed for post-deploy tests, such as Functional and Smoke tests.
Failsafe, as the name implies handles failures safely, by always returning a success exit code (as you have experienced in your build).
The reason behind this is for post-deploy tests you don't want test failures to immediately break the build because you may have spun up infrastructure, or seeded some test data into the system, which needs clean-up before the build is failed.
Therefore Failsafe splits the failure checking into the failsafe:verify goal which happens in the verify phase.
If you run Maven without parameters (assuming the normal jar lifecycle) you'll notice a phase between integration-test and verify called, post-integration-test. This is the phase which was designed for the tear down of any infrastructure or test data you may have injected into the system to run the tests.
... pre-integration-test, integration-test, post-integration-test, verify, ...
The phase before integration-test, pre-integration-test is designed for the said seeding and infrastructure creation.
To sum it all up, with pre-deploy tests you want to fail the build as early as possible, and so surefire guarantees this by exiting with a failure.
Whereas with post-deploy tests there are often external dependencies which need clean-up before you fail the build. For example in Kubernetes you may want to issue a rollback of the deploy and so failsafe:integration always returns a build success to guarantee you can perform the clean-up before failing the build by checking the status with failsafe:verify.
(The currently accepted answer just quotes the manual and doesn't explain why there are two commands and there are differences which is why I feel this answer is needed.)
I'd like my console to be as quiet as possible when I run my tests with mvn test, unless something goes wrong. Then again, when I'm writing tests in Eclipse (in other words, when I run single junit tests inside Eclipse), it's ok for them to be pretty verbose.
So I would need a way to have different log4j/logback settings when running all my tests with surefire than when I run them one by one in Eclipse. Is there a way to accomplish this?
You can configure surefire to use a different logger settings file. That will be ignored by Eclipse.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Dlogback.configuration=[wherever]</argLine>
</configuration>
</plugin>