I'm running builds and tests on a CI server (Bamboo). I'm using TestNG to run my integration tests.
When I run my build and tests (using Maven), I see the following in the logs (this is an example, there are other classes affected too)
build 27-Dec-2013 19:06:35 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$18
build 27-Dec-2013 19:06:36 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:06:36 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.194 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$18
build 27-Dec-2013 19:06:36 Running c.v.p.i.c.ExerciseLifecycleForStaticAssetIntegrationTest$1
build 27-Dec-2013 19:06:36 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:06:36 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.188 sec - in c.v.p.i.c.ExerciseLifecycleForStaticAssetIntegrationTest$1
build 27-Dec-2013 19:06:36 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$25
build 27-Dec-2013 19:06:36 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:06:36 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.188 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$25
build 27-Dec-2013 19:06:37 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest
build 27-Dec-2013 19:06:37 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:07:57 Tests run: 21, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 80.748 sec <<< FAILURE! - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest
build 27-Dec-2013 19:07:57 testGetPostDescriptorForVersion(c.v.p.i.c.MultiChannelContentServiceIntegrationTest) Time elapsed: 0.066 sec <<< FAILURE!
build 27-Dec-2013 19:07:57 java.lang.AssertionError: expected:<1.0> but was:<null>
build 27-Dec-2013 19:07:57 at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:101)
build 27-Dec-2013 19:07:57 at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:108)
build 27-Dec-2013 19:07:57 at c.v.p.i.c.MultiChannelContentServiceIntegrationTest.checkFieldsThatShouldBeTheSameForAllVersions(MultiChannelContentServiceIntegrationTest.java:891)
build 27-Dec-2013 19:07:57 at c.v.p.i.c.MultiChannelContentServiceIntegrationTest.testGetPostDescriptorForVersion(MultiChannelContentServiceIntegrationTest.java:832)
build 27-Dec-2013 19:07:57
build 27-Dec-2013 19:07:58 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$24
build 27-Dec-2013 19:07:58 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:07:58 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.186 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$24
build 27-Dec-2013 19:07:58 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$15
build 27-Dec-2013 19:07:58 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:07:58 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.183 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$15
build 27-Dec-2013 19:07:58 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$11
build 27-Dec-2013 19:07:59 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:07:59 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.2 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$11
build 27-Dec-2013 19:07:59 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$2
build 27-Dec-2013 19:07:59 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:07:59 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.206 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$2
build 27-Dec-2013 19:07:59 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$22
build 27-Dec-2013 19:07:59 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:08:00 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.189 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$22
build 27-Dec-2013 19:08:00 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$7
build 27-Dec-2013 19:08:00 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:08:00 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.19 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$7
build 27-Dec-2013 19:08:00 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$13
build 27-Dec-2013 19:08:00 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:08:00 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.199 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$13
build 27-Dec-2013 19:08:00 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$16
build 27-Dec-2013 19:08:01 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:08:01 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.214 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$16
build 27-Dec-2013 19:08:01 Running c.v.p.i.c.ExerciseLifecycleIntegrationTestTemplate
build 27-Dec-2013 19:08:01 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:08:01 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.229 sec - in c.v.p.i.c.ExerciseLifecycleIntegrationTestTemplate
build 27-Dec-2013 19:08:01 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$14
build 27-Dec-2013 19:08:01 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:08:02 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.184 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$14
build 27-Dec-2013 19:08:02 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$28
build 27-Dec-2013 19:08:02 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:08:02 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.186 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$28
build 27-Dec-2013 19:08:02 Running c.v.p.i.c.MultiChannelContentServiceIntegrationTest$5
build 27-Dec-2013 19:08:02 Configuring TestNG with: TestNGMapConfigurator
build 27-Dec-2013 19:08:02 Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.191 sec - in c.v.p.i.c.MultiChannelContentServiceIntegrationTest$5
What I'm confused about is the instances of c.v.p.i.c.MultiChannelContentServiceIntegrationTest$XX.
I haven't intentionally or explicitly requested that the classes by run multiple times, nor does it seem that they are being run multiple times (since they report Test runs: 0 for all but a single instance).
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-testng</artifactId>
<version>2.16</version>
</dependency>
</dependencies>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
<configuration>
<argLine>
-javaagent:"${user.home}/.m2/repository/org/springframework/spring-instrument/${org.springframework.version}/spring-instrument-${org.springframework.version}.jar"
-Djava.awt.headless=true -Dehcache.rest.port=9888
</argLine>
<includes>
<include>c/v/p/integration/**/*.java</include>
</includes>
<reuseForks>false</reuseForks>
<forkCount>1</forkCount>
<argLine>-Xms512m -Xmx3096m -XX:PermSize=384m -XX:MaxPermSize=512m</argLine>
</configuration>
</execution>
</executions>
</plugin>
Can anyone point me towards what might be happening here. The phenomenon doesn't really lend itself to Googling. Most searches for "multiple test runs with TestNG" explain how to do it on purpose.
Are there any anonymous inner classes in your test class? That's what
MultiChannelContentServiceIntegrationTest$22 etc. seems to indicate.
Apparently, the pattern
<include>c/v/p/integration/**/*.java</include>
includes these anonymous classes, so Surefire tries to run them as test classes.
Try **/*Test.java or simply use the default.
Related
I've got a problem with maven-surefire-plugin reports. Maven version is 3.3.9, maven-surefire-plugin 2.18.1, version of JUnit is 4.12.
Tests run in Jenkins, there should be info after each test, but sometimes something goes wrong and there is no info in Console and there are no surefire reports.
Should be something like that
11:07:42 Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 194.569 sec - in com.example.content.SomeTest
11:07:42
11:07:42 Results :
11:07:42
11:07:42 Tests run: 8, Failures: 0, Errors: 0, Skipped: 0
11:07:42
11:07:42 [INFO] -------------------------------------------------------------------
11:07:42 [INFO] BUILD SUCCESS
11:07:42 [INFO] -------------------------------------------------------------------
11:07:42 [INFO] Total time: 03:17 min
11:07:42 [INFO] Finished at: 2017-11-15T11:07:39
11:07:42 [INFO] Final Memory: 15M/377M
11:07:42 [INFO] -------------------------------------------------------------------
But as I said sometimes there are no these lines. Previous and next test classes work fine and have all info and reports as it supposed to be
Is this problem really caused by surefire-plugin or maybe it's something different?
I'm using Maven 3.3.3 with Java 1.8. How do I get my integration tests to run in alphabetaical order, by class name (or file, whatever's easier)? I thought "runOrder" might help me so I created this plugin configuration ...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<reuseForks>true</reuseForks>
<argLine>-Xmx4096m -XX:MaxPermSize=512M -noverify -XX:-UseSplitVerifier ${itCoverageAgent}</argLine>
<runOrder>alphabetical</runOrder>
<skipTests>${skipAllTests}</skipTests>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
but as the integration tests are run, they are still not being run in order. Notice a file beginning with "V" precedess one beginnign with "G" ...
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier; support was removed in 8.0
Running org.mainco.subco.antivirus.queue.repo.VirusScanQueueDaoIT
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.196 sec - in org.mainco.subco.antivirus.queue.repo.VirusScanQueueDaoIT
Running org.mainco.subco.antivirus.queue.service.VirusScanQueueServiceIT
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.055 sec - in org.mainco.subco.antivirus.queue.service.VirusScanQueueServiceIT
Running org.mainco.subco.classroom.repo.GroupDaoIT
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.181 sec - in org.mainco.subco.classroom.repo.GroupDaoIT
Running org.mainco.subco.classroom.service.GroupServiceIT
Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.114 sec - in org.mainco.subco.classroom.service.GroupServiceIT
Running org.mainco.subco.core.repo.CategoryDaoIT
The ITs are run with respect to the alphabetical order... of the fully qualified name of the class. The following list
org.mainco.subco.antivirus.queue.repo.VirusScanQueueDaoIT
org.mainco.subco.antivirus.queue.service.VirusScanQueueServiceIT
org.mainco.subco.classroom.repo.GroupDaoIT
org.mainco.subco.classroom.service.GroupServiceIT
org.mainco.subco.core.repo.CategoryDaoIT
follows this ordering. There are currently no run orders working with just the class name.
Good day! I would like to ask about the error that I'm encountering in Jenkins:
"Build step 'Invoke top-level Maven targets' marked build as failure"
I'm trying to run a selenium script (with JMeter script inside) in Jenkins. I'm getting a BUILD SUCCESS in console output in Jenkins but it marks the job failed because of that error. Kindly see below the console output:
Running TestSuite
Start: 11/04/2016 5:19:29 PM
RawConfigFileScenario3 is created
11/04/2016 5:19:49 PM [PASSED] No Error encountered on the Jmeter Script
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.962 sec - in TestSuite
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 48.165 s
[INFO] Finished at: 2016-11-04T17:19:50+08:00
[INFO] Final Memory: 14M/137M
[INFO] ------------------------------------------------------------------------
Build step 'Invoke top-level Maven targets' marked build as failure
Finished: FAILURE
I also tried to run the same script in Eclipse and I'm getting a SUCCESS result. Please see below the logs:
Running TestSuite
Start: 11/04/2016 4:26:26 PM
RawConfigFileScenario3 is created
11/04/2016 4:26:41 PM [PASSED] No Error encountered on the Jmeter Script
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.739 sec - in TestSuite
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 37.743 s
[INFO] Finished at: 2016-11-04T16:26:41+08:00
[INFO] Final Memory: 13M/136M
[INFO] ------------------------------------------------------------------------
I would like to ask for your guidance regarding this matter. Thank you in advance for the answers. :)
i have encounter this problem. and i just change the sdk configuration from java8 to java7.
hope this can help you.
I'm having this error while running my test suite from jenkins and when i run in eclipse via Maven it's working fine,
I'm using the same pom.xml for jenkins which is giving this error:
Running TestSuite
INFO [main] (BrowserManager.java:124) - Connecting to http://chromedriver.storage.googleapis.com/ to check lastest chromedriver release
INFO [main] (BrowserManager.java:112) - Latest driver version: 2.9
INFO [main] (Downloader.java:56) - Binary driver previously downloaded /var/lib/jenkins/.m2/repository/webdriver/chromedriver/linux64/2.9/chromedriver
INFO [main] (Downloader.java:60) - Exporting webdriver.chrome.driver as /var/lib/jenkins/.m2/repository/webdriver/chromedriver/linux64/2.9/chromedriver
Tests run: 4, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 2.904 sec <<< FAILURE! - in TestSuite
Setup(webautomation.GetApp) Time elapsed: 2.218 sec <<< FAILURE!
java.lang.NoClassDefFoundError: org/openqa/selenium/ElementNotSelectableException
at webautomation.GetApp.Setup(GetApp.java:57)
Caused by: java.lang.ClassNotFoundException: org.openqa.selenium.ElementNotSelectableException
at webautomation.GetApp.Setup(GetApp.java:57)
Results :
Failed tests:
webautomation.GetApp.Setup(webautomation.GetApp)
Run 1: GetApp.Setup:57 ยป NoClassDefFound org/openqa/selenium/ElementNotSelectableExce...
Run 2: PASS
Tests run: 3, Failures: 1, Errors: 0, Skipped: 2
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18.671 s
[INFO] Finished at: 2016-10-06T17:27:37+05:00
[INFO] Final Memory: 26M/200M
[INFO] ------------------------------------------------------------------------
I'm using Maven 3.0.3, JUnit 4.8.1, and Spring 3.1.1.RELEASE. I have my Surefire plugin configured this way
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.14.1</version>
<configuration>
<reuseForks>false</reuseForks>
<argLine>-Xmx2048m -XX:MaxPermSize=512M</argLine>
</configuration>
</plugin>
I notice when my unit tests fail because of IllegalStateExceptions, Maven never prints out the test names that failed at the end of execution. Instead I'll see
mvn clean install
...
Tests in error:
? IllegalState Failed to load ApplicationContext
? IllegalState Failed to load ApplicationContext
? IllegalState Failed to load ApplicationContext
? IllegalState Failed to load ApplicationContext
How can I get Mavne to print out more specific info about test failures? Right now I have to go digging through the surefire-reports directory to figure out what failed. Most of my JUnit tests look like this:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration({ "classpath:test-context.xml" })
public class CleverTeacherServiceTest extends AbstractImportServiceTest
{
I think your Spring test config is failing before your tests run. You should normally see the failed tests listed in the console, like:
T E S T S
Running TestSuite
..
Tests run: 234, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.019 sec <<< FAILURE!
fooTest(BarTest) Time elapsed: 0.011 sec <<< FAILURE!
fooTest(BarTest.java:23)
Results :
Failed tests:
BarTest.fooTest:23 null
Tests run: 234, Failures: 1, Errors: 0, Skipped: 0
You could run your tests setting the surefire plugin property useFile == false to have all the plugin output sent to the console. I wouldn't configure that in the pom file, but run it like so:
mvn -Dsurefire.useFile=false clean test
See Surefire Plugin docs for config information.