I have developed TestNG framework and implemented Maven
When I run the script, the results are generated and in the target folder, when I open index.html, all the scripts are displayed as Pass though it failed.
I have used java verifications (reporter.log)
Can someone suggest me on how to change the java verifications to TestNG to view the failures?
Thank in advance
Reporter.log is just for logging statements in the report. You need to use assertions to do your verifications. Try out the Assert class in Testng.
Related
I have some tests that need to check if the main code did a System.exit(...). This works very nicely with suggestions from https://stackoverflow.com/a/309427/1023341. But when running these tests in Jenkins (in stead of in my IDE Eclipse) and later when trying them on the command-line using Maven-Surefire (as Jenkins does) the tests fail without telling me why. It only tells me: Error occurred in starting fork, check output in log.
When setting a SecurityManager during JUnit (5) using System.setSecurityManager and using Surefire plugin, you should restore the SecurityManager after the test.
SecurityManager origSecurityManager = System.getSecurityManager();
try {
// ... code under test here ...
} finally {
System.setSecurityManager(origSecurityManager);
}
or some other more suitable form. This makes sure that Maven-Surefire-plugin stays happy.
Edit for suggested pre-baked solutions:
There are two pre-baked libraries for this:
junit5-system-exit
system-lambda
As the name suggests: the system-lambda is a Java 8+ solution. Both are JUnit 5 compatible. My personal preference lies with the lambda solution.
More background information
I recently started working on creating jacoco reports for maven projects including unit and integration tests and they seem to work out correctly.
Now I have encountered a different scenario which I am not sure how to approach.
I have one workspace which consists of integration test cases - application A, but the source code does not exist in the same workspace/code base. The source code which actually runs on invoking these integration test scripts are in a different workspace/code base - application B(they are invoked using rest api calls with the localhost urls. The jboss server is started for application B so that the localhost context is up) from the integration tests.
The aim is to invoke these integration tests from application A, which in turn calls the source code of these tests in application B generating the jacoco report of the code coverage for application B.
I am not actually sure how to achieve this.
Can someone provide some input.
Thanks.
If I understand you correctly, you actually have 2 different processes in your scenario:
The "client" process that runs the integration tests and for which jacoco can be easily applied, but it's not what you need
The "server" process that runs the actual JBoss server and executes the actual code.
Client process contacts the server via HTTP.
In this case, I'm afraid jacoco won't be able to provide a coverage for you if you're running the tests from maven/gradle, because jacoco instruments only bytecode on the running JVM. So you have to be "creative" here :)
I'll list here some possible approaches
Disclaimer: I haven't tried them though (didn't work with jboss/java ee), but maybe you'll be able to at least borrow some ideas
The first approach would be running the tests together with the application somehow, like its done for example in spring tests (I'm not sure whether JBoss provides similar capabilities).
The idea is simple:
You run the integration test, it runs the jboss "embedded in the same jvm" and you can inject beans / EJB session beans into the test (like autowiring with spring).
The advantage of such a method is that you'll be able just to use jacoco maven plugin and it will instrument everything for you
I don't know how easy will be achieving this architecture technically, I know that recent jboss versions support embedded mode, So maybe you'll find This link to be a useful foundation
Another direction is to take a look at Arquillian project. They have some jacoco extension that probably will help, but I've never tried it.
And the last approach I can think of is running the jboss server with jacoco agent directly instead of relying on the build system that runs jacoco for you.
The idea here is to stream the results of covered server code into some file / tcp endpoint. So you run the jboss with -javaagent:[yourpath/]jacocoagent.jar and it starts streaming the results wherever you need it to stream. After the tests you should gather these results and prepare a report. You can find Here more information about this approach
I've donwloaded JMeter 3.1 and am trying to run a junit sampler.
Per these instructions
jmeter.apache.org/usermanual/junitsampler_tutorial.pdf
I've added a ThreadGroup to test plan, i select add->sampler, but do not find Junit.
This is what I see. What am I missing?
no jmeter on menu
Here is an example photo I found on the tutorial.
enter image description here
I do have a junit class in a jar in the lib/junit folder.
This is weird, my JMeter 3.1 (installation comes with the JUnit Request sampler out of the box.
So please double check the following:
jmeter.log file - normally it contains enough information to guess the problem cause
you should have ApacheJMeter_junit.jar file under "lib/ext" folder of your JMeter installation. If it is not there - most probably something went wrong and your installation is corrupt
Be aware that you can add JUnit support (as well as any other JMeter Plugin) using JMeter Plugins Manager
For JMeter 5.0, the JUnit sampler did not appear for me when I incorrectly placed all my JUnit dependency jars in JMeter's lib/junit. It looks like it got stuck parsing all those jars looking for tests.
After placing JUnit dependency jars in JMeter lib folder and only the unit test jar in lib/junit, it worked correctly and JMeter GUI also started a lot faster.
I do have a junit class in a jar in the lib/junit folder.
This may be the problem in you case, if JMeter does not find any tests (or fails to find in my case), the JUnit sampler is simply not added to the UI
I have a Spring project (Apache CXF, Spring, Hibernate, Maven ...) hosted on BitBucket and I'm trying to use Bamboo as my CI server. My idea is deploying the code directly to Heroku from Bamboo so that deploying time is automated.
I made a plan with a couple of tasks to achieve this. First I have a Source Code Checkout task and a builder task. Both of them are working, code is compiling and test are passing, I can see that in the task log. The problem is that Bamboo doesn't seem to recognize the tests (it marks the task are testless).
I have also tried to create a new JUnit test task and it's even worst. Log shows that everything is working properly but Bamboo marks the plan as a failure after the test task is executed.
Any ideas?
Not sure which version of Bamboo you're using, but in the version that we have, you have to turn on unit test result evaluation on the Builder tab. Please see the attached screenshot, and make sure that this is enabled, and the directory setting is pointing to the directory where Maven Surefire creates the test results (in XML format).
I'm using Maven 2 w/ cargo to deploy to different remote tomcats depending on the maven profile used.
I also have integration tests (junit w/ remote webservice calls) that I would like to automatically run.
Question: How do I setup my test code to be able to read either the cargo.hostname (preferred, changed property value based on maven profile) or the maven profile actived so it knows which url to go run the tests against?
Specifically, this is in java in the test case.
Thanks!
Either you define a fixed value for the cargo.hostname (not the best; but sometimes it workds well, cause using a fixed test server) or better define an appropriate property in Maven for it and put the information also into a properties file which will be filtered by the build process in the src/test/resources folder which can be read before the real integration tests.