Went through the serenity documented for extracting the test outcomes
below isthe code, it didn't work
OutcomeFormat format = OutcomeFormat.XML; TestOutcomes outcomes =
TestOutcomeLoader.loadTestOutcomes().inFormat(format)
Tried with below code and its working,
OutcomeFormat format = OutcomeFormat.JSON; TestOutcomeLoaderBuilder
outcomes= TestOutcomeLoader.loadTestOutcomes().inFormat(format);
TestOutcomes out =outcomes.from(new File(""));
Issue is i need the test outcomes in #AfterScenario, but the thing is serenity reports gets generated after the entire execution tried changing the pom but didn't help. Is there any other way using which we can extract the test results?
Serenity uses JSON format by default now. Why are you trying to obtain the test outcomes? (i.e. what problem are you trying to solve?)
Created a separate java class for report extracting and added that in maven plugin which will get executed after serenity report is generated.
As #John smart has mentioned JSON and HTML are default output formats.
Still if you want to access Outcomes after test execution.
You can create a custom listener and listen to serenity event bus.
TestRunFinished event will be published with Outcome as a parameter.
You can use the outcome for getting required details.
For creating custom listener you can follow this page
Related
Is there anyway to identify which is the current running cucumber tag in Cucumber-Java for API testing?
I am using Cucumber-Java, along with TestNG. For grouping and executing in different environments, I am using cucumber tags.
The tags are given in feature level. Multiple tags have been specified for same feature. Like:
#regression-staging #regression-production
Feature: Add to cart
Scenario:
.
.
.
Same code is using for staging and production. And as build took I am using Maven. Since TestNG is used, the test is triggered from RunTestNGTest class that I wrote. Inside the RunTestNGTest, I have #BeforeSuite and #AfterSuite methods.
The test is run using mvn command
mvn test -Dcucumber.filter.tags=#regression-staging
or
mvn test -Dcucumber.filter.tags=#regression-production
Is there anyway I can get which tag is currently running? (In my case, either one will be active at a time). I want to log which tag is currently using, and also want to use the same in the HTML report. I tried scenario.getSourceTagNames();, but that returns all the tags for the scenario, not the currently running one.
You can simply access cucumber.filter.tags from your code like you would access any other system property:
String myCurrentTag = System.getProperty("cucumber.filter.tags");
Now you can parse the value (split by commas or additionally remove # symbol, etc.)
This is question about unit test (jest + #testing-library/react)
Hi. I started using #nrwl/react these days.
This is amazing products and I'm excited monorepos project with nx.
Btw, there is afterEach(cleanup); in generated template test file.
This is my sample project.
https://github.com/tyankatsu0105/reproducibility-react-test-nx/blob/master/apps/client/src/app/app.spec.tsx#L7
However react-testing-library doesn't need cleanup when using jest.
https://testing-library.com/docs/react-testing-library/api#cleanup
Please note that this is done automatically if the testing framework you're using supports the afterEach global (like mocha, Jest, and Jasmine). If not, you will need to do manual cleanups after each test.
In fact, I see error when remove afterEach(cleanup); from test files.
Found multiple elements with the text:
thanks!
I am currently seeking information around the available reporting capabilities for OpenTest. I would need information regarding the following:
Portability of reports/logging - can these results be published in various formats
Granularity of reports/logging - is there a way of getting very detailed in what is reported on and/or strategies to ensure that enough information is logged to allow for debugging of automated tests and System Under Test(SUT)
Screenshots - is there current functionality to allow for screenshots to be taken and published/posted to external systems?
Portability of reports/logging
You can obtain the test session results by using the API, either in JSON format (which contains a lot of detail) or the JUnit XML format:
http://localhost:3000/api/session/<SESSION_ID>?format=json
http://localhost:3000/api/session/<SESSION_ID>?format=junit
The detailed log of the test session can be retrieved in: JSON or human-readable format:
http://localhost:3000/api/session/<SESSION_ID>/log?format=json
http://localhost:3000/api/session/<SESSION_ID>/log?format=pretty
Granularity of reports/logging
The test results in JSON format will tell you everything you need to know about the pass/fail status for each test and each individual test action within a test, arguments that were used for test actions, the name of the screenshot captured for each test action, execution times and many other useful information.
When you want to troubleshoot a failed test, most of the time you'll need the detailed log information, which can be retrieved using the APIs I mentioned above. Aside from the log information generated by OpenTest itself, you can always log additional information that is specific to your application or test scenario using the $log JavaScript API.
Screenshots
Screenshots are automatically captured for Web and UI tests, whenever a test action fails. If you need to capture additional screenshots during your test, you can do so using the TakeScreenshot keyword for either Web testing or mobile testing. You can also capture a screenshot after any test action by using the $screenshot global test action argument:
- description: Click product 1 and capture a screenshot
action: org.getopentest.selenium.Click
args:
locator: { id: product1 }
$screenshot: true
You can download screenshots using this API:
https://localhost:3000/api/screenshot/SID1554380072_WEB_T05_SG01_ST01_after_03.png
SID1554380072_WEB_T05_SG01_ST01_after_03.png is the name of the screenshot file, which you can find in the test execution results in JSON format.
Integrating with custom reporting solutions
At some point you will need to integrate with a dedicated reporting product that can give you all the nice features that OpenTest is not be able to provide out-of-the-box. This is possible to do using the APIs I described. In order to notify interested parties of the current status of test sessions, OpenTest also provides a WebSocket API. You can use that to be notified when a test session has finished, then you can extract all the information you need via APIs. You can find a Java project that does all that here: https://github.com/adrianth/opentest-monitor. This project is intended as a starting point for your own custom integration.
in version 2.0.0 of maven-cucumber-reporting plugin is it possible to hide #Before and #After hooks from the report?
so for example in the html report I see:
Before BusinessStepDefs.prepare()
Scenario: Business scenario
Given business specific input
When performing action
Then result is success
After HelperStepDefs.cleanUp()
I am using the version 2.0.0, using an older version did not have those lines. Any ideas ?
I would like this to appear as
Scenario: Business scenario
Given business specific input
When performing action
Then result is success
See this issue for a discussion and possible resolution.
I am trying to figure out how to run unit tests, using Google Test, and send the results to TeamCity.
I have run my tests, and output the results to an xml, using a command-line argument --gtest_output="xml:test_results.xml".
I am trying to get this xml to be read in TeamCity. I don't see how I can get XML Reports passed to TeamCity during build/run...
Except through XML report Processing:
I added XML Report Processing, added Google Test, then...
it asks me to specify monitoring rules, and I added the path to the xml file... I don't understand what monitoring rules are, or how to create them...
[Still, I can see nowhere in the generated xml, the fact that it intends to talk to TeamCity...]
In the log, I have:
Google Test report watcher
[13:06:03][Google Test report watcher] No reports found for paths:
[13:06:03][Google Test report watcher] C:\path\test_results.xml
[13:06:03]Publishing internal artifacts
And, of course, no report results.
Can anyone please direct me to a proper way to import the xml test results file into TeamCity ? Thank you so much !
Edit: is it possible that XML Report Processing only processes reports that were created during build ? (which Google Test doesn't do?) And is ignoring the previously generated reports, as "out of date", while simply saying that it can't find them - or are in the wrong format, or... however I should read the message above ?
I found a bug report that shows that xml reports that are not generated during the build are ignored, making a newbie like me believe that they may not be generated correctly.
Two simple solutions:
1) Create a post build script
2) Add a build step that calls the command line executable with the command-line argument. Example:
Add build step
Add build feature - XML report processing
I had similar problems getting it to work. This is how I got it working.
When you call your google test executable from the command line, prepend %teamcity.build.checkoutDir% to the name of your xml file to set the path to it like this:
--gtest_output=xml:%teamcity.build.checkoutDir%test_results.xml
Then when configuring your additional build features on the build steps page, add this line to your monitoring rules:
%teamcity.build.checkoutDir%test_results.xml
Now the paths match and are in the build directory.