I have some basic unit tests in VS2010 that I'm currently running. I'd like to be able to add some additional information to the Test Result Details screen (When a unit test completes, right click on it in the Test Results screen, and select Test Result Details). Currently it just lists Common Results, and some standard information.
Is there a way to include some additional values to test result? For example, PreCleanup, initialization, Test, Post Cleanup, etc?
Apparently, anything written to the System.Diagnostics.Debug.WriteLine will be displayed on an Debug Trace Pane..
Related
When I run the test in GUI, i see the Average, Min, Max in GUI. But when I run in console, is there a way to add these to the csv file?
These values are being calculated so you will be able to see the values only when you open .jtl results file after test finishes in the listener of your choice, i.e. Aggregate Report or Summary Report.
If you want to see the interim statistics while your test is being executed you have the following choices:
JMeter Summarizer output. JMeter reports some numbers into stdout while your test is being running
You can get some extended information if you run your JMeter test using Taurus tool as a wrapper
Both console and web interface options are available, in order to see current test execution stats in browser start your test like:
bzt yourtest.jmx -report
And finally you can use Backend Listener to send your results into database, message queue or web service and use custom plotting application to print out either raw or parsed statistics, here you are limited only by your fantasy:
More information:
JMeter: Real Time Results
How to Use Grafana to Monitor JMeter Non-GUI Results
JMeter produces some basic fields/result_field. JMeter doesn't create everything you see in different types of Listeners.
You can give this a try.
Create a plan
Generate atleast 100 samples (As large amount of data is required for some listeners), using a single sampler (request)
Use as many Listeners as you want of different types (say 15 types of listeners)
Run the plan....
Now in the filename field of all the listeners give series of names of files like a1.jtl a2.jtl
and so on....
see the screen shot
enter image description here
Now again run the plan. Go to the files and open them in some good editors like notepad++.
For your surprise you will find the same data in all the files irrespective of the type of listener generating the file.
Crux of the matter is : JMeter gathers only handful of information from the run, the rest information which is shown in different Listeners is COMPUTED by the JMeter.
So you can read the *.jtl file into any of the listener.
In JMeter, the new way since 3.0 to have results is to use the Web report generated at end of test:
http://jmeter.apache.org/usermanual/generating-dashboard.html
I want to execute a specific step only once before each cucumber feature files. A cucumber feature files can have multiple scenarios. I don't want Background steps here which executes before each scenario. Every feature file can have a step (which is different in each feature) which will execute only once. So i can't use that step into before hook as i have a specific step for every 20 features. Sample Gherkin shows below:
Scenario: This will execute only once before all scenario in this current feature
When Navigate to the Page URL
Scenario: scenario 1
When Some Action
Then Some Verification
Scenario: scenario 2
When Some Action
Then Some Verification
Scenario: scenario 3
When Some Action
Then Some Verification
I hope you guys understand my Question. I am using Ruby Capybara Cucumber in my framework.
Cucumber doesn't really support what you are asking about. A way to implement this with cucumber hooks would be to use these two pieces of doc:
https://github.com/cucumber/cucumber/wiki/Hooks#tagged-hooks
https://github.com/cucumber/cucumber/wiki/Hooks#running-a-before-hook-only-once
You would tag all your feature files appropriately and you can implement tagged Before hooks that execute once on a per feature tag basis.
It's not beautiful but it accomplishes what you want without waiting on a feature request (or using a different tool).
This can be achieved by associating a Before, After, Around or AfterStep hook with one or more tags. Examples:
Before('#cucumis, #sativus') do
# This will only run before scenarios tagged
# with #cucumis OR #sativus.
end
This must be in the top 5 most frequent questions on the Cucumber mailing list. You can do what you want with hooks. However you almost certainly should not do what you want. The execution time you save by taking this approach is totally outweighed by the amount of time and effort it will take to debug the intermittent failures that such an approach generally leads to.
One of the foundations of creating automated tests is to start from a consistent place. When you have code that setups key things in scenarios, but that is not run for every scenario you have to do the following:
Ensure your setup code creates a consistent base to start from (this is easy)
Ensure that every scenario that uses this base, does not modify the base in any way at all (this is very very difficult)
In your example you'd have to ensure that every action in every scenario ends up on your original page URL. If just one scenario fails to do that, then you will end up with intermittent failures, and you will have to go through every scenario to find your culprit.
In general it is much easier and more effective to put your effort into making your setup code FAST enough so that you are not worried about running it before each scenario.
Yes, This can be done by passing the actual value in you feature file and using "(\\d+)" in you java file. Look at below shown code for better understanding.
Scenario: some test scenario
Given whenever a value is 50
In myFile.java, write the step definition as shown below
#Given("whenever a value is (\\d+)$")
public void testValueInVariable(int value) throws Throwable {
assertEqual(value, 50);
}
you can also have a look at below link to get more clear picture:
https://thomassundberg.wordpress.com/2014/05/29/cucumber-jvm-hello-world/
Some suggestions have been given, especially the one quoting the official documentation which uses a global variable to store whether or not initial setup has been run.
For my case, where multiple features were executed one after another, I had to reset the variable again by checking whether scenario.feature.name has changed:
$feature_name ||= ''
$is_setup ||= false
Before do |scenario|
current_feature_name = scenario.feature.name rescue nil
if current_feature_name != $feature_name
$feature_name = current_feature_name
$is_setup = false
end
end
$is_setup can then be used in steps to determine whether any initial setup needs to be done.
I'm looking for the way how I can manage ruby cucumber scenarios in runtime. I'd like to filter some scenarios out having some information about SUT which I can gather in runtime.
For instance, I've the following scenarios
#automated
Scenario: As a customer I want to run ... scenario 1
Given ...
#automated #debug
Scenario: As a debugger I want to run ... scenario 2
Given ...
#automated
Scenario: As a customer I want to run ... scenario 3
Given ...
#automated #release
Scenario: As a releaser I want to run ... scenario 4
Given ...
I'm able to determine whether a debug or release application is testing now. And for the debug one I want to see scenarios 1,2,3 to be run but for the release app I want to see 1,3,4 to be run.
I know how to do it using rake or any other wrapper script but it'd be better to find solution without such wrapper scripts.
Also, cucumber profiles might not be a good choice here because there are several parameters each with a number of values. So it might require some crazy number of their combinations.
In my test plan I want to make java request and check how it was done by making jdbc request, But I don't exactly know when java request will change DB. I want to make several requests with waiting after each. And if JDBC request doesn't special changes after certain time stop whole test. After that I want to see progress where test plan stopped in view results tree.
BUT
If I use any stop-thread functions, there are no data in View Results Tree.
Haw can I do what I want with jmeter?
Thanks.
you can use a similar mechanism to the one described here:
http://www.sourcepole.ch/2011/1/4/waiting-for-a-page-change-in-jmeter
I have a question about TestComplete. My automated tests jump sometimes into a different window in the tested application and they become stuck there. This is caused by unknown controls for the specified test (it is searching for f.e. combobox which doesn't exists on the window). I was wandering if there is some way to avoid this situation and just skip to another test?
The problem is that TC stays in endless loop of searching for the not existing object.
Thanks in advance for your responses.
Josef
You need to organize your tests with test items. In this case, you will be able to specify the Test Item value for your test items' Stop on error property and TestComplete will start executing the next test if an error occurs during execution of the current test. You can find more information on this in the Tests and Test Items and Stopping Tests on Errors and Exceptions help topics.
It does not jump there alone, does it? Be sure to have the right buttons pressed. If 2 windows are similar and in one window there is the ComboBox you want to test and in the other window not then I would go for something like this:
if(Aliases.GenericWindow.WaitAliasChild("ComboBoxInQuestion", customTimeoutInMilliseconds).Exists)
Log.Message("Do something with ComboBox");
The timeout can be set in the WaitAliasChild function. This waits customTimeoutInMilliseconds and if there is no ComboBox found it just skips the tests that were made for the ComboBox.