I am new to taurus testing. I have a set of tests inside of a taurus project.
Within the scenario 1300_Azuresubscriptions.yaml I have the following list of labels:
0100-Authorization
1300_ListAzureSubscriptions
1310_CreateAzureSubscriptions
1320_UpdateAzureSubscriptions
1330_FetchAzureSubscription
1340_CreateAVWGateway
1341_CreateVirtualWanSite
1342_ListVirtualWanSites
1343_UpdateVirtualWanSite
1344_FetchVirtualWanSite
1345_DeleteVirtualWanSite
1346_DeleteAVWGateway
1350_DeleteAzureSubscription
1351_ListADSubscription
1352_CreateAzureSubscriptions
1353_FetchADSubscription
1354_ValidateADSubscriptions
1355_GetADGroups
1356_ADSyncConfigurations
1357_ADSync
1358_CheckADCLientCreation
1359_DeleteADSubscription
1360_CheckADCLientDeletion
1361_ListAzureSubscriptionsWithInvalidAuthHeader
1362_ListAzureSubscriptionsWithNoAuthHeader
1363_CreateAzureSubscriptionsWithInvalidAuthHeader
1364_CreateAzureSubscriptionsWithNoAuthHeader
1365_UpdateAzureSubscriptionsWithInvalidAuthHeader
1366_UpdateAzureSubscriptionsWithNoAuthHeader
1367_FetchAzureSubscriptionWithInvalidAuthHeader
1368_FetchAzureSubscriptionWithNoAuthHeader
1369_CreateAVWGatewayWithInvalidAuthHeader
1370_CreateAVWGatewayWithNoAuthHeader
However when I run bzt test-cases/1300_AzureSubscriptions.yaml, only the following list of labels are actually getting tested:
0100-Authorization
1300_ListAzureSubscriptions
1310_CreateAzureSubscriptions
1320_UpdateAzureSubscriptions
1330_FetchAzureSubscription
1340_CreateAVWGateway
1341_CreateVirtualWanSite
1342_ListVirtualWanSites
1343_UpdateVirtualWanSite
1344_FetchVirtualWanSite
1345_DeleteVirtualWanSite
1346_DeleteAVWGateway
1350_DeleteAzureSubscription
1351_ListADSubscription
1352_CreateAzureSubscriptions
1353_FetchADSubscription
1354_ValidateADSubscriptions
1355_GetADGroups
1356_ADSyncConfigurations
1357_ADSync
1358_CheckADCLientCreation
1359_DeleteADSubscription
1363_CreateAzureSubscriptionsWithInvalidAuthHeader
1364_CreateAzureSubscriptionsWithNoAuthHeader
1365_UpdateAzureSubscriptionsWithInvalidAuthHeader
1366_UpdateAzureSubscriptionsWithNoAuthHeader
1367_FetchAzureSubscriptionWithInvalidAuthHeader
1368_FetchAzureSubscriptionWithNoAuthHeader
1369_CreateAVWGatewayWithInvalidAuthHeader
1370_CreateAVWGatewayWithNoAuthHeader
Why isn't it running the labels 1360-1362? I don't understand the problem. Any help would be greatly appreciated. Thanks!
Taurus is just a wrapper for underlying load and functional testing tools, if something is not executed there could be various reasons for this, for example:
The particular request is not enabled in YAML configuration file
The underlying test executor is failing on attempt to run the particular request
There is a lack of test data for the requests
Check out bzt.log file and any logs generated by the underlying tool. By default Taurus uses JMeter executor so if it's your case the reason could be found in jmeter.log file (it also worth checking jmeter.out and jmeter.err for any suspicious entries)
More information: Navigating your First Steps Using Taurus
You can also reach out to Taurus developers, maintainers and users at Taurus support forum
The issue was the indentation in the yaml. I would recommend that if you're running these tests you'd better install a linter to check your yaml formatting. One tab off and the error is invisible. It just ignores the tests.
Related
I am trying to execute a .jmx script in the GitHub. The login page requires a OTP, which I am generating using org.jboss.aerogear. However, it seems when I am executing the script in GitHub the script is not generating this OTP and throwing 406 (Not Acceptable). Can anyone please guide me on this issue please.
This is running perfectly in JMeter but getting error in GitHub.
Do I need to add this specific driver and how?
Whenever you face any "error" first of all take a look at jmeter.log file, normally it contains the reason or at least a clue so you can figure out or guess the error cause. If it doesn't - increase JMeter logging verbosity for the test elements you're using.
Most probably you need to add the .jar file which provides this Totp class (along with dependencies, if any) to JMeter Classpath to JMeter installation in "Github" (whatever it means) and the error will go away
I am trying to clear cookies after each iteration in Taurus yaml.
I tried the solution from below thread but it didn't work:
[Google Group] https://groups.google.com/g/codename-taurus/c/wa4xIbxkUYI?pli=1
Above thread says:
For reference I did it like this:
modifications:
set-prop:
"Cookies>CookieManager.clearEachIteration": "true"
I have tried the same but didn't work.
Using JMeter I can clear the cookies but need the same behavior for Taurus. Any help.
What do you mean "for Taurus"? Taurus just provides YAML configuration of some of JMeter test elements. The "solution" you're referring is for running existing .jmx script using Taurus
Taurus doesn't support 100% of JMeter functionality when it comes to building test plan from config, refer the documentation to see what is supported.
If you want to use some JMeter functionality which is not supported by Taurus the options are in:
Reach out to Taurus developers via the forum you're referencing and ask them to implement the functionality you need
Implement it yourself, Taurus is an open source project
Switch to the existing .jmx script execution
Switch to JMeter and stop using Taurus as a wrapper
Use JR223 Test Elements to perform the removing of cookies, i.e. you could add a JSR223 PreProcessor to the first sampler in the Thread Group and call sampler.getCookieManager().clear() function there where sampler stands for HTTPSamplerBase
I created JMeter Test and under first "HTTP Request" I created a Beanshell listner script which works fine when using GUI but 8 out of 10 times the script totally get ignored in non-GUI mode.
I am also running these test in Gitlab CI using Docker Image "justb4/jmeter:latest" and Beanshell script also get ignored there. I don't know whats wrong there it is working fine with GUI
There is no such thing as "get ignored" in JMeter world, it either passes or fails, in case of failure you should see the relevant message(s) in the jmeter.log file
Also be aware that you should not be using Beanshell at all, starting from JMeter 3.1 you should be using JSR223 Listener and Groovy language for scripting, one of Beanshell's disadvantages is that it's being interpreted each time while Groovy scripts can be compiled and cached providing the most optimal performance. See Apache Groovy - Why and How You Should Use It article for more details.
Also be informed that we cannot efficiently help without seeing your code and the aforementioned jmeter.log file.
The whole morning I have been trying to setup e2e tests reporting via SonarQube's Generic Execution, by using the Generic Test Data -> Generic Execution feature.
I created a custom xml report that gets added to the scan properties like this:
sonar.testExecutionReportPaths=**/e2e-report.xml
So far, SonarQube seems to completely ignore this property and I no attempt to parse the file in the logs. Has anyone made it work?
These are links by Sonar about the Generic Execution feature:
https://docs.sonarqube.org/display/SONAR/Generic+Test+Data
https://github.com/SonarSource/sonarqube/blob/master/sonar-scanner-engine/src/main/java/org/sonar/scanner/genericcoverage/GenericTestExecutionSensor.java
This is a SonarQube 6.2+ feature. Make sure to use an appropriate SonarQube version.
In addition sonar.testExecutionReportPaths does not allow matchers (like *).
Please provide relative or absolute paths, comma separated.
See also:
The official documentation of the Generic Test Data feature
The source code, that looks up the generic execution files
What I try to achieve is to get JMeter results in 2 formats, for the same test execution. CSV is the one I'm mostly interested in, unless there are failures - then I may need to use data which can be saved only to XML. I can't use XML for most occasions, due to misc reasons.
I was looking at jmeter.save.saveservice.output_format cfg entry, it doesn't seem to accept both as valid entry. I was also looking at JMeter code itself, it also clearly separates both options.
Did anyone invent a hack to get both JMeter outputs at the same time?
Solution for both standalone JMeter and jmeter-maven-plugin is welcome.
If you use JMeter Maven plugin, as per Five Ways To Launch a JMeter Test without Using the JMeter GUI guide the default output has to be XML.
If you need CSV output as well - just add Simple Data Writer listener. If you looking for CSV data to be in line with JMeter default CSV values configure it as follows: