Using SpecFlow together with MS Test v2, how can I use [DoNotParallelize] attribute from MS Test framework to mark a SpecFlow scenario or a feature as one that I don't want to run in parallel?
Seems that this feature is already implemented here https://github.com/techtalk/SpecFlow/pull/996 but I cannot figure out how to use it.
Thanks in advance!
With trial and errors I managed to figure out how it works.
So the annotation you need to use in .feature file is "#mstest:donotparallelize"
If you want to make a scenario not run in parallel:
If you want to make the whole feature not run in parallel:
In the .feature.cs file it generates something like this:
As you can see it also adds a category (tag) to scenarios that are marked with this attribute but I guess it's not a problem at all!
Related
I used to use Cypress 9 on previous projects.
By default, when running cypress open or cypress open --browser chrome used to run all tests for all React components.
However I installed Cypress 10 for the first time on a project that didn't have e2e tests yet. I added test specs, but I don't see any option to run all tests altogether.
It seems I have to run the tests one by one, clicking on each of them.
Can anyone please suggest how do I run all the tests automatically?
It's been removed in Cypress v10, here are the change notes related
During cypress open, the ability to "Run all specs" and "Run filtered specs" has been removed. Please leave feedback around the removal of this feature here. Your feedback will help us make product decisions around the future of this feature.
The feedback page to register your displeasure is here
You can create a "barrel" spec to run multiple imported specs.
I can't vouch for it working the same as v9 "Run all tests", but can't see any reason why not.
// all.spec.cy.js
import './test1.spec.cy.js' // relative paths
import './test2.spec.cy.js'
...
As #Constance says, reinstated in v11.20.
But still a very handy technique if you want to run a pre-defined subset of your tests.
In Cypress version 11.2.0 the Run All button has been reinstated.
You need to set experimentalRunAllSpecs to true in cypress.config.js.
Please see Configuration - End-to-End Testing
If Cypress Test Runner is not a must, I suggest to utilize the CLI/Node Cmd approach
You can trigger all the test(s) by npx cypress run(Still the video recording & screenshot on failed steps would be saved in the respective folders) to run all or with any other cypress flags to filter out specific spec files, or browser etc.
As per the feedback discussion there is a workaround the same as #Fody's answer that will achieve the same result as v9. Also worth noting though is the section on Continuous Integration and the Update 1 that includes a fix for preventing this workaround creating issues with the cypress run command.
Are there any current workarounds?
Yes. If you are impacted by the omission of this feature, it is possible to achieve the same level of parity as 9.x with a workaround Gleb Bahmutov explains here: https://glebbahmutov.com/blog/run-all-specs-cypress-v10/
This will still inherit the same problems as the previous implementation (which is why it was removed) but it will work in certain cases where the previous implementation was not problematic for your use case
https://github.com/cypress-io/cypress/discussions/21628#discussion-4098510
It was removed because people used it wrong.
The Test Runner is for debugging single tests. But by running all tests, then performance will quickly become a problem and crash the entire suite.
Running all tests should only be performed from the CLI.
Sources
https://github.com/cypress-io/cypress/issues/681
https://github.com/cypress-io/cypress/discussions/21628
Here's the background:
I've wrote my tests in protractor#visual studio and got it running successfully with the help of this guide;
Checked the project into TFS;
Now, my intention is to associate my protractor tests to microsoft case tests, then i intent to run against a environment (SCVMM).
I tried Coded Ui Test, but i found then a little bit slow.
Is there a way that i can get this done?
Thanks.
What I did was use Generic Tests to accomplish this.
Each Generic Test you create can represent a Protractor test. I just named them similarly. Then you can link the Generic Test to the MTM test. Simple as that.
I took it a step further and set the Generic Test up in such a way that it runs the Protractor test. This is really nice because Visual studio will recognize these tests in Test Explorer. So no more command prompt :)
Here's an example of one of mine:
I have created a project in Eclipse for Selenium automation using testNG framework. I have never used any user extension file in the past, but I heard that we need to use an user extension file. As since I am unsure of the proper use of this file I have never used it and my project runs smoothly. The question now arises is that later on will I have any issues because of not using an user-extension file? Please give me some ideas on the user extension file and let me know whether its mandatory to use in selenium RC project.
thanks in advance
No, the user-extensions.js file is not required. But you may indeed need to use it, if your existing tests depend on an extension that is in it. Obviously, at that point you need to use the version of the file that the test author used.
I am not quite sure what your question is but I think the answer is No anyway, you do not need to have any "user extension files" in order to use selenium API in any JAVA testing environment. You just call API from your testNG or other testing frameworks. see http://release.seleniumhq.org/selenium-remote-control/1.0-beta-2/doc/java/
While writing my tests this week for a new project, I encountered the TestContext property using the Microsoft Unit Testing Framework available with Visual Studio Team System.
I was wondering:
1. How this property could be used concretely?
2. In other words, what is its purpose?
3. Anyone has a concrete example to provide to ease my understanding?
I have written many tests so far in my career, and have never ever found a use for this particular property.
Any clue anyone?
Thanks in advance! =)
This property can be used to run parametrized tests, like if you have constants you don't want to hardcode in your test project that you want to load and share among all the tests. By default it will be populated with useful information, like the path where your test is running.
Then it is one of the key to run the same test several times with different values. The MSTest framework can be used to link your test project to a database or a csv file and run the tests once for all the rows in the said DB. This is called data driven testing and you'll find many sample on the net. Here is one how to.
Is there an API for running Visual Studio Unit Tests programmatically?
Running MSTests.exe with Process.Start() does not work in the current scenario. What I'm looking for is something like the NUnit SimpleTestRunner.
Any ideas?
/Erik
You're correct in that there's no public API for the mstest framework. I wrote a manual replacement for mstest one day to see how hard it was, and it's not as simple as it looks (particularly if you want to take advantage of more than one CPU core), so beware of going down this path.
Personally I've always just run mstest.exe programatically and then parsed the resulting .trx XML file. Are there any particular reasons why you can't use Process.Start to run it?
P.S. Some of the strange behaviour of mstest.exe are solved if you pass the /noisolation command line parameter - give that a go if you feel so inclined :-)
Update: Erik mentions he wants to run the test API in the current thread so he can set the thread culture for globalization issues.
If you run a unit test under the debugger, you'll notice that mstest creates a bunch of threads, and runs all your tests in different threads, so this isn't likely to work even if you could access the API.
What I'd suggest doing is this:
From your test "runner" application, set an environment variable
Run mstest pointing it at the specific tests
Add a [ClassInitialize] (or [TestInitialize]) method which reads this environment variable and sets the culture
Profit!
After taking a deep dive with reflector into MSTest.exe and further down into the Visual Studio Unit Test stack, I found that the API used by MSTest is sealed up and made private so that i cannot be used from the outside.
Why not using Reflector and seeing how NUnit SimpleTestRunner is running the tests... And then use this technique...
You can make use of the Microsoft REST API's for TFS to run ms tests. Please refer to the documentation here.
I've linked to "Call a Rest API" so that you can see how you'd go about calling one of the REST API's for TFS.
Note that if your tests are linked to the build, they should run automatically every time a build is queued.
Here is the link to Run Functional Tests.
I've also discovered an article on using the TFS SDK API to run tests. Here is that link as well: Link to API Article