So, I have multiple selenium test projects in separate VS solutions and they all have the same Setup() methods which are executed before scenarios are run as well as all having the same TearDown() methods for after. Currently if a change is required for these methods, they have to be updated separately so I was looking in to centralising these methods to be used in all of the test projects/solutions.
I'm relatively new so apologise in advance but does anyone have experience of this with suggestions on approaches I could take? Is it even possible? My tests do not currently run in parallel so is this something I'd need to look in to?
How if you change code be
[OneTimeSetUp] and [OneTimeTearDown]?
in my opinion you should create one class for setup include setup and tear down then in yor test add setup like this public class HREmployeeList:Setup
Hopefully help or not just check this below link
http://toolsqa.com/selenium-webdriver/c-sharp/how-to-write-selenium-test-using-nunit-framework/
One thing you could do is create a new project that contains the functions for setup and teardown and then include that compiled dll in all the other projects. If you need to make a change to setup/teardown, you make the change and compile a new dll and the code change is passed to all the other projects.
Keeping long story short - you need to create a Meta framework.
By concept Meta frameworks provide a method of solving the problem with automating multiple pieces as part of a larger automation strategy. Testers define independent utility classes which can be generically used with any automation tool and can be reused between different automation projects as well. The framework provides an abstraction layer that allows the separate automation pieces to be executed and have their results reported back in a standardized way.
I have a post on the topic, so feel free to take what you need form there.
Since you've tagged VisualStudio, I'll share first the approach my team did use for sharing common functionality across Test projects. What you need is a private Nuget server. Each team compiles and supports a Nuget pack based on the service it provides. For example Selenium code, API calls etc.
Next and probably closer to your case solution would be to utilize git submodules and share the Test harness engine between your projects.
Both approaches will benefit of a Fixture Setup Patterns like Shared Fixture Construction.
Related
My goal is to be able to write core testing that I can use within a unit testing framework as well as UI testing with selenium.
For simple test like:
Scenario: Add two numbers
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press add
Then the result should be 120
I would create both unit tests to prove that my core API would pass as well as a Selenium test that would prove my UI is doing the correct thing as well.
I briefly tried to find anyone doing something similar through Google, but couldn't find any examples. So I guess my question is, has anyone here done anything similar?
On approach I had thought of was simple adding the feature files to a project or directory and using the add existing item as link as the solution.
Update: Adding feature files to a common directory and adding them as a link appears to be working great. The feature bindings regenerates for each project the feature file was included in so I can run unit tests in one and Selenium UI tests in the other.
First, lets start with why you might want to do this. Its laziness of the good kind.
The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful, and document what you wrote so you don't have to answer so many questions about it. Hence, the first great virtue of a programmer. Also hence, this book. See also impatience and hubris. (p.609)
Larry Wall, Programming Perl
Except it isn't, because we aren't going to reduce our overall energy expenditure.
When you are using SpecFlow, the easy part to keep up to date is the plain text. You will find yourself refactoring the [Binding]s again and again, but the scenarios tend to be quite easy to work with, and need very little revision once they have been agreed.
In addition the [Binding]s are global. Load them in from any assembly and they are available to the SpecFlow runner. In respect of what you are trying this actually makes things harder as you need to put effort in to keep the UI bindings from being mixed up with the non-UI bindings.
Also consider the way that SpecFlow actually runs the tests from feature files. It's a two stage process.
When you save the .feature file the SpecFlow VS plugin generates a .feature.cs file.
When you run your test engine (e.g. NUnit) it ignores the plain text and uses compiled code from .feature.cs
So if you start using linked .features I have no idea if the SpecFlow plugin will generate .feature.cs for both instances of the file. (If you try this please let us know)
Second lets consider the features themselves. I think you will constantly finding yourself compromising your tests to make them fit the other place they are used. Already in the example you have given you have on the screen. If you are working with just the core API then there won't be a screen, so do we change this to fit better in a non-UI scenario?
Finally you have another thing to consider, just how useful will your tests be. If you have already got a test that tests the Core API, then what will it mean to run same test via Selenium. All you will really test is the UI layer. In my current employment we have a great number of regression tests that perform this very kind of testing, running up a client that connects to a server and manipulating the UI to get the desired scenarios enacted. These are the most fragile tests we have due to their scale. They constantly break and we basically have to check our entire codebase to find the line that broke them. Often something like 10-100 of them break just for a one line change. If these tests weren't so important to the regression cycle then the effort in maintaining them would just be too much. In my own personal projects I tend to remove these tests completely and instead with UIs, I avoid testing the View layer. With WPF MVVM, I execute Commands and test for results in ViewModels. If somebody then decides the TextBox should be a ComboBox or that it will work better in mauve, then my testing is isolated.
In short, there is a reason you can't find anything about this on Google :-)
In general (see http://martinfowler.com/bliki/TestPyramid.html), one should limit the number of automated tests that test the UI directly, and prefer tests that start at the presentation layer (just below the view layer), or below.
SpecFlow is agnostic; the tests can be implemented using e.g. Selenium at the UI layer or just MSTest or NUnit at any of the layers below.
However having said that I appreciate that you will have situations where you are doing ATDD and want to implement SpecFlow scenarios to match each of the acceptance criteria. Some of the criteria will be perfectly fine to test at a lower architectural level, but one or two of them may be specific to the GUI-- for example testing Login and ensuring that the user is redirected to the home page after successful login. If using Angular2 or React routing (see https://en.wikipedia.org/wiki/Single-page_application), that redirect is likely done in the GUI layer itself.
I don't have a perfect answer yet, but as a certified SpecFlow trainer, I have a vested interest in this! The way I am currently leaning is to use a complementary tool like CucumberJS for the front-end specific tests (such as testing React router redirects) and SpecFlow for tests at lower architectural layers. Our front-end uses Node.JS/Express and our backend is .NET Core. The idea is that the front-end tests mostly use the front-end only with mocked out AJAX calls to the backend (see sinonjs), and the back-end tests use EF Core with the in-memory option (see docs.efproject.net/en/latest/providers/in-memory/. So the tests all run fast.
Of course, you still need a few tests that actually go all the way through, but those are different-- we should call those integration tests. I do not believe that acceptance tests need to be integration tests. That way, you have a suite of acceptance tests from doing ATDD, plus a relatively small set of integration tests that test all the way front-to-back. The integration tests run more slowly and require more maintenance, so you separate them out into a different part of the CI/CD build chain.
I hope this makes sense. It is not so much solving the problem as avoiding the problem.
I have looked for similar questions, but could find none other than the difference between solutions and projects. Mine is on the same level, I suppose, but is slightly different.
I'm a previous Java developer thrust into C# recently and I am the sole individual charged with setting up source control, project standards, etc. etc. and this is my first go with Visual Studio (using 2010 Pro).
I understand a solution is a "container" for related projects, but I am unsure the best practices of adding projects to a solution which are related, but are of a different type.
For example, would I place a web project in the same solution with a desktop application or mobile app if they are related (rather near identical) in function? They are basically the same app, but in different formats. They may use the same classes (for a contrived example, a Person class).
To me, they seem obviously related, but are different applications, so it would seem they should be in different solutions.
I appreciate any feedback offered.
Thanks in advance.
Those apps should be in one solution, expecialy if they share functionality, common projects etc. Quite good approach is to group projects within a solution using solution folders, for example "Common", "Web", "Mobile", "Setup" etc. This way you can have logical groups inside solution.
I would place them in the same solution, since this makes it easier to have a common class library as one project in that solution.
There are no rules for this so there is no right or wrong answer. It all comes down to how you want to organize your code. We commonly have web apps and console apps in the same solution because they are functionally tied together and share code so the type of project really does not matter.
I would place them in the same solution. You can create different configurations to build each application or build both of them at the same time. This allows you to change the class and verify changes made to the class will work for both applications.
Also if you start to see functionality start to duplicate across applications it is easier to create new classes and project that can apply to both applications.
You can throw everything in one solution. This is convenient if you often have to debug the library and application code together, but if the solution has many projects it can become unwieldy.
One approach is to treat the solution as 'that which will be installed as a unit.' Then your common library will go in one solution, and your mobile and desktop apps in their own solutions. This keeps solutions smaller, but it can be inconvenient when you are developing everything together-- so it works best when the library is very stable.
I was wondering if there is an off the shelf tool to help create a 'web application' directory structure with a basic application up and running? The source code would already be pre-written I'm guessing or based on the options some files may/may-not be generated.
I'm not sure what you'd call it but say we have a custom framework* which are to be used for web application development - rather than 'creating' a directory structure all the time we could just create it once and have a console like interface similar to the play framework to generate a basic application or an empty one as per the developer's choice.
We could just give various types of 'zip' files and ask folks to unzip it and import it in their IDE of choice and continue. However, we'd prefer to have an 'installable' to run from the command line (or GUI but no such preference) to have a basic application up and running without everyone wanting to do it all over again.
How does the Play framework do it? What do they use? (I'm guess similar things exist for RoR, Groovy/Grails.)
*It's not custom per se, but similar to having all the spring/hibernate/restlet/freemarker etc files pre-configured, up and running and a directory structure with packages for the various components by convention
I think one of the key points here regarding the Play framework is that it uses the concept of convention over configuration. Your applications are forced to follow the same pattern for the different parts of your application, or it will not work. I personally like this because it makes working on different projects easier, as the rules are always the same, rather the somewhat unwritten rules of best practice.
Java EE on the other hand takes the concept of configuration over convention. Therefore all your files and structures are defined in your relevant XML documents that specify your frameworks, classpaths, etc. There do exist some tools to try to bridge the gap. For example
IDE's will have project creation tools for your chosen framework, so will create a Struts or Spring MVC project structure with a few simple wizard steps. Eclipse does this for sure as one example.
Spring MVC also has Roo. This is a boilerplate code generation tool that creates large parts of your initial project for you.
From your description it seems you have a few different frameworks that you want to have auto-generated, but I don't think any tool currently will serve your purpose. Your concept of a zip file is your best bet here.
If you want a kind of scaffolding in the Java EE world, take a look at Appfuse which provides some archetype with several implementations on the views layer (JSF, Spring MVC, Struts 2...).
While writing my tests this week for a new project, I encountered the TestContext property using the Microsoft Unit Testing Framework available with Visual Studio Team System.
I was wondering:
1. How this property could be used concretely?
2. In other words, what is its purpose?
3. Anyone has a concrete example to provide to ease my understanding?
I have written many tests so far in my career, and have never ever found a use for this particular property.
Any clue anyone?
Thanks in advance! =)
This property can be used to run parametrized tests, like if you have constants you don't want to hardcode in your test project that you want to load and share among all the tests. By default it will be populated with useful information, like the path where your test is running.
Then it is one of the key to run the same test several times with different values. The MSTest framework can be used to link your test project to a database or a csv file and run the tests once for all the rows in the said DB. This is called data driven testing and you'll find many sample on the net. Here is one how to.
I was wondering what would be a good UI to specify test cases.
Currently we use macros with excel to specify our test cases and generate an xml out of it and export it to the script generator.
Excel is good and really flexible and allows testers to enter their test cases very quickly.
However the xml generated is sometimes not well formed and the system has a huge learning curve.
I want to change the UI from excel to something else that would allow testers to enter test cases quickly and provide flexibility.
A nice TDD tool is SLIM/FitNesse. It is a wiki system which allows to enter special tables and/or commands which trigger test methods. These test methods can be written in Java and .NET (other languages might be supported). Also there are various plug-ins for doing DB testing or Selenium web tests. Here is a first tutorial video.
I've used Test Link for this sort of task. It's an opensource php project.
You might check out Fitnesse, which does a similar thing. http://fitnesse.org/