I am trying to use Specflow with Playwright in order to do BDD on a portal app developed but I am facing a small problem.
The Specflow project is a separate project with the ASP.Net core server that has the Api of the portal app (it is in Vue). Since the tests are pointing to a specific URL (currently localhost), before running the tests, I need to run the ASP.Net core & Vue project locally. Otherwise, Specflow & Playwright will not be able to do the test (as it will not find the localhost).
Is it any way I can force the run of the Web Server project? I tried to run it from outside Visual Studio with dotnet build and then dotnet run commands but somehow they are missing parameters (that exist while running it from inside VS) and apart from that, these commands must somehow be triggered while trying to run the tests.
I have seen solutions like creating a Docker image from a Docker Compose file in order to pack a .Net project & server in it before running the Specflow tests. Then in the BeforeTestRun hook using the FluentDocker to spin-up the server but I am not quite sure it is the easier (or best) solution.
Does anyone know how I can trigger running the .net core project (with the Vue pages)?
This is actually a pretty big question, with a pretty big answer, however this is well-trodden ground. The issue isn't so much a "specflow" issue as a general automated testing issue. Development practices like continuous integration and continuous delivery can help. Each one is too big for a single question, however I can answer this in more general terms.
In its simplest form, running automated tests locally involves these steps:
Build the application
Deploy the application to a real web server
Run tests
I'm going to assume you are developing in a Windows environment, however every operating system has some sort of command line scripting solution available. The scripting language might change, but the overall idea will not.
Configure a web server. In Windows, this would be Internet Information Services (IIS).
Add a new "application" (or "IIS app" as some people call it) to your localhost web server. Point the physical directory to the root directory for the web project. Repeat this for each web site or web app your system requires.
Write a PowerShell script that gives you an easy way to build and deploy the applications to your local web server.
This script should use publish profiles set up in Visual Studio, which allows you to publish directly from Visual Studio before invoking tests manually through Test Explorer.
Write a PowerShell script used has a "harness" script to coordinate building, deploying locally, and then invoking dotnet test.
Running tests locally just requires a single line of PowerShell to invoke your test harness script:
.\Scripts\Run-Tests.ps1 -solutionDir . -tags BlogPosts,Create
# Skip deploying in case web apps haven't changed:
.\Scripts\Run-Tests.ps1 -solutionDir . -tags BlogPosts,Create -deploy:False
Related
I found this question. Right now, my deployment works as follows:
Hit the web depoly button in VS. In the old ASP.NET world, this was all that had to be done.
Open the developer command prompt.
Run npm run build
Open the project folder and find the index.html file and the dist folder.
Open another Explorer and navigate to the IIS's shared folder.
Copy the files onto the server.
My question is, if there is a simple way to integrate steps 2 to 6 into the UI web deploy command of Visual Studio. I guess, that I am not the only person on the planet who is looking for such a one-click solution, but unfortunately I can't find anything. But I'm pretty sure, that I'm just searching for the wrong keywords.
The most simple solution would be to write a simple script that does steps 3-6, and then find a way to make VS Code run it. However, if you are making regular changes, you should probably commit your code to a repository like git and use CI/CD to make the deployment a part of your code commit. Even if you aren't branching your project - the purpose is to automate your build and deploy workflow. With Continuous Integration (CI), as soon as you commit your code, it will then be automatically deployed. In the long run, following this kind of workflow will save you time.
Some projects that you could use for this kind of workflow:
Jenkins - https://www.jenkins.io/
Gitlab CI/CD - https://docs.gitlab.com/ee/ci/
Netlify - https://www.netlify.com/ - includes CI/CD with free static web hosting
I have a doubt.
How can i create scritps to :
Get my code from repository (GitHub, GitLab...)
Build
Publish
Test
Run in IIS
This script should run in windows or linux OS, and consider that i have a empty VM.
This application is an .Net Core WebApi.
I searched in web but not found an template geting code from repository.
This is doable with scripts like #Scott said and you should consider using solutions for this because there are some great free ones out there like teamcity with octopus integration. Here is what you need to consider if you decide on making scripts for this.
The vm you have is empty so the runtimes need to be installed and
checked are they compatible with code you are trying to deploy to
them.
The scripts for some parts of deployment will need to be run under user with sufficient privileges
You will need to handle the webserver configuration with the scripts as well for all of this
And those only a few things that are on the list for that path. Now having said that there is the path of containers which handle most of this through code and can be deployed to all of environments you mentioned before and you only need to worry that there is a container service on those vm-s you want to deploy to and it will be much easier to handle since like i mentioned it is all in code and is easily changed unlike some scripts.
I'm trying to do something incredibly trivial I thought but apparently this needs to be hard. And yes there are bits and pieces throughout stack overflow but they're either out of date or don't actually work.
I've got an asp.net core site that I've dockerized with the add/docker/linux command.
In VSTS I can build the image and publish it with 2 docker-compose items.
And then I can release the image with the release management.
What I can't figure out how to do:
run dotnet test on my image and report the results to VSTS
Setup environment variables on Azure App Service Container that get properly passed into the image when its run.
On #1, I cannot find any up-to-date documentation on how to set it up so that while developing unit tests don't run unless specifically specified (and if I tell it to run tests in visual studio they should run in the docker image! I can get them to run always, but that's a waste of time while developing if they run every time you start debugging!).
And I cannot figure out how to use either docker-compose or the new VS.net 2017 15.8 way with just docker run commands to run the tests. It seems to me that I would need a new dockerfile just for the tests to run and have it generate and then discard the image that was created. But I can't figure out how to do this or even if this is the right way.
How should this be setup to do unit tests? (I've gone through 5 pages of google search results and none of them work right.)
On #2, setting and application setting in the App Service does not pass the values in docker run. I've tried everything and they never get passed. How do you pass environment variables on Azure so that the run command gets the right -e parameters?
For#1 you could use dotnet test command. This will generate a .trx file that VSTS can pick up and render a nice test report. You just need to setup the “Publish Test Results” task.
dotnet test --logger trx --results-directory /var/temp
More details please take a look at this blog: Running your unit tests with Visual Studio Team Services and Docker Compose
For#2 not totally get your point, if you want to override environment variable values on VSTS and use the value on Azure App Service Container. Please try this solution through powershell script: How to override values of environment variables on VSTS tasks
Beside suggest you also go through this blog shows how Docker Deployment to Azure App Service (Linux) using VSTS including both CI and CD. Which maybe helpful to you.
I am looking to be able to run my postman scripts using newman during a TeamCity build.
Instead of deploying the build to a test environment, I'd like to run the postman scripts on that particular build, so it isn't deployed to an environment used by other developers which could potentially break it.
My current build chain in TeamCity is:
Build main project (contains the REST Api and all required code)
Run Postman scripts using Newman on that project
I have the collection and environment file, along with the CLI command to call it. When I try and point the environment for a local build, it does not work.
I am thinking of running an IIS Express server on the agent and then with that active port, run the tests but I have been unsuccessful.
Any ideas on how to approach this would be appreciated!
I have looked at How do I integrate my Postman Integration Tests with TeamCity and this uses a test environment, which is not what I am after.
I looked at https://ie.com.au/a-how-set-up-automated-api-testing and this was helpful, but I think this is still reliant on setting up a test envrionment.
TeamCity isn't really equipped to handle what you are trying to do. You are trying to run API tests against a build, in order to do that, you'll need an environment. You need something to run your project in order to query against it.
The only potential path you might try looking at is containerizing your project, in docker or something similar, then running your image after it's built and querying against that. However this isn't a great practice and bloats the build time.
A good practice would be to build your project > deploy it to a test environment, you should set up a separate 'test' or 'dev' environment that is ok being broken > after deploy trigger a service to run your tests against the 'dev'
I have a solution with a web application and test project. The test application has coded ui tests of the web application. How can I get the web application to start up when the tests execute?
I've tried
adding a TestSetup method to start the web server via System.Diagnostics.Process. I don't like this because it seems clunky and the coded ui tests don't lend themselves to inheritance real well
adding a setup script in the testsettings to start the web server (same command line) via a batch file. However the batch file never exits and eventually times out. I've tried starting the web server via the start command, but that didn't work -- it works great when you run it from the command line, but not from the testsettings setup script.
I've thought about maybe just running IIS in the background and pointing it to a specific folder. Then deploy the web application to that folder. And let the test project test against IIS.
This seems like a lot of work though. There's got to be an easier way. I want to do this on several developer's machine and also on the TFS build machine via scheduled or continuous integration.
Any thoughts?
Thanks
I configure the web project (in the project properties) to be hosted on IIS. Its just 3 or 4 clicks away. It will prompt to create a virtual application. After that, build and run your project and it will be permanently hosted on IIS. So you can start Coded UI tests without manually starting the web server.
You could do what Gerardo Grignoli suggested and just host the site in IIS so that it's always on.
You could also just start the web project (Without Debugging if you want to debug your tests) and then run/debug the tests. In your tests, just use BrowserWindow.Launch("localhost:appPort/whateverPageToStartOn"); just fill in the port instead of appPort based on your IIS express settings.
In regards to running it on the TFS build machine, that seems like probably a questionable thing to do. You can test the deployed site (dev, test, prod, whatever) from a testing lab. Microsoft Test Manager (MTM) is one of the products that can do this.