I have a tests directory containing several tests packages. I want to see if I am able to have a central main_test.go file which executes all tests packages internally?
This way I would be able to treat all packages as unified and be able to get a unified coverage report etc, without being worried about test setup or without adding any bash scripting etc.
Related
I tried to read https://docs.ansible.com/ansible/latest/dev_guide/ but I couldn't find any instructions about how to run all tests related to a specific ansible module.
As an occasional contributor I do want to locally run tests around the module I am trying to patch. Running the entire test suite for such a big project is extremely hard.
For example lets say I touched a lib/ansible/modules/cloud/docker/docker_image.py file, what do I need to run in order to test my change, before pushing it and creating a PR to test it on CI.
When a private agent build starts in VSTS, it gets assigned a directory, e.g. C:\vstsagent_work\1\s
Is there a way to set this to a different path? On other CI servers, like Jenkins, I can define a custom workspace for a job. I'm dealing with a huge monorepo and have dozens of build definitions around the same repository. It makes sense (to me anyway) to share a single directory on the build agent computer.
The benefit to me is that my builds can use pre-built components from upstream repositories, if they have already been built.
Thanks for any help
VSTS build always creates a working directory per build definition. This leaves you two options:
Create a single build definition and use conditionals on steps to skip certain steps in order to only run what is needed. This allows you to use the standard steps and may require a powershell script to figure out which steps to run and which ones to skip. Set variables from powershell using the special logging commands.
Disable the get sources step and add a step that manually fetches sources. You'll need to clean the working directory, checkout the right commit, basically replicating the actions in the get sources step manually. It may require some fidgeting to get all the behavior correctly for normal build, pull request builds etc. That way you can take full control over the location where sources are checked out.
I'd also recommend you investigate the 2017 project formats that use the new <packageReference> in the project files to fetch packages. The new system supports configuring a version range which can always fetch the latest available version of packages. It's a better long-term solution.
No, it isn’t available in VSTS build system.
You can change working directory of agent (C:\vstsagent_work) (Re-configure it and specify another working folder), but it won’t uses the same source folder for different build definitions, the folder would be 1, 2, 3 ….
I've been trying to get Jmeter load tests to run in VSTS thus far without avail. I've been back and forth (very slowly!) with the Microsoft support team about this, but as the issues are ironed out I would like to at least run a small set of load tests on our build machine using Jmeter and then have the results uploaded somehow to VSTS so they are easier to track. I have part 1 of this working: From the VSTS release definition I run a batch file that runs the load tests locally, and then generates an aggregate spreadsheet with results.
The question is - how can I get those results loaded into VSTS?
In our case we had to export the results to xml using the jmeter.test.xmlouput configuration. Then we had a script to transform the xml in a proper Xunit result file and we finally used a publish test results to gather this file and add the results to the release. (this approach would work with build definitions too).
It's a little bit complicated, requires some scripting and surely would be easier if a dedicated task was available.
We have a build definition for our project that's set to continuous integration and which runs all the unit tests on every check in. It was all running fine until recently, when it began to reject a handful of tests that worked on developer machines.
A moment's examination revealed that this was because the tests were dependent on an external file template for which the path was specified in the app.config file. It was a network drive which developer machines had access to, but which didn't exist on the build server.
The project is a WPF application. Ideally I'm going to try and refactor the code to see if I can bypass that external dependency. But if I can't, or just to satisfy my curiosity, is there a way of editing the build itself to use or deploy a different config file?
You have a much bigger problem: Your code and your tests are dependent on some random, external, uncontrolled file. That file should be something that's source controlled and deployed along with your application.
You also have an external dependency in your tests: It has to read that file from the file system. You may want to break that dependency using standard inversion of control techniques so that tests can run in true isolation, without any need for a file to be present in the file system.
SlowCheetah should work for this use case.
Nuget.org - SlowCheetah
We are currently running a Jenkins (Hudson) CI server to build and package our .net web projects and database projects. Everything is working great but I want to start writing unit tests and then only passing the build if the unit tests pass. We are using the built in msbuild task to build the web project. With the following arguments ...
MsBuild Version .NET 4.0
MsBuild Build File ./WebProjectFolder/WebProject.csproj
Command Line Arguments ./target:Rebuild /p:Configuration=Release;DeployOnBuild=True;PackageLocation=".\obj\Release\WebProject.zip";PackageAsSingleFile=True
We need to run automated tests over our code that run automatically when we build on our machines (post build event possibly) but also run when Jenkins does a build for that project.
If you run it like this it doesn't build the unit tests project because the web project doesn't reference the test project. The test project would reference the web project but I'm pretty sure that would be butchering our automated builds as they exist primarily to build and package our deployments. Running these tests should be a step in that automated build and package process.
Options ...
Create two Jenkins jobs. one to run the tests ... if the tests pass another build is triggered which builds and packages the web project. Put the post build event on the test project.
Build the solution instead of the project (make sure the solution contains the required tests) and put post build events on any test projects that would run the nunit console to run the tests. Then use the command line to copy all the required files from each of the bin and content directories into a package.
Just build the test project in jenkins instead of the web project in jenkins. The test project would reference the web project (depending on what you're testing) and build it.
Problems ...
There's two jobs and not one. Two things to debug not one. One to see if the tests passed and one to build and compile the web project. The tests could pass but the build could fail if its something that isn't used by what you're testing ...
This requires us to know exactly what goes into the build. Right now msbuild does it all for us. If you have multiple teams working on a project everytime an extra folder is created you have to worry about the possibly brittle command line statements.
This seems like a corruption of our main purpose here. The tests should be a step in this process not the overriding most important thing in this process. I'm also not 100% sure that a triggered build is the same as a normal build does it do all the same things as a normal build. Move all the correct files in the same way move them all into the same directories etc.
Initial problem.
We want to run our tests whenever our main project is built. But adding a post build event to the web project that runs against the test project doesn't work because the web project doesn't reference the test project and won't trigger a build of this project. I could go on ... but that's enough ...
We've spent about a week trying to make this work nicely but haven't succeeded. Feel free to edit this if you feel you can get a better response ...
In Jenkins/Hudson, it's quite ok to have many jobs. some for doing compilation triggered version control changes, some for running (unit) tests triggered by successful builds, some for doing more tests (integeration) trigered by successful earlier tests, some for deploying, triggered by successfully passing all tests.
Look at plugins like join, build pipeline, parametrized trigger and more to help out with this.
This will also allow things to happen in parallel, by using multiple nodes. Trying to cram everything in one job is not the way to go.