So - I have a set of tests that test something - They are sensitive to the current environment - so testing locally tests locally - testing on the build machine tests the deployed server.
However I want to have one method locally that allows me to call the deployed server - but of course I don't want it to run if I run all the local tests - and I don't want it to run on the build machine - I only want it to run when it is the only test being run
ie: I want have something like the following code
[Fact]
public void TestA()
{
}
[FactAlone]
public void TestB()
{
}
[Fact]
public void TestC()
{
}
and if I go into resharper or vs test runner, I see that they are all tests - and if I right click and say "run all" - it runs A and C.... but if I right click on B and say run test - I want B to run.
I tried to make a custom fact attribute and a custom IXunitTestCaseDiscoverer... but to make it work I need to know the complete set of tests that will be run - and that wasn't available in the discoverer. I know I could probably do it it if every attribute was mine - but this is a big codebase, and I can't stop people using "Fact", so that's not an option.
at the moment I just comment out the fact, and then uncomment it when I want to run it - which is of course a terrible solution.
Related
My setup is:
Selenide
Junit5
gradle
I using parametrized tests to run same tests with different users.
On top of that, I want to add BrowserStack.
So eg. having test
#ParameterizedTest(name = "{index}; user: {0}")
#ArgumentsSource(...)
public void exampleTest(TestUser user) {
I want to end up with, eg.
(0) exampleTest; user 1; Firefox
(1) exampleTest; user 2; Firefox
(2) exampleTest; user 1; Chrome
(3) exampleTest; user 2; Chrome
Problem is that BrowserStack example for junit5 is (a) written for plain selenium, not selenide and (b) doesn't seem to work with ParameterizedTest.
Have anyone attempted such setup, and can point be to right direction? I don't want to reinvent the wheel if I can use out-of-the-box solution ;)
I'm also open to use SauceLabs instead of Browserstack.
I think, a good approach is to isolate your test domains
Browsers switching in test runtime is not good solution since Selenide has static configuration
You have to split test configuring and test running
1 part - your configuration domain: external browsers configuration (you can implement this approach via JUnit5 extensions) Example - JUnit5 extension that configure your test libs (Selenide, etc) according to some external env variable
2 part - your test domain - parameterized tests via your test data (users, conditions, etc)
I think you can use the Official Browserstack documentation for running the Selenide test parallel on various browsers.
https://www.browserstack.com/docs/automate/selenium/getting-started/java/selenide/run-tests-in-parallel
https://www.browserstack.com/docs/automate/selenium/getting-started/java/selenide#introduction
Also, you can take help from their GitHub Repository for Selenide with TestNG, where they have used #parameters with their test cases.
https://github.com/browserstack/selenide-browserstack.git
I hope this helps.
Thanks
I have an existing flutter application, that I would like to configure with a couple of simple unit tests. I followed instructions from https://codelabs.developers.google.com/codelabs/flutter-app-testing#0 with pointers on how to configure a project with unit tests, and applied those steps to my own project.
After applying those steps flutter test is not behaving like how I would expect, and blatantly just doesn't work. In specific, it yields the following output Connection closed before test suite loaded.
While trying to resolve the error a couple of oddities:
When I ran unit tests (from vscode) I noticed that the application was "paused" on an unhandled exception.
When I looked into the exception, it was about a dependency my actual application relies on. Specifically, about SQLite not being able to find the .so/.dll
When I looked into the callstack, I noticed that it was basically just loading my actual application instead of the unit tests own entry point. Specifically, it was loading all of my app providers.
After I removed that specific exception, it would continue to fail on other parts. For example about unhandled exceptions over missing assets. This is something I could work around with --[no-]test-assets to force the compilation of assets.
The behavior I have witnessed above. I sort of came to a conclusion that it's trying to execute the actual application instead of the tests altogether (hence the various missing dependencies etc). So as a theory, I removed the main function from my main.dart. Running flutter test then fails to compile some autogenerated generated_main.dart, because the entrypoint is missing.
//
// Generated file. Do not edit.
// This file is generated from template in file `flutter_tools/lib/src/flutter_plugins.dart`.
//
// #dart = 2.6
import 'package:liquid/main.dart' as entrypoint;
import 'dart:io'; // flutter_ignore: dart_io_import.
import 'package:path_provider_linux/path_provider_linux.dart';
import 'package:path_provider_windows/path_provider_windows.dart';
... //stripped for brevity
typedef _UnaryFunction = dynamic Function(List<String> args);
typedef _NullaryFunction = dynamic Function();
void main(List<String> args) {
if (entrypoint.main is _UnaryFunction) {
(entrypoint.main as _UnaryFunction)(args);
} else {
(entrypoint.main as _NullaryFunction)();
}
}
From what I observed, most of the issues seem to come from the fact that's choosing to run my actual application instead of the unit test. That first line import 'package:liquid/main.dart' as entrypoint; is basically referencing the application entrypoint.
According to https://github.com/flutter/flutter/issues/8516 there used to be a time where flutter test wasn't supported but it's already been closed. So I assume there are no fundamental broken issues "just because windows platform".
However, it appears that something is not completely right on my setup and I am not really sure what I need to do to get a simple test case working.
I would like to ask a question regarding UI Test when continuously navigation from one page to another and so on. If I launch a UI Test, do I need to continuously navigate from one page to another for one specific test or can it be chopped into pieces without launching the app again and again? Thanks
e.g.
[Test]
public void Navigation()
{
app.Tap("button1NavigateToPage1");
app.WaitForElement("ElementInPage1"); //If Success
app.Tap("buttonNavigateToPage2");
app.Tap("buttonNavigateToPage3");
}
The test should start from normal state. That is why the app is relaunched every test.
It shouldn't take a lot of time and you don't have to watch it doing the things it does.
It will show you a summary in the end of failed tests and you can rerun those and check them again.
You can also take a screenshot to make sure that the flow went as intended
app.Screenshot("First screen.");
I'm taking over a project and am trying to write some tests for CodeIgniter to run on the command line with continuous integration and there seem to be very little documentation online on testing with CodeIgniter and getting tests to run for controllers seem particularly difficult.
There's some previous answers suggesting to use FooStack, but FooStack is for CodeIgniter 1.x as best as I can gather.
I'm at the point not where I'm considering running selenium over the whole site. Has anyone managed to test controllers with CodeIgniter?
Related answers:
How to test controllers with CodeIgniter?
Using my-ciunit.
Grab a copy of https://bitbucket.org/kenjis/my-ciunit
Install PHPUnit 3.7.x
Run the installation instructions.
Presumably you didn't name the dadabase on the command line so go to application/config/testing.database.php and name your database with _test
Create your database.
Go to the tests folder and delete all the folders except controllers. All these files are examples and obviously you won't have those fixtures.
You only need the SomeControllerTest.php as a starting point. Edit it to hit a controller you actually have. I wanted to just add a basic access test.
class HomepageControllerTest extends CIUnit_TestCase {
public function setUp() {
$this->CI = set_controller('homepage');
}
public function testIndexController() {
$this->CI->index();
$out = output();
$this->assertNotEmpty($out);
}
}
I wrote two articles on testing Codeigniter both Unit and Acceptance tests. I use PHPUnit and Selenium for that.
Here are the links:
Unit tests: http://taiar.com.br/2013/11/08/testing-codeigniter-applications-with-phpunit/
Acceptance tests (Selenium): http://taiar.com.br/2014/04/21/acceptance-tests-on-codeigniter-with-phpunit-and-selenium/
I'm planning to write an article on tests at all (with testing theory) but my two first articles should help!
I have different tests with different deployment items, like this:
[TestMethod]
[DeploymentItem("item1.xml")]
public void Test1(){...}
[TestMethod]
[DeploymentItem("item2.xml")]
public void Test2(){...}
If I run the tests one by one it works, however if I run them all togheter all the deployment items are copied, so Test2 will run wit item1.xml and item2.xml in the output folder.
What I am trying to do is run each test only with the specified deployment items in the output folder.
Is there any way to clean deployment items after each test?
My workaround is deploying the items to an output subfolder for each test, but I dont like it.
If you define the following in your test class, you may be able to accomplish what you're looking for:
[TestInitialize()]
public void Setup()
{
//Delete all files from the deployment directory
}
I tend not to use files as part of 'unit' tests, so I'm not positive about this, but it's probably worth looking into if you'd really prefer the files to be isolated in that directory, and not in sub-directories.