In an existing ios app and project, how do I enable an existing test target and classes to support UI testing and the record UI test button?
I can add a new test target and I'll see the record button if I do that, but what if I wanted to add some UI testing functions to an existing test class and target?
UI Testing is in a separate target from normal unit test cases, so you can't just add UI testing to you existing test cases.
You should use new UI test target to add/execute your ui tests for clean build and maintenance purpose.However, assuming you are trying to call XCUI* APIs in your unit test classes and enable UI recording with unit test target, I see couple of issues cropping up there,
If you are using swift2 for unit testing and including your main app modules using #testable import {module}, when you call XCUI* menthods from this testcase, swift will throw an error
Module {module} was not compiled for testing
Since UI test run as separate process, it can't load you app code dependencies.
Since XCUI* api depends on Obj-C libraries, invoking UI test methods would expect you to include Objective-C Bridging Header file in your unit test target. It would not error out but it pollutes your target.
I also verified that you need to add your test case file to UI test target in order to enable UI test recording button.
There are other benefits of running UI tests as a separate target, e.g flexibility of specifying the target to only run unit test or UI test from the command line tools and ease of maintaining clean state for UI tests execution.
Related
We have the following project structure:
Workspace:
- app project
- cocoapods project
App scheme:
- app target (run)
- ui test target (test)
- unit test target (test)
After upgrading to Xcode 10, Xcode insists on building the entire project, including pods and ui tests, every time I run a single unit test, which is a quite heavy and slow process.
If i create a seperate scheme which only include the unit and/or ui tests, it only rebuilds the tests when I run them. Just as I want.
However, as they are no longer member of the main app scheme, I can no longer press Test on the main scheme, as it no longer contains test targets.
My question is then: Is it normal or recommended to have seperate schemes for tests, or can I prevent the entire project from being rebuild in another way, when running a single unit test?
(note: I have set the Host Application setting to None on the unit tests, so I don't get why it always builds the entire project anyway?)
Why not both? In your "Test All the Things" scheme, add each test target. This scheme should be shared.
But when I'm working in one target, I make a scheme for it alone. (More accurately, AppCode creates one for me.) Such schemes are not shared.
I'm working on adding UITest to my iOS app. I've gone through the steps to create the new UI Test Target, but when I try to even run the sample test, it seems to be pointing to the same target as my Unit Test (which inevitably fails)
Where do I go in Xcode to configure the target for UI tests?
Thanks!
I created a build definition that runs automated tests using MTM build environments and test suites. I recently created a Visual Studio Load Test, which can be added to a test suite just like any test method marked with the [TestMethod] attribute. However, when I run the build, I get no errors and it appears the aggregate tests don't run. Is there a way to make this work?
I found this article: https://blogs.msdn.microsoft.com/testingspot/2013/01/22/how-to-automatically-run-a-load-test-as-part-of-a-build/ which describes a way to do it, but I can't find a build template that matches what he describes, and it appears this only allows you to run a single load test.
Also, when you configure a test controller, there is an option to configure it for load testing, but to do this, you must unregister it from the Team Project Collection. If this is done, it appears the controller can no longer be used in an environments to run project automated tests. This defeats the purpose of what I want to do and makes it seem that Load Tests and Team Projects are mutually exclusive. Is this the case? If so, this is a big oversight. Load tests are the kind of thing you would like to run automatically. Thanks for the help.
You are unfortunately right. A test controller used for load testing cannot be used for other automated test execution 'at the same time'. In your scenario I would recommend that you setup a different test controller and agent for load testing and you would be able to queue it as a part of your build to achieve what you are looking for.
There is no special build process template for this case.
Is there a way to run the unit tests in a debug build and performance tests in a release build without manually selecting and running individual schemes?
I have a unit test and a performance test scheme. In the test configuration for the unit test scheme, I selected debug build, and for the performance test scheme I selected release build. If I run each scheme individually, I get a debug build and a release build respectively.
If I create another scheme that runs both of these schemes, then that new scheme will have its own build configuration. If I set a build configuration of debug for this new scheme, then I will get a debug build for my performance tests as well.
You could use different test bundles and include/exclude what you want.
What I mean is creating your own custom test targets (bundles) and using the Xcode Test Navigator.
So for example creating a MyUnitBundleTests target and a MyPerformanceBundleTests target. They would be two separate test 'bundles' where you choose to included/exclude classes, methods etc.
The Test Navigator displays a hierarchical list of the test bundles and associated, classes, and methods etc included in a project.
You can enable and disable test bundles, classes, and methods selectively by Control-clicking the items in the test navigator list and choosing Enable or Disable from the shortcut menu, thereby enabling or disabling the items in the scheme.
Testing with Xcode - Quick Start
Testing with Xcode - Running Tests and Viewing Results
There are several additional interactive ways to run tests. Xcode runs tests based on what test targets are included and enabled in a scheme. The test navigator allows you to directly control which test targets, classes, and methods are included, enabled, or disabled in a scheme without having to use the scheme editor.
Not sure if this is what you're wanting but this is the only option I can think of for testing without going up to the Scheme Editor.
There might be a way to achieve the desired behaviour but it is more of a hack.
The main problem I see is the desired build configuration for the test action of the scheme. Since the build configuration applies to all targets in the scheme you need to find a way to trick Xcode into building your target with both Debug and Release configuration. That being said, here is the idea:
Export the release configuration for your target into an xcconfig file (xcodebuild -scheme "schemeName" -showBuildSettings >> release.xcconfig and make sure it only contains the release settings)
Create another target for your performance tests that builds the same source files
Set the debug configuration of this new target to the aforementioned xcconfig file of the release configuration (that is basically the part where we fool Xcode)
Add the original and the newly created target to the build action of your scheme and add the test bundles (assuming they reside in a separate test bundle)
Select the debug build configuration for the test action.
If you test your scheme it should now build the sources in both debug and release and should run your unit tests on the debug build configuration and the performance tests on the release configuration.
The setup is a bit fragile as you need to add new source files to both targets. You should be able to automate the process using a ruby script and the xcodeproj gem though.
Hope that helps.
I'm new to load testing in Visual Studio/MSTest, and I created a new Load Test recently to validate some high-traffic scenarios for a WCF service. I want to add this to the tests project for the service, but I don't want the test to be executed whenever I "Run All Tests in Solution" nor as part of our Continuous Integration build-verification process because a) it takes 5 minutes to run, and b) the service call that it is testing generates many thousands of email messages. Basically, I'd like to do the equivalent of adding the [Ignore] attribute to a unit test so that the load test is only executed when I explicitly choose to run it.
This MSDN Article ("How to: Disable and Enable Tests") suggests that the only to disable the test is to use Test Lists (.vsmdi files), but I don't have much experience with them, they seem like a hassle to manage, I don't want to have to modify our CI Build Definition, and this blog post says that Test Lists are deprecated in VS2012. Any other ideas?
Edit: I accepted Mauricio's answer, which was to put the load tests into a separate project and maintain separate solutions, one with the load tests and one without. This enables you to run the (faster-running) unit tests during development and also include the (slower-running) load tests during build verification without using test lists.
This should not be an issue for your CI Build Definition. Why?
To run unit tests as part of your build process you need to configure the build definition to point to a test container (usually a .dll file containint your test classes and methods). Load tests do not work this way, they are defined within .loadtest files (which are just xml files) that are consumed by the MSTest engine.
If you do not make any further changes to your CI Build definition the load test will be ignored.
If you want to run the test as part of a build, then you need to configure the build definition to use the .loadtest file.
Stay away from testlists. Like you said, they are being deprecated in VS11.
Edit: The simplest way to avoid running the load test as part of Visual Studio "Run All" tests is to create a different solution for your load tests.
Why don't you want to use Test Lists. I think is the best way to do that. Create different Test Lists for each test type (unit test, load test...) and then in your MSTest command run the Test List(s) you want:
MSTest \testmetadata:testlists.vsmdi \testlist:UnitTests (only UnitTests)
MSTest \testmetadata:testlists.vsmdi \testlist:LoadTests (only LoadTests)
MSTest \testmetadata:testlists.vsmdi \testlist:UnitTests \testlist:LoadTests (UnitTests & LoadTests)