We have the following project structure:
Workspace:
- app project
- cocoapods project
App scheme:
- app target (run)
- ui test target (test)
- unit test target (test)
After upgrading to Xcode 10, Xcode insists on building the entire project, including pods and ui tests, every time I run a single unit test, which is a quite heavy and slow process.
If i create a seperate scheme which only include the unit and/or ui tests, it only rebuilds the tests when I run them. Just as I want.
However, as they are no longer member of the main app scheme, I can no longer press Test on the main scheme, as it no longer contains test targets.
My question is then: Is it normal or recommended to have seperate schemes for tests, or can I prevent the entire project from being rebuild in another way, when running a single unit test?
(note: I have set the Host Application setting to None on the unit tests, so I don't get why it always builds the entire project anyway?)
Why not both? In your "Test All the Things" scheme, add each test target. This scheme should be shared.
But when I'm working in one target, I make a scheme for it alone. (More accurately, AppCode creates one for me.) Such schemes are not shared.
Related
How do you test the shared portion of code when you have a multiplatform project? I see tests for iOS and tests for macOS, but nothing for the shared pieces. I want to add unit tests for the platform-independent portion of my app.
Required
The most important part is to import the module to the test target:
The #testable word means that this test can access the internal stuff of the imported module.
Also, note that the MultiplatformAPP is the name of my project. You should look for the name of your project, instead.
✅ Now you can access to the shared section and test it as you like. But there are some extra optional works you can do:
Optionally
You can build a custom bundle for your shared test:
Note that is doesn't matter what template group you are choosing this template from. It's a template! you can change it to anything else anytime. So we started with tvOS 🤷🏻♂️
Then you can nake it independent to any of the targets or depends on any of them you prefer to test with:
Note that I have named it Tests shared to match the pattern of the default tests naming convention of the multiplatform app.
Think we managed to figure this out.
If you go to File > New > Target ...
In the Multiplatform section there isn't many options, but in iOS there is a Testing bundle named "Unit Testing Bundle"
If you add that and name it "Tests Shared"
You can then unit test against files that are part of the Shared target
My Xcode project is very large, in terms of number of source files, and lines of code.
When I make a change to only the XCTest code, and hit run on a single test, a single test file, or even the whole test suite, Xcode re-compiles my host application first, even if no files have changed in the host application. It does so every time I make any change to any test file.
This is posing an efficiency challenge with writing tests, because small adjustments in the test code take a long time to re-run and re-validate. (upwards of 3 minutes per run - yes it's that big of a project, and we are already taking many steps to cut down the build time, by inspecting the build logs, etc...)
Using Xcode 10.3 with Swift 4.2
the tests are using:
#testable import MyModuleName
Is there a way I can prevent Xcode from re-building the whole project, if I've only changed tests?
I found the problem.
We have some custom scripts under the Project's Build Phases section that actually change some of the compilable assets in the codebase. We also have a Carthage dependency copy stage.
Commenting out both of these stages from Build phases makes it so that the whole project does not recompile when only changing tests. These scripts were actually making changes to the compilable code, making XCode force re-compilation.
Moral of the story: Check your build phases for any steps that could be modifying your source code, including possibly resource files that are part of the target. We will try to do this in a more automated way in the future, without having to comment out these build steps.
I'm working on adding UITest to my iOS app. I've gone through the steps to create the new UI Test Target, but when I try to even run the sample test, it seems to be pointing to the same target as my Unit Test (which inevitably fails)
Where do I go in Xcode to configure the target for UI tests?
Thanks!
Is there a way to run the unit tests in a debug build and performance tests in a release build without manually selecting and running individual schemes?
I have a unit test and a performance test scheme. In the test configuration for the unit test scheme, I selected debug build, and for the performance test scheme I selected release build. If I run each scheme individually, I get a debug build and a release build respectively.
If I create another scheme that runs both of these schemes, then that new scheme will have its own build configuration. If I set a build configuration of debug for this new scheme, then I will get a debug build for my performance tests as well.
You could use different test bundles and include/exclude what you want.
What I mean is creating your own custom test targets (bundles) and using the Xcode Test Navigator.
So for example creating a MyUnitBundleTests target and a MyPerformanceBundleTests target. They would be two separate test 'bundles' where you choose to included/exclude classes, methods etc.
The Test Navigator displays a hierarchical list of the test bundles and associated, classes, and methods etc included in a project.
You can enable and disable test bundles, classes, and methods selectively by Control-clicking the items in the test navigator list and choosing Enable or Disable from the shortcut menu, thereby enabling or disabling the items in the scheme.
Testing with Xcode - Quick Start
Testing with Xcode - Running Tests and Viewing Results
There are several additional interactive ways to run tests. Xcode runs tests based on what test targets are included and enabled in a scheme. The test navigator allows you to directly control which test targets, classes, and methods are included, enabled, or disabled in a scheme without having to use the scheme editor.
Not sure if this is what you're wanting but this is the only option I can think of for testing without going up to the Scheme Editor.
There might be a way to achieve the desired behaviour but it is more of a hack.
The main problem I see is the desired build configuration for the test action of the scheme. Since the build configuration applies to all targets in the scheme you need to find a way to trick Xcode into building your target with both Debug and Release configuration. That being said, here is the idea:
Export the release configuration for your target into an xcconfig file (xcodebuild -scheme "schemeName" -showBuildSettings >> release.xcconfig and make sure it only contains the release settings)
Create another target for your performance tests that builds the same source files
Set the debug configuration of this new target to the aforementioned xcconfig file of the release configuration (that is basically the part where we fool Xcode)
Add the original and the newly created target to the build action of your scheme and add the test bundles (assuming they reside in a separate test bundle)
Select the debug build configuration for the test action.
If you test your scheme it should now build the sources in both debug and release and should run your unit tests on the debug build configuration and the performance tests on the release configuration.
The setup is a bit fragile as you need to add new source files to both targets. You should be able to automate the process using a ruby script and the xcodeproj gem though.
Hope that helps.
I'm new to load testing in Visual Studio/MSTest, and I created a new Load Test recently to validate some high-traffic scenarios for a WCF service. I want to add this to the tests project for the service, but I don't want the test to be executed whenever I "Run All Tests in Solution" nor as part of our Continuous Integration build-verification process because a) it takes 5 minutes to run, and b) the service call that it is testing generates many thousands of email messages. Basically, I'd like to do the equivalent of adding the [Ignore] attribute to a unit test so that the load test is only executed when I explicitly choose to run it.
This MSDN Article ("How to: Disable and Enable Tests") suggests that the only to disable the test is to use Test Lists (.vsmdi files), but I don't have much experience with them, they seem like a hassle to manage, I don't want to have to modify our CI Build Definition, and this blog post says that Test Lists are deprecated in VS2012. Any other ideas?
Edit: I accepted Mauricio's answer, which was to put the load tests into a separate project and maintain separate solutions, one with the load tests and one without. This enables you to run the (faster-running) unit tests during development and also include the (slower-running) load tests during build verification without using test lists.
This should not be an issue for your CI Build Definition. Why?
To run unit tests as part of your build process you need to configure the build definition to point to a test container (usually a .dll file containint your test classes and methods). Load tests do not work this way, they are defined within .loadtest files (which are just xml files) that are consumed by the MSTest engine.
If you do not make any further changes to your CI Build definition the load test will be ignored.
If you want to run the test as part of a build, then you need to configure the build definition to use the .loadtest file.
Stay away from testlists. Like you said, they are being deprecated in VS11.
Edit: The simplest way to avoid running the load test as part of Visual Studio "Run All" tests is to create a different solution for your load tests.
Why don't you want to use Test Lists. I think is the best way to do that. Create different Test Lists for each test type (unit test, load test...) and then in your MSTest command run the Test List(s) you want:
MSTest \testmetadata:testlists.vsmdi \testlist:UnitTests (only UnitTests)
MSTest \testmetadata:testlists.vsmdi \testlist:LoadTests (only LoadTests)
MSTest \testmetadata:testlists.vsmdi \testlist:UnitTests \testlist:LoadTests (UnitTests & LoadTests)